What self-hosted AI agents really mean for compliance teams

In April 2026, United States federal banking regulators issued SR 26-2, the long-awaited update to fifteen-year-old model risk management guidance. The new guidance explicitly excludes generative and agentic AI from its formal scope, citing technology that is "novel and rapidly evolving" (Federal Reserve, 2026).

For compliance teams evaluating AI agent platforms, that gap is the starting point, not a footnote. Regulators have not finished writing the rulebook for agentic AI, and that absence does not remove an enterprise's obligation to govern these systems responsibly. It shifts the burden onto whoever deploys them.

"Self-hosted AI agents" means the orchestration engine, and ideally the model inference layer itself, runs inside infrastructure the enterprise controls rather than inside a vendor's cloud. That single architectural choice determines where prompts, documents, and reasoning traces physically reside, who can access them, and which jurisdiction's courts can compel disclosure.

This piece examines what the term actually changes, technically and legally, and what it does not.

The regulatory rules compliance teams are actually facing today

Multiple binding rules already apply to AI-driven automation in regulated industries, even though comprehensive agentic AI regulation remains years away. DORA (Digital Operational Resilience Act) has applied to every EU financial entity since January 17, 2025.

It treats AI vendors as ICT third parties requiring formal risk assessment (EIOPA, 2025). Compliance teams need a "what applies today" list, not a forecast.

EU rules: DORA and the AI Act

DORA requires EU financial entities to maintain a Register of Information covering all ICT third-party relationships, including AI vendors (EIOPA; ESMA, 2025). This is not guidance under consideration; it is a binding operational requirement with an existing enforcement mechanism.

The EU AI Act layers on top of DORA on a staggered timeline. Prohibited-practice rules took effect February 2, 2025, and GPAI transparency obligations followed on August 2, 2025 (European Commission, 2025).

Under the May 7, 2026 "Digital Omnibus" agreement, the high-risk system deadline covering many financial-sector uses moved from August 2026 to December 2, 2027, and product-embedded systems were pushed to August 2028 (Holland & Knight; Travers Smith, 2026).

Penalties did not move with the deadlines. Prohibited practices still carry fines of up to €35 million or 7% of global annual turnover, whichever is higher (artificialintelligenceact.eu, 2025).

US rules: a fragmenting state-by-state picture

Colorado's SB 26-189, signed May 14, 2026 and effective January 1, 2027, regulates automated decision-making with pre-use notices and adverse-outcome explanations (Colorado General Assembly; Norton Rose Fulbright, 2026). It's the most detailed state framework so far, and other states are watching it closely.

Texas and California are moving on their own tracks. Texas TRAIGA took effect January 1, 2026, with penalties up to $200,000 per violation (Norton Rose Fulbright, 2025). California's CCPA automated decision-making rules take effect January 1, 2027 (Baker Botts, 2026). No two states are converging on the same requirements.

Insurance-specific rules: the NAIC bulletin

Insurance carriers face a parallel, faster-moving track. The NAIC Model Bulletin on Insurers' Use of AI, adopted in December 2023, requires a written AI governance program covering underwriting, rating, claims, and marketing.

More than 20 states had adopted it by mid-2026, and the NAIC began piloting an AI evaluation tool for market-conduct exams in 12 states as of March 2026 (NAIC; Quarles Law; Plante Moran, 2026).

Security and risk concerns, not regulatory uncertainty, now rank as the top barrier organizations cite when scaling agentic AI. Average Responsible AI maturity scores rose to 2.3 in 2026, up from 2.0 the prior year, and just 14% of organizations report fully integrated data readiness (McKinsey, 2026). None of this waits for federal clarity.

What "self-hosted" actually changes, technically

Self-hosting changes where four specific control decisions get made: data residency, audit trail location, sub-processor exposure, and encryption key custody. Each shifts from a vendor-controlled default to an enterprise-controlled configuration, which is precisely what most 2026-era compliance frameworks now ask organizations to demonstrate.

Data residency: deployment decision vs. inference gamble

For self-hosted systems, data residency is fixed at deployment time. For cloud-hosted agentic systems, it becomes a live, per-request gamble. Agentic architectures make dynamic routing decisions: external tool calls, cross-region vector database replicas, sub-agent delegation. These can move data outside a defined region even when the source database never left home.

This means residency claims made about a cloud agent platform at contract signing may not hold true six months later. New tools or sub-agents added to the workflow can shift where inference actually runs.

Audit trail control

Cloud LLM APIs place prompt and response logs, along with reasoning traces, on vendor infrastructure by default. That moves the audit trail outside the enterprise's defined security perimeter. Self-hosting keeps every prompt, retrieved document, and generated answer inside infrastructure the enterprise already audits under existing controls. A separate vendor logging system, requiring extra access requests, becomes unnecessary.

Sub-processor chain depth

Sending data to a cloud LLM vendor extends trust to its sub-processors, and to whichever sub-processors those sub-processors add later. Anthropic's Commercial Data Processing Agreement grants general pre-authorization for new sub-processors with only fifteen days' notice (Heuking, 2026). Self-hosting removes this chain, since no data leaves enterprise-controlled infrastructure.

Encryption key ownership: BYOK vs. HYOK

The distinction between BYOK and HYOK is where most procurement conversations stop too early. BYOK, Bring Your Own Key, still stores the customer's key inside the provider's own key management service, meaning the provider retains technical access. HYOK, Hold Your Own Key, keeps keys exclusively in infrastructure the customer controls (IBM, 2026).

Property	BYOK	HYOK
Key storage location	Inside the provider's own key management service	Inside infrastructure the customer controls
Provider technical access	Provider retains technical access to the key	Provider has no technical access to the key
Compliance implication	Data can still be technically decrypted by the provider	Decryption capability sits entirely outside the provider's reach

BYOK is frequently marketed using language nearly identical to HYOK. This is why compliance reviewers need to ask the follow-up question directly: where does the key physically live, and who can technically retrieve it.

Why vendors gloss over the orchestration-vs-inference distinction

The single most consequential detail in a "self-hosted" AI claim is whether inference is self-hosted, not just orchestration. A workflow engine can run entirely inside enterprise infrastructure while every model call still travels to an external LLM API.

Self-hosting the orchestrator alone solves none of the residency problem unless the inference layer is locked down separately, typically by routing to a self-hosted or on-premises model instead of a cloud AI endpoint.

This gap shows up across self-hosted workflow and AI agent tooling broadly, not in one isolated case. Vendors rarely volunteer the distinction, because "self-hosted" sounds complete on its own.

In practice, orchestration and inference are two separate architectural decisions, each carrying its own residency, logging, and sub-processor implications. A compliance reviewer needs to evaluate them independently rather than assuming one implies the other.

Plenty of platforms market self-hosting explicitly for GDPR, HIPAA, and SOC 2 needs, but local orchestration infrastructure does not, by itself, guarantee local inference; a self-hosted workflow can still route every model call to an external LLM endpoint unless that is separately locked down. The orchestrator sits inside the perimeter; the reasoning frequently does not.

VPC-isolated LLM deployment addresses this gap directly, because it keeps inference itself, not just storage or orchestration, inside a controlled network boundary. That single property satisfies GDPR data residency, PHI isolation, and financial-services accountability requirements simultaneously.

Compliance teams evaluating any platform should ask specifically where the model executes, not only where the workflow logic runs. This is the distinction Lunnoa is built around: orchestration and inference both run inside the customer's own infrastructure, not just one of the two.

Why a SOC 2 badge does not answer the question a regulator will ask

A SOC 2 report attests to a vendor's internal process controls; it does not attest to where customer data or AI inference actually resides. SOC 2 contains no AI-specific controls for training-data absorption, inference logging, or model-weight exposure (Linford & Co, 2026). A shared cloud environment with SOC 2 controls remains, structurally, a shared environment.

This gap matters because AI-specific breach risk is measurably different from the risk SOC 2 was built to address. Shadow AI, meaning AI tools used without organizational approval, illustrates that difference clearly.

Breach type	Average cost
Standard breach	$4.44M
Involving shadow AI	$4.63M

Source: IBM Cost of a Data Breach Report, 2025 (600 organizations surveyed, March 2024–February 2025)

Source: IBM, "Cost of a Data Breach Report," 2025

Breaches involving shadow AI cost $4.63M on average, $670,000 more than standard incidents. One in five organizations reported a shadow AI breach, and only 37% have policies to detect or manage it (IBM, 2025).

Source: IBM, "Cost of a Data Breach Report," 2025

Separately, 13% of organizations reported breaches of AI models or applications directly, and 97% of those lacked proper AI access controls (IBM, 2025). A SOC 2 badge on a cloud AI vendor says nothing about whether an employee routed sensitive data through an unapproved AI tool. It also says nothing about whether the vendor's own AI access controls were adequate (Linford & Co, 2026).

Two misconceptions getting in the way of a clear-eyed decision

Self-hosted AI is not inherently behind cloud frontier models

The capability gap between self-hosted and cloud frontier models has narrowed enough that most 2026 analyses describe pragmatic hybrid routing rather than an absolute tradeoff. Predictable, high-volume workloads run self-hosted, while burst or frontier-reasoning tasks route to cloud APIs as needed. Treating self-hosting as a permanent capability sacrifice misreads how enterprises are actually deploying it today.

Self-hosting is cheaper at real production volume

Cloud AI platforms typically bill per token or per execution, so the invoice scales with every workflow run. Self-hosting removes that recurring meter: infrastructure cost stays fixed regardless of usage, so the cost advantage compounds as usage grows.

For any organization running AI agents in production, not piloting a handful of workflows, self-hosting is the cheaper path over time, on top of being the more controlled one.

What a genuinely self-hosted platform looks like in practice

The pattern across this piece points to one bar: a platform is self-hosted in the full sense only when orchestration, inference, audit logging, and key custody all run inside infrastructure the customer already owns, with no component quietly carved out for a vendor's convenience.

Partial self-hosting, where only the workflow layer sits on-premises while model calls and logs still leave the perimeter, does not clear that bar even when the marketing says "self-hosted."

Lunnoa was built around that full-stack definition from the start. Every component, including the workflow engine, document processing, and LLM routing, runs on the customer's own AWS, GCP, Azure, or on-premises infrastructure. Compliance teams evaluating any platform, Lunnoa included, should apply the same checklist below rather than take a vendor's self-description at face value.

A practical checklist for evaluating a "self-hosted" claim

A vendor's use of the word "self-hosted" is not a specification; it is a marketing claim that needs verification against six concrete architectural questions. Compliance and security reviewers should require written answers to each before sign-off, not verbal assurance during a sales call.

This list is not theoretical. Lunnoa is deployed inside a private equity firm's compliance function today, running self-hosted automation of internal compliance processes, and the questions below reflect what that deployment actually required answering, not a checklist assembled from vendor marketing pages.

1. Does inference run inside enterprise infrastructure, or only orchestration? Confirm model calls never leave the network boundary.
2. Where physically do encryption keys live, and who controls the key management service handling them? Confirm HYOK, not BYOK: the provider should have no technical access to the key at any point, including during key rotation or backup.
3. Are prompts and reasoning traces logged inside the enterprise's own audit system, without a separate vendor log?
4. What is the full sub-processor list, and how is the enterprise notified of changes? Fifteen days' notice, as in Anthropic's Commercial DPA, is not pre-approval.
5. Can data residency be verified per-request, not just at contract signing? Dynamic tool calls and sub-agent delegation can shift residency after deployment.
6. Does the enterprise, not the vendor, control model updates and version changes? Unannounced model changes complicate audit trails and consistency requirements.

The bottom line

Regulators have not finished defining how agentic AI should be governed in regulated industries. SR 26-2's explicit exclusion of generative and agentic AI makes that gap official rather than theoretical.

Self-hosting does not close every open regulatory question. It does put data residency, audit trails, sub-processor exposure, and key custody back under enterprise control while those questions remain open.

The distinction that matters most is not whether a platform is labeled "self-hosted." It is whether inference, not just orchestration, actually runs inside infrastructure the enterprise controls.