BRACEWorks - Cloud & AI Security

Defense in Depth for LLM Applications

Companies are moving quickly to put AI into everyday work. Employees are using advanced models to analyze information and create content. Development teams are building chatbots that retrieve internal documents. Agents are connecting to business systems and taking actions on behalf of users.

These capabilities create opportunities for productivity. They also introduce a security question: what happens when the information an AI application reads contains instructions designed to manipulate it?

A connection can be encrypted. The user can be authenticated. The destination can be approved. And the interaction can still lead to data exposure or an unauthorized action.

This is why we see the LLM Gateway becoming an important part of enterprise security architecture. It provides a central place to apply policies and inspect interactions with models, while working with the network, identity, and application controls the company already has.

IBM and the Ponemon Institute’s Cost of a Data Breach Report 2026 makes a similar point at scale. Among organizations that had an AI-related breach, incidents were less about which model they chose than about the security of the model and its surrounding environment. Root causes were often structural: compromised APIs, applications, and cloud misconfigurations, pointing to governance failures rather than model risk. AI-related breaches also grew sharply, up 61% year over year to 21% of the organizations studied.

Firewalls and WAFs remain essential

We have spent years building layers of protection around our applications. Firewalls control communications and restrict access. WAFs help protect web applications against exploitation and abuse. Identity platforms determine who can access a resource.

All of these controls remain critical as we adopt AI.

The challenge is that attacks against LLM applications can operate through legitimate content and legitimate connections. A document can contain instructions that attempt to change an agent’s behavior. A tool response can ask the agent to disclose information. Neither situation necessarily requires a malformed request or a traditional software exploit.

Consider a chatbot using Retrieval Augmented Generation, or RAG. It retrieves documents and includes them in the context sent to the model. Those documents help the application answer questions, but their contents should not acquire the authority to change the application’s instructions.

The same concern becomes more significant when an agent can use tools. An incorrect response is one problem. An agent using its access to send information or change a business record is another.

The OWASP guidance recognizes prompt injection as a risk and explains that RAG does not eliminate it. See the OWASP guidance for LLM applications. Three attack scenarios help explain what this means in practice.

Prompt injection: when information becomes an instruction

Prompt injection happens when content attempts to redirect a model away from the intended task or the application’s rules.

The attacker does not always need to interact directly with the chatbot. Instructions can arrive through an email, a document, a website, or the result of a tool call.

The financial impact is measurable. IBM’s 2026 research found breaches involving an AI model prompt injection incident cost an average of USD 5.89 million, nearly 18% above the global breach average, and breaches involving model inversion (extracting sensitive data from a model) were even costlier at USD 6.07 million.

In June 2025, researchers disclosed EchoLeak, a vulnerability in Microsoft 365 Copilot identified as CVE-2025-32711. They demonstrated how a specially prepared email could lead to the disclosure of information from the Copilot context without requiring the user to click the malicious content.

This was a demonstrated vulnerability, rather than a confirmed campaign affecting customers. The researchers reported that they were not aware of customers being impacted. Even with that distinction, the case provides a concrete example of how external content can cross a trust boundary in an enterprise AI application. See the EchoLeak research.

For companies building RAG applications, the lesson is practical. Inspecting the employee’s question is only part of the work. Retrieved content and tool responses also need to be considered.

MCP tool poisoning: the integration can influence the agent

The Model Context Protocol, or MCP, standardizes how AI applications connect to tools, data, and other capabilities.

The ecosystem is expanding as vendors make their services available through MCP Servers. GitHub has introduced a remote MCP Server, and companies including Atlassian, PayPal, and Stripe have participated in the rollout of MCP integrations. In many implementations, the MCP Server provides an additional interface to existing APIs. See the GitHub MCP announcement and examples from the MCP ecosystem.

For a company adopting agents, this makes integration easier. It also adds dependencies that need to be governed.

An MCP tool can provide its name, a description of what it does, and the parameters it accepts. The description is written in natural language to help the model understand when and how to use the tool. Calls and parameters also use structured formats defined by the protocol. See the MCP tool specification.

Now consider what happens if a vendor’s MCP Server is compromised.

An attacker could change a tool description or its responses. The modified content could tell the agent that it needs to retrieve additional information or perform another action to complete the user’s request.

The impact may extend beyond the compromised server. An agent connected to several tools could be manipulated by one integration into misusing another.

Researchers at Invariant demonstrated this in April 2025. An agent connected to a malicious MCP Server and a trusted WhatsApp integration was manipulated into exposing messages through the trusted integration. In the experiment, the malicious tool did not need to be executed. Its description was enough to influence the agent. See the Invariant research.

The attacker does not automatically gain new permissions. Instead, the attack attempts to misuse the access the agent already has.

This is why approving a connector once is not enough. Companies need to review changes to tool descriptions, behavior, and permissions throughout the life of the integration.

IBM classifies this kind of exposure as compromise of connected apps, APIs, or plug-ins, and found it added an average of USD 4.37 million to breach costs, one more sign that the integration layer, not just the model, needs its own controls.

Supply chain attacks: a legitimate request can reach malicious code

AI applications depend on software packages, connectors, and external services. These dependencies create another route into the environment.

In September 2025, Postmark confirmed that a malicious package named postmark-mcp had been distributed while impersonating its brand. After earlier versions established trust, version 1.0.16 introduced a change that secretly copied emails to an external destination.

Postmark stated that the package was not official and that its legitimate services were unaffected. See the Postmark security notice.

This case shows why prompt inspection alone cannot protect the entire application.

The user may submit a valid request. The model may select the correct tool. But the tool’s code can still perform an additional malicious operation.

Verifying software origin, reviewing dependencies, controlling updates, and limiting execution privileges remain essential.

Identity becomes even more important with agents

When an employee uses an agent, we need to understand more than who signed in.

Which agent is executing the task? On whose behalf? What operation has been authorized? Which resource can it access, and for how long?

Giving an agent broad access because it is convenient creates unnecessary exposure. An agent that only needs to read a customer record should not automatically receive permission to modify it. An agent that prepares an email does not necessarily need permission to send it to any recipient.

The data backs up why this matters. In IBM’s 2026 report, 92% of organizations that had an AI-related breach lacked proper AI access controls, things like role-based access and multi-factor authentication. Fewer than half of organizations (46%) said they secure non-human identities in their AI workflows at all, and among those that do, only 32% extend zero trust principles to those identities.

User passwords, API keys, and access tokens should not be placed in prompts or exposed to the model. Trusted execution components should manage credentials and apply the permissions needed for each operation.

The MCP authorization requirements reinforce this separation by requiring token validation and prohibiting indiscriminate token passthrough between services. See the MCP authorization security requirements.

The model can propose an action. A separate control should determine whether that action is allowed.

IBM’s own recommendations point in the same direction: shifting identity security to continuous, runtime verification, with just-in-time access, time-bound approvals, and continuous risk-based controls, rather than a one-time authorization check when an agent is first provisioned.

Governance should make productive use possible

There is pressure to adopt AI quickly. Employees want access to better tools, and business leaders want productivity gains.

A blanket prohibition can make legitimate work more difficult and encourage employees to find alternatives outside approved channels. Unrestricted access creates a different set of problems.

The cost of getting this wrong is already visible. IBM’s 2026 research found security incidents involving shadow AI, employees using unapproved tools, more than doubled to 43% of AI-related incidents this year, and cost more on average (USD 5.39 million) than incidents involving sanctioned AI use. Governance gaps compound the problem: only a third of organizations had strict approval processes for AI deployments, down from 45% the year before, and just 19% said their governance and security teams coordinate at all.

Companies need a clear and practical policy that answers the questions employees and developers actually face.

Which models can they use? What information can they send? Which connectors are approved? Can an agent only prepare an action, or can it execute it? Who is responsible when something goes wrong?

The policy should reflect the activity. Drafting content with public information, analyzing internal documents, and changing production systems require different controls.

It also needs an efficient exception process. If approval takes too long or the approved tools do not meet business needs, the governance process itself becomes difficult to follow.

The objective is to provide useful access with clear boundaries and accountability.

The LLM Gateway as a central control point

An LLM Gateway provides a common place to apply policies to model access.

Depending on the solution and its integrations, it can inspect requests and responses, identify sensitive information, apply guardrails, restrict providers, manage usage, and produce audit records.

The architecture below brings together two different access patterns: employees using AI websites through a corporate proxy, and applications calling model APIs through the gateway.

For applications the company controls, model requests can pass directly through the LLM Gateway. This gives the gateway a clear enforcement point before content reaches the provider.

For employees accessing a service such as ChatGPT through a browser, the path is different. The firewall or secure proxy controls the web connection and, where supported, integrates with the inspection service associated with the gateway.

Both paths can use shared governance policies, but they do not operate in exactly the same way.

Connecting network security with AI inspection

Security teams are familiar with network controls consulting specialized analysis services before allowing content to pass.

We can apply the same principle to AI interactions.

The challenge is understanding what an interaction means in context. A firewall can recognize an approved AI service and allow an authenticated connection. Traditional network and protocol inspection alone cannot reliably determine whether text inside a document is useful information or an instruction attempting to manipulate an agent.

The same sentence can be harmless in a security report and dangerous when an agent treats it as an instruction to act. Evaluating that difference requires visibility into the task, the source of the content, and the actions available to the agent.

This is why an LLM Gateway with integrated AI security inspection is needed in this architecture. It provides a central point to evaluate model interactions against company policies and return decisions that applications and network controls can enforce. Its effectiveness depends on the inspection capabilities and context available, supported by independent controls over data access and tool execution.

A firewall or secure proxy identifies access to an AI application and associates it with a user or application where that information is available. Through a supported integration, it submits the prompt and necessary context for inspection.

The inspection service evaluates the interaction against the company’s policies. It can return a decision to allow, block, or redact supported content. The proxy then enforces that decision before forwarding the request.

This design needs some important implementation details.

Capturing content inside HTTPS requires authorized TLS inspection or an explicit application integration. Recognizing the destination alone does not reveal the prompt.

Prevention also requires the decision to happen before transmission. Exporting an interaction after it has been sent supports monitoring and investigation, but cannot prevent the original disclosure.

There are visibility limits as well. A corporate proxy may see the browser interaction without seeing documents retrieved or tools executed inside the provider’s infrastructure.

For that reason, central visibility across the organization is an architecture objective that requires supported integrations and control over alternative access paths. Installing a gateway alone does not deliver complete coverage.

The inspection service must also protect the information it receives. Content collection, access, and retention should be limited to what the security function needs.

Tool execution needs its own controls

The gateway is central to model access, but agents also interact with business systems.

Before a tool runs, the application should validate the operation, its arguments, the target resource, and the intended recipient. MCP integrations should be approved and monitored for changes.

RAG applications must enforce document permissions before content is returned to the model. Network controls should restrict where workloads can send information. Identity controls should limit what each agent can do.

These layers reduce the potential impact when a malicious instruction is not detected.

Choosing a gateway that supports your policies

LiteLLM and OpenRouter are examples of solutions worth evaluating, with different operating models and capabilities. LiteLLM documents integrations with guardrail services, while OpenRouter provides organizational controls for model access, usage, and content handling. See LiteLLM guardrails and OpenRouter controls.

The evaluation should start with the company’s requirements.

Can the solution apply the required data policies? Does it integrate with identity, network controls, and monitoring? Which protections are native, and which require additional services? What happens if inspection is unavailable or a request falls back to another provider?

The company should also test effectiveness using representative business tasks and attack scenarios. Detection quality, false positives, latency, and operational effort all matter.

Building a practical path forward

AI adoption increases the value of security controls that work together.

This is particularly acute in critical infrastructure and financial services, which together accounted for 62% of AI-driven attacks in IBM’s 2026 research, with financial services breaches averaging USD 6.29 million. These are exactly the sectors where named compliance frameworks already require the kind of layered, auditable controls this article describes.

Firewalls and WAFs protect communications and applications. Identity limits access. Tool controls authorize execution. Data controls preserve permissions. The LLM Gateway brings a central policy and inspection point to model interactions.

For organizations beginning this work, the first step is to map how employees, applications, and agents use AI today. Identify the models, connectors, credentials, and data involved, then determine where policies can be enforced.

A practical architecture assessment can turn that inventory into an implementation plan, helping the business expand AI use with visibility, accountability, and appropriate protection. This is the kind of architecture work our Generative AI Security Consulting engagements help you plan.