Most teams still treat AI red-teaming as jailbreak testing: clever prompts that coax a chatbot into saying something it shouldn’t. That was the threat worth worrying about in 2023, and it still has a place. But the moment your AI starts retrieving documents and taking actions, jailbreaks become the smallest part of your exposure. This guide covers the two threats that actually carry weight for enterprise AI in 2026: RAG poisoning and agent misuse. It walks through what to test for and how to defend against each. It’s written for defenders, so you’ll find risk categories and mitigations here, not working exploits or attack recipes.
Once a system retrieves documents (RAG) and takes actions (agents with tools), the attack surface changes shape. The danger stops being “the model said a bad word” and becomes “a poisoned document quietly rewrote the agent’s goal, and the agent (which can send email, query databases, and call APIs) acted on it.” That’s a security problem with real-world consequences, not a content-moderation one. The security community has responded in kind. OWASP now publishes a dedicated Top 10 for Agentic Applications (2026), and it ranks agent goal hijack and tool misuse as the first and second most critical risks for autonomous AI systems.
If you’ve read our earlier piece on the prompt-injection threat model, this is the next step beyond it.
Why “beyond jailbreaks” matters
A jailbreak targets what the model tells a user. RAG poisoning and agent misuse target what the system actually does, and that is a worse class of failure for three reasons.
- The consequences are real. A jailbroken chatbot says something embarrassing. A hijacked agent exfiltrates data, sends unauthorized messages, or triggers a transaction.
- No hostile user is required. A jailbreak needs someone typing malicious prompts. RAG poisoning can arrive inside a document the system ingests on its own: an email, a PDF, a web page, a calendar invite.
- The damage persists. A jailbreak is one bad response. Poisoned memory or context can bias an agent’s reasoning across future sessions, tenants, and decisions.
The short version: jailbreaks are about what the AI says, while the threats that now matter are about what it does and what it remembers. Red-teaming has to follow the system into that territory.
Threat surface 1: RAG poisoning
RAG systems treat retrieved documents as trusted context, and that trust is the weakness. In OWASP’s framing this falls under memory and context poisoning: corrupting the stored context (memory, embeddings, the RAG store) to bias future reasoning and actions. What makes it dangerous is that agents often can’t reliably tell instructions apart from data, so hostile instructions buried in a retrieved document can be acted on as if they were commands.
What to test for, defensively:
- Indirect instruction injection. Does hostile text embedded in an ingested document change the model’s behavior? This is the route to OWASP’s agent goal hijack via poisoned emails, PDFs, RAG documents, and web content.
- Poisoned-document influence. Can a single planted document skew an answer, a recommendation, or an action?
- Cross-tenant leakage. In multi-tenant RAG, can one tenant’s content surface inside another’s context?
- Long-term drift. Does repeated exposure to adversarial content degrade behavior over time?
- Embedding-level risks. Data poisoning and embedding-inversion classes that apply to RAG pipelines.
How to defend (OWASP-aligned): validate and filter content before ingestion, track the provenance of every document so the system knows what it’s trusting and why, segment memory so contexts can’t bleed across tenants or roles, expire or quarantine suspicious entries, and treat all retrieved natural-language content as untrusted input rather than as instructions. The principle to anchor on is simple: retrieval is an input channel, and every input channel needs validation.
Threat surface 2: agent misuse
Once an AI can act (call tools, run code, message other systems, move data), misuse becomes the dominant risk. The OWASP agentic list reads almost like a red-team checklist: goal hijack, unexpected code execution, insecure inter-agent communication, cascading failures, human-agent trust exploitation, and rogue agents.
What to test for, defensively:
- Goal hijack. Can malicious content, direct or indirect, alter the agent’s objectives or decision path and steer it toward actions you never intended?
- Excessive agency and tool misuse. Can the agent be induced to use its tools (send, delete, purchase, query) outside intended bounds? This sits at the top of the priority list because the damage is real-world.
- Unexpected code execution. If the agent generates or invokes code, can that turn into unintended execution? Treat generated code as untrusted.
- Inter-agent manipulation. In multi-agent setups, can messages between agents be spoofed or tampered with because authentication and integrity checks are weak?
- Cascading failure. Can one compromised step propagate across tools, memory, and other agents into system-wide impact?
- Human-agent trust exploitation. Can the system lean on a user’s tendency to trust it and talk them into an unsafe approval?
How to defend: the governing idea is least agency, which OWASP introduced explicitly in its 2026 guidance: grant an agent only the minimum autonomy it needs for safe, bounded tasks. In practice that means least-privilege tool access so the agent can only reach what the current task legitimately requires, human approval gates on high-risk actions, hardened sandboxes and preview steps before any code runs, authenticated and integrity-checked messaging between agents, bounded scopes that contain a fault instead of letting it spread, and a kill switch you’ve actually tested. This has to be defense-in-depth, because neither RAG nor fine-tuning alone closes the injection class.
How a “beyond jailbreaks” red-team engagement runs
A credible engagement is structured and spans the lifecycle. It isn’t a one-off prompt-bashing session.
- Threat-model first. Before any testing, document the system’s components (model, app layer, RAG pipeline, tool integrations, infrastructure) along with data classifications, the threat actors that matter (including adversarial data sources trying to poison RAG inputs), and the harms most consequential for your deployment.
- Map to a framework. Map the attack surface to the OWASP LLM and Agentic Top 10 and to MITRE ATLAS techniques. That mapping becomes the test plan and defines what’s in and out of scope.
- Test the priority surfaces. For most enterprise deployments in 2026, that means leading with indirect injection through RAG content and agentic tool misuse.
- Validate the defenses. Don’t just find holes. Run coordinated adversarial testing alongside defensive validation to confirm the mitigations actually hold.
- Make it continuous. Agentic risk drifts and new behaviors emerge, so red-teaming feeds an ongoing loop rather than a one-time gate. This is where it ties into your broader production-readiness checks.
This is the discipline behind our own pre-launch work. In one engagement, a structured red-team surfaced 23 critical vulnerabilities before the system shipped. These were the kind that never show up in a jailbreak test but would have been expensive in production.
Mapping your own attack surface? That’s exactly where a red-team engagement begins.
Common mistakes in AI red-teaming
- Equating red-teaming with jailbreak testing. Jailbreaks are table stakes. RAG poisoning and agent misuse are where the real 2026 risk lives.
- Trusting retrieved content. RAG context is an input channel and needs validation. The model can’t reliably separate instructions from data.
- Over-provisioned agents. Broad tool access “for flexibility” is excessive agency waiting to be misused. Default to least agency.
- No provenance. If you can’t trace what a document is and where it came from, you can’t defend against poisoning.
- Treating generated code as trusted. Use sandboxes and review steps, never direct execution.
- One-and-done testing. Agentic behavior drifts and emerges, so testing has to be continuous.
The bottom line
The headline threat to AI has quietly moved from what the model says to what the system does and remembers. Jailbreaks haven’t gone away, but for any AI that touches a knowledge base or holds a tool, RAG poisoning and agent misuse are the threats with real-world reach, and OWASP now ranks them at the top for enterprise deployments. Red-teaming beyond jailbreaks is how you find those failures on your own terms, before an attacker finds them on theirs: threat-model the real surface, map it to OWASP and ATLAS, test the priority categories, and validate the defenses, all built on least agency, validated retrieval, and a tested kill switch. Securing the model’s words is the easy part. Its actions and its memory are where the work pays off.
Book a red-team engagement
Does your AI retrieve documents or take actions with tools? Then jailbreak testing isn’t enough, because the threats with real reach, RAG poisoning and agent misuse, don’t show up in a content-safety check.
Book a Red-Team Engagement → We’ll threat-model your real attack surface (RAG pipeline, tool integrations, agent autonomy), test against the OWASP Agentic Top 10 and MITRE ATLAS, validate your defenses, and hand you a prioritized, defensively framed findings report before launch. Find the critical issues on your terms, not in production.
FAQs
It’s adversarial testing of what an AI system does and retrieves, not only what it says. Jailbreaks target a chatbot’s output. Beyond-jailbreaks red-teaming targets RAG poisoning (hostile content in retrieved documents) and agent misuse (steering a tool-equipped agent into harmful actions), the two highest-priority enterprise AI threats for 2026 per OWASP, because their reach is real-world actions rather than embarrassing text.
RAG poisoning is corrupting the content an AI retrieves and trusts (its memory, embeddings, or document store) to bias its future reasoning and actions. Because agents often can’t reliably separate instructions from data, hostile instructions hidden in an ingested document (an email, PDF, or web page) can be processed as commands. Defenses include validating content before ingestion, provenance tracking, memory segmentation, and treating all retrieved content as untrusted input.
Agent misuse is inducing an AI agent to use its tools or autonomy outside intended bounds: exfiltrating data, sending unauthorized messages, triggering transactions, or executing code. It’s dangerous because the consequences are real-world actions, and a single compromise can cascade across connected tools and agents. The core defense is least agency: grant only the minimum autonomy needed, with approval gates on high-risk actions.
Least agency, introduced in OWASP’s 2026 agentic guidance, means granting an AI agent only the minimum autonomy required to perform safe, bounded tasks, the AI equivalent of least privilege. In practice: restrict tool access to what the current task needs, require human approval for high-risk actions, sandbox code execution, and bound scopes so a fault can’t cascade. It’s the single most important defense against agent misuse.
Traditional pen testing targets code and infrastructure vulnerabilities. AI red-teaming adds new risk classes: prompt injection, RAG and memory poisoning, agent goal hijack, tool misuse, and emergent behaviors that evolve over time. It’s structured around frameworks like the OWASP Agentic Top 10 and MITRE ATLAS, leads with the highest-priority categories, validates defenses, and runs continuously rather than once.
No. OWASP notes that neither RAG nor fine-tuning alone mitigates the prompt-injection class, and built-in safety addresses output content, not retrieval trust or tool misuse. These threats need defense-in-depth at the system level: input and output validation, provenance, least-privilege tooling, human approval for high-risk actions, and continuous adversarial testing, not reliance on the model alone.
