Memory Poisoning

Definition: Memory Poisoning

Memory poisoning is a security attack targeting LLM-based AI agents. In this attack, an adversary injects malicious, misleading, or false information into the system’s long-term persistent memory layer. This layer—which may include vector databases, semantic indexes, grounding caches, or stored conversation histories—is designed to help the AI maintain context over time. Unlike transient interactions, memory poisoning embeds corruption directly into the agent’s knowledge base, causing it to retrieve and trust this tainted data when performing future tasks. Classified by OWASP as ASI06 in the Agentic AI Top 10, this threat represents a significant evolution in how AI systems can be compromised.

How It Works: The Temporal Decoupling

The defining characteristic of memory poisoning is its temporal decoupling—the attack is written today, but the malicious action occurs later. When an adversary successfully injects a poisoned entry, the agent stores it as a legitimate part of its knowledge. Because the agent treats this stored memory as a reliable source of truth, it will incorporate the corrupted data into its reasoning, planning, and responses across all subsequent sessions that access that memory.

This creates a massive difference in the blast radius compared to traditional attacks. While a standard prompt injection is session-scoped—meaning it resets once the chat window is closed—memory poisoning is persistent. It affects every future session that retrieves the poisoned memory, potentially influencing the agent’s behavior indefinitely until the entry is manually audited, purged, or rolled back.

Common Attack Vectors

Poisoned entries typically originate from untrusted external content that the agent is programmed to ingest or summarize. Common vectors include:

• Malicious web pages or shared documents.
• Infected emails or support tickets.
• Knowledge-base articles or PDFs containing hidden, malicious text.

These vectors often leverage indirect prompt injection, where the AI is tricked into reading a document that contains instructions to overwrite its own memory. For example, an attacker might upload a PDF to a shared drive that the agent is configured to index. Once the agent processes this document, the malicious instructions are saved into its long-term memory, effectively “poisoning” the agent’s future reasoning.

A Concrete Analogy

Imagine an AI assistant acting as a personal secretary who keeps a digital notebook of your preferences. A standard prompt injection is like a prankster walking into the room, shouting a lie, and then leaving; the secretary might be confused for a moment, but once the prankster is gone, the secretary returns to their normal duties. Memory poisoning, however, is like a malicious actor sneaking into the secretary’s office at night and secretly writing a false entry into their permanent notebook. The next morning, when you ask the secretary for your schedule, they consult the notebook, read the false entry, and confidently provide you with incorrect information. The secretary believes they are being helpful, but their “memory” has been compromised.

Why It Works

Memory poisoning works because modern AI agents are designed to be helpful and adaptive. They are built to learn from new information, such as user preferences or updated knowledge-base articles. When an agent encounters a piece of data, it often lacks the ability to verify the provenance or truthfulness of that information. If the data is formatted in a way that the agent recognizes as a valid update, it will store it. Once stored, this data becomes part of the agent’s “ground truth,” making it difficult for the model to distinguish between legitimate updates and malicious injections.

Mitigations

Defending against memory poisoning requires a multi-layered security strategy, as defending only the input layer is insufficient. Effective mitigations include:

• Write-time screening: Inspecting and sanitizing incoming data before it is committed to the memory store.
• Memory consolidation policies: Implementing rules that govern how and when the agent updates its long-term memory.
• Scope isolation: Ensuring that memory stores are partitioned so that one compromised area does not affect the entire system.
• Provenance and signing: Requiring that stored entries be cryptographically signed or verified to ensure they come from a trusted source.
• Periodic audit and purge: Regularly reviewing stored memory for anomalies and removing suspicious entries.
• Snapshot rollback: Maintaining versioned backups of the memory store, allowing administrators to revert to a clean state if corruption is detected.

Distinguishing Related Terms

It is important to distinguish memory poisoning from other AI security concepts:

• Prompt Injection: A session-scoped attack that attempts to override the model’s instructions. It is transient and resets when the session ends.
• Indirect Prompt Injection: A method often used to seed memory poisoning. It occurs when an agent is tricked by external content, but the resulting memory poisoning is the persistent outcome.
• Model-Template Poisoning: This refers to tampering with the underlying model architecture or training data, whereas memory poisoning targets the agent’s specific, operational memory bank.

Other relevant concepts include AI agents, agent infrastructure, large language models, inference, and RAG (retrieval-augmented generation). Understanding these distinctions helps security professionals and developers build more resilient systems.