Agent memory and state: what an AI agent remembers, and where

People talk about an AI agent "remembering" as though it were one thing. It is at least three, with different lifespans, costs and risks. Knowing which is which is the difference between an agent that picks up where it left off and one that repeats itself, or that quietly keeps data it should not.

Layer 1: the context window

Everything a model uses to produce its next response has to fit in its context window: instructions, the conversation so far, and the results of any tools it has called. As a long task proceeds, that fills up. Anthropic describes context editing as a way to clear stale tool calls and results as an agent approaches its token limits, keeping it focused on relevant material ^[1].

The context window is the cheapest thing to reach for and the most fragile. When it overflows or resets, whatever was only there is gone.

Layer 2: session state

State is the saved record of where a task stands. LangGraph, a popular open-source agent framework, formalizes this: a "thread" is the scope of a conversation's memory, and a checkpointer persists the thread's state as saved snapshots, so a later run can continue from where the previous one stopped ^[3]. Its documentation lists conversation continuity, human-in-the-loop, time travel and fault tolerance as what this enables ^[3].

Business examples are easy to find. A claims-intake agent can pause while it waits for a customer to upload a document, then resume. A reviewer can approve a step before the agent proceeds. If a server fails mid-task, the work is not lost.

Layer 3: long-term memory

Long-term memory persists across conversations. In LangGraph's terms, this is a separate "store" holding application-defined data such as user preferences or shared reference knowledge, and most applications use both a checkpointer for the current thread and a store for durable information ^[3].

Anthropic's memory tool is one concrete design. Claude can create, read, update and delete files in a memory directory that persists between sessions, reading them back on demand instead of loading everything up front ^[2]. It runs client-side: the model requests file operations and your application executes them, so memory lives in infrastructure you control ^[2].

Does it help?

Anthropic's September 2025 announcement of these features gave results from an internal evaluation of complex, multi-step agentic search tasks: memory combined with context editing improved performance by 39% over baseline, and context editing alone by 29% ^[1]. In a separate 100-turn web search evaluation, context editing cut token consumption by 84% and let workflows finish that would otherwise have failed from context exhaustion ^[1]. The source does not give the sample size or baseline configuration ^[1], so treat the numbers as indicative, not as a benchmark you can expect to reproduce.

Memory that improves itself

Vendors are now adding automation on top. At its Code w/ Claude conference in San Francisco, Anthropic announced a feature for its Managed Agents service called Dreaming: a scheduled process that reviews past agent sessions, surfaces patterns and curates memory, so agents "improve between runs" ^[4]. The company said it is available to all developers ^[4]. The idea is sound, and it adds a governance question: something other than a person is now deciding what the agent keeps.

What to govern

Anthropic's documentation is direct about the responsibilities that come with memory you host ^[2]:

• Sensitive information. Claude usually refuses to write sensitive information to memory, but the guidance recommends validation that strips it before saving for stronger guarantees ^[2].

• Size. Track and cap how large memory files can grow ^[2].

• Expiration. Periodically delete memory files that have not been accessed in a long time ^[2].

• Access. The documentation warns that a malicious path such as one using ../ can reach files outside the memory directory, and says implementations must validate every path ^[2].

Add to that a business-level question the documentation cannot answer for you: how a customer, or an employee, asks that remembered information about them be corrected or deleted.

Practical implication: For any agent you deploy, write down what it remembers, where that is stored, who can read it, and when it is deleted, before you turn memory on.