Anthropic: agent memory works when models know what to forget
Anthropic's tests show selective notes beat exhaustive archives: a Claude that saved 413 Pokémon files earned one badge, while a leaner successor earned four, and 'Dreaming' review passes fix faulty memories.

Why it matters
- Claude Sonnet 3.7 created 413 files in 225 folders over ~10,000 Pokémon Red steps and earned one badge; Claude Fable 5 kept a few files and earned four.
- In a legal test, Claude scored 87 percent with a wiki versus 73 percent with search alone on multi-document questions.
- Claude built a linked wiki from about 350,000 internal documents from Stainless, a company Anthropic acquired.
- Anthropic launched Dreaming in May as a research preview for Claude Managed Agents, supporting Claude Opus 4.7 and Sonnet 4.6 and up to 100 past sessions.
- OpenAI says its June ChatGPT memory overhaul raised temporal accuracy from 52.2 to 75.1 percent.
A Claude model that saved 413 files across 225 folders during roughly 10,000 Pokémon Red game steps finished with a single badge; a leaner successor that kept only a few files earned four. That contrast anchors Anthropic's new research finding: the key to agent memory is knowing what to forget, not how much to store.
In a research post, Anthropic describes how Claude retains knowledge across sessions while the underlying model stays unchanged. Claude writes notes to a folder it can read back later. The core problem, according to Anthropic, is selection: before an agent knows what comes next, it must decide in the moment what might matter later, so every note is a bet on the future.
The stakes extend beyond a Game Boy cartridge. Anyone working with an AI agent on a multi-day project hits a wall fast, because every new session starts from scratch. Memory systems that work reliably would let agents carry strategies, rules and institutional knowledge forward — a precondition for agents that operate on long-horizon tasks.
Why do agents that save everything fail?
In Anthropic's Pokémon Red test, the older Claude Sonnet 3.7 treated everything as potentially important. Across roughly 10,000 game steps, it created 413 files in 225 folders, often a separate one for each battle and item it found. Because the game state keeps changing, the archive goes stale fast. At times, the model did nothing but reorganize its notes instead of playing. It finished with just one badge.
The newer Claude Fable 5 limited its memory to a few files and earned four badges. Fable 5 did not record information already visible on screen. Instead it wrote down plans, strategies and rules it had tested. Anthropic notes a caveat: the comparison covers only a single playthrough per model.
Anthropic recommends explaining the purpose of memory to Claude and letting the model decide how to use it. In one benchmark, rigid instructions such as "only record mistakes" wiped out most of the advantage for stronger models, according to Anthropic. Older models, by contrast, benefited from clear instructions — for example, being told to actually read their notes before every response.
What is Dreaming and how does it fix false memories?
The bigger risk comes from faulty notes that the agent blindly trusts later. Dreaming, an idea that builds on research from Letta and UC Berkeley, is designed to address this. Much like the human brain processes experiences during sleep, Claude reviews transcripts of earlier sessions during idle periods and checks its notes against them.
The Pokémon test shows the failure mode concretely. Claude had incorrectly noted which side of a mountain a landmark was on. Relying on that faulty note, Claude fell into the same trap three times, eventually wanted to give up, and missed its goal. After a Dreaming pass, Claude caught the error by checking the note against the transcript and found the right path.
Dreaming can also organize larger knowledge bases. Claude built a linked wiki from about 350,000 internal documents from Stainless, a software company Anthropic acquired. Dreaming generates new pages while fixing gaps, duplicates and broken links. The payoff was greatest when Claude could combine the wiki with search, so the wiki worked as a complement rather than a replacement.
A separate legal test by Anthropic quantifies that effect. For questions whose answers were spread across multiple documents, Claude scored 87 percent with the wiki, compared to 73 percent with search alone. For simple lookup tasks, the wiki performed roughly on par with search. In one case, the wiki even grew larger than the source documents.
Anthropic introduced Dreaming in May as a research preview for Claude Managed Agents, the company's cloud platform for AI agents. The service reads an existing memory store and up to 100 past sessions, removes duplicate and outdated entries, and creates a reorganized store, leaving the original untouched. At launch, Dreaming supported only Claude Opus 4.7 and Sonnet 4.6.
What problems remain unsolved?
Anthropic acknowledges open questions. Taking notes mid-task distracts the agent from its work. Dreaming costs extra compute, and its results arrive only after a delay. Wikis and notes need regular upkeep, or errors and hallucinations pile up. Anthropic also warns against automatically loading too many memories into every session, because they eat up context and pass any bad note along to every future session.
All of the tests come from Anthropic itself, a limitation worth weighing when evaluating the claims. The company's overall takeaway: a useful agent memory depends less on storing as much as possible than on keeping the right information and correcting mistakes later.
How do OpenAI and Google approach the same problem?
OpenAI also calls ChatGPT's memory system, which it overhauled in June, "Dreaming." That system maintains a user profile in the background and automatically corrects outdated information. According to OpenAI, the overhaul raised temporal accuracy from 52.2 to 75.1 percent.
Researchers at Google and Google DeepMind also let AI agents dream, but they use dreaming to improve an agent's search strategy. Their agent works through alternative solutions using results it has already stored, without calling the model again. Google has also built WikiSkill, a wiki for agents. As in Anthropic's system, WikiSkill keeps raw transcripts, distilled knowledge and specific instructions in separate layers.
The convergence matters. Three major AI labs are independently arriving at the same architecture: selective note-taking, layered knowledge stores and offline review passes that correct errors. If Anthropic's single-playthrough Pokémon results hold up under broader testing, the discipline of forgetting — not the capacity to remember — may become the defining constraint on how far an agent's memory can reach.
By Marcus Bennett
Original: anthropic.com
More from Marcus Bennett
Show full bio
Senior reporter covering companies, deals and strategy in artificial intelligence and machine learning at AI In Context.
718 articles
Related articles
- Graphon Labs Raises $35M Series A to Scale Persistent AI Memory
- Nvidia's SoL-Pi Cuts Coding-Agent Token Use by 49% by Reshaping the Harness
- Study Finds AI Agents Break Rules Even When Compaction Keeps Them
- Ataraxos Beats Stratego's Greatest Player, Ending a Human Stronghold
- MIT Put 13 AI Agents on a Desert Island and They Invented Play