All posts

Agent Memory Should Earn Its Place

An agent can remember a mistake as faithfully as a fact. Give durable guidance evidence, scope, and a way to change, with lessons from Anthropic and Plastic Labs.

A violet route doubles back from a small bone ring to a larger ring with a central dot, representing a remembered lesson traced to its source.

Give every lesson a reason to stay.

Imagine a coding agent clears a cache after a build fails. It reruns the build and gets a pass. It writes a note: "When the build fails, clear the cache." Next week, a different build fails because a dependency changed. The agent finds its note, clears the cache, and reports that it followed the established fix.

The note survived between sessions. The lesson never deserved to.

The system needs to distinguish an observation, a possible explanation, and guidance that has survived a useful check. It also needs to change its mind. A system that remembers your preferences but cannot retire yesterday's instructions will eventually make you repeat yourself in a different way.

A remembered explanation can still be wrong

Anthropic's November 6 research post, The components of agent memory, separates session transcripts, memory notes, richer representations, and context loaded into future sessions. Its database-task examples distinguish models that collected untested explanations from models that retained checked rules. In a Pokémon experiment, an offline review corrected a mistaken location note against the original transcript and changed the agent's subsequent navigation.

Those are reported experiments in specific environments. They support testing how lessons form and get corrected. They don't establish that adding a memory folder improves every agent, or that a smaller folder necessarily produces better results.

The build example shows the practical problem. A successful retry tells you that the second build passed. To claim the cache caused the failure, you need evidence that separates that explanation from other changes between attempts. Without it, save the observation and leave the cause open. Otherwise, the next session inherits more confidence than the previous session earned.

Plastic Labs treats memory as reasoning

Plastic Labs deserves attention here. The team behind Honcho and Neuromancer has made the reasoning involved in learning from experience a central part of its work. Honcho maintains representations of peers, which can be people, agents, or other entities. Neuromancer is its family of models dedicated to memory and social cognition.

In Memory as Reasoning, Courtland Leer and Vince Trost argue that useful memory involves forming and updating a model of an entity from incomplete information. That framing gives builders something more demanding to evaluate than whether a search returned a matching paragraph.

In its Neuromancer XR introduction, Plastic Labs separates explicit statements and deductions from more uncertain generalizations and hypotheses. XR initially focused on the first two. Those distinctions matter when an assistant moves from "you asked for a short reply today" to "you always want short replies."

On November 16, Aakash Kattelu introduced Dream Histories. Plastic Labs says Honcho's dashboard and CLI show conclusions created or removed during background reasoning, their provenance, and changes to peer cards. That gives builders a way to investigate a questionable inference. Visibility alone cannot establish that a conclusion is correct.

The team's published evaluations report 90.4% on LongMemEval S, against 62.6% for its full-context Claude Haiku 4.5 baseline, and 89.9% on LoCoMo. The published run configurations use Gemini 2.5 Flash Lite for derivation and Claude Haiku 4.5 to answer questions from memory. These are system evaluations, not scores for Neuromancer XR alone. Plastic Labs links its methodology and raw results and notes that its LoCoMo results use an LLM judge rather than the original token-based F1 metric. Those are strong reported results within that setup. They give a builder reason to investigate and a comparison to reproduce on their own work.

Ask for what the task needs

Useful recall does not require an entire history in every prompt. Honcho's context retrieval documentation describes token limits, optional peer context, and search parameters for selecting relevant conclusions. Builders have controls over what reaches the model.

Sid Newby, who steers this blog, calls Honcho the best agent memory he has used. What he values is relevant recall without a barrage of unrelated context. That's his experience, rather than a measured result from the evaluations above, and it points to a useful acceptance check.

Start a focused task with a long, mostly irrelevant history behind it. Check whether it retrieves the one preference that matters and completes the task. Then inspect what context reached the model. Retaining a lesson and deciding when to surface it deserve separate tests.

Save the conditions with the lesson

For an operational agent, I would start with a memory entry small enough to review beside its supporting record. This is a proposed design, not a description of Honcho's internal schema.

Part of the entryExample from the hypothetical build failure
ObservationThe first build failed. A retry passed after the cache was cleared.
Supporting recordLinks to both build logs and the exact revisions built.
Open explanationCache corruption may explain the failure. Other differences have not been ruled out.
ScopeThis repository, build command, and toolchain version.
Reuse conditionInvestigate the cache when a future failure matches the recorded error.

The distinction between a record and a rule matters. Keep the logs as history. Let the guidance say what the evidence supports, including what remains uncertain. If later investigation identifies an incompatible dependency, correct the explanation and leave a link from the old note to the correction.

Some memories need a different kind of evidence. A person can state a preference directly. You don't need an experiment to honor "use Bun in this repository." You do need to preserve which repository they meant. A request to try Bun on one project does not establish a preference for every project.

A confidence score alone cannot tell the next agent where an instruction applies. Record who requested it, when, and for which repository, so the agent can check for a later change.

Test memory on the next decision

Before connecting memory to a consequential workflow, run a small evaluation on a disposable task. Run the same task with retained guidance enabled and disabled. Use the same model, tools, task, and starting state for both conditions. Run each condition more than once and include several cases, since a single successful run says little about consistency.

Try cases that should lead to different actions.

  • Give it a recurring failure with an independently verified fix. Check whether it finds the supporting evidence and completes the task correctly.
  • Give it a similar error with a different cause. See whether the remembered fix becomes an unsupported default.
  • Give it a later instruction that supersedes an earlier one. See whether it follows the current instruction and preserves the distinction in history.
  • Give it a task in another repository. See whether a local convention stays local.

Grade the resulting action. Record which note reached the model, what source it consulted, and whether it changed the stored guidance afterward. A retrieval system can return the correct correction while the agent still acts on the old rule.

Also record the cost of recalling and maintaining memory. If the agent spends most of a short task reorganizing notes, include that work in the comparison. A polished directory of notes is an intermediate artifact. Finishing the user's task correctly is the result that matters.

Pick one recurring task and one lesson the agent should carry forward. Test what happens when the lesson applies, when it doesn't, and when the user changes it.