Agent Memory Without the Mythology

We talk about AI memory as if the goal were to make a machine remember us the way a person does.

That framing is emotionally intuitive and architecturally unhelpful.

Production agents do not need one mysterious faculty called memory. They need several different systems that preserve different kinds of information for different lengths of time. A coding agent needs the current plan while it works, repository conventions across sessions, and a record of why an architectural choice was made. A support agent may need customer preferences, the state of an open case, and a legally governed history of prior actions.

Putting all of that into a growing conversation produces something that feels like memory until it becomes a liability.

The right question is not “How do we make the agent remember everything?” It is “Which facts should survive, for whom, in what form, and under whose control?”

That moves memory out of mythology and into systems design.

Memory Is a Collection of Responsibilities

Different information has a different job.

Working state keeps the current task coherent: the goal, plan, open questions, active files, and latest evidence. It changes quickly and should disappear when the task is complete unless part of it is promoted.

Semantic memory preserves durable facts and conventions: the supported deployment regions, the repository's test command, a customer's chosen language, or the meaning of an internal term. It changes more slowly and should have an owner.

Episodic memory records what happened: a failed migration, a previous support interaction, an agent's unsuccessful approach, or the evidence behind a past decision. Episodes are useful when the present resembles the past, but they are not automatically instructions.

Procedural memory describes how work should be done: runbooks, skills, policies, review checklists, and tool-use patterns. This is closer to versioned operating knowledge than recollection.

Artifact memory preserves the products of work: patches, reports, plans, test results, approvals, and decision records. These artifacts often matter more than the dialogue that created them.

When all five are stored and retrieved in the same way, the agent cannot tell a durable rule from an interesting anecdote. A useful memory architecture begins by separating their responsibilities.

The Transcript Is Not Memory

A transcript is valuable evidence of an interaction. It is not a reliable representation of truth.

Consider a debugging conversation. The agent proposes three possible causes, rejects two, modifies a file, discovers a hidden constraint, reverses the change, and finally fixes a different module. The transcript contains every step. Retrieving it later may bring back the rejected causes with the same rhetorical confidence as the final diagnosis.

Conversation history also carries accidental information: copied secrets, irrelevant personal details, stale tool output, and instructions that applied only to one moment. Retaining it indefinitely creates cost and risk without guaranteeing usefulness.

A stronger system extracts durable products from the interaction. The current task state records what is still active. A decision record preserves what was chosen and why. A patch captures the change. Test output supplies evidence. The transcript can remain available under an appropriate retention policy, but it is no longer the only place where truth lives.

This is the difference between remembering that a conversation happened and preserving the state required to continue the work.

Writing Memory Is a Consequential Action

Most memory designs focus on retrieval. The more dangerous operation is writing.

If an agent can silently promote anything from a conversation into long-term memory, mistakes become durable. A temporary preference becomes permanent policy. A model inference becomes a customer fact. Malicious content from a retrieved document becomes an instruction shown to future sessions.

Memory writes need a contract.

The system should know:

• which memory class is being written
• where the information came from
• whether it was observed, inferred, or explicitly provided
• who or what the memory describes
• how long it should live
• who may read or update it
• what evidence would invalidate it

Some writes should require confirmation. If a user says, “For this report, use a formal tone,” the system should not necessarily remember a permanent global preference. If an agent infers that a service is deprecated from one failing test, that conclusion should remain a hypothesis until an authoritative source confirms it.

Durability should be earned.

Provenance Makes Memory Correctable

A remembered fact without provenance is difficult to challenge.

“This repository uses Node 20” may have come from the current engine declaration, a year-old setup guide, or a previous run on one developer's laptop. Those sources deserve different confidence and different expiration behavior.

Useful memory carries its origin, timestamp, scope, and status. It can point back to the source artifact or decision that created it. When a newer authoritative fact appears, the system can supersede the old memory instead of letting both compete indefinitely.

This makes memory auditable and correctable. A human can ask why the agent believes something. An evaluator can determine whether a wrong answer came from reasoning or stale state. A cleanup process can find memories whose sources no longer exist.

Provenance is not decorative metadata. It is the mechanism that keeps memory from hardening error into identity.

Retrieval Is a Policy Decision

Even correct memories can be harmful when retrieved into the wrong task.

A customer's preference from one product may not belong in another. An incident from staging may be misleading during a production diagnosis. A code pattern from an old architecture may be valid history and terrible current guidance.

Retrieval should therefore consider more than semantic similarity. It should respect scope, recency, authority, task role, privacy, and the cost of a false match.

For a code change, repository-local conventions should usually outrank a general preference learned elsewhere. For a regulated decision, current policy should outrank a similar past case. For a personal assistant, the system should distinguish information the user asked it to remember from information merely mentioned in passing.

The retrieval layer is deciding which past gets to influence the present. That is an exercise of power, not just search.

Memory Needs Forgetting

A system that never forgets does not become wise. It becomes cluttered, expensive, and dangerous.

Some information should expire because the world changes. Some should be deleted because the user asks. Some should be compacted because the detailed episode has served its purpose. Some should never have been retained at all.

Forgetting can take several forms:

• Expiration: remove temporary facts after a defined lifetime
• Supersession: keep history but mark a newer fact as authoritative
• Compaction: replace detailed records with a durable decision or summary
• Scope closure: discard working state when the task is finished
• User deletion: remove information from every system that can retrieve it
• Quarantine: prevent disputed or suspicious memory from influencing work

These operations need to be designed alongside storage. “We use a vector database” is not a memory policy any more than “we use a hard drive” is a records-management strategy.

The ability to forget deliberately is part of trustworthy memory.

Shared Memory Can Become Shared Contamination

Multi-agent systems make memory boundaries even more important.

It is tempting to give every worker access to one common pool. Shared memory promises coordination without repeated explanation. It can also spread a bad assumption across the entire system.

A researcher records an unverified claim. A planner retrieves it as fact. An implementer turns it into code. A reviewer sees the same shared memory and treats the implementation as consistent with the plan. What looks like agreement is one error echoing through several roles.

Agents should share durable artifacts and verified state where possible, not an undifferentiated stream of impressions. Private scratch space can hold tentative reasoning. Task state can be shared inside one workflow. Curated semantic memory can serve a broader system after it passes a stronger write policy.

Isolation is not only about context limits. It preserves the possibility of independent judgment.

A Practical Memory Architecture

I would build agent memory as a small set of explicit stores rather than one universal brain.

The task store holds the current goal, plan, checkpoints, and unresolved state. It is optimized for reliable continuation.

The artifact store holds outputs and evidence: documents, patches, structured results, tool receipts, and approvals. It is optimized for audit and reuse.

The knowledge store holds versioned, scoped facts with provenance and ownership. It is optimized for retrieval into future work.

The procedure store holds skills, runbooks, and policies. It is reviewed like code because it directly shapes behavior.

The episode log preserves important histories under an explicit retention policy. It is searched when analogy is useful, not injected by default.

An access layer decides what the current capability can read and write. A promotion process decides when temporary state becomes durable knowledge. An expiration process removes or supersedes what no longer deserves influence.

This architecture is less magical than “persistent agent memory.” That is one of its strengths.

Evaluate Memory on the Second Encounter

Memory quality is not measured when information is stored. It is measured when the system meets a related situation later.

Did retrieval bring back the right fact? Did it preserve scope? Did the agent recognize that the memory was stale? Did the system avoid exposing one customer's information to another? Could a human correct the record? Did deletion actually prevent future retrieval?

Useful evaluations should include false memories, conflicting updates, ambiguous preferences, malicious content, expired facts, and tasks where the correct behavior is to retrieve nothing.

Recall matters. Precision may matter more. A memory system that occasionally fails to surface a convenience is frustrating. One that confidently injects the wrong instruction can be destructive.

Memory Should Make the System More Honest

The promise of memory is continuity. The risk is false familiarity.

An agent that remembers well should not merely sound as though it knows the user or the project. It should preserve the right state, retrieve it under the right conditions, expose where it came from, and allow it to be corrected or forgotten.

That requires architecture: separate memory classes, governed writes, scoped retrieval, durable artifacts, explicit retention, and tests built around later use.

We do not need agents that remember everything. We need systems that know what deserves to survive.