A Practical Guide to Agent Memory: From Context to Long-Term Learning

Traditionally, organizations stored structured information in a few systems of record: “500 pillows in inventory” lived in the ERP, while “Mark is a prospect” lived in the CRM. But much of what makes an organization actually work (why a decision was made, how a process really works, or what someone learned from a failed project) remained in people’s heads. As agents take on more work, making this knowledge available to them through memory systems becomes increasingly important.

This field is now advancing rapidly. However, with many different names for similar things, the field can be confusing. Here’s a breakdown to help you navigate it and find the right solution for you. We'll cover the fundamental components of memory, the types of memory and their use cases, and, further below, how providers package them together.

TLDR

• Agent memory is more than a database. A memory system combines how information is ingested, stored, and retrieved or used.
• Short-term memory keeps agents on track now. It includes working context, session history, and coordination between agents and humans. Also known as runtime.
• Long-term memory helps agents operate across time. It can store identity (who), procedures (how), facts (what), events (when), and reasoning (why).
• Different tools cover different parts of this stack. Models, coding harnesses, databases, and memory providers are complementary rather than interchangeable.
• Most agent memory today focuses on retrieval: remembering facts and past events. The emerging frontier is computational memory that also preserves reasoning, evidence, causality, and changing beliefs so agents can learn over time.

Components of Agent Memory

Agent memory is any system that lets an agent retain and use information beyond what is immediately available in its current context. Different memory systems serve different purposes, but most can be understood through three fundamental layers:

• Ingestion: a pipeline that loads the data (and sometimes transforms or mines it into the desired format).
• Storage: either local or cloud-based, short- or long-term.
• Retrieval & use: how the right information is found and brought back into the agent’s work. This can include keyword or semantic search, temporal queries, or advanced modalities like graph traversal and logical inference, or automated triggers and workflows.

Until recently, what type of search a system did was a core differentiator. Increasingly, memory systems combine multiple ingestion, storage, and search methods. Hence, it’s now more meaningful to segment memory systems by the function they perform (the type of memory).

The 8 Types of agent memory and what each does

A broad way to organize agent memory is by how long the information needs to remain useful.

Short-term memory keeps an agent oriented while work is happening: what it is doing, what happened in a recent session, and what other agents or humans are working on.

Long-term memory persists knowledge across tasks and sessions: who the agent is, how work should be done, what is known, what happened in the past, and why particular beliefs or decisions exist

Types of Short-Term Agent Memory:

• 1. Working Memory: The information actively available to the agent while completing its current task: its plan, recent messages, relevant variables, intermediate results, and tool outputs. It usually lives inside the model’s context window. Modern models can support very large context windows, but simply fitting information into the context does not mean the model will reliably use all of it, so effective working memory is usually much smaller than the theoretical maximum (often starts to degrade past 100 thousand tokens).
• 2. Session Replay: A record of what happened during a previous agent run: messages, tool calls, intermediate steps, outputs, and sometimes a condensed summary. Unlike episodic memory, which stores meaningful events extracted from experience, session replay preserves the execution history itself. It is useful for debugging and for resuming interrupted work, although loading an entire replay can quickly consume working memory.
• 3. Coordination: The information used to transfer work between agents or between agents and humans: handoff summaries, delegated tasks, status updates, ownership, and dependencies. Coordination is not always classified as memory itself, but it serves a memory function in multi-agent systems by preserving state across actors and sessions.

Types of Long-Term Agent Memory:

• 4. Identity (WHO): The persistent information that tells an agent who it is and how it should behave: its role, goals, constraints, preferences, permissions, and available tools or capabilities. Identity is long-lived, although relevant parts are typically loaded into working memory during execution.
• 5. Procedural (HOW): answers HOW. This combines workflows (sequences of steps; more macro), specific processes (more micro, e.g., how to use the database), and guiding principles and rules.
• 6. Semantic (WHAT): The facts and concepts that represent the knowledge about the world.
• 7. Episodic (WHEN): Stored events like an order being fulfilled, a decision made, a meeting taking place, a task being completed, a campaign launched, etc.
• 8. Epistemic (WHY): Stores why something is believed or why a decision was made: the evidence, assumptions, arguments, alternatives, tradeoffs, and causal relationships behind facts and decisions. For example, semantic memory might store “we should target developers,” while epistemic memory stores the evidence and reasoning that led to that belief, and what new evidence would cause it to change.

Which solution do I need? Understanding the Stack

The field is evolving rapidly, which can be confusing. However, a few patterns are starting to emerge from how providers bundle their offerings:

The Models themselves (Sol, DeepSeek Flash, Fable, etc):

Models contain knowledge and learned behaviors in their weights, but those weights are not the same thing as an agent’s writable memory. During normal operation, an agent cannot simply add a new fact to the model’s weights and retrieve it later. Updating weights requires training or fine-tuning, while agent memory is designed to be written, updated, retrieved, and attributed during operation.

The labs use data from agent sessions to train their next model. As a result, increasingly universal workflows, instructions for common tools, and general principles can become part of the model's learned behavior.

Coding Harnesses (Claude Code, Codex, Pi, Cursor, etc.):

The runtime around a coding agent: the layer that gives the model access to files, tools, commands, instructions, and workflows. Harnesses typically provide some (short-term) memory themselves while allowing developers to connect additional memory systems.

• Short-term memory (Working memory, Session Replay, and increasingly some Coordination)
• Procedural memory (Skills) in Markdown files and additional functionality for workflows via hooks, APIs, and MCPs.
• Often possible to configure Identity memory via Agents.md or plugins.
• Semantic, Episodic, and Epistemic memory are generally not first-class capabilities, although they can be added through external memory systems via MCP, APIs, hooks, or other integrations.

Databases (Neo4j, Postgres, FalkorDB, GitHub, etc):

These aren't agent memory systems by themselves, but they often serve as the underlying storage layer.

(GitHub and similar systems offer broader functionality than a database and arguably belong in their own category).

Consumer Agents (Claude Web, ChatGPT, Grokbot, Muse, etc.):

Simplified all-in-one (ish) packages designed for personal use

• Native short-term memory (Working memory, Session Replay, and Coordination/Handovers).
• Longer-term memory with a focus on Identity and some Semantic memory for user-specific facts and preferences. The exact memory architecture is rarely disclosed, and capabilities vary considerably between products.
• Compatibility with external tools and memory systems varies, as does the ability to enforce memory usage within workflows.

Company Brains (Glean, Lore, etc.):

All-in-one type of solutions for companies, combining a harness, memory, and integration to popular tools like Email, Slack, etc.

• Short-term memory (Working memory, Identity, Session Replay, and Collaboration).
• Semantic, Episodic, and Procedural memory.
• Include the agents’ harness and many integrations.
• Epistemic memory is generally not a first-class capability.

Knowledge Management Systems (Obsidian, Confluence, Roam, etc):

Human-first; longer-term knowledge management tools before agents became prevalent.

• Semantic and Procedural memory via Markdown files and relational databases.
• Some compatibility with agents via APIs.

Agent Memory providers (Mem0, Honcho, Zep/Grafiti, Hindsight, Gbrain, Supermemory, etc):

Agent-first systems designed to persist information across sessions and retrieve it when relevant. They typically extract facts, preferences, entities, or events from agent interactions, store them outside the model, and retrieve relevant memories during future tasks. Implementations range from files and vector databases to relational and graph-based systems, often combining several retrieval methods.

• Optimized for Semantic (fact retrieval) and, often but not always, Episodic (event retrieval).
• Some include Session Replay; others don’t.
• Designed to work together with harnesses: don’t include Working Memory or Coordination, and often no Procedural Memory either, nor Identity.
• Epistemic memory is generally not a first-class capability.

Computational Memory providers (Tortoise, RSIAgent):

An emerging category of long-term memory that goes beyond retrieving stored facts and events. These systems process relationships between evidence, beliefs, decisions, outcomes, and sometimes causal claims, allowing new information to update more than a single stored fact. Instead of asking only “what should I retrieve?”, they also address “what does this new information change?”. Computational Memory systems actively process information to learn, enabling better reasoning and higher agent autonomy.

• Generally no short-term memory, but included if there’s an agent harness attached.
• Semantic and Episodic.
• Some Procedural memory, but designed to work in tandem with harnesses.
• Epistemic memory as a first class capability.

Summary table

Legend:

• ✓ = core/native capability
• \~ = some providers / partial capability
• (blank) = generally not included

Note: Capabilities vary significantly within each category. The table describes the typical or intended role of each provider type, not a strict feature checklist for every product.

Provider type	Working	Session Replay	Coordination	Identity	Procedural	Semantic	Episodic	Epistemic
Consumer agents	✓	✓	✓	✓	✓	\~	\~
Coding harnesses	✓	✓	✓	✓	✓	\~
Knowledge Management Systems		\~			✓	✓
Agent Memory Providers		\~			\~	✓	✓
Company Brains	✓	✓	✓	✓	✓	✓	✓
Computational / World-Model Memory		\~			\~	✓	✓	✓

The road ahead for Agent Memory

Today, much of agent memory is optimized around WHAT and WHEN: retrieving relevant facts and past events. As agents move from answering questions to managing long-running tasks and making consequential decisions over multiple cycles of experimentation and learning, HOW and especially WHY become more important. Agents need to preserve not just what happened or what is believed to be true, but the procedures, evidence, assumptions, and tradeoffs behind those beliefs and decisions, and compute all this information to decide the next action.

This shifts memory from a retrieval problem to a learning and reasoning problem. The infrastructure is moving from storing context to representing knowledge, relationships, reasoning, and change over time.

About us

We’re Premise Labs, the team building Tortoise, a computational memory system for agents.

Tortoise gives agents a persistent model of what is known, why it is believed, what decisions have been made, and what changes when new evidence arrives. Instead of only storing and retrieving facts, Tortoise computes the relationships between facts, events, evidence, decisions, and beliefs.

This allows agents to update their world model as new information arrives, preserve reasoning across cycles, and make complex decisions without forgetting important variables or being swayed by the most recent context.