Picture an AI agent managing a complex procurement workflow: it has been running for three days, coordinating between suppliers, validating contract clauses, and tracking approval chains across a dozen systems. On day four, it needs to reference a decision made on day one. There is a problem. The original context is gone evicted from the model’s working memory, unavailable for retrieval, effectively forgotten. The agent does not know what it does not know. It proceeds anyway, and the decision it makes is inconsistent with the one it made three days ago.
This is not an edge case. It is the defining engineering challenge for any organization deploying agentic AI in multi-day, multi-step business workflows and it is far less visible in demos than it is in production.
1. The Context Window Is Not What You Think It Is
Every Large Language Model (LLM) – the AI foundation powering agentic systems – operates within a context window: the maximum amount of text it can hold in active ‘working memory’ during a single inference call. Modern frontier models advertise context windows of 128,000 to over 1 million tokens (roughly 100,000 to 750,000 words). These numbers sound enormous. In practice, long-running business agents can exhaust the useful context budget faster than expected, and the costs, latency, and quality risks of filling very large windows grow non-linearly rather than simply disappearing as windows get larger.
The numbers are suggestive, but not all equally strong. Industry and practitioner analyses increasingly point to context drift, stale retrieval, and memory loss as recurring causes of agent failure in multi-step reasoning, but the commonly repeated ~65% figure appears in secondary/vendor analyses and should be treated as directional rather than as a neutral benchmark. On edge devices running multi-agent systems, the constraint appears even earlier: at 8K context per agent in standard precision, a 10-agent workflow on Apple M4 Pro hardware can fit only 3 agents simultaneously before cache eviction forces costly re-computation – 15.7 seconds per agent for full re-prefill at 4K context (Shkolnikov, arXiv 2026).
The deeper problem is not raw capacity. It is information discipline. Practitioner literature and recent research increasingly frame the context problem as a context-engineering problem: modern models can handle very large inputs, but performance depends on selecting, structuring, pruning, summarizing, and isolating the information the model uses to reason. What goes into the window, when, and how it is compressed or evicted: these are design decisions, not defaults. Most enterprise deployments treat them as afterthoughts.
2. Four Memory Types, Four Engineering Challenges
Agent memory literature and current agent frameworks commonly distinguish four memory categories, each with distinct storage requirements and failure modes:
- In-context (working) memory: the active prompt window — fast, expensive, and finite. Everything the agent can “see” right now
- Episodic memory: timestamped records of past interactions, decisions, and observations — analogous to human autobiographical memory. Enables the agent to recall that a specific decision was made on a specific date with specific inputs
- Semantic memory: structured facts, entity relationships, and domain knowledge — what the agent knows independent of any specific event
- Procedural memory: learned routines, cached plans, and successful workflows that can be reused without full re-reasoning
Customer service is one of the most common reported agent use cases: LangChain’s State of Agent Engineering survey found customer service was the leading use case among respondents, at 26.5%, followed closely by research and data analysis. It illustrates why all four types matter: the agent needs to recall this customer’s history (episodic), understand product facts (semantic), know how to handle escalation (procedural), and track the current conversation (in-context) – simultaneously, across potentially days of interaction.
The failure mode when any layer is missing is not a clean error. The agent silently produces inconsistent or stale outputs, often without flagging the gap. This is where most long-running deployments break.
3. The Episodic Memory Gap and Why It Matters Most
Of the four memory types, episodic memory remains one of the least settled engineering areas for long-horizon agents – and one of the most consequential for long-running tasks. A 2025 position paper by Pink et al. (arXiv 2025) makes the case directly: ‘Episodic Memory is the Missing Piece for Long-Term LLM Agents.’ The argument draws on cognitive science: episodic memory supports single-shot learning of instance-specific contexts, enabling agents to reason from specific past events rather than blending them into generalized semantic representations that lose critical temporal and causal detail.
The research gap shows up in production failures. Current mainstream approaches — full conversation history injection, summarization, or semantic embedding retrieval — all sacrifice something the agent needs. Full history injection burns the context budget. Summarization destroys precise causal chains. Embedding retrieval loses temporal ordering and nuanced context.
Newer architectures like EM-LLM (Fountas et al., arXiv 2024), E-mem (Wang et al., arXiv 2026), and CraniMem (Mody et al., arXiv 2026) are addressing this directly: event-based segmentation, goal-conditioned gated retention, and multi-stage consolidation that mirrors how biological memory converts short-term experience into durable long-term storage without destroying the original episode. These are research prototypes and early design primitives, not yet mature enterprise standards; most production teams still rely on simpler combinations of conversation history, summaries, vector retrieval, and external state stores.
4. The Latency Problem: Memory Retrieval at Scale
Even when memory is stored correctly, retrieving it at runtime introduces a second class of engineering problems. An agent that must scan thousands of past interaction records to answer “what did we decide about supplier X in Q4?” faces retrieval latency that compounds across every step of a long workflow.
The AMV-L (Adaptive Memory Value Lifecycle) research (Bamidele, arXiv 2026) identifies the core issue: age-based retention policies (TTL — time-to-live) bound item lifetime but not computational footprint. As retained items accumulate, retrieval candidate sets and vector similarity scans grow unpredictably, “yielding heavy-tailed latency and unstable throughput.” An agent that performs within SLA in week one may drift outside acceptable response times by week eight, not because it is processing more requests, but because its memory store has grown.
The production-hardened answer is tiered memory with active lifecycle management hot memory for recent high-relevance items, warm storage for compressed summaries, cold archives for compliance retention. Memori (Borro et al., arXiv 2026) demonstrates an approach: treating memory as a data structuring problem rather than a context stuffing problem, using structured augmentation pipelines that reduce redundancy while preserving reasoning context, and cutting inference costs substantially versus raw conversation injection.
5. Compliance in the Memory Stack: The Tension Nobody Plans For
Long-running agent memory creates a compliance surface that most enterprise teams discover late. Two requirements collide directly.
GDPR’s Article 17 right to erasure can require personal data to be deleted when the legal conditions are met. The EU AI Act, which applies progressively and is broadly applicable from August 2026 with important exceptions and later dates for some high-risk categories, creates separate obligations around technical documentation, logging, monitoring, and record-keeping for high-risk AI systems. The common shorthand that the Act requires ’10-year audit trails’ is imprecise: technical documentation may need to be kept for 10 years, while automatically generated logs for high-risk systems generally have a minimum six-month retention period unless other Union or national law requires otherwise. An agent managing financial decisions, HR processes, or healthcare coordination may therefore face overlapping deletion, documentation, and evidentiary duties.
These are not edge cases at the enterprise scale. They require memory architecture decisions made before deployment, not retrofitted after: separating personal-data-bearing episodic records from anonymized decision logs, maintaining auditable action traces independent of conversational memory that may need to be purged, and verifying that memory deletion does not leave inconsistent state in the agent’s semantic layer.
The Zombie Agent security research (Yang et al., arXiv 2026) adds another dimension: persistent agent memory across sessions creates a novel attack surface. Untrusted content observed during a benign session can be stored as memory and later treated as instruction a “persistent injection” that survives session boundaries. Governance of what enters long-term memory is as important as governance of what the agent does with it.
Conclusion
The engineering challenges of long-running business agents are not primarily about intelligence. Model capability alone is not sufficient. The hard problems are memory architecture, retrieval latency, state consistency across multi-day workflows, and the compliance obligations that persistent memory creates.
Organizations deploying agentic AI in sustained business processes are, in most cases, working with memory systems designed for single-session interactions. The gap between what those systems can handle and what multi-day enterprise workflows require is not a vendor roadmap problem it is a design problem, one that requires intentional architectural decisions about episodic storage, lifecycle management, and compliance separation before the first production workflow goes live.
As long-horizon agent deployments move from pilots toward production, memory architecture will emerge as the primary engineering differentiator between systems that hold up and systems that quietly fail. The question is not whether your agent can reason well. It is whether it can still reason well on day forty-seven.
Novelis perspective
Memory architecture is one of the least visible and most consequential design decisions in agentic AI deployments. At Novelis, we have built production systems where state management across multi-day workflows was the defining engineering challenge and where getting it wrong had real operational consequences. The difference between a reliable long-running agent and an unpredictable one often comes down to how memory was engineered from the start.
Further Reading
- Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents — Pink et al. (2025): five key properties of episodic memory essential for adaptive LLM agents
- AMV-L: Lifecycle-Managed Agent Memory for Tail-Latency Control — Bamidele (2026): adaptive memory value lifecycle management for production LLM systems
- CraniMem: Gated and Bounded Memory for Agentic Systems — Mody et al. (2026): neurocognitively-motivated multi-stage memory for long-running agent workflows
- Zombie Agents: Persistent Control via Self-Reinforcing Injections — Yang et al. (2026): security risks of persistent cross-session memory in self-evolving LLM agents
- State of AI Agent Memory 2026 – Mem0: vendor-published benchmark and architecture overview; useful for market context, but best read alongside independent academic memory-system papers
- Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers – 2026 survey of memory mechanisms, evaluation methods, and open engineering challenges for autonomous agents
If the challenges described here resonate with where your organization is today, we are always open to comparing notes. The problems are real, the solutions are evolving fast, and the teams working through them together tend to move further.
