Memory in the Loop.
In-process retrieval as extended working memory for language agents. My paper, on arXiv.
A language agent runs a loop: observe, reason, act. The memory it reasons over usually sits outside that loop, queried at most once a turn. This paper studies what happens when memory moves inside the loop, read and written on every step.
The standing objection is latency: networked vector stores answer in tens to hundreds of milliseconds. That cost is a property of where the store lives, not of the pattern. An in-process store answers in about 100 microseconds, and at that speed a fast, always-available memory stops being a tool the agent consults and starts working like extended working memory.
I coined the term memory in the loop, a deliberate extension of human in the loop, to name that shift. The paper is the first to name the regime and study it systematically across models.
Latency is a location, not a law
An in-process store answers in ~100µs, three orders of magnitude under a networked store. At that speed, reading memory on every reasoning step is effectively free.
The effect is causal
Hold the per-turn memory budget fixed and vary only the store’s answer speed. Redundant actions climb monotonically as the store slows down.
In-loop memory recovers recall
A bounded context window drives recall to zero across four GPT-5-class models. In-loop memory with a write-side dedup gate brings it back near perfect.
Read the full paper
The dedicated paper site, with the complete write-up, figures, and results.