The software factory · memory

Memory for AI agents

The default advice is "bolt on a vector database." We didn't. Here are the five memory tiers our factory actually runs — all zero-infra — and the seven memory systems we looked at hard and said no to, with the exact trigger that would change each answer.

Every "give your agent memory" tutorial ends the same way: spin up a vector store, embed your history, retrieve on each turn. It's the reflex answer, and for a running software factory — a fleet of agents shipping code day after day — it's usually the wrong one. Not because retrieval is bad, but because most "memory" failures are write-quality failures wearing a retrieval costume. If the agent never wrote down the decision, no embedding model will find it. So we started from a different question: what is the least infrastructure that makes the factory remember the things that actually matter?

The answer came out as five tiers, none of which is a database we run. Tap through them.

Why almost none of it is "AI memory"

Notice what's missing: there's no semantic vector search anywhere in the stack. Recall of past work is lexical full-text search over every session transcript — the same FTS index SQLite ships for free, answering in tens of milliseconds at zero marginal cost. It turns out that when you're looking for "that time we fixed the auth redirect," you remember a word that was in it. Lexical gets you there. Semantic search earns its keep when the query and the memory share meaning but no words — and that's a specific, nameable failure, not a default.

Global memory holds invariants only. Everything else lives where it's diffable, attributable, and reversible: in git.

The load-bearing decision is that the tiny always-loaded memory file is capped on purpose — a couple thousand characters, holding maybe half a dozen invariants. A cap forces a question on every write: is this a permanent truth about how the factory works, or is it project knowledge that belongs in that project's repo? Almost everything is the latter. Per-project decisions live in a decisions log inside the project's own git history, where they're versioned, attributable, and revertible like any other artifact. The cap isn't a limitation we tolerate; it's the mechanism that keeps global memory from rotting into a junk drawer.

The systems we rejected (and what would change our mind)

We didn't reject external memory systems out of NIH reflex — several are genuinely good, and one we fully expect to adopt later. We rejected them for now, each against a written trigger. That's the honest artifact: not "vector DBs are bad," but a standing list of the conditions under which each becomes worth its weight. Tap any of them.

The one rule

Add an external memory provider only when you can name a specific recall that failed. Not "it might scale better," not "everyone uses RAG for this" — a real moment where the agent needed a fact it had genuinely stored and the current stack couldn't surface it. Until that moment arrives, every extra memory system is a daemon to babysit, a database to back up, and an LLM bill that accrues whether or not it ever earns its place. The trigger-based list above isn't procrastination; it's the difference between architecture and accumulation.

This is the same discipline the rest of the factory runs on: the machine earns each new capability against evidence, and the build stays cheap to kill. Memory is where that discipline is easiest to abandon and most expensive to get wrong — a forgotten vector store is the classic "why are we paying for this" line item a year later.