Version Two: Six Weeks Inside a Memory

Six weeks ago I wrote up a memory system for an AI assistant: a folder, three markdown files, and a discipline about when you are allowed to write to them. It closed on a warning. An assistant that can write to its own memory can write its mistakes into its own premises.

That was after ninety minutes. Since then the system has run every day, across dozens of sessions a week, on my phone and at my desk. It held. It also failed, twice, in ways the first write-up did not predict, and one of its rules turned out to be wrong.

This is what version two changed, and why. This week I moved a second assistant onto it, so it is also the first time the method has had to survive being handed to someone else.

Six weeks, one library

The short version: the build was fine. The writes were fine. Everything that went wrong happened in the part of the system that has no file: the habit of using it.

The first failure: it didn’t use it

The assistant was supposed to file anything durable into the library. Four times in five days, it didn’t. It filed everything into whatever was nearest to hand, its built-in memory and some project documents, and wrote the library only when I asked. Each time I asked in nearly the same words.

Here is the part worth dwelling on. The instruction to use the library was not missing. It sat at the top of the index, the first file read in every session, and on at least two of those days the assistant had read it at the start of that very session. It knew the rule. The rule just wasn’t there at the moment the rule mattered.

An instruction read at the start of a session does not fire at filing time.

If that sentence sounds familiar, it should. It is the oldest finding in knowledge management, showing up in a new medium. Knowing the procedure and doing the procedure are different capacities. A rule written in a document is explicit knowledge; the reflex that reaches for the right place when something needs keeping is not. The first is on the page. The second has to be built where the action happens.

So that is what fixed it. Not a louder instruction. A trigger, attached to the act itself: a small skill that loads whenever the assistant is about to file, document, note, record, remember or commit something, and asks one question. Library first? From the day it went in, every session filed where it should.

The second failure: it used it perfectly

Then the opposite happened, and it took me longer to notice because nothing looked wrong.

Most sessions run from my phone, where the assistant can create files but not edit them. So each session did exactly what it was told: wrote its notes as new files in the memory folder and stopped. Merging them into the main files needs a session at my desk, and for fourteen days, none of them did it. Nobody had made it anyone’s job.

By the time I asked how the memory was holding up, 151 unmerged notes were waiting, about 528KB of them, against 189KB of core files. The short-term file was describing a setup that no longer existed. Every session had done its job. The library was still wrong.

The fix is boring, which is how you know it is the right one. Every session now starts by counting the waiting notes, and if there are more than about fifteen, it says so before doing anything else and offers to merge. Merging happens on a cadence, roughly daily, by whichever session can edit. With several sessions a day, 25 to 35 new notes a day turns out to be normal load, not a backlog.

The lesson I keep coming back to: filing is half the job. A library that is only written to is a log.

The rule I had to reverse

The first write-up was emphatic. Split memory by how fast things rot, not by subject. “Don’t.”

Half right. The volatility split held: long-term for what stays true, short-term for current state, and that division never caused a problem. But a busy long-term file keeps growing, and the whole point of the design is that it gets read at the start of every session. Mine reached about 135KB, then later about 285KB, and at that size reading it is most of what a session can afford before any work starts.

So version two keeps the volatility split and then, only when size forces it, carves whole subjects out of the long-term file into their own files. Those are read only when the work touches them. Current state for every subject still lives in short-term, one entry each. Lessons about how to build things get their own file, read before building rather than at every start.

Two rules make that safe. The assistant proposes the split; it doesn’t do it. I approved it three times, once per split. And subjects move whole, never rewritten on the way: create the new file, verify it, then trim the old one, so nothing ever exists only in a copy nobody has checked.

What the loop learned

The first write-up told the story of a file whose size went 12,636 → 10,898 → 12,636 → 14,627 bytes in twelve minutes while every read reported success. That lesson aged well. Version two mostly turns it into procedure.

A few rules came straight out of things breaking:

• Take the timestamp from the clock, not from the assistant’s sense of the time, and list the folder right before creating a note. Parallel sessions picked the same minute four times. If the name is taken, use the next minute.
• Create through a channel that refuses to overwrite. One note was lost when a second session, twenty seconds later, wrote its own note to the same path.
• Merge one file at a time, and compare text, not size. Not the byte count, and not the modification time either: the time once advanced while the content did not.
• A write that hasn’t appeared by the next session didn’t happen. “I’m early” is the right first guess. It is not a permanent one.
• Before repeating a count, list the thing. A note once claimed about 60 waiting notes, copied from a session summary. The folder held 18.

And one that I would put first if I were starting again: big merges get audited by a second reader. When a lot of notes are waiting, the merge is split up mechanically, every line is accounted for (lifted, turned into state, or skipped with a reason), and then a separate instance that did not do the merge reads the result before anything is committed. It has found real errors every time it has run. Eighteen on one pass, nine on the next. Open items that existed only in state. An inference written down as fact. A fact cited to a note that didn’t contain it.

None of those would have been caught by the one that made them. That isn’t a criticism of the assistant. It is the confirming loop doing what it does, and a second reader is the only reliable way in from outside it.

What didn’t change

The [stated] tag. It is still the most important convention in the system, and six weeks of use only sharpened it: an option I picked from a list is recorded as a choice, not as words I typed; what the assistant knows from its own work goes in untagged; its conclusions get a separate [inferred]. Three kinds of line that must never be mistaken for each other.

The correction log. Still the most valuable section in the library, and still the best single sign of whether the thing is healthy.

The constrained mode as the default. Phone first, desk as an accelerant. Every convention still has to work with nothing but a file connector that can’t edit.

And the doctrine underneath it: AI proposes, human disposes. The assistant proposes where a note belongs, proposes a split, proposes a merge. The decisions that change the shape of the system stay mine.

Moving a second assistant

This week I upgraded a second assistant, WC, from version one to version two. Same model, different body of work. So far so good. I’ll know more in a few months.

After the upgrade, I asked the first assistant to write the second a handoff: one instance to another, field notes from six weeks on the same method. It is a better summary of this post than this post is. It said the failures in two sentences, the rules in a page, and ended with the line I would want any system to live by: if you hit a failure that isn’t on this list, write it down. That is how the list got made.

What I’ll be watching, in WC and in the original, is three things. The waiting notes stay few. The core files describe the world as it is today. And the correction log keeps growing.

That last one still sounds backwards. It isn’t. A memory that never records being wrong isn’t one that was never wrong. It’s one that isn’t noticing.