Memory files prime LLM style: the four-op audit and the deterministic-then-LLM pattern

Written with Claude.

Every session loads the gotchas file, and the gotchas file has a style

I kept needing to invoke /minto or ELI7 to break down Claude’s responses into plain language. Every session. Reliably. The root cause, I eventually understood, wasn’t the model: CLAUDE.md and the memory-bank files don’t just carry constraints, they prime the style the model writes back in. Dense comma-stacked gotcha bullets in, dense comma-stacked responses out.

I’d read a 14-line reply about event-source substrate behavior and half of it was comma-chained prose that read like a warning label. Technical, accurate, and somehow unreadable. The model treats the context window as an example of good writing, and if the dominant example is incident-prose archaeology, that’s the register it defaults to. Cleaning the substrate is the fix: a one-time memory audit that drops archaeology and clarifies surviving prose has a compounding effect on every subsequent session.

Cull, promote, compact, clarify: four operations that don’t interfere

A memory audit isn’t one operation. Running it as a single rewrite pass on everything is the wrong shape: it mixes four distinct questions that have different costs, different tools, and different failure modes.

Cull: delete entries whose protected failure mode is no longer reachable, or whose content is already covered by a canonical source elsewhere. A gotcha about a tool bug that was patched eighteen months ago is dead weight that occupies context and primes toward outdated patterns.

Promote (substrate-fix): migrate an entry to a more durable surface. A timing rule that could be enforced by a hook should be a hook, not a bullet that the model reads and forgets to apply. The question is always: is there a specific tool change, hook event, or wrapper script that would make this entry unnecessary? If yes, the right fix is that change, not a better-worded bullet.

Compact: drop archaeology without changing the rules. Session references (S867), issue numbers (#1320), version catalogues, tombstone phrases (“X was deleted in…”), and “Background:” prose blocks serve the person writing them, not the person reading them later. A rule that needs three paragraphs of provenance to explain why it exists probably needs to be split into the rule and a commit message.

Clarify: rewrite surviving prose so it reads at the right calibration level. Not tutorial mode (over-explains context any senior engineer infers). Not embedded-SWE dense (the starting condition). The target is a peer engineer in an adjacent specialty who holds the domain terms after first use but doesn’t want to parse four nested clause qualifications to find the operative instruction.

These four run sequentially and don’t interfere. Cull first. No point compacting a dead entry. Promote before compact: a rule that’s going to become a hook doesn’t need rewriting. Compact before clarify: drop the archaeology, then rewrite what’s left.

The structure of the audit skill I built mirrors this: Phase 1 gathers and scores, Phase 2 dispatches verdict passes in parallel (necessity ladder + primitives), Phase 3 assembles a report, Phase 4 applies deterministic verdicts first and LLM rewrites second.

The necessity ladder beats a rewrite pass

The 2.152.0 refactor (commit 552eaa6, May 22 13:03) replaced a five-pass taxonomy with a two-pass design organized around one question: should this entry exist?

The necessity pass uses a five-rung ladder to find the bedrock answer:

Rung 1 - Reachability: Is the protected failure mode still reachable today?
  No -> `delete` (rationale names the gone precondition)

Rung 2 - Substrate: Could a substrate change eliminate this entry?
  FORCING FUNCTION: name a specific tool change, hook event, settings key,
  wrapper script, or alias in ONE sentence. If you cannot, advance to rung 3.
  Yes -> `substrate-fix` (rationale names the concrete change).

Rung 3 - Coverage: Is this covered elsewhere already?
  Yes -> `delete` (rationale names the canonical source).

Rung 4 - Trigger: Does the entry fire at a future decision moment?
  No -> `rewrite` if a trigger is nameable; `delete` if no trigger possible.

Rung 5 - Bedrock: The substrate genuinely cannot change.
  -> `keep` (rationale names the unchangeable constraint)

The prior version tried to classify entries by quality shape: routing, bloat, freshness, primitive shape. That produced 15 verdict codes and 8 BLOAT subtypes, each requiring the model to decide which quality dimension was most degraded. The ladder replaces that with sequential yes/no gates. Most entries answer at rung 1 or rung 5; rungs 2 and 3 are where the architecture catches entries that look load-bearing but aren’t.

After the 2.152.0 refactor, a /code-review pass found 19 issues. Thirteen were fixed by deletion. The commit message for 2.155.0 (5824f3f) records this directly: “13 of 19 fixed by deletion (simplification cascade).” The memory audit skill was itself carrying archaeology.

Cheap classifiers in front of expensive rewrites

The compact and clarify operations are LLM-expensive: each section dispatch costs roughly 10K tokens and runs a full analysis+rewrite cycle. But per-section yield varies widely. A section dense with session references, issue numbers, and “Background:” prose might compress 23%. A section of tight, current rules with no archaeology might compress 2%.

If every section gets dispatched regardless, most of the token spend returns almost nothing. The pre-filter is what makes the audit economically viable.

The mechanical scan scores each section on two dimensions before deciding whether to dispatch:

The dispatch rule:

dispatch if (archaeology_markers >= T1_low) OR
            (clarity_markers >= T2_high AND section >= 8 lines)

OR with asymmetric thresholds, not AND. A section with fifteen archaeology markers but tight clarity still dispatches; a section with four clarity markers but only two lines skips.

The asymmetry matters because false positives and false negatives aren’t symmetric. A false positive (dispatching a section that didn’t need work) costs roughly $0.05 in tokens, generates a VERDICT: PRESERVE from the analysis subagent, and logs as a dispatch. A false negative (skipping a section that needed work) is silent. The dense section stays in CLAUDE.md, continues priming every session toward dense responses, and is indistinguishable from “the model is just jargony.” Bias toward over-dispatch.

The pattern generalizes beyond memory audits. Compilers parse before optimizing. CI runs linters before integration tests. Drug discovery pipelines run in-silico filters before wet lab. Anywhere an expensive operation has high variance in per-input value, a cheap classifier in front captures most of the savings before the expensive stage sees a single input.

One bullet loses 1,700 characters and stays the same rule

The ### Event-source substrate traps (post-cutover) section of the project CLAUDE.md is a good concrete example. It’s eight bullets covering the append-only file attributes, torn-write recovery, and the event-source write primitives. Before the audit pass it scored:

• Archaeology-density: 1.70 markers per line (17 markers across 10 lines: one session reference, seven issue numbers, one arxiv citation, a version catalogue, three tombstone phrases, one “Background:” prose block, two date-stamps)
• Clarity-density: high (three bullets exceeding 700 characters each, four distinct rules packed into bullet five, no causal transitions in five of eight bullets)

Both thresholds crossed, so it dispatched to Phase 4 Tier 2 (the bundled compact+clarify subagent). The verdict table from Phase 2:

All eight bullets survive necessity. But four need compaction and all need the archaeology stripped.

The biggest single entry was bullet one, the chattr +a rule. At 1,708 characters, it was encoding not just the current rule but the history of why the rule exists – six em-dashes chaining inline parentheticals through three separate “Background:” and “Resolved S867” archaeology blocks. Before (excerpt from CLAUDE.md L203):

events.jsonl is OS-enforced append-only via chattr +a (dev-pi). Background: the former protection idea (migrate-to-event-source.py step 8 chown meap-web:meap-web) was a silent no-op […] Resolved S867 (Phase 3+4 T14): each migrated project’s events.jsonl gets sudo chattr +a […] On the dev-pi ext4 host this binds even the file owner: the gateway can still append (verified), but jq -i, vim, echo >, truncate, rm, and rename are EPERM for chris and all tooling. […] KNOWN FOLLOW-UP (T14, filed): the gateway’s auto-rotate-on-corrupt […] cannot clear +a itself – the unit has NoNewPrivileges=yes, so the service can’t sudo – meaning a real torn write makes the gateway’s quarantine rename hard-fail.

After:

events.jsonl is OS-enforced append-only via chattr +a (dev-pi). Each migrated project’s events.jsonl carries the Linux append-only attribute (chattr +a), set during migration. On the dev-pi ext4 host this binds even the file owner: the gateway can still append (verified), but jq -i, vim, echo >, truncate, rm, and rename are all EPERM for chris and all tooling.

Because chattr +a blocks rename and unlink of events.jsonl itself, any rotation operation must clear the attribute first, then restore it:

sudo chattr -a <events.jsonl>
# perform rotation
sudo chattr +a <events.jsonl>

--force-redo handles this automatically. Manual recovery does not.

The rule is identical. The history is gone. The code block makes the procedure scannable instead of buried in a sentence. Full section: 7,992 characters to 6,150 after the compact+clarify pass, a 23% reduction, with eight bullets collapsing to six via two sibling merges.

Gate verification runs deterministically after the LLM rewrite: the audit design specifies 61 critical strings that must survive verbatim and 25 forbidden phrases (tombstones, session refs, provenance markers) that must be absent. Both gates pass before the rewrite is accepted.

The threshold can’t carry the LLM’s judgment

The central architectural risk for any cheap-classifier-plus-expensive-judge design is trying to make stage one perfect. Encoding full LLM judgment in regex means the threshold has to anticipate every case where clarity-density alone warrants dispatch. It won’t.

The fix is to stop trying. The deterministic threshold handles the obvious-skip cases: sections with zero archaeology and tight, current rules. The LLM handles borderline judgment: sections where the markers are ambiguous or where clarity problems are structural rather than lexical. The threshold’s job is to skip the easy cases cheaply, not to be right about the hard ones.

The 2.157.0 release (a4abb0a, May 22 15:11) was driven by running the audit on the workspace that generated it. Of 36 mechanical findings on the meap2-it corpus, 11 were false positives: P001 was triggering on ellipsis literals in documentation, and F001 was missing coverage for the deploy-path command inventory. Both fixed; 11 findings dropped. The 2.158.0 release (89e12b2, May 22 15:26) came 15 minutes later, after we looked at the audit report summary and couldn’t tell from the output how many lines were actually going to get cut. The commit message records the feedback: “you’re showing me a bunch of jargon and it seems you’re only saying -6 LOC.” Phase 3 now leads with lines pruned and percentage of memory surface.

Five releases in one session, 13:03 to 15:26. 2.152.0 to 2.158.0. The audit tool was calibrated by dogfooding it against the memory substrate that motivated building it.

Threshold calibration is ongoing, not a one-time setup. Dispatch rate, VERDICT: PRESERVE frequency, and mean reduction per dispatched section are the three signals to watch. Below 10% dispatch rate means the threshold is probably too high; above 50% means too low. A VERDICT: PRESERVE rate above 30% of dispatches means the threshold is letting noise through. These targets tune on the full workspace corpus, not one file: asymmetric archaeology density across projects means single-file calibration silently underfits on everything else.

The discipline is to keep stage one cheap, keep stage two responsible for the hard cases, and treat a false negative as the more expensive failure by at least an order of magnitude. A session that loaded a dense, archaeology-heavy memory substrate and generated unreadable responses was the diagnostic signal. Getting the threshold wrong and missing a section costs one pass of token spend; getting the threshold wrong and treating it as unsolvable costs every future session.