Fabrication-Pattern Regression-Guard Automation: A Script That Hunts the Exact Shape of Nine Real Defects

Fixing nine real occurrences of the same underlying defect pattern is real, valuable work — but a fix applied nine times by hand says nothing about whether a tenth instance, written by someone unaware of the previous nine, would ever be caught before it ships. A pattern that recurred nine times independently across one codebase clearly reflects something structural about how that pattern gets introduced, not nine unrelated coincidences — and the only way to genuinely close that gap is a mechanism that keeps checking for the pattern itself, automatically, rather than relying on everyone who touches the code afterward to remember nine specific historical incidents.

A one-time fix addresses the nine specific lines of code where a defect pattern was found — it does nothing about the eight, or eighty, other places in an actively developed codebase where the identical pattern could be written next, by a different person, months later, with no memory of the original incident. The only way to make a fix genuinely durable against a pattern that recurred nine times independently is to encode the pattern itself as something a machine checks for automatically, on every file, every time — because a pattern real enough to appear nine separate times on its own is exactly the kind of pattern that will appear an unknown number of additional times if nothing is specifically watching for it.

The same internal audit that rewrote Axiom's proof-of-action mechanism and connected its safety scanner to the real AI path (both covered on this pillar's companion pages) produced a standalone script whose entire purpose is refusing to let any of the nine specific fabrication patterns it found reappear anywhere in the module. Rather than attempting to judge intent — the audit's own account is explicit that none of the nine were malicious, all began as legitimate demo scaffolding that simply never got told when it became permanent — the guard searches for the concrete, mechanical signatures each defect actually left in the code.

Among the checks: any code where a random-number function feeds directly into a variable later used as a pass/fail decision, a score, or a count presented to a user as a measurement (with a short, explicitly justified allowlist for the small number of genuinely legitimate uses of randomness, like generating an internal identifier — each exemption has its reason written down, on the principle that an exemption without a documented reason is the same failure mode as the original defect: someone decided, and nobody else knows why); the precise bit-shifting pattern of the specific homemade hash algorithm the fake "proof" feature used, plus any comment admitting a hash implementation is a placeholder; any storage path referencing the fake `mock://` scheme; and any severity or risk ranking computed by looking up a value's position in an array (the exact fail-open pattern that let an unrecognized threat severity silently rank as harmless in the Safety Scanner). The script also directly checks that specific, named wiring stays intact going forward — that the AI conversation path still calls the safety scanner's real enforcement function, that action execution still calls the proof-capture functions, and that a set of legacy, dangerous SQL migration files still carry their obsolete-and-do-not-apply marker.

The script's own honesty about its limits is worth taking at face value: it deliberately strips out comments before scanning code, specifically because several of its own explanatory comments quote the exact bad code being guarded against (to explain what was wrong), and without stripping comments first, the guard would flag its own documentation of the fix as evidence the defect had returned — which would create a genuinely bad incentive, effectively punishing whoever had bothered to explain the fix in a comment.

That design decision was directly, adversarially tested, and the test found a real gap in the guard's own first version — not a hypothetical one. One rule was built to catch a fixed-delay timer used to fake real processing time, and its detection method searched for the specific comment associated with that pattern ("Simulate LLM delay"). But the guard's own comment-stripping step ran first, and comment-stripping removes exactly that kind of comment before any rule ever sees the code — so when the fixed delay was deliberately reintroduced without its identifying comment, as part of testing the guard by trying to sneak 13 previously-fixed violations back in one at a time, that rule missed it, while the other 12 reintroduced violations were all caught. The rule was rewritten to search for the delay's actual executable PATTERN — a literal timer call with a fixed delay above a minimum threshold — rather than relying on a comment that a script had already deleted by the time the rule ran. Re-testing after the fix caught all 13 reintroduced violations.

A future change to the codebase, made by someone with no memory of the original nine-defect audit, reintroduces a random-number-based placeholder score somewhere new, planning to replace it with a real calculation "later." Running the guard script catches this immediately and by name, rather than letting it ship silently the way the first nine instances did before anyone was specifically looking.

The nine specific defects found in one audit are now backed by a standing, repeatable check for the pattern itself — not just nine individual fixes that could each be quietly undone or independently reintroduced elsewhere without anyone noticing. And because the guard was adversarially tested against its own supposed targets and had a real, found-and-fixed gap in its first version, its current pass result carries more weight than a check that has only ever been run against code nobody deliberately tried to sneak a violation past.

This guard is a standalone script invoked directly, confirmed by direct inspection to have no automatic CI or build-pipeline wiring anywhere in the repository today — it does not run itself on every change without someone or something explicitly executing it. A team expecting an automatic, always-on gate blocking every commit will need to add that wiring themselves; what exists today is a working, adversarially-tested detector, not a self-triggering one.

A detection rule built around a comment or a name is only as durable as that comment or name — and a comment is far easier to delete than the underlying code pattern it describes, especially when a different processing step (like the guard's own comment-stripping) removes it before the rule ever runs. Centriu's guard was corrected specifically to search for the executable pattern itself wherever comments alone weren't reliable, learned directly from a real, documented gap its own first adversarial test uncovered rather than assumed to be airtight from the start.