LLM Serving Emergency Kill Switch Integrity Automation: The Switch Was On, and Nothing Stopped

An emergency stop is only as good as its aim. When it is implemented as a check inside one specific function, it protects exactly that function — not the underlying capability the function used to be the only way to reach. The moment a product adds a second way to reach the same capability, whether through a new customer-credential path, a new agentic loop, or simply a refactor that moved the calling code, the original check keeps working exactly as written. It just stops being in anyone's way. Nothing about that failure looks like a bug from the inside: the function still checks its flag, the flag still flips, the tests that exercise that exact function still pass. The gap only shows up if someone asks the one question the tests never asked — not "does this function respect the switch," but "does anything that can reach a model respect the switch."

The emergency stop for AI message serving was originally a single `if` statement inside the platform's own OpenAI client, checked at the moment that client built a request using Centriu's own shared, platform-level credential. That was a reasonable place to put it at the time it was written — it was the one and only way any customer conversation reached a model.

The product then moved to requiring each customer's own connected AI credential for real conversations. The orchestrator's own logic for that case — resolve the customer's credential, then call the provider directly with it — returns before ever reaching the original platform client, let alone the `if` statement inside it. Separately, a tool-calling agentic loop was added to let the assistant use tools mid-conversation; it calls the provider through its own dedicated function, which was never wired to consult the switch at all, because the switch simply didn't exist yet in the part of the codebase this new loop was built in.

Measured directly in production: with the emergency switch set to its "on" state, a full census of the real, currently-live call sites that can reach a model found eight — six of them behind the two functions that carry actual customer traffic (a single-message path and the tool-calling loop), one behind the now-mostly-unused platform-credential path, and one in a streaming code path confirmed to have no route exposing it at all. Zero of the six live, customer-traffic call sites consulted the switch. The emergency brake existed, was engaged, and stopped nothing that mattered.

A companion control compounds the same failure from a different angle: a per-customer override for the same switch, stored as a flag in the database, had existed since months before this fix with zero lines of code anywhere ever reading it. A control that exists only as a stored value, never consulted, is not a weaker version of a working control — it is indistinguishable from having no control at all, except that it looks, to anyone auditing the database schema, like protection is already in place.

The fix introduces one shared assertion function, called at the earliest point in the request lifecycle that already knows which customer is asking — specifically before that customer's own AI credential is even retrieved, so a blocked request never gets far enough to touch a secret. The same function is also called as a backstop from inside both of the adapter functions that actually build provider requests, specifically so that a future shortcut added by a new caller cannot silently reopen the same gap by skipping the earlier check. The global switch is evaluated first, without touching the database at all; the per-customer flag is evaluated second, and now genuinely read. And where the previous per-customer flag read would have quietly treated a database error as "flag is off" (a reasonable default for an ordinary feature flag), the new emergency-stop read deliberately fails CLOSED on any read error — blocking the call — because an emergency brake that disarms itself the moment the database has a bad moment is a brake in name only.

An operator, responding to a real incident, sets the emergency kill switch to stop all AI message serving platform-wide. Before this fix, that action changes a value that exactly one, now-mostly-unused code path checks — every real customer conversation, running through the customer-credential path or the tool-calling loop, continues exactly as before, because neither path was ever wired to look at that value. The dashboard shows the switch as "on." The bill for every model call that ran anyway keeps arriving. After the fix, the same action is evaluated by a single guard consulted at the true entry point of every real call path — the switch being "on" and a model actually being called become, for the first time, mutually exclusive states.

Centriu Synapse's emergency stop for AI message serving now covers every real call path to every supported provider — evaluated at the point closest to the decision, backed up inside the two functions that build the actual provider requests, and confirmed by a test suite that counts real invocations rather than trusting that a thrown error means the block worked.

This automation governs Centriu's own internal emergency-stop mechanism for AI message serving — it is not a customer-facing feature to configure, is not a rate limit or usage quota, and does not replace authentication, authorization, or tenant isolation, all of which are separate, independently maintained controls.

Placing a safety check inside the one function that happens to be the current gateway to a capability works exactly as well as that function stays the only gateway — and no promise about that holds once a product keeps shipping. A guard called from a single shared entry point, with backstops inside every real implementation as insurance against a future shortcut, does not depend on staying the only path; it depends only on new code choosing to call it, which is a much easier property to keep true and to verify with a test that counts real calls instead of trusting an exception.