Hypothesis and Experiment Prioritization Automation: Testing Cheap Instead of Guessing Expensive

Once a team accepts that testing cheap beats guessing expensive, the next problem is deciding what to test first — and a backlog with no consistent scoring system tends to get worked in whatever order feels most urgent, most exciting, or most politically convenient that particular week. A genuinely high-impact idea that happens to be hard to execute competes unfairly against an easy, low-impact idea that just happens to be quick to ship, and without a forced, closing decision on every finished experiment, the backlog also fills with tests nobody ever circled back to interpret.

Scoring frameworks like ICE and RICE are well known conceptually, but applying them consistently requires someone to actually assign four separate numbers to every single idea and trust the resulting math over gut feeling — a discipline that tends to erode under deadline pressure unless the tool itself makes scoring the path of least resistance. And forcing every experiment to close with a real decision, rather than just quietly ending, requires the system to make "still pending" visibly different from "actually finished" — most informal tracking (a spreadsheet, a doc, a chat thread) has no such enforced state.

A hypothesis is created with a required statement plus optional origin, supporting evidence, an expected metric, and an expected outcome — and four scoring inputs a person assigns directly: impact (1-5), ease (1-5), reach (1-10), and confidence (1-5), alongside a required risk level (low, medium, high, or critical). From those inputs, a pure scoring function computes an ICE score (impact times confidence times ease, 1 to 125), a reach-weighted RICE score (reach times impact times confidence, divided by the inverse of ease), a PIE score kept for record alongside ICE, and a separate risk-learning score (the risk level's weight multiplied by six minus confidence) that surfaces hypotheses where testing would teach the most precisely because confidence is currently low. The same inputs classify every hypothesis into exactly one of four quadrants by crossing impact against ease: quick wins (high impact, high ease), big bets (high impact, low ease), fill-ins (low impact, high ease), or thankless (low impact, low ease) — plotted on a visual matrix so a backlog's shape is visible at a glance, not buried in a sorted table. A hypothesis moves through six explicit states — proposed, prioritized, in experiment, validated, refuted, or archived — and each experiment created against it declares one of seven kinds (A/B test, paid channel, page/landing, offer, message/copy, qualitative, or other) plus a design, audience, channel, primary metric, and explicit success and failure criteria declared up front, before the test runs. Every experiment is required to close with one of six decisions — pending, escalate, iterate, pivot, kill, or no action — alongside a result, an interpretation, and recorded learnings, so a finished test can never just silently vanish from the backlog without anyone stating what was concluded. Before any experiment is saved, the system verifies server-side that its declared hypothesis genuinely belongs to the same project and organization the experiment is being created in — a cross-object integrity check that stops a hypothesis from one engagement from silently absorbing an experiment meant for another.

A consultant logs a bold repositioning idea — impact 5, ease 2, reach 8, confidence 2 — alongside a small landing-page copy tweak — impact 2, ease 5, reach 9, confidence 4. On ICE alone (impact times confidence times ease) the repositioning idea scores 20 and the copy tweak scores 40 — the "smaller" idea actually wins on pure ICE, because low confidence and low ease both drag the bolder idea down. But the risk-learning score tells a different part of the story: the repositioning idea's low confidence (2) against a high declared risk level pushes its risk-learning score well above the copy tweak's, meaning testing it cheaply would resolve real uncertainty, not just add a small, well-understood win. The quadrant view shows the repositioning idea as a "big bet" (high impact, low ease) and the copy tweak as a "quick win" (low impact, high ease) side by side — both worth doing, for genuinely different reasons the raw ICE ranking alone would have obscured.

A testing backlog gets worked in an order two different people would actually agree on, because the scoring inputs and the resulting math are the same for everyone, not a matter of individual gut feeling. A bold, uncertain idea and a safe, incremental one both get a fair, differentiated hearing instead of the safer idea always winning by default. And a finished experiment always leaves behind an explicit decision and a recorded learning, so the same already-tested idea never quietly resurfaces as if it were new.

A team running dozens of simultaneous, statistically powered A/B tests with automated traffic-splitting and significance calculations needs a dedicated experimentation platform — this lab is a structured hypothesis-and-decision backlog for a consulting engagement's handful of concurrent tests, not a high-volume statistical testing engine.

An unscored list of "things we could try" gets worked by instinct, and a finished test's conclusion lives in someone's memory at best. Centriu Oracle's lab instead scores every hypothesis on the same four dimensions, classifies it visually by impact and ease, and refuses to let an experiment disappear without an explicit closing decision — so the backlog's priority order is something two people would actually agree on, and nothing tested is ever silently forgotten.