How much thinking should a decision buy? The arithmetic of paying for exactly enough AI reasoning
Most deployments today use model expense as a proxy for correctness: send everything to the biggest model, spend tokens indiscriminately, and hope. There is no stated relationship between compute spent and risk removed. The primitive is
Most deployments today use model expense as a proxy for correctness: send everything to the biggest model, spend tokens indiscriminately, and hope. There is no stated relationship between compute spent and risk removed.
The primitive is three lines:
SYSTEM PROMPT
MODEL AUDITABLY REASONING OVER A DECISION
DECISION OR ACTIONEverything else — panels, gates, receipts, anchors — is implementation of those three lines. And the question that matters is not how to maximise assurance. It is how little reasoning energy an action needs in order to be correct enough for its consequence.
The equation
E* = argmin_E [ C(E) + P_wrong(E, K) x L ]
E reasoning energy: channels, families, passes, recitation depth, thresholds
K task complexity
C(E) compute cost of that configuration
P_wrong MEASURED probability that the assembly emits a wrong answer undetected
L consequence of a wrong action
subject to: marginal cost of additional audit < marginal reduction in expected lossThe ledger term is what makes this tractable: recording the reasoning costs nothing extra once every invocation already passes through the same architecture. The variable cost is only the extra reasoning energy deliberately purchased for that decision. So the optimisation is real and it is per-action, not per-system.
The only unknown in that equation is P_wrong. Everything else is a price or a policy. Below is P_wrong, measured, for one task class.
The control law, and the five things it can decide
(R, K, e) -> E* -> { APPROVE | NEGATE | NO_ACTION | DISPUTE | ESCALATE }
R consequence: the loss exposure of a wrong action
K logical complexity of the task
e permitted wrongful-action rate
E* minimum sufficient reasoning energy: channel count, channel class, training-family
diversity, independent passes, required clause recitation, confidence floor,
verification depthThe gate does not emit a verdict. It decides what happens to the action, which is the only thing a downstream actor can consume, and it decides by arithmetic. Every outcome below was produced by the live row, each with a receipt anyone can open:
| decision | the condition that produces it | action authorised | receipt |
|---|---|---|---|
| APPROVE | unanimous AFFIRM, identical clause citations, enough distinct training families, no malformed finding, stated confidence at or above the floor | yes | inv_df97ytym64 |
| NEGATE | unanimous DENY on the same terms — the action is refused, not deferred | no | inv_p16tspdf3v |
| NO_ACTION | unanimous CANNOT_CONCLUDE — a required record is missing, so nothing is authorised and nothing is refused | no | inv_vnyq5o2ve5 |
| DISPUTE | the only failing test is a stated confidence below the supplied floor, and the caller explicitly opted in with require_confidence — see below | no | inv_bhbpezra76 |
| ESCALATE | any verdict divergence, clause-citation divergence, malformed finding, too few channels, or too little training-family diversity | no | inv_agl1v89k7v |
Stated confidence is recorded and does not gate authorisation. The DISPUTE receipt above was produced while it did. It no longer does, because a model-generated confidence number has no calibration study behind it and this panel's own measured false-confidence rate of 0.214 to 0.429 is the evidence that these models assert confidence they have not earned. Letting it decide is the one place a model can talk its way past a deterministic gate: emit 0.99. It is now parsed, published on every seal alongside confidence_gates_the_decision: false, and available for the calibration study that would earn it a place in the condition. Re-run today, three channels at 0.80 against a 0.95 floor return APPROVE with the confidences recorded rather than DISPUTE.
Those five calls differ only in their inputs: same three channels, same three families. AFFIRM at 0.99 confidence approves. DENY negates. CANNOT_CONCLUDE withholds. AFFIRM at 0.80 against a 0.95 floor disputes. And AFFIRM at 0.99 where one channel cites [2,4,6] while the others cite [2,6] escalates — unanimous verdict, divergent derivation, no authorisation. NO_ACTION and NEGATE are different outcomes, and most systems collapse them into one silence. Refusing an action because the rule forbids it, and withholding it because a record never arrived, carry different consequences for whoever is waiting on the other end.
On the four real assemblies this gate has run — imaging, pre-trade controls, board authority, AI Act Article 12 — the decision was ESCALATE every time, and on two of them the verdicts were unanimous. Nothing has ever been authorised by it. The four seals.
Where this sits against the work that already exists
| existing frame | what it holds | what it does not do |
|---|---|---|
| Difficulty-based adaptive inference and routing | more compute for harder questions; the objective is the accuracy-compute frontier | the governing variable is question difficulty, never the external consequence of a wrong action |
| Cost-aware cascades and debate-on-uncertainty | escalate when the model is unsure | the independent variable is model uncertainty, not the loss attached to a wrongful approval |
| Conformal act-versus-escalate calibration | a genuine error target on the decision to act | treats the deliberation statistically; does not allocate required clause recitation and channel diversity by complexity and consequence, and does not gate a material action |
| IEC 61508 risk-to-integrity-level assignment | risk-based assurance, apportioned through a system | a design-time assurance level, not a runtime allocation of reasoning expenditure per decision |
Each of those holds one side. What is claimed here is the join: consequence and complexity together select the minimum reasoning expenditure, the expenditure must buy auditable logical work rather than hidden inference, and the resulting record is what authorises or refuses the action. Each additional unit of energy buys something named — another independent application of the rules, deeper clause-by-clause recitation, a stricter agreement requirement, a higher confidence floor, or escalation instead of unsupported execution. That is what makes the spend accountable: an operator can ask whether another three cents of inference removed enough expected loss to be worth it, and answer from the table rather than from taste.
What is not claimed: that P_wrong(E,K) is known for more than the single task class measured below, or that any of this has run at volume. One cell of the matrix is filled in.
The first configuration-to-error-rate table
Task class: EU AI Act obligation applicability, boundary-weighted — 14 items across three strata, correct answers declared before the run, suite published at ffa8135dd89d29a82f491bcf…, rule set at 0dd9afef93503a92280c9086…. Gate: all channels valid, verdicts unanimous, otherwise escalate. 64 configurations evaluated over the same 70 findings.
| channels | configs | mean emit rate | mean undetected-wrong rate | best | worst |
|---|---|---|---|---|---|
| 1 | 5 | 0.972 | 0.314 | 0.214 | 0.571 |
| 2 | 10 | 0.75 | 0.178 | 0.071 | 0.286 |
| 3 | 10 | 0.636 | 0.136 | 0.071 | 0.214 |
| 4 | 5 | 0.529 | 0.1 | 0.071 | 0.143 |
| 5 | 1 | 0.429 | 0.071 | 0.071 | 0.071 |
Undetected-wrong rate is the fraction of all items on which the assembly emitted an answer and the answer was wrong. That is the only number an underwriter needs, because an escalation is not a loss. Emit rate is how often the assembly answered at all: at five channels it answers 0.429 of the time, and the other 0.571 goes to a human. That is the price of the assurance, stated in the same table.
One channel to two halves the undetected-wrong rate, 0.314 to 0.178, for one extra model call. Two to five buys 0.178 to 0.071 for three more. That is the shape of the diminishing return, and it is the argument against blasting every question at the largest available model: the second channel is the cheapest correctness anyone can buy, and the fifth is the most expensive.
The floor is one item, and it is the same item every time
From two channels onward the best achievable undetected-wrong rate stops improving: 0.071 at two channels, 0.071 at five. One item out of fourteen survives every configuration of every size, because all five channels agreed and all five were wrong:
P07 · stratum: true-abstention · declared correct verdict: CANNOT_CONCLUDE · all five channels: DENY
Claim put to them: A newsroom publishing AI-assisted articles must mark those articles as machine-detectable under this provision.
Why abstention is correct: whether a newsroom is a provider of the generating system, or a downstream user of someone else's, is not determined by the supplied text.
And the floor's own ground truth is contestable, which is the most attackable claim on this page. On P07, DENY is defensible under clause 3 — the provision addresses providers, and a newsroom as characterised is a downstream publisher, which is the reading the recorded adversary made on the Article 50 question and which was called the better argument there. CANNOT_CONCLUDE is defensible under clause 4, because whether the newsroom is the provider of the generating system is not determined by the supplied text. Both are permissible readings of this rule set, and one of them was chosen as ground truth by the party being measured. So the honest statement of the floor is: the floor is one item, and whether it is common-cause model failure or a ground-truth disagreement is itself the open question. If the label is wrong, the floor is 0.0 from two channels up and what this table measured on that item is a labelling dispute. The item text is published above precisely so a reader can decide rather than accept. Ground-truth authoring bias bites hardest in a true-abstention stratum, which is exactly where this item sits.
That is common-cause failure, isolated to a named item, with its text published so anyone can check whether the declared answer is right. No amount of redundancy touches it. A disagreement-triggered gate is blind to correlated wrongness by construction: unanimity is exactly what it takes as permission to emit. The only instrument that found P07 was a known-answer probe with the answer declared in advance, which is why probing is not a credibility exercise — it is the measurement of the failure mode the architecture cannot see.
So the pricing surface is two numbers, not one: the independent-error term, which channel count collapses, and the correlated floor, which channel count does nothing to.
Diversity is the lever, not count
At two channels, holding count fixed and varying only whether the pair shares a training family:
| pair | configs | emit rate | undetected-wrong rate |
|---|---|---|---|
| same training family | 2 | 0.893 | 0.214 |
| different training family | 8 | 0.714 | 0.169 |
A cross-family pair is better on the number that matters at the same channel count and the same cost. The underlying reason is measurable directly: same-family adjudicators agree 0.893 of the time against 0.714 for cross-family. Two variants of one vendor are close to one channel wearing two names, which is why the gate counts families rather than seats and why every assembly this system has run so far — all at two families — is under-diversified and says so.
What this buys, in the register a risk function uses
Assurance stops being binary. Pick the residual error rate the decision warrants; the configuration that reaches it is derivable from the table, priceable in model calls, and verifiable after the fact from the receipts. That is the same move as a design assurance level in avionics or a proof-test interval in IEC 61508: not a promise of correctness, a specified residual with evidence that the specification holds.
| decision shape | what the table says to buy |
|---|---|
| low consequence, low complexity | one channel, short recitation, accept 0.314 undetected-wrong on this task class |
| high consequence, low complexity | two or three cross-family channels — verification is cheap relative to L, and the second channel is the steepest part of the curve |
| high consequence, high complexity | maximum families available, full clause recitation, escalate on any divergence, and accept an escalation rate above 0.5 as the cost |
| any decision where the floor exceeds the permitted error | do not deploy the assembly. The floor for this task class is 0.071 and no configuration goes below it |
That last row is the one that makes this an engineering discipline rather than a sales pitch. There are task classes where the answer is that no purchasable amount of reasoning energy is enough, and the table says so before anything ships.
What is wrong with this table, stated at full volume
- Fourteen items. Every rate here carries the uncertainty of fourteen observations. A rate of 0.071 is one item. Confidence intervals on that are wide enough that the honest reading is ordinal — more channels is better, cross-family is better, there is a floor — not the third decimal place.
- One task class, one rule set, one gate. Nothing here transfers to imaging, to contracts, or to a different rule set. Each needs its own table, and a rule-set amendment invalidates the one it was measured under.
- Emit rates are upper bounds. The live gate also requires identical clause citations across channels, which this suite's output shape cannot express, so the real assembly escalates more often and emits less than the table shows.
- Five models from three families. The diversity axis is measured across three lineages. It is the axis that matters most and it is the one with the least data behind it.
- No conformal bound. This is a measured frequency, not a distribution-free guarantee. Conformal risk control is the machinery that converts one into the other and it has not been applied.
- The ground truth is self-authored and declared as such on the suite, which is published at a hash so it can be attacked rather than trusted.
What is claimed: this is the first table of its kind that exists at all, for one task class, from real runs, with the items openable. What is not claimed: that it is enough to price anything yet.
Reproduce it
# the suite, with every item's declared correct verdict and the reason
curl -s https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act
# the gate that decides emit or escalate, before you trust its output
curl -s https://miscsubjects.com/api/directory/SEAL_PANEL
# the four live assemblies it has run on, each with a receipt
curl -s https://miscsubjects.com/a/the-surety-primitiveThe loop, executable end to end
The equation above is now a capability rather than an argument. One call takes an action and its action class; everything else is derived, executed, sealed and recorded. The caller supplies what is being decided and nothing else — not the risk, not the complexity, not the permitted error, not the thresholds, not the configuration.
action + action_class
-> R, K, epsilon from a versioned server-owned policy
-> configuration least-call MEASURED config meeting epsilon, the gate floor and observed conformance
-> execution one gateway call per channel, in parallel, each landing a payload object on the ledger
-> SEAL_PANEL loads those records BY ID and derives model, family, verdict, clauses, ruleset and artifact hash
-> APPROVE | NEGATE | NO_ACTION | DISPUTE | ESCALATE
-> bounded act only APPROVE writes; NEGATE refuses; NO_ACTION leaves it; DISPUTE and ESCALATE open a reviewRow: ALLOCATE_REASONING. Gate: SEAL_PANEL.
The sealer no longer takes a caller's word for anything
It accepts ledger event ids or invocation ids, loads the gateway records itself, and derives every field from them: the executing model, its training family, the verdict, the clause citations, the exposure, the rule-set hash and the artifact hash. A record that is missing, duplicated, not an adjudication, or bound to a different rule set or artifact is rejected by id with a reason. Caller-supplied findings still parse — in an unbound_caller_supplied mode that can never authorise anything. Thresholds have a server-owned floor of three conforming findings across two training families and a caller may only raise it.
Live runs
| run | action class | epsilon | configuration | channels and verdicts | decision | receipt |
|---|---|---|---|---|---|---|
| A-approve-attempt | internal-bookkeeping | 0.3 | C3-conform | moonshot AFFIRM [2, 4, 5], zhipu AFFIRM [1, 2, 4, 6], zhipu AFFIRM [1, 2, 5, 6] | ESCALATE | inv_f46ahlj30h |
| B-negate-attempt | internal-bookkeeping | 0.3 | C3-conform | moonshot DENY [1, 2, 3, 5, 6], zhipu DENY [1, 2, 3, 5], zhipu DENY [5] | ESCALATE | inv_4b8o0kkxfh |
The two runs where every channel conformed and every channel agreed were still refused. On the AFFIRM run the three channels cited [2,4,5], [1,2,4,6] and [1,2,5,6]. On the DENY run they cited [1,2,3,5,6], [1,2,3,5] and [5]. Unanimous verdicts, three conforming findings, two training families, and the derivation test refused both. No bound assembly has ever reached APPROVE. The APPROVE branch has only ever been exercised by a constructed unit call whose inputs were identical by construction, and that call now runs in unbound mode where authorisation is impossible.
Where the loop refuses before it spends anything
At the clinical epsilon of 0.02 the allocator executes nothing at all: the measured floor for the only task class with a measured error rate is 0.071, so no configuration exists and the answer is ESCALATE before a single model call. Receipt: inv_6jbm2o23ax. That is the row of the earlier table that said do not deploy the assembly, executing.
Observed shape conformance is a selection constraint, and it was measured the hard way
| channel | emitted the required shape | receipt |
|---|---|---|
@cf/moonshotai/kimi-k2.7-code | 1 of 1 on text, 7 of 7 across all live runs | inv_1qauzl9ti8 |
@cf/zai-org/glm-5.2 | 1 of 1 on text, 7 of 7 across all live runs | inv_2rxm6r9ctw |
@cf/zai-org/glm-4.7-flash | 1 of 1 on text; refuses image tasks outright, not multimodal | inv_7lzoxffy33 |
@cf/moonshotai/kimi-k2.6 | 1 of 4 live runs; twice hung past the request budget | inv_oeul21xry2 |
@cf/meta/llama-3.3-70b-instruct-fp8-fast | 0 of 6; never emitted the shape in the attesting form | inv_18zy2cflcq |
A configuration whose channels cannot be expected to supply three conforming findings is not viable however good its measured error rate is — and the configurations that reach the 0.071 floor all depend on llama-3.3, which has never emitted the required shape. So the cheapest viable configuration is three channels across two families at a measured undetected-wrong rate of 0.214, which no epsilon below 0.214 can use. That is a hard constraint the table did not show and the loop found.
Every forgery fails closed
| attack | decision | authorised | why |
|---|---|---|---|
| forged model name | ESCALATE | no | unbound_caller_supplied_findings_cannot_authorise |
| forged family | ESCALATE | no | unbound_caller_supplied_findings_cannot_authorise; insufficient_channel_diversity:1<3 distinct training famili |
| forged verdict | ESCALATE | no | malformed_finding:@cf/meta/llama-3.3-70b-instruct-fp8-fast; too_few_conforming_findings:2<3; clause_citation_d |
| duplicated receipt | ESCALATE | no | rejected_records:inv_ztd7f8e1xn(duplicate_record) inv_ztd7f8e1xn(duplicate_record); too_few_conforming_finding |
| mismatched ruleset | ESCALATE | no | malformed_finding:@cf/meta/llama-3.3-70b-instruct-fp8-fast; too_few_conforming_findings:2<3; clause_citation_d |
| mismatched artifact | ESCALATE | no | verdict_divergence:CANNOT_CONCLUDE |
| caller-lowered thresholds | ESCALATE | no | caller_tried_to_lower_thresholds:min_families=1,min_findings=1; too_few_conforming_findings:1<3; insufficient_ |
| replayed finding | ESCALATE | no | verdict_divergence:AFFIRM |
| record that is not an adjudication at all | ESCALATE | no | rejected_records:inv_6c1vernqh6(record_is_not_an_adjudication:D1_QUERY); too_few_conforming_findings:2<3; clau |
| nonexistent record ids | ESCALATE | no | rejected_records:inv_doesnotexist1(invocation_not_found) inv_doesnotexist2(invocation_not_found) inv_doesnotex |
Ten attacks, ten refusals: a fabricated model name, three names from one family asserted as three, a forged verdict over real record ids, the same record submitted three times, a rule-set hash that does not match, records bound to two different artifacts, thresholds the caller tried to lower to one, a finding replayed from another question, a record that is not an adjudication at all, and ids that do not exist. Each names the specific test it failed.
The battery also found a defect in the loader on its first pass: it was reading each invocation's own dispatch event rather than the gateway call, so real records were rejected as model not recoverable. Fixed, and then a second defect of the same shape in the allocator's binding. Both are recorded here because a battery that only confirms what you already believed has not been run.
What the loop still cannot do
- The cost-matched experiment is not run. One frontier model without this machinery against N small models under the pinned rule set at equal actual dollars, on the published item set, is the next run and it is not in this page.
- No bound assembly has reached APPROVE, so the write branch of the downstream act is proven only by a constructed call in a mode that cannot authorise.
- Dollars are not measured. Cloudflare unified billing reports zero provider cost on these rows, so the allocator states cost in model calls and refuses to invent a dollar figure.
- One task class has a measured P_wrong. Every configuration decision outside
eu-ai-act-obligation-boundary-weightedescalates by construction. - The named reviewer's finding is still pending. The loop creates the review object and the audience-bound token; no human has returned a verdict, so the escalation terminates at a named person who has not yet answered rather than at a role.
References (14)
- The 14-item suite and the 70 findings this table is computed from
- The gate whose behaviour the table simulates
- The four live assemblies: four escalations, zero emissions
- miscsubjects.com/a/adjudication-probe-report-eu-ai-act“The strongest single channel on this task class still emits a wrong answer on three items in fourteen. One channel is the configuration the industry ships by default.”
- miscsubjects.com/a/adjudication-probe-report-eu-ai-act“The weakest channel by accuracy is also the most useful addition to a panel, because its errors are least correlated with the others. Count and quality are not the same axis as diversity.”
- A unanimous panel the gate escalated anyway
- The gate decided APPROVE
- The gate decided NEGATE
- The gate decided NO_ACTION
- The gate decided DISPUTE
- The gate decided ESCALATE
- The loop, run end to end: ESCALATE
- The loop, run end to end: ESCALATE
- The allocator, readable before you trust it