A toolkit for building narrow, calibrated decision models — trained on synthetic data with a known answer key, so the confidence they report is trustworthy by construction, not patched on afterward.
pip install certo
research preview
MIT
non-generative · typed probabilities
Inspired by Jev / “System-1” decision models. Independent project — not affiliated with TypeSafe.
A decision model earns its keep when it says “route to billing, 0.82” and the 0.82 is real — so you can escalate on doubt or trade quality against cost. Train on hard labels (the usual way) and the model still picks well but turns overconfident; you’re left patching calibration after the fact.
certo trains against the full answer distribution (a proper scoring rule), so probabilities mean what they say — calibrated by construction.
Because the data is generated from a known posterior, every model is scored against the exact answer — not just accuracy, but fidelity.
You define a decision as a synthetic world whose answer is computable. certo renders it to natural language, and a small encoder learns to read the state and each option and emit calibrated, typed probabilities. The answer key both trains and grades.
Measured against the exact posterior (KL, lower is better) on held-out wording. Training on the full distribution lands essentially on the answer; single-label targets sit near “ignore the evidence.”
The interface certo ships (research preview — API stabilizing).
# 1 — describe your decision as a known-answer synthetic world from certo.synth import EvidenceWorld world = EvidenceWorld(classes=["billing", "tech", "refunds"], features=12) data = world.generate(n=20_000) # (state, options, EXACT target) # 2 — train, calibrated by construction (soft/distributional targets) from certo import train model = train(data, backbone="ModernBERT-base", target="soft") model.save("my-router") # 3 — use it locally: state + runtime options -> calibrated probabilities from certo import DecisionModel m = DecisionModel.load("my-router") r = m.decide(state="card charged twice, refund please", options=[{"name":"billing", "description":"charges & payments"}, {"name":"tech", "description":"app problems"}], kind="choice") r.probs # {"billing": 0.86, "tech": 0.08, "OUT": 0.06} — calibrated r.abstain # True when the top probability is below your threshold
Certo-R1: a Qwen3-4B fine-tuned for concise, verified reasoning — behind the same API.
A calibrated encoder is cheap and trustworthy, but hard, reasoning-heavy decisions (multi-step arithmetic, timezone math, nested exceptions) need generation. Certo-R1 reasons in a sentence or two and returns the chosen option — non-inferior to full verbose reasoning at ~15× fewer tokens and ~8–11× lower latency. The natural architecture is a cascade: the calibrated encoder decides the easy majority; low-confidence cases escalate to Certo-R1.
from certo import ReasoningDecisionModel m = ReasoningDecisionModel.load("altslate/certo-r1-qwen3-4b", device="cuda") r = m.decide( state="A parcel weighs 2.4 kg. Shipping is $3/kg, any part of a kg rounds up. Charged $9.", options=[{"id":"yes", "description":"charged correctly"}, {"id":"no", "description":"charged incorrectly"}], instructions="Was the customer charged correctly?") r["top"] # 'yes' r["rationale"] # 'Round 2.4->3 kg; correct = 3x3 = $9; paid $9 -> correct. FINAL ANSWER: A'
easy 0.98 · standard 0.90 · hard 0.60 — reasons far above an encoder on the hard tier (it emits an answer, trading calibration for intelligence).
concise verified reasoning: ≈32 vs ≈489 generated tokens per decision, ~8–11× lower routed latency, at non-inferior accuracy.
ReasoningDecisionModel.load().decide(state, options) —
drop-in with the calibrated encoder; escalate on low confidence.
The unified decider: one Qwen3-4B backbone, one scoring head — Choice, Score and Noul, probabilities-first.
A single head answers all three decision shapes — pick one of many (Choice), rate on a scale (Score), or yes/no (Noul) — and always returns a probability distribution, not just a label. It can answer in one fast forward pass, or write a short rationale first; both paths return a calibrated distribution, so reasoning improves the answer without giving up trustworthy confidence. Software reads the distribution into the typed response you asked for.
On the decision types it’s trained for: Choice acc 0.93–0.97 (incl. unseen domains), Noul 0.88–0.92, Score up to 1.00 — and it holds this on new domains and wordings.
Probabilities-first, calibrated by construction: ECE 0.008–0.05 on matched data. The number means what it says — so you can abstain, escalate, or trade quality against cost.
A cheap prompt-token read for the easy majority;
a brief rationale when it helps. Both emit a distribution. decide(state, options, kind, mode).
Where it shines: train it on the decision types you’ll actually serve — routing, triage, intent, eligibility, ordinal scoring, verification — and it is accurate and well-calibrated across all three shapes from one endpoint. It’s built for the decisions you can define, not for zero-shot transfer to arbitrary unseen specifications; give it examples of a decision type and it does that type well.
We checked this on a different source (tool-calling + science decisions it never trained on): picking the right action from described options transferred almost perfectly zero-shot (0.99), while a brand-new decision type — “call a tool now, or ask first?” — needed a few examples before it became reliable and calibrated (0.57 → 0.88). Choice over described options generalizes; new rubrics and new decision types are learned, not assumed.