A field guide · October 2026
Match the tool to the work.
A decision model reads text and returns typed answers with probabilities: which of these options, how likely is this true, where on this scale. It is fast and costs fractions of a cent per decision. It sits between your code and a big language model, and it handles the judgment calls that don't need an essay. This site teaches that idea from a five-year-old's version up to the engineering, using what we measured in two real projects.
Pick your level
Explain like I'm five
A mailroom, three bins, and a grown-up upstairs for the letters nobody is sure about. Five minutes, no jargon.
101 · What a decision model is
Choice, Noul and Score. State and questions. Why calibrated probabilities matter, and how a decision model differs from an LLM.
201 · Asking well, using answers
Contrastive choices that name the near misses, exact propositions, answers as features, and thresholds that mean something.
301 · Engineering a decision tier
Honest evaluation, thresholds keyed to (model, runtime, rubric), drift, adapters across vendors, determinism and operations.
The big ideas
The cascade
Code first, a decision model for bounded judgment, an LLM for open questions, a human for stakes. Every escalation should end with a fix in a lower tier.
ArchitectureNovelty and doubt
The model's own "this one isn't mine" signals. High doubt was right every time it fired. The bugs sat just under the line.
Case studyThe Melvor agent
Four pages on the real testbed: an idle game played for six days, the cascade built for it in color-coded pseudo-code, and the 65 push-downs that taught it.
EvidenceFour ways compared
Jev, OpenAI's Decisions API, open-weight decision models and Claude Haiku 5.5, measured on wine reviews, a real inbox and browser pages.
EvidenceThe cost ladder
With Jev as 1×: Decisions API 2.4×, Haiku about 6×, Sonnet about 140×, Opus about 290×, Fable about 710×. Includes a calculator.
EvidenceLandscape refresh
What's new as of October 2026: the Claude lineup, the Decisions API, and whether the open-weight options have gotten better.
BuildBuild your own
The Decision Skeleton: a facilitated tutorial you run in Claude Code, one small module at a time, ending with a cascade you measured yourself.
The ten-line version
- Put each kind of decision in the cheapest tier that can make it correctly: code first, a decision model for judgment, the LLM for novelty, the human for preference.
- When the decision model is wrong in a consistent direction, or the question has a computable answer, move it to code.
- The decision model judges the decision against the state it is shown. It can't see a consequence the state doesn't contain, so code must put that consequence in the state.
- Doubt is good evidence when it is high. Bugs often sit just below the escalation line, though, and the worst ones never reach the model at all: they are planner gaps where every option offered was bad.
- A playbook line turns a novelty escalation into a confident decision next time. One sentence of context moved doubt from 0.71 to 0.35.
- Verifiers must check outcomes (the count went up), not starts. They must also accept deliberate stops as success.
- In an agent loop, LLM wakes are the scarce resource. Batch sizes, restock sizes and how often "nothing to do" happens decide the wake rate.
- Prefer one contrastive Choice whose options name the near misses. Grade every question against its own concept.
- Use answers as features in a learned combiner, not as hard thresholds. Validate on a new time window, not just cross-validation.
- Safety belongs in the fastest tier, and the environment (tabs, focus, disconnects) is its own failure class.
Or sing it
What Are We Asking For? is a 4½-minute karaoke video that teaches the same cascade: deterministic code for rules you can write down, a decision model for bounded judgments, a language model for open questions, and a human for stakes and values. The chorus is the whole idea: match the tool to the work we do, and let the real world tell us what holds true.
Want it sharper? Download the 1080p version (523 MB).
Song made with Suno (voice and piano). Animation made with Claude Code on ClaudeAnimationBase.
What Are We Asking For? (Name the Job, Then Light the Spark) is the pop-punk version: 3¼ minutes, new lyrics, the same lesson, set in a mid-2000s rock club. Clawd is the frontman (the language model), a drum machine plays the code, a guitar tuner that can only snap to the options it was given plays the decision model, and the stage manager makes the human call on the pyro. Name the job before we light the spark, and let the real world keep the score.
Song made with Suno (pop-punk). Animation made with Claude Code on the same kit; thumbnail made with gpt-image-2.
Where the evidence comes from
An email first pass
A personal inbox, studied across several held-out months. Every message got a set of decision-model questions plus deterministic rules, and was sorted into buckets that set how much LLM attention it gets. Junk cost zero LLM tokens. Concept labels were made blind.
A long-running operations agent
An agent that ran for days in a web browser: it scheduled production jobs, bought and restocked supplies, managed limited storage, and kept a health metric above a floor. Code planned and verified, the decision model judged priorities on every tick, and Claude woke on escalations. Several hundred logged ticks and about 65 resolved escalations. The decision model cost under $0.10 a night.
The testbed was an idle game (Melvor Idle) chosen because it runs unattended for days. The general pages describe it in plain operations terms: jobs, supplies, stock, storage. The Melvor case study shows the real thing.
The decision model in both was Jev, TypeSafe's hosted model (jev-1.13.0). In October we ran the same harnesses against OpenAI's Decisions API, two open-weight decision models and Claude Haiku 5.5. The results are on Four ways compared.