Architecture Escalation signals
Novelty and doubt
A decision model that acts in a loop needs a way to say "this one isn't mine." In the long-running operations agent, it answered two extra questions on every tick, and either one could send the decision up to the LLM. They ask different things, and they point to different fixes.
| Check | Asked how | Question | Escalates when |
|---|---|---|---|
novel_situation | Inside the decide call, alongside the option scores | Is this state unlike what the goal, playbook and milestones describe? | > 0.7 |
doubt_action | A separate verifier call, after the action is chosen, with that action in view | Would a careful operator doubt this specific action here? | > 0.7 |
Novelty is about the situation
High novelty means the context the model judges against doesn't cover this state. The fix is more context: a playbook line, a milestone, a standing rule.
Doubt is about the action
High doubt means the chosen action looks wrong for this state. The fix is in the options or the plan: a planner rule, a code policy, or the LLM overriding this one action.
What we observed
Four sessions, several hundred ticks, about 60 logged escalations.
1. In-call novelty is weak; a separate doubt verifier is strong
Asked inside the decide call, novelty sat in a narrow band (p10/p50/p90 = 0.32/0.40/0.50) and rarely fired. Doubt, asked as its own call on the chosen action, ranged widely (about 0.1–0.85) and tracked real problems. A model grading its own pick in the same breath is less useful than a second look at a concrete action.
2. When doubt fires high, it is usually right
Every firing above about 0.75 pointed at a real problem:
- 0.79: a plan that depended on a 2%-per-attempt outcome, burning about 50 attempts' worth of supplies on average.
- 0.78: sell an item every cycle while also producing it.
- 0.84: take on a risky, under-equipped job with a 4.5% chance of payoff, to advance an unrelated goal.
The one false alarm (0.74, a large sale that was fine) went away after one line of playbook context. The same kind of sale scored 0.35 next time.
3. Novelty fires on first occurrences, and context retires it
Novelty fired at 0.71 twice in one session, both genuinely new for the agent: retiring an obsolete supply tier for the first time, and switching tools to gather a raw material. After one playbook line ("older tiers are surplus once the better tier is in use"), retiring an older piece of equipment went through at novelty 0.35. Firings do their job when they're rare, and when each one leaves behind context that prevents the next.
4. Doubt measures how well the context fits
| Decision | Doubt |
|---|---|
| Equipping the milestone's target item | 0.08–0.19 |
| Steps along a named milestone chain | 0.15–0.35 |
| Ambiguous picks against a vague goal | 0.5–0.65 |
That makes doubt a cheap metric for context changes. When a playbook line conflicted with the only safe option, doubt sat at 0.60–0.63 on three identical picks. After the line gained a safety condition, the same pick scored 0.51, then 0.46.
5. The patterns under the threshold carry information
- Rising doubt across repeated identical decisions points to a loop. A purchase loop's doubt climbed to 0.64 over repeats and never fired.
- Flat, persistent mid-band doubt on the same pick (0.6 ± 0.03, several times) points to a context conflict.
- A single mid-band reading on a new kind of action is usually fine. Let it run and look at the outcome.
A fixed 0.7 gate tuned to stay quiet hides all three. Treat 0.5–0.7 as a watch band: log it, and have the LLM review it at its next scheduled wake rather than waking it on purpose.
6. Doubt can't see problems in the options it was given
This is the most important limit. Several bad plans arrived with doubt at only 0.27–0.35: making 30 units when the batch needed 300; making a short batch instead of first making the scarce input; running a job on leftovers that covered one unit; gathering for 100 minutes when equivalent stock sat in storage.
The verifier judges whether the chosen action is reasonable among the options it was shown, in the state it was shown. When the planner offers only bad options, the best bad option looks fine. Doubt is a check on the choice, not on the option generator. Planner gaps have to be caught another way: the LLM reads the plan's stated reasons, outcome checks run after each step, and code raises efficiency signals ("this batch will cover only 4 units").
How to use the two signals
| Signal | Reading | Who acts | Typical fix |
|---|---|---|---|
| Doubt > 0.7 | This action looks wrong | LLM, now | Override; then fix the planner or add a code policy so the option isn't offered again |
| Novelty > 0.7 | This state is outside the context | LLM, now | Approve or redirect; add a playbook line or milestone |
| Doubt 0.5–0.7, flat over repeats | Context conflict | LLM, next wake | Find the conflicting line and add the missing condition |
| Doubt rising over repeats | Possible loop | LLM, next wake | Check whether the outcomes are changing state at all |
| Doubt low, plan inefficient | Planner gap; neither check can see it | LLM audit, outcome checks | Fix the option generator |
| Doubt after a context change | Did the fix work? | Measurement | Compare the same kind of decision before and after |
Design notes
- Use a separate verifier call for doubt. One concrete chosen action gives a sharper reading than self-assessment inside the scoring call. It costs one extra call per decision, which is cheap at decision-model prices.
- Keep the two questions separate. Merging them into one "uncertain?" signal loses the routing.
- Log every reading, not just firings.
- Close the loop. If the same check keeps firing on the same kind of case, the lower tier hasn't learned yet.
- Don't expect the decision model to audit its inputs. It can say "this choice looks wrong." It can't say "you forgot to offer the right choice."
// The shape of one tick (pseudocode)
const options = planner.legalOptions(state); // code
const scores = await dm.decide(state, options, { // decision model
extra: { novel_situation: noul("Is this state unlike the goal, playbook and milestones?") }
});
const pick = combine(scores, weights); // code, explicit weights
const doubt = await dm.ask(state, noul(`Would a careful operator doubt this action here: ${pick}?`));
log({ tick, pick, doubt, novelty: scores.novel_situation });
if (doubt > 0.7 || scores.novel_situation > 0.7) return escalate(pick); // LLM, now
if (doubt >= 0.5) watchBand.push({ tick, pick, doubt }); // LLM, next wake
act(pick); verifyOutcome(pick); // code