A bounded
decision engine.

These small, hand-authored tasks isolate a decision that a larger agent could delegate: choosing a tool, selecting a complete trajectory, checking relevance, checking completion, or choosing files. Correct labels follow the written request and evidence. They do not establish production agent reliability.

The fixtures were published in the earlier report. Model training overlap was not audited. High scores here therefore cannot establish generalization to unseen production traces.

File-context selection has 18 relevant files among 225 judgments. Rejecting every file would score 92% accuracy while recovering no relevant context. Positive F1, recall, false positives, and false negatives therefore matter more than the combined accuracy across all tasks.

A false positive keeps an irrelevant result, claims an incomplete subgoal is complete, or adds an irrelevant file. A false negative discards useful evidence or misses a completed subgoal. Brier measures probability error; lower is better. It uses the returned yes probability and the original 0.5 decision threshold.

For 72 related pairs, changes in option order or incidental wording preserved the semantic answer in 71/72 Kev pairs and 71/72 Jev pairs. For trajectory choices, semantic agreement compares the selected text after option reordering, not the option key.

The 72 pairs are related copies of 72 base tasks, not 144 independent observations. The 225 file judgments are grouped into 15 search tasks. Client timing includes local versus hosted network paths.

The original 40 ticket requests and four fixed team descriptions were sent serially to both models in the same fixed order, first as new states and then as an exact replay. The replay uses no client answer cache. Server caches may apply; Kev retains only four state prefixes by default, so cycling through 40 states is not the same cache condition as CLM's earlier replay.

These CLM results are from September 24 and were not rerun for this report. The builder verifies that the narrow-decision request hash is identical. The comparison can show how the models answered the same fixture; historical timing remains tied to the original serving stack.