Brief hygiene from run 172: the 68k-char apply brief, late task ticks, the judge's unclear contradiction, each judge writing its own validator, reviewers re-reading pasted text #386
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What happens
failandunclearfail the round" and "the verdict isfailwhen any criterion failed ... pass otherwise"; the r3 judge spent ~7k reasoning tokens reconciling them. Each judge round wrote its ownnode -eschema validator (3 times).// spec:tags inside the test bodies instead of above theit; rule B6 failed and 2 min went to moving them: the rule has no literal example.agent.review.combinedre-read spec files that were pasted into its brief ("they were pasted, but just...") and rewrotespec.jsontwice;agent.code-reviewre-read 22 test files the write-spec-tests lint had already passed (1.8 min) and simulated the README and Dockerfile by hand (1.2 min).$TMPDIRscripts, which pushed the apply agent's harness into the repository.Fix
.osf/brief/specs.md,.osf/brief/rules.md), keeping the brief to the task and the pointers.testreport: when the report names the tasks' tests, the step ticks them in the same turn.unclearwording; ship the judge-report validator as a tool or a scripted check.// spec: cap/scenariodirectly aboveit(.Done when
The apply session starts under 30k tokens of brief, ticks follow green runs, and a judge round has one verdict rule and no validator of its own. ~25 min on run 172.
From the run 172 transcript review (Scratch on vLLM + MTP k=4, 2026-11-04; report in the owner's
~/repos/vLLM/reports/). Related: braid#378, braid#379.Shipped in PR #390, production
ddd4b6b.fixloop check-reportas a numbered step of ending, so no judge writes its own validator.carried_fromshown as a number, a literal example for B6, ticks on a green test report, and fewer second reads.Dropped after measuring: moving the delta specs out of the brief. The 81k that apply started at was mostly inherited fork history: its fresh session started at 25.9k, brief included. The move would undo #170. The fork rule in #383 deals with the 81k.
Evidence:
tests/pipeline/test_check_report.py. In a Free replay of round 3, the check caught a false carry before the report was final.agent.code-reviewtook 32.7 minutes andagent.review.combined1.8.Did not clearly hold:
agent.review.combinedstill made a batch of reads on that model, so the line on second reads is unproved.