A requirement can ship unbuilt: run 171's animations had a ticked task and no code, and nothing checked them #374

Closed
opened 2026-11-03 18:47:38 -05:00 by cmoriarty · 1 comment
Owner

What happens
Run 171's spec says the client "SHALL animate dealing, the trump flip, cards sliding to center, the trick sweeping to the winner, and a scratch mark", and task 6.3 is ticked. The client has no @keyframes and no animation library, only three transition rules. Nothing caught it:

  • The requirement's one scenario is Reduced motion, which passes with no animation at all.
  • The spec-test step handed web-interface/Animations and web-interface/Responsive phone play to the Visual check, but those are requirements' names, not scenarios', so with_visual dropped them without a word. The check was told six of its eight, and the phone layout was never judged.
  • The code review missed it, and the Visual check called the look "good polish for an MVP", on a brief that isn't one.

Fix

  • An id in .osf/test/visual.json that names a requirement reaches the check as that requirement. One that matches nothing fails the step.
  • The proposal's review flags a requirement whose scenarios don't exercise its main SHALL.

Done when
A sim: a requirement handed over by name gets a verdict, or the round fails. Findings are on #365.

**What happens** Run 171's spec says the client "SHALL animate dealing, the trump flip, cards sliding to center, the trick sweeping to the winner, and a scratch mark", and task 6.3 is ticked. The client has no `@keyframes` and no animation library, only three `transition` rules. Nothing caught it: - The requirement's one scenario is `Reduced motion`, which passes with no animation at all. - The spec-test step handed `web-interface/Animations` and `web-interface/Responsive phone play` to the Visual check, but those are requirements' names, not scenarios', so `with_visual` dropped them without a word. The check was told six of its eight, and the phone layout was never judged. - The code review missed it, and the Visual check called the look "good polish for an MVP", on a brief that isn't one. **Fix** - An id in `.osf/test/visual.json` that names a requirement reaches the check as that requirement. One that matches nothing fails the step. - The proposal's review flags a requirement whose scenarios don't exercise its main `SHALL`. **Done when** A sim: a requirement handed over by name gets a verdict, or the round fails. Findings are on #365.
Author
Owner

Shipped in PR #390, production ddd4b6b.

  • An id left to the Visual check that names a requirement is judged. One that names nothing fails the round.
  • The review brief flags a requirement whose scenarios would pass with it unbuilt.

Evidence: tests/pipeline/test_visual_scenarios.py and sim judge-requirement-by-name. A Free replay of run 172's agent.review.combined raised the new flag, in 1.8 minutes.

Not proved: run 171's game was not rebuilt, so the next full run is the first real test of a ticked task with no code behind it.

**Shipped** in PR #390, production ddd4b6b. - An id left to the Visual check that names a requirement is judged. One that names nothing fails the round. - The review brief flags a requirement whose scenarios would pass with it unbuilt. **Evidence:** `tests/pipeline/test_visual_scenarios.py` and sim `judge-requirement-by-name`. A Free replay of run 172's `agent.review.combined` raised the new flag, in 1.8 minutes. **Not proved:** run 171's game was not rebuilt, so the next full run is the first real test of a ticked task with no code behind it.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#374
No description provided.