cmoriarty/braid
1
0
Fork 0
Code Issues 15 Pull requests Projects Releases Packages 1 Wiki Activity Actions

A streaming answer draws a list number as text until its . arrives, and the agent-view spec fails under load #130

New issue
Closed
opened 2026-10-02 22:38:41 -04:00 by cmoriarty · 1 comment
cmoriarty commented 2026-10-02 22:38:41 -04:00
Owner
Copy link

ui/e2e/agent-view.spec.ts:154 ("a streaming answer › flows a few characters a frame, formatted as it is written, and settles without moving", @slow) fails with the answer got shorter at some frame: 268.69 px after 291.08 px. It failed twice in a row in the full lane on 2026-10-02 while #124 and #129 were verified, it fails on main (2ec079a) too, and it passes when run alone.

It is the UI, not the measurement

Every frame of a failing run, recorded by a local probe in the spec (the same heights as the lane):

frame the visible text ends with answer height
216 …older than two d 246.3 px
217 …older than two days, a blank line, 1 291.1 px
218 …older than two days, then Add 268.7 px

The answer's source there is a bulleted list ending - Prune heartbeat rows older than two days, a blank line, then 1. Add the timer. When the reveal has written up to the 1, repairStreaming (ui/src/lib/markdown.ts) keeps it, so it is drawn as a paragraph one blank line below the list. A frame later it is a list item with no gap above it, and the line jumps up 22 px. repairStreaming already holds back a last line that is only the start of a block (#, -, 1., one or two backticks), but not the digits before the ..

Walking every prefix of the fixture answer through StreamingMarkdown in a real browser, a character at a time, finds 7 places where the answer gets shorter, every one a list number drawn as text until its . arrives: the 1 below the bullets and below "Checks", and 2, 3, 4 drawn as an extra line of the item above. The same walk over other shapes finds the same defect twice more:

  • a ~ that will open a ~~~ fence is drawn, then hidden at ~~;
  • a table header | File | Change | with a | under it is two lines, until |- makes it a one-row header, 13 px shorter.

Why it fails only under load

The wrong layout is drawn on every run: in 9 of 10 solo runs a frame showed a 2 or a 4 as an extra line of the item above. But the frame after it was the finished item, which is the same height, so the spec cannot see it. The reveal's frames land on nearly the same prefixes every run, locked to the fake's delivery cadence; CPU contention moves them, and sometimes onto the 1 below the bullets, where the next frame is a line shorter. With seven busy loops, 2 of 8 runs one after another failed, at the same two heights.

In production the deltas are token-sized (1 and . are separate tokens, drained at 30 Hz), so the visible text stops on a list number whenever the reveal catches up with the model, for as long as the next token takes.

The trace's screencast does not show the frame: under load Chrome left a 217 ms gap in it around the drop. The frames on either side are there.

The --repeat-each=8 --workers=8 reproduction is a different failure

None of its 16 failures (8 in each of the two worktrees) was this assertion. They were 20 s timeouts waiting for the live answer, and "a frame added a block of text". The eight copies share the fake osfd's one scenario clock and restart each other's stream, as the spec's header warns.

Also seen, not fixed here

The parser merges a numbered list that follows a bulleted list into the bulleted one. The fixture's 1. Add the timer, 2. Update the query and 3. Update the panel are drawn as - items of the list above, with the blank line between the lists gone. That deserves its own issue.

Fix

Hold back a last line that can still become a block's syntax, as #, - and 1. already are: list numbers before their ., ~ and ~~, and a line of pipes, colons and dashes under a line with a pipe. Add a unit test that walks every prefix of an answer, so a case like this fails on every run rather than only on some runs under load.

`ui/e2e/agent-view.spec.ts:154` ("a streaming answer › flows a few characters a frame, formatted as it is written, and settles without moving", `@slow`) fails with `the answer got shorter at some frame`: 268.69 px after 291.08 px. It failed twice in a row in the full lane on 2026-10-02 while #124 and #129 were verified, it fails on main (2ec079a) too, and it passes when run alone. ## It is the UI, not the measurement Every frame of a failing run, recorded by a local probe in the spec (the same heights as the lane): | frame | the visible text ends with | answer height | |---|---|---| | 216 | `…older than two d` | 246.3 px | | 217 | `…older than two days`, a blank line, `1` | 291.1 px | | 218 | `…older than two days`, then `Add` | 268.7 px | The answer's source there is a bulleted list ending `- Prune heartbeat rows older than two days`, a blank line, then `1. Add the timer`. When the reveal has written up to the `1`, `repairStreaming` (`ui/src/lib/markdown.ts`) keeps it, so it is drawn as a paragraph one blank line below the list. A frame later it is a list item with no gap above it, and the line jumps up 22 px. `repairStreaming` already holds back a last line that is only the start of a block (`#`, `-`, `1.`, one or two backticks), but not the digits before the `.`. Walking every prefix of the fixture answer through `StreamingMarkdown` in a real browser, a character at a time, finds 7 places where the answer gets shorter, every one a list number drawn as text until its `.` arrives: the `1` below the bullets and below "Checks", and `2`, `3`, `4` drawn as an extra line of the item above. The same walk over other shapes finds the same defect twice more: - a `~` that will open a `~~~` fence is drawn, then hidden at `~~`; - a table header `| File | Change |` with a `|` under it is two lines, until `|-` makes it a one-row header, 13 px shorter. ## Why it fails only under load The wrong layout is drawn on every run: in 9 of 10 solo runs a frame showed a `2` or a `4` as an extra line of the item above. But the frame after it was the finished item, which is the same height, so the spec cannot see it. The reveal's frames land on nearly the same prefixes every run, locked to the fake's delivery cadence; CPU contention moves them, and sometimes onto the `1` below the bullets, where the next frame is a line shorter. With seven busy loops, 2 of 8 runs one after another failed, at the same two heights. In production the deltas are token-sized (`1` and `.` are separate tokens, drained at 30 Hz), so the visible text stops on a list number whenever the reveal catches up with the model, for as long as the next token takes. The trace's screencast does not show the frame: under load Chrome left a 217 ms gap in it around the drop. The frames on either side are there. ## The `--repeat-each=8 --workers=8` reproduction is a different failure None of its 16 failures (8 in each of the two worktrees) was this assertion. They were 20 s timeouts waiting for the live answer, and "a frame added a block of text". The eight copies share the fake osfd's one scenario clock and restart each other's stream, as the spec's header warns. ## Also seen, not fixed here The parser merges a numbered list that follows a bulleted list into the bulleted one. The fixture's `1. Add the timer`, `2. Update the query` and `3. Update the panel` are drawn as `- ` items of the list above, with the blank line between the lists gone. That deserves its own issue. ## Fix Hold back a last line that can still become a block's syntax, as `#`, `-` and `1.` already are: list numbers before their `.`, `~` and `~~`, and a line of pipes, colons and dashes under a line with a pipe. Add a unit test that walks every prefix of an answer, so a case like this fails on every run rather than only on some runs under load.
cmoriarty commented 2026-10-03 01:26:50 -04:00
Author
Owner
Copy link

Shipped in 5addb7a, deployed on 2026-10-03 at 01:16 (Actions run 96, manual, together with #132, #133 and #134). Run 95 had been cancelled so osfd would not restart during #124's timing measurement. Archived in 0285a3d as openspec/changes/archive/2026-10-03-streaming-answer-never-shrinks. This change's own session ended after the push, and the #133 session finished the deploy check and the archive.

What changed

  • repairStreaming holds back a last line that is only the start of a block's syntax until the next characters decide what it is. That covers a list marker at any indent (a bullet, or a number with or without its . or )), a heading's #s, and one or two backticks or tildes. It also covers a line of |, :, - and spaces under a line with a pipe, which can still become a table's delimiter row. Inside an unclosed fence nothing is held.
  • A unit test walks every prefix of 16 answer shapes, drawn as StreamingMarkdown draws them, and requires the rows they take in the terminal register never to fall.
  • When the agent-view heights check fails, it now names the answer's text just before and just after the frame that got shorter.
  • agent-transcript's "Streaming text is drawn in its settled form" now says nothing on screen is taken back while text is written, with three new scenarios.

Tested

  • agent-view.spec.ts:154 passed 16 of 16 runs under seven busy loops. None of their 7,589 frames ends in a bare list number, bullet or tilde, and none is shorter than the one before. On the old renderer, 23 of 33 runs drew such a frame, and 2 of them failed.
  • A walk over every prefix in Chromium, over 21 texts, found no drop in the spec's answer (1,299 prefixes) or in any of the unit test's shapes. The two drops left are the known limit below.
  • The fast lane passed in 104 s and the full lane in 152 s (2,363 backend, 524 vitest, 153 browser).

Verified on production

  • /api/status reports 7b6a101, which contains 5addb7a. /healthz has a new boot_id, boot_01M4031RD5W1RT9B98G2MNF1EE.
  • The served index-DPU8Rmow.js holds both new rules, BLOCK_START and DELIMITER_START, verbatim.
  • Not checked on production: a live answer streaming in the console.

Left over

  • Known limit: in a table written without leading pipes, a body row is a paragraph until its first | arrives, and then joins the table, 13 px shorter.
  • The list merge this issue noted is fixed by #133. The other load-sensitive agent-view spec, "carries its reveal across settling", is #135.
Shipped in 5addb7a, deployed on 2026-10-03 at 01:16 (Actions run 96, manual, together with #132, #133 and #134). Run 95 had been cancelled so osfd would not restart during #124's timing measurement. Archived in 0285a3d as `openspec/changes/archive/2026-10-03-streaming-answer-never-shrinks`. This change's own session ended after the push, and the #133 session finished the deploy check and the archive. ### What changed - `repairStreaming` holds back a last line that is only the start of a block's syntax until the next characters decide what it is. That covers a list marker at any indent (a bullet, or a number with or without its `.` or `)`), a heading's `#`s, and one or two backticks or tildes. It also covers a line of `|`, `:`, `-` and spaces under a line with a pipe, which can still become a table's delimiter row. Inside an unclosed fence nothing is held. - A unit test walks every prefix of 16 answer shapes, drawn as `StreamingMarkdown` draws them, and requires the rows they take in the terminal register never to fall. - When the agent-view heights check fails, it now names the answer's text just before and just after the frame that got shorter. - `agent-transcript`'s "Streaming text is drawn in its settled form" now says nothing on screen is taken back while text is written, with three new scenarios. ### Tested - `agent-view.spec.ts:154` passed 16 of 16 runs under seven busy loops. None of their 7,589 frames ends in a bare list number, bullet or tilde, and none is shorter than the one before. On the old renderer, 23 of 33 runs drew such a frame, and 2 of them failed. - A walk over every prefix in Chromium, over 21 texts, found no drop in the spec's answer (1,299 prefixes) or in any of the unit test's shapes. The two drops left are the known limit below. - The fast lane passed in 104 s and the full lane in 152 s (2,363 backend, 524 vitest, 153 browser). ### Verified on production - `/api/status` reports 7b6a101, which contains 5addb7a. `/healthz` has a new boot_id, `boot_01M4031RD5W1RT9B98G2MNF1EE`. - The served `index-DPU8Rmow.js` holds both new rules, `BLOCK_START` and `DELIMITER_START`, verbatim. - Not checked on production: a live answer streaming in the console. ### Left over - Known limit: in a table written without leading pipes, a body row is a paragraph until its first `|` arrives, and then joins the table, 13 px shorter. - The list merge this issue noted is fixed by #133. The other load-sensitive agent-view spec, "carries its reveal across settling", is #135.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
archive/prefect-kestra-experiments
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cmoriarty/braid#130
No description provided.