Deleting many runs times out at 60 s and stops halfway #542

Closed
opened 2026-11-15 12:25:57 -05:00 by cmoriarty · 1 comment
Owner

What happens

Deleting 20 runs from the ledger ends with "osfd did not answer in time … close other Braid tabs". The tabs are not the cause.

POST /api/runs/delete deletes the runs one after another inside one request. Each purge is a slow write (1 s for 2k events, 23 s for 15k), so 20 runs take minutes, and the console gives up at 60 s. On production on 2026-11-15, 10 runs went in 65 s, then the deleting stopped with the dropped request. The dialog's "it may still be deleting" was wrong: the other 10 stayed until the delete was sent again.

Fix

Delete in the background: accept the request at once and let the dialog count the runs off as their osf.run.deleted tombstones arrive, so a long selection finishes whether or not the tab waits. Make the timeout message name the slow request, not the tabs.

Done when

A test deletes more runs than fit in the client's wait, and every one goes, with the dialog showing progress to the end.

## What happens Deleting 20 runs from the ledger ends with "osfd did not answer in time … close other Braid tabs". The tabs are not the cause. `POST /api/runs/delete` deletes the runs one after another inside one request. Each purge is a slow write (1 s for 2k events, 23 s for 15k), so 20 runs take minutes, and the console gives up at 60 s. On production on 2026-11-15, 10 runs went in 65 s, then the deleting stopped with the dropped request. The dialog's "it may still be deleting" was wrong: the other 10 stayed until the delete was sent again. ## Fix Delete in the background: accept the request at once and let the dialog count the runs off as their `osf.run.deleted` tombstones arrive, so a long selection finishes whether or not the tab waits. Make the timeout message name the slow request, not the tabs. ## Done when A test deletes more runs than fit in the client's wait, and every one goes, with the dialog showing progress to the end.
Author
Owner

POST /api/runs/delete answers 202 at once and deletes in a background job, one job at a time, and GET /api/runs/delete/{id} gives its progress and, once it ends, its results. The dialog sends the whole selection and follows the job to the end. A request osfd does not answer in time now names what was asked and no longer blames other tabs.

Evidence: a backend test where the request answers before any run is purged and the job counts them off (it fails on the old code), and a browser journey that deletes 12 runs to the end.

Shipped in pull request 550 (merge 3bed64d), change console-fixes-and-session-cap, archived in pull request 551.

`POST /api/runs/delete` answers 202 at once and deletes in a background job, one job at a time, and `GET /api/runs/delete/{id}` gives its progress and, once it ends, its results. The dialog sends the whole selection and follows the job to the end. A request osfd does not answer in time now names what was asked and no longer blames other tabs. Evidence: a backend test where the request answers before any run is purged and the job counts them off (it fails on the old code), and a browser journey that deletes 12 runs to the end. Shipped in pull request 550 (merge `3bed64d`), change `console-fixes-and-session-cap`, archived in pull request 551.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/braid#542
No description provided.