The Playing card counts swaps but can't show them: what each one dropped, why, which stream it landed on, and what it will try next #90

Closed
opened 2026-11-05 22:14:53 -05:00 by cmoriarty · 2 comments
Owner

What happens

The Playing card counts a channel's swaps but can't show them. On Ravens vs. Jaguars (2026-11-05) the card said 11 swaps. Its recent list showed only the last four events, and none of them said why the old link was dropped or which stream the new one was.

To find out what the 11 were, I had to put together the event log, the container log and the splice lines' source timestamps. Each stream has its own clock, so the same clock value behind two sites means they carry the same stream.

The 11 swaps, tuned 9:49 PM ET on buff-streams.buzz:

# Time (ET) How Landed on Stream Picture Previous link lasted
1 9:51:02 recovery givemestreams.cfd A 1024×576, 25 fps 1 m 44 s
2 9:52:12 recovery shd247.world D 640×360, 30 fps 1 m 10 s
3 9:53:10 recovery, after 20 s of filler shd247.world (the same link) D 640×360, 30 fps 58 s
4 9:53:28 degrading swap nfl-castnow.lat A 1024×576, 25 fps 18 s
5 9:56:50 recovery s2watch.cyou B 1280×720, 59.94 fps 3 m 22 s
6 10:01:31 recovery sportsframe.site A 1024×576, 25 fps 4 m 41 s
7 10:01:56 recovery fast-streams.biz B 1280×720, 59.94 fps 25 s
8 10:06:16 recovery topsurge.st C 1152×648, 59.94 fps 4 m 20 s
9 10:07:54 degrading swap, after 2 filler splices buff-streams.buzz C (probably; it left after 9 s, before a splice was logged) 1152×648, 59.94 fps 1 m 38 s
10 10:08:03 recovery givemestreams.cfd A 1024×576, 25 fps 9 s
11 10:09:16 recovery, after 4 s of filler givemestreams.cfd (the same link) A 1024×576, 25 fps 1 m 13 s

The 11 swaps used 9 sites, but behind them were only 4 streams. Stream A (now hls.meth-streams-hd.cfd) was landed on five times, through three sites. buff-streams.buzz and topsurge.st carry the same stream: their clocks matched to 0.1 s at an earlier swap (9:37 PM). From the card, the viewer saw a long list of different sites, but the channel was really cycling through the same few streams.

What the app knows but doesn't keep:

  • Why a link was dropped. The channel computes it (Health.failed_by: no new segment, playlist reload failures, segment errors, refused segments; degrading_by: stalling, behind, frames, late). It is shown only on the live card, while it is true, and then it is lost:
    • The swapped/recovered events don't record it.
    • Nor does the container log. Its lines are only channel <id>: recovered on <player page> and channel <id>: swapped a degrading upstream for the standby. Nothing is logged when the failure is detected (_start_recovery and _maybe_swap are called silently). The last upstream error (last_error, e.g. "playlist: upstream unreachable: ConnectTimeout") is never logged either.
    • So after the fact, neither log can say why any of the 11 links was dropped. The only clue is the filler splices (clock: seq N splices (… filler)) before some of them.
  • Which stream a link carries. stream_identity already exists (#74), but it is used only to choose standbys.
  • The count mixes two things. "11 swaps" was 2 degrading swaps and 9 recoveries (Channel._adopt counts both).

What it costs

When a game keeps swapping, the card can't answer the questions that decide what to do:

  • Is one bad stream being retried, or are all of them bad?
  • Is the trouble upstream, or is it the app?
  • Would a manual Swap help?
  • Is the swap logic doing what it should: is the failed link set aside, and what will it try next?

The idea

  1. The swap count opens the list. Clicking it shows every swap since the tune, newest first. Each row has the time, how it happened (recovery, degrading swap, standby, re-read, ladder), the site it landed on, a chip for the stream, the picture, and how long the previous link lasted.

  2. Why it left. Each row shows the reason in a few words, such as "no new segment for 12 s", "3 segment errors", "stalling" or "behind". The longer explanation sits behind an info icon.

  3. Show the streams. Each distinct stream behind the links gets a small chip (a letter or a colour) on each row and on the card's current link and standby. That way a repeat is obvious at a glance. When the channel lands on a stream that failed in the last few minutes, the row is marked.

  4. Split the count. Show recoveries and degrading swaps separately, for example "9 + 2", with the meaning in a tooltip.

  5. The link list, live, to watch the logic work. The card shows the game's links in the order the app will try them (Ladder.order), updating as things happen:

    • At the top, the links in use: playing, standby and backup.
    • Then the links that can be tried, in their order. A link on a host with a bad record, or one that keeps finding nothing, sinks below the rest of its tier and is marked so.
    • At the bottom, the cooled links, greyed out, each with a countdown until it comes back ("back in 1:13"). Hosts backed off after a challenge are marked as well, with their own countdown.
    • On every row, the stream chip from item 3, so copies of a failed stream stand out.

    A failure should be visible as it happens: the failed link drops to the bottom with a 10:00 countdown, the standby moves up to playing, the backup moves up to standby, and the next standby read takes the first link that can be tried. Tonight's list had 28 links. At 10:16 PM topsurge.st and buff-streams.buzz were cooling with about a minute left. Nine other sites had failed earlier in the game and were back in the order. The panel showed none of this; it only says "N of M links cooling" in the problems list. When #91 lands, a failed stream's other sites sink here too, and that can be seen.

  6. Backend, the event log: the swapped and recovered events record:

    • the reason the old link was dropped;
    • the old and new link's stream identity and stream host;
    • the picture.

    They should also stay readable after the game, so the History section can show the same rows.

  7. Backend, the list. The admin view carries the game's ladder in order. Each link has:

    • its host and tier;
    • its slot (playing, standby or backup);
    • whether it is cooling, and the seconds left;
    • whether it is sunk, empty, backed off or rejected;
    • the stream it last resolved to (Ladder.streams).

    The ladder is per game and source. When there is more than one source, the list takes the one playing.

  8. Backend, the container log. Each swap can be explained from docker logs alone, at INFO:

    • When the failure is detected, before the recovery starts, one line names the link, its stream host, the rule it tripped with its figure, and the last upstream error. For example: channel nfl-401873037: givemestreams.cfd (hls.meth-streams-hd.cfd) failed: no new segment for 12 s; last error: playlist ConnectTimeout. A degrading swap's line names its degrading_by reason in the same way (stalling, behind, frames, late).
    • When the channel lands, the existing recovered on and swapped lines add how it landed (standby, re-read, ladder), the new stream host and stream identity, and the picture.
    • Each recovery step that is tried and fails (standby not ready, re-read failed) gets a line too, so a long recovery shows where its time went.

Defaults (per the front-end preferences):

  • One screen, minimal text, explanations behind icons, and no new section.
  • The swap list sits under the card's existing recent-events disclosure and replaces its four-item cap for the current tune.
  • The link list sits behind its own disclosure on the card. Closed, it shows a one-line summary: the next link to try and how many are cooling.
  • Stream letters are assigned per tune, in the order the streams first appear.

Not part of this: changing what the recovery picks. Here, the recovery went back to stream A after it had just failed. That belongs in its own issue.

## What happens The Playing card counts a channel's swaps but can't show them. On Ravens vs. Jaguars (2026-11-05) the card said **11 swaps**. Its recent list showed only the last four events, and none of them said why the old link was dropped or which stream the new one was. To find out what the 11 were, I had to put together the event log, the container log and the splice lines' source timestamps. Each stream has its own clock, so the same clock value behind two sites means they carry the same stream. **The 11 swaps, tuned 9:49 PM ET on buff-streams.buzz:** | # | Time (ET) | How | Landed on | Stream | Picture | Previous link lasted | |---|---|---|---|---|---|---| | 1 | 9:51:02 | recovery | givemestreams.cfd | A | 1024×576, 25 fps | 1 m 44 s | | 2 | 9:52:12 | recovery | shd247.world | D | 640×360, 30 fps | 1 m 10 s | | 3 | 9:53:10 | recovery, after 20 s of filler | shd247.world (the same link) | D | 640×360, 30 fps | 58 s | | 4 | 9:53:28 | degrading swap | nfl-castnow.lat | A | 1024×576, 25 fps | 18 s | | 5 | 9:56:50 | recovery | s2watch.cyou | B | 1280×720, 59.94 fps | 3 m 22 s | | 6 | 10:01:31 | recovery | sportsframe.site | A | 1024×576, 25 fps | 4 m 41 s | | 7 | 10:01:56 | recovery | fast-streams.biz | B | 1280×720, 59.94 fps | 25 s | | 8 | 10:06:16 | recovery | topsurge.st | C | 1152×648, 59.94 fps | 4 m 20 s | | 9 | 10:07:54 | degrading swap, after 2 filler splices | buff-streams.buzz | C (probably; it left after 9 s, before a splice was logged) | 1152×648, 59.94 fps | 1 m 38 s | | 10 | 10:08:03 | recovery | givemestreams.cfd | A | 1024×576, 25 fps | 9 s | | 11 | 10:09:16 | recovery, after 4 s of filler | givemestreams.cfd (the same link) | A | 1024×576, 25 fps | 1 m 13 s | The 11 swaps used 9 sites, but behind them were only **4 streams**. Stream A (now `hls.meth-streams-hd.cfd`) was landed on five times, through three sites. buff-streams.buzz and topsurge.st carry the same stream: their clocks matched to 0.1 s at an earlier swap (9:37 PM). From the card, the viewer saw a long list of different sites, but the channel was really cycling through the same few streams. **What the app knows but doesn't keep:** - **Why a link was dropped.** The channel computes it (`Health.failed_by`: no new segment, playlist reload failures, segment errors, refused segments; `degrading_by`: stalling, behind, frames, late). It is shown only on the live card, while it is true, and then it is lost: - The `swapped`/`recovered` events don't record it. - **Nor does the container log.** Its lines are only `channel <id>: recovered on <player page>` and `channel <id>: swapped a degrading upstream for the standby`. Nothing is logged when the failure is detected (`_start_recovery` and `_maybe_swap` are called silently). The last upstream error (`last_error`, e.g. "playlist: upstream unreachable: ConnectTimeout") is never logged either. - So after the fact, neither log can say why any of the 11 links was dropped. The only clue is the filler splices (`clock: seq N splices (… filler)`) before some of them. - **Which stream a link carries.** `stream_identity` already exists (#74), but it is used only to choose standbys. - **The count mixes two things.** "11 swaps" was 2 degrading swaps and 9 recoveries (`Channel._adopt` counts both). ## What it costs When a game keeps swapping, the card can't answer the questions that decide what to do: - Is one bad stream being retried, or are all of them bad? - Is the trouble upstream, or is it the app? - Would a manual Swap help? - Is the swap logic doing what it should: is the failed link set aside, and what will it try next? ## The idea 1. **The swap count opens the list.** Clicking it shows every swap since the tune, newest first. Each row has the time, how it happened (recovery, degrading swap, standby, re-read, ladder), the site it landed on, a chip for the stream, the picture, and how long the previous link lasted. 2. **Why it left.** Each row shows the reason in a few words, such as "no new segment for 12 s", "3 segment errors", "stalling" or "behind". The longer explanation sits behind an info icon. 3. **Show the streams.** Each distinct stream behind the links gets a small chip (a letter or a colour) on each row and on the card's current link and standby. That way a repeat is obvious at a glance. When the channel lands on a stream that failed in the last few minutes, the row is marked. 4. **Split the count.** Show recoveries and degrading swaps separately, for example "9 + 2", with the meaning in a tooltip. 5. **The link list, live, to watch the logic work.** The card shows the game's links in the order the app will try them (`Ladder.order`), updating as things happen: - **At the top,** the links in use: playing, standby and backup. - **Then the links that can be tried,** in their order. A link on a host with a bad record, or one that keeps finding nothing, sinks below the rest of its tier and is marked so. - **At the bottom,** the cooled links, greyed out, each with a countdown until it comes back ("back in 1:13"). Hosts backed off after a challenge are marked as well, with their own countdown. - **On every row,** the stream chip from item 3, so copies of a failed stream stand out. A failure should be visible as it happens: the failed link drops to the bottom with a 10:00 countdown, the standby moves up to playing, the backup moves up to standby, and the next standby read takes the first link that can be tried. Tonight's list had 28 links. At 10:16 PM topsurge.st and buff-streams.buzz were cooling with about a minute left. Nine other sites had failed earlier in the game and were back in the order. The panel showed none of this; it only says "N of M links cooling" in the problems list. When #91 lands, a failed stream's other sites sink here too, and that can be seen. 6. **Backend, the event log:** the `swapped` and `recovered` events record: - the reason the old link was dropped; - the old and new link's stream identity and stream host; - the picture. They should also stay readable after the game, so the History section can show the same rows. 7. **Backend, the list.** The admin view carries the game's ladder in order. Each link has: - its host and tier; - its slot (playing, standby or backup); - whether it is cooling, and the seconds left; - whether it is sunk, empty, backed off or rejected; - the stream it last resolved to (`Ladder.streams`). The ladder is per game and source. When there is more than one source, the list takes the one playing. 8. **Backend, the container log.** Each swap can be explained from `docker logs` alone, at INFO: - **When the failure is detected,** before the recovery starts, one line names the link, its stream host, the rule it tripped with its figure, and the last upstream error. For example: `channel nfl-401873037: givemestreams.cfd (hls.meth-streams-hd.cfd) failed: no new segment for 12 s; last error: playlist ConnectTimeout`. A degrading swap's line names its `degrading_by` reason in the same way (stalling, behind, frames, late). - **When the channel lands,** the existing `recovered on` and `swapped` lines add how it landed (standby, re-read, ladder), the new stream host and stream identity, and the picture. - **Each recovery step that is tried and fails** (standby not ready, re-read failed) gets a line too, so a long recovery shows where its time went. **Defaults** (per the front-end preferences): - One screen, minimal text, explanations behind icons, and no new section. - The swap list sits under the card's existing recent-events disclosure and replaces its four-item cap for the current tune. - The link list sits behind its own disclosure on the card. Closed, it shows a one-line summary: the next link to try and how many are cooling. - Stream letters are assigned per tune, in the order the streams first appear. **Not part of this:** changing what the recovery picks. Here, the recovery went back to stream A after it had just failed. That belongs in its own issue.
cmoriarty changed title from The Playing card counts swaps but can't show them: what each one dropped, why, and which stream it landed on to The Playing card counts swaps but can't show them: what each one dropped, why, which stream it landed on, and what it will try next 2026-11-05 22:21:49 -05:00
Author
Owner

Merged in PR #112 (46a39c3) with the Playing card (#105), deployed 2026-11-08 08:21Z (container 0b677e66cda0) while nobody watched; the change is archived in dfbe572.

What now records each swap:

  • Why it left. When a stream fails or a degrading swap is decided, the channel keeps the rule with its figure and the last upstream error: no new segment for 12 s; last error: playlist: …, 3 segment errors in a minute, 2 segments refused in a row (403), the player ran dry 2 times in 2 min / got 0.8× real time, fell behind its source, 3 of the last 5 segments short of frames, 3 of the last 5 segments came late. Swap now, Go back, Wrong game and the background trade are causes too.
  • The events. swapped and recovered carry cause, why, how (standby, reread, ladder, degrading, operator, back, trade, wrong_game), from_stream / to_stream (the stream mark), from_stream_host / to_stream_host, and quality.
  • The container log, at INFO:
    • channel <id>: givemestreams.cfd (hls.meth-streams-hd.cfd) failed: no new segment for 12 s; last error: … when the failure is detected, before the recovery starts;
    • channel <id>: p1 (stream host) degrading: 3 of the last 5 segments came late; swapping to the standby p2;
    • channel <id>: recovered on <url> via the standby, stream <mark> on <stream host>, 1280×720, 59.94 fps;
    • channel <id>: the standby was not ready, channel <id>: re-reading <host> found nothing (<reason>).
  • The card: the runs since the tune on the Tonight timeline (each run's site, stream letter, times and why it ended on hover), the count split as recoveries + other swaps, stream letters per tune in the order streams appear, and every link of the game in the order the app tries it, with its last outcome and its cooling countdown (#105).

Not part of this, as the issue said: what the recovery picks.

Not seen live yet. Left open for the live check: the next recovery in production logs a failed: line and a landing line with via …, stream ….

Merged in PR #112 (46a39c3) with the Playing card (#105), deployed 2026-11-08 08:21Z (container 0b677e66cda0) while nobody watched; the change is archived in dfbe572. What now records each swap: - **Why it left.** When a stream fails or a degrading swap is decided, the channel keeps the rule with its figure and the last upstream error: `no new segment for 12 s; last error: playlist: …`, `3 segment errors in a minute`, `2 segments refused in a row (403)`, `the player ran dry 2 times in 2 min` / `got 0.8× real time`, `fell behind its source`, `3 of the last 5 segments short of frames`, `3 of the last 5 segments came late`. Swap now, Go back, Wrong game and the background trade are causes too. - **The events.** `swapped` and `recovered` carry `cause`, `why`, `how` (standby, reread, ladder, degrading, operator, back, trade, wrong_game), `from_stream` / `to_stream` (the stream mark), `from_stream_host` / `to_stream_host`, and `quality`. - **The container log**, at INFO: - `channel <id>: givemestreams.cfd (hls.meth-streams-hd.cfd) failed: no new segment for 12 s; last error: …` when the failure is detected, before the recovery starts; - `channel <id>: p1 (stream host) degrading: 3 of the last 5 segments came late; swapping to the standby p2`; - `channel <id>: recovered on <url> via the standby, stream <mark> on <stream host>, 1280×720, 59.94 fps`; - `channel <id>: the standby was not ready`, `channel <id>: re-reading <host> found nothing (<reason>)`. - **The card:** the runs since the tune on the Tonight timeline (each run's site, stream letter, times and why it ended on hover), the count split as recoveries + other swaps, stream letters per tune in the order streams appear, and every link of the game in the order the app tries it, with its last outcome and its cooling countdown (#105). Not part of this, as the issue said: what the recovery picks. Not seen live yet. Left open for the live check: the next recovery in production logs a `failed:` line and a landing line with `via …, stream …`.
Author
Owner

Closed at the owner's word (2026-11-08): shipped in PR #112, deployed 08:21Z, archived in dfbe572. Reopen if the live check on a watched game shows otherwise.

Closed at the owner's word (2026-11-08): shipped in PR #112, deployed 08:21Z, archived in dfbe572. Reopen if the live check on a watched game shows otherwise.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/sports-breaker#90
No description provided.