A swap carries on from the same moment of the game #168

Closed
opened 2026-11-14 19:25:46 -05:00 by cmoriarty · 2 comments
Owner

What happens

When the channel changes stream, the new one starts three segments from its end, wherever that falls in the game (Timeline.append_upstream). The owner, 2026-11-14: "it's annoying when it goes back in time because you have to re-watch something, and confusing when it jumps forward because you missed something."

On recordings of two games from that afternoon, the same swap was built two ways:

Game As the app lands it today Cut on the same moment
North Florida vs. The Citadel, to a faster stream forward 7.3 s the sound 0.04 s apart, every second of the game clock shown once
Gonzaga vs. LSU, to a slower stream back 19.9 s, a basket taken off the board the sound 0.02 s apart, every second of the game clock shown once

Clips, frames and scripts: /Users/cmoriarty/Library/Application Support/Claude/side-session-notes/local_daf105a8-87c5-4403-af2c-b78517c66b67/stream-swap-research/. The owner's page: https://claude.ai/artifact/TCY5qaAe54rzrNnmjajbok.

This is step 2 of 3 in the milestone. It needs #167, and comes after #136, which edits the same file (backend/app/channel.py).

The idea

When the channel changes stream and knows where the new one is (#167), it ends the old stream and starts the new one on the same moment T.

  • T is one of the new stream's full pictures. One site has a keyframe every 2 s in 5 s segments, so every second segment starts 1 s before one. A join on such a segment's start holds the picture still for a second. The first test clip did exactly that, and today's swaps onto that site do too.
  • Both sides are cut on the packets, nothing re-encoded. The old stream keeps everything before T, the new one everything from T on (cut_tail and cut_head in the research scripts, pasted in a comment below). The join is a discontinuity as today, and the channel's clock runs on (tsclock).
  • Where T can be: inside what the channel holds of the old stream and has not listed to players (the head start), and inside what the new stream still keeps (12 to 24 s on the sites measured).
  • The same stream on another site: T comes from the timestamps. No listening.

What the viewer gets:

  • New stream ahead (leaving the slower one): no gap, as long as the new stream still keeps the moment. Every pair measured did: they ran 5 to 16 s ahead, and the closest call was 16 s ahead of a stream that keeps 18 s.
  • New stream behind: the channel drops what it holds past the new stream's end and carries on. What is left in hand is the head start less the gap. When that is less than the new stream needs (the watch grades each stream's need, 6 to 15 s), the channel repeats the difference: a few seconds, where today it is 20 to 34.
  • Out of reach (further ahead than it keeps, or further behind than is held): only the shortfall is skipped or repeated.
  • It never catches up. After a move to a faster stream the channel stays where it was, with more in hand.

When it may aim

The owner, 2026-11-14:

  • "it is fairly common for the streams to just be wrong, and this new feature will need to be robust to that scenario."
  • "sometimes the streams are from different markets and have different commercials, but they are showing the same game. We will need to be robust to this scenario as well."

So:

  • No relation, no aim. Another game, or another broadcast of this one, never matches (#167). The swap lands as today, and its record says "not aimed: no match". The same for Wrong game: there is no place to keep.
  • Checked at the join. The new stream's first segments, which the channel downloads to play anyway, are compared with the old stream's sound before the join is listed to players. A sample that matched in the last minute counts as the same check.
    • Found where expected: join there.
    • Found somewhere else: join there. The relation moved, and the fresh reading wins.
    • Not found (the two are in different commercials, or the new stream turned to something else): join on the relation only if it is trusted and neither stream has restarted since. The record says "aimed, not confirmed". Otherwise land as today.
  • Measured afterwards. After a swap that could not be confirmed, the comparison goes on while the stream that was left is still kept. Once both are back in the game, the record gets the seconds the picture really moved.
  • An off switch. A row on the Settings page, on by default, so aiming can be turned off without a deploy. Off means every swap lands as today.

Settings

  • Start behind live (playback.head_start, #117) is the room for meeting a slower stream: 15 s by default, up to 30 s. The owner, 2026-11-14: "The 15s/30s decision should be a setting, lets start with 15s by default." It is that setting already, so nothing new is built. Its help text gains one line: at 30 s, none of the pairs measured would repeat anything.

Data it needs

  • On each change-of-stream event: aimed (yes, unconfirmed or no, with the reason when not), moved_seconds as measured, and in_hand_seconds after the join.
  • The game page says the same on the Log line and the Link lane.

Done when

In the unit lane, with the generated streams of #167, reading the served playlist and segments the way a player does:

  • To a faster stream within reach: picture and sound run on within 0.1 s, and no picture is held or repeated at the join. Read every 0.2 s across it: frames one second apart hid a held picture in the research.
  • To a slower stream within what is held: the same.
  • Onto a stream whose segments start between full pictures: the join is on a full picture.
  • To the same stream on another site: carries on with nothing repeated (today 3 to 9 s).
  • Another programme as the new stream: lands as today, and the record says not aimed.
  • Different commercials, swap during the break: joins on the trusted relation, the record says unconfirmed, and after the break it shows the measured move.
  • The new stream restarted since the last match: lands as today unless the check at the join finds it.
  • Out of reach either way: only the shortfall moves, and the record says how much.
  • Media servers: a read with ffmpeg -f hls … -c copy -copyts across an aimed join has no timestamp going backwards and loses no frame. The web player (hls.js) plays through it.
  • With the switch off, every case lands as today.

Live: a Swap by hand between two matched streams records 0.0 s (within 0.1), and the owner sees no jump on the web player and on Emby.

Specs: stream-supervision, admin-page, event-log.

## What happens When the channel changes stream, the new one starts three segments from its end, wherever that falls in the game (`Timeline.append_upstream`). The owner, 2026-11-14: "it's annoying when it goes back in time because you have to re-watch something, and confusing when it jumps forward because you missed something." On recordings of two games from that afternoon, the same swap was built two ways: | Game | As the app lands it today | Cut on the same moment | | --- | --- | --- | | North Florida vs. The Citadel, to a faster stream | forward 7.3 s | the sound 0.04 s apart, every second of the game clock shown once | | Gonzaga vs. LSU, to a slower stream | back 19.9 s, a basket taken off the board | the sound 0.02 s apart, every second of the game clock shown once | Clips, frames and scripts: `/Users/cmoriarty/Library/Application Support/Claude/side-session-notes/local_daf105a8-87c5-4403-af2c-b78517c66b67/stream-swap-research/`. The owner's page: https://claude.ai/artifact/TCY5qaAe54rzrNnmjajbok. This is step 2 of 3 in the milestone. It needs #167, and comes after #136, which edits the same file (`backend/app/channel.py`). ## The idea When the channel changes stream and knows where the new one is (#167), it ends the old stream and starts the new one on the same moment T. - **T is one of the new stream's full pictures.** One site has a keyframe every 2 s in 5 s segments, so every second segment starts 1 s before one. A join on such a segment's start holds the picture still for a second. The first test clip did exactly that, and today's swaps onto that site do too. - **Both sides are cut on the packets,** nothing re-encoded. The old stream keeps everything before T, the new one everything from T on (`cut_tail` and `cut_head` in the research scripts, pasted in a comment below). The join is a discontinuity as today, and the channel's clock runs on (`tsclock`). - **Where T can be:** inside what the channel holds of the old stream and has not listed to players (the head start), and inside what the new stream still keeps (12 to 24 s on the sites measured). - **The same stream on another site:** T comes from the timestamps. No listening. What the viewer gets: - **New stream ahead** (leaving the slower one): no gap, as long as the new stream still keeps the moment. Every pair measured did: they ran 5 to 16 s ahead, and the closest call was 16 s ahead of a stream that keeps 18 s. - **New stream behind:** the channel drops what it holds past the new stream's end and carries on. What is left in hand is the head start less the gap. When that is less than the new stream needs (the watch grades each stream's need, 6 to 15 s), the channel repeats the difference: a few seconds, where today it is 20 to 34. - **Out of reach** (further ahead than it keeps, or further behind than is held): only the shortfall is skipped or repeated. - **It never catches up.** After a move to a faster stream the channel stays where it was, with more in hand. ## When it may aim The owner, 2026-11-14: - "it is fairly common for the streams to just be wrong, and this new feature will need to be robust to that scenario." - "sometimes the streams are from different markets and have different commercials, but they are showing the same game. We will need to be robust to this scenario as well." So: - **No relation, no aim.** Another game, or another broadcast of this one, never matches (#167). The swap lands as today, and its record says "not aimed: no match". The same for Wrong game: there is no place to keep. - **Checked at the join.** The new stream's first segments, which the channel downloads to play anyway, are compared with the old stream's sound before the join is listed to players. A sample that matched in the last minute counts as the same check. - Found where expected: join there. - Found somewhere else: join there. The relation moved, and the fresh reading wins. - Not found (the two are in different commercials, or the new stream turned to something else): join on the relation only if it is trusted and neither stream has restarted since. The record says "aimed, not confirmed". Otherwise land as today. - **Measured afterwards.** After a swap that could not be confirmed, the comparison goes on while the stream that was left is still kept. Once both are back in the game, the record gets the seconds the picture really moved. - **An off switch.** A row on the Settings page, on by default, so aiming can be turned off without a deploy. Off means every swap lands as today. ## Settings - **Start behind live** (`playback.head_start`, #117) is the room for meeting a slower stream: 15 s by default, up to 30 s. The owner, 2026-11-14: "The 15s/30s decision should be a setting, lets start with 15s by default." It is that setting already, so nothing new is built. Its help text gains one line: at 30 s, none of the pairs measured would repeat anything. ## Data it needs - On each change-of-stream event: `aimed` (`yes`, `unconfirmed` or `no`, with the reason when not), `moved_seconds` as measured, and `in_hand_seconds` after the join. - The game page says the same on the Log line and the Link lane. ## Done when In the unit lane, with the generated streams of #167, reading the served playlist and segments the way a player does: - **To a faster stream within reach:** picture and sound run on within 0.1 s, and no picture is held or repeated at the join. Read every 0.2 s across it: frames one second apart hid a held picture in the research. - **To a slower stream within what is held:** the same. - **Onto a stream whose segments start between full pictures:** the join is on a full picture. - **To the same stream on another site:** carries on with nothing repeated (today 3 to 9 s). - **Another programme as the new stream:** lands as today, and the record says not aimed. - **Different commercials, swap during the break:** joins on the trusted relation, the record says unconfirmed, and after the break it shows the measured move. - **The new stream restarted since the last match:** lands as today unless the check at the join finds it. - **Out of reach either way:** only the shortfall moves, and the record says how much. - **Media servers:** a read with `ffmpeg -f hls … -c copy -copyts` across an aimed join has no timestamp going backwards and loses no frame. The web player (hls.js) plays through it. - **With the switch off,** every case lands as today. Live: a Swap by hand between two matched streams records 0.0 s (within 0.1), and the owner sees no jump on the web player and on Emby. Specs: `stream-supervision`, `admin-page`, `event-log`.
Author
Owner

Reference from the research, not production code: the two cuts as the scripts did them (splice_demo.py and splice_demo3.py in the notes folder). They are here so the method does not depend on a scratch folder.

PACKET = 188

def pes_pts(data, off):
    """The PTS of the PES packet starting in this TS packet, or None."""
    if not (data[off + 1] & 0x40) or not (data[off + 3] & 0x10):
        return None
    p = off + 4
    if data[off + 3] & 0x20:
        p += 1 + data[off + 4]
    if p + 14 > off + PACKET or data[p:p + 3] != b"\x00\x00\x01":
        return None
    if not (data[p + 7] & 0x80):
        return None
    q = p + 9
    return (((data[q] >> 1) & 0x07) << 30 | data[q + 1] << 22 | ((data[q + 2] >> 1) & 0x7F) << 15
            | data[q + 3] << 7 | (data[q + 4] >> 1) & 0x7F)

def cut_tail(data, cut_pts):
    """Everything from cut_pts on dropped, per stream: each elementary stream keeps its
    packets up to its first PES packet at or past the cut. Tables and everything before
    stay byte for byte, so packet ids and counters run on."""
    out = bytearray(); stopped = set()
    for off in range(0, len(data) - PACKET + 1, PACKET):
        if data[off] != 0x47:
            continue
        pid = ((data[off + 1] & 0x1F) << 8) | data[off + 2]
        if pid in stopped:
            continue
        pts = pes_pts(data, off)
        if pts is not None and pts >= cut_pts:
            stopped.add(pid)
            continue
        out += data[off:off + PACKET]
    return bytes(out)

def cut_head(data, cut_pts):
    """Everything before cut_pts dropped, per stream: each elementary stream starts at
    its first PES packet at or past the cut. Packets that carry no PES stream (the
    tables) are kept."""
    pes_pids = {((data[o + 1] & 0x1F) << 8) | data[o + 2]
                for o in range(0, len(data) - PACKET + 1, PACKET)
                if data[o] == 0x47 and pes_pts(data, o) is not None}
    out = bytearray(); started = set()
    for off in range(0, len(data) - PACKET + 1, PACKET):
        if data[off] != 0x47:
            continue
        pid = ((data[off + 1] & 0x1F) << 8) | data[off + 2]
        if pid in pes_pids and pid not in started:
            pts = pes_pts(data, off)
            if pts is None or pts < cut_pts:
                continue
            started.add(pid)
        out += data[off:off + PACKET]
    return bytes(out)

What the research saw with them:

  • Tail cut: the old stream's last segment ended 0.013 to 0.019 s past T in video and 0.019 to 0.049 s in sound (one frame of each). A playlist with a tail-cut segment in it, read with ffmpeg -fflags +igndts+genpts -f hls -i … -c copy -copyts -start_at_zero, came out whole: 25.36 s of 25.36 s, the largest video step 0.08 s, no timestamp going backwards.
  • Head cut at a keyframe: the new stream's first segment started exactly on T in video and 0.007 to 0.012 s after it in sound, and decoded with no error. cut_pts has to be the keyframe's own timestamp. In decode order every picture before a keyframe is earlier than it, so the first video packet at or past T is the keyframe.
  • Not tried: a head-cut segment inside an HLS playlist read by ffmpeg or hls.js. Only the tail cut was read that way. The head cut was decoded as one stream with the segments after it.
  • Re-muxing a cut segment with ffmpeg changes its packet ids, and an HLS reader then drops it. The cut has to be on the packets.
  • Testing a join: read each side's segments as one stream, the way a player does. Decoding segments one by one loses the start of every segment that begins between two keyframes. Then read the game clock, or any counter in the picture, every 0.2 s across the join: frames one second apart hid a picture that held still for 0.8 s.
  • Keyframes seen 2026-11-14: one site every 2.0 s in 5 s segments (so every second segment starts 1.0 s before one), another one per 3 s segment at its start, a third every 0.35 s.
Reference from the research, not production code: the two cuts as the scripts did them (`splice_demo.py` and `splice_demo3.py` in the notes folder). They are here so the method does not depend on a scratch folder. ```python PACKET = 188 def pes_pts(data, off): """The PTS of the PES packet starting in this TS packet, or None.""" if not (data[off + 1] & 0x40) or not (data[off + 3] & 0x10): return None p = off + 4 if data[off + 3] & 0x20: p += 1 + data[off + 4] if p + 14 > off + PACKET or data[p:p + 3] != b"\x00\x00\x01": return None if not (data[p + 7] & 0x80): return None q = p + 9 return (((data[q] >> 1) & 0x07) << 30 | data[q + 1] << 22 | ((data[q + 2] >> 1) & 0x7F) << 15 | data[q + 3] << 7 | (data[q + 4] >> 1) & 0x7F) def cut_tail(data, cut_pts): """Everything from cut_pts on dropped, per stream: each elementary stream keeps its packets up to its first PES packet at or past the cut. Tables and everything before stay byte for byte, so packet ids and counters run on.""" out = bytearray(); stopped = set() for off in range(0, len(data) - PACKET + 1, PACKET): if data[off] != 0x47: continue pid = ((data[off + 1] & 0x1F) << 8) | data[off + 2] if pid in stopped: continue pts = pes_pts(data, off) if pts is not None and pts >= cut_pts: stopped.add(pid) continue out += data[off:off + PACKET] return bytes(out) def cut_head(data, cut_pts): """Everything before cut_pts dropped, per stream: each elementary stream starts at its first PES packet at or past the cut. Packets that carry no PES stream (the tables) are kept.""" pes_pids = {((data[o + 1] & 0x1F) << 8) | data[o + 2] for o in range(0, len(data) - PACKET + 1, PACKET) if data[o] == 0x47 and pes_pts(data, o) is not None} out = bytearray(); started = set() for off in range(0, len(data) - PACKET + 1, PACKET): if data[off] != 0x47: continue pid = ((data[off + 1] & 0x1F) << 8) | data[off + 2] if pid in pes_pids and pid not in started: pts = pes_pts(data, off) if pts is None or pts < cut_pts: continue started.add(pid) out += data[off:off + PACKET] return bytes(out) ``` What the research saw with them: - **Tail cut:** the old stream's last segment ended 0.013 to 0.019 s past T in video and 0.019 to 0.049 s in sound (one frame of each). A playlist with a tail-cut segment in it, read with `ffmpeg -fflags +igndts+genpts -f hls -i … -c copy -copyts -start_at_zero`, came out whole: 25.36 s of 25.36 s, the largest video step 0.08 s, no timestamp going backwards. - **Head cut at a keyframe:** the new stream's first segment started exactly on T in video and 0.007 to 0.012 s after it in sound, and decoded with no error. `cut_pts` has to be the keyframe's own timestamp. In decode order every picture before a keyframe is earlier than it, so the first video packet at or past T is the keyframe. - **Not tried:** a head-cut segment inside an HLS playlist read by ffmpeg or hls.js. Only the tail cut was read that way. The head cut was decoded as one stream with the segments after it. - **Re-muxing a cut segment with ffmpeg changes its packet ids,** and an HLS reader then drops it. The cut has to be on the packets. - **Testing a join:** read each side's segments as one stream, the way a player does. Decoding segments one by one loses the start of every segment that begins between two keyframes. Then read the game clock, or any counter in the picture, every 0.2 s across the join: frames one second apart hid a picture that held still for 0.8 s. - **Keyframes seen 2026-11-14:** one site every 2.0 s in 5 s segments (so every second segment starts 1.0 s before one), another one per 3 s segment at its start, a third every 0.35 s.
Author
Owner

Merged in pull request #172 as 584162b. The live checks (a swap by hand recording 0.0 s, no jump on the web player or Emby) are next.

Merged in pull request #172 as 584162b. The live checks (a swap by hand recording 0.0 s, no jump on the web player or Emby) are next.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/sports-breaker#168
No description provided.