Upstream: reuse connections instead of a fresh one per playlist poll and segment #121

Closed
opened 2026-11-08 15:39:09 -05:00 by cmoriarty · 1 comment
Owner

Follow-up to #118. With gluetun's DNS filter off (#120), the VPN finds and plays the same streams as the home connection. What it still costs is connection setup under bursts, and the app opens a new connection for every request it makes.

What happens

  • One connection per request: upstream._client() builds a new httpx.AsyncClient for every request (open_upstream, fetch_manifest, fetch_bytes), and closes it afterwards. So every playlist poll and every segment download pays a fresh TCP connection and TLS handshake, and no connection is ever reused.

  • Who makes those requests:

    • each channel's puller, a playlist poll every half segment length plus each segment;
    • the stream proxy (proxy.py);
    • verification and re-checks;
    • the watch's polls of up to 24 links (WATCH_LINKS_LIMIT), every 1–3 s.
  • What it costs through the tunnel (measured 2026-11-08 for #118):

    480 fresh TCP+TLS connections, 4 at a time to each of edgestream3.pro, hls.meth-streams-hd.cfd, hls.lshls.st and s1.saltext.ch Direct Tunnel
    Timed out (5 s) 0 10 (2.1%)
    p95 setup 250–470 ms 1.3–3.4 s
    • One connection at a time through the tunnel was clean: 239 of 240, p95 243–354 ms.
    • Bursts to nearby Cloudflare were clean too.
    • The loss shows under bursts of new connections to the far stream servers.
  • When it bites:

    • 2026-11-08 19:13–19:17Z: while a probe added about 13 streams to the same tunnel, several channels failed at once with "no new segment for 9–11 s; last error: playlist: upstream unreachable: ConnectTimeout", on two CDNs.
    • With the app alone it happened once in 20 minutes (19:18:51).
    • More warm channels, or a watched game plus background ones, push the app toward that load.
  • The handshake itself is slow: a TLS handshake to these servers takes 200–500 ms on either path, so even without loss each fresh connection adds a quarter to half a second to every fetch of a 2 s segment.

The idea

  1. One long-lived pooled client: keep-alive on, for example at most 50 kept connections that expire after 30 s.
    • Keep it separate for the TLS-lenient hosts (verify=False), so tls_lenient keeps working.
    • Pass each call's own timeout per request (MANIFEST_TIMEOUT, UPSTREAM_TIMEOUT).
  2. Callers stop closing the client: open_upstream's callers (proxy.py in three places, fetch_manifest, fetch_bytes) close only the response. A streaming response still has to be closed or read to the end, so its connection goes back to the pool.
  3. Tests still swap the transport: set_transport() rebuilds the shared clients, so the lane's scripted upstream still applies (15 tests use it).
  4. Retry a dropped connection once: a pooled connection the server has closed in the meantime should cost a retry, not a failed reload. Use the transport's connect retries (AsyncHTTPTransport(retries=1)) and test that path.
  5. Shut down cleanly: the app's shutdown closes the shared clients.

Done when

  • In the unit lane:
    • with the scripted upstream, ten playlist polls of one stream open one connection, not ten;
    • a burst of segment fetches to one host opens no more connections than the pool allows;
    • a TLS-lenient host still works;
    • the stream proxy still streams and returns its connection to the pool;
    • a pooled connection closed by the server is retried, not counted as a reload failure.
  • Live, behind the VPN:
    • the channels' "no new segment … ConnectTimeout" failures drop over a game window;
    • the watch's polls no longer show up as a burst of new connections.
Follow-up to #118. With gluetun's DNS filter off (#120), the VPN finds and plays the same streams as the home connection. What it still costs is connection setup under bursts, and the app opens a new connection for every request it makes. ## What happens - **One connection per request:** `upstream._client()` builds a new `httpx.AsyncClient` for every request (`open_upstream`, `fetch_manifest`, `fetch_bytes`), and closes it afterwards. So every playlist poll and every segment download pays a fresh TCP connection and TLS handshake, and no connection is ever reused. - **Who makes those requests:** - each channel's puller, a playlist poll every half segment length plus each segment; - the stream proxy (`proxy.py`); - verification and re-checks; - the watch's polls of up to 24 links (`WATCH_LINKS_LIMIT`), every 1–3 s. - **What it costs through the tunnel** (measured 2026-11-08 for #118): | 480 fresh TCP+TLS connections, 4 at a time to each of edgestream3.pro, hls.meth-streams-hd.cfd, hls.lshls.st and s1.saltext.ch | Direct | Tunnel | |---|---|---| | Timed out (5 s) | 0 | 10 (2.1%) | | p95 setup | 250–470 ms | 1.3–3.4 s | - One connection at a time through the tunnel was clean: 239 of 240, p95 243–354 ms. - Bursts to nearby Cloudflare were clean too. - The loss shows under bursts of new connections to the far stream servers. - **When it bites:** - 2026-11-08 19:13–19:17Z: while a probe added about 13 streams to the same tunnel, several channels failed at once with "no new segment for 9–11 s; last error: playlist: upstream unreachable: ConnectTimeout", on two CDNs. - With the app alone it happened once in 20 minutes (19:18:51). - More warm channels, or a watched game plus background ones, push the app toward that load. - **The handshake itself is slow:** a TLS handshake to these servers takes 200–500 ms on either path, so even without loss each fresh connection adds a quarter to half a second to every fetch of a 2 s segment. ## The idea 1. **One long-lived pooled client:** keep-alive on, for example at most 50 kept connections that expire after 30 s. - Keep it separate for the TLS-lenient hosts (`verify=False`), so `tls_lenient` keeps working. - Pass each call's own timeout per request (`MANIFEST_TIMEOUT`, `UPSTREAM_TIMEOUT`). 2. **Callers stop closing the client:** `open_upstream`'s callers (`proxy.py` in three places, `fetch_manifest`, `fetch_bytes`) close only the response. A streaming response still has to be closed or read to the end, so its connection goes back to the pool. 3. **Tests still swap the transport:** `set_transport()` rebuilds the shared clients, so the lane's scripted upstream still applies (15 tests use it). 4. **Retry a dropped connection once:** a pooled connection the server has closed in the meantime should cost a retry, not a failed reload. Use the transport's connect retries (`AsyncHTTPTransport(retries=1)`) and test that path. 5. **Shut down cleanly:** the app's shutdown closes the shared clients. ## Done when - **In the unit lane:** - with the scripted upstream, ten playlist polls of one stream open one connection, not ten; - a burst of segment fetches to one host opens no more connections than the pool allows; - a TLS-lenient host still works; - the stream proxy still streams and returns its connection to the pool; - a pooled connection closed by the server is retried, not counted as a reload failure. - **Live, behind the VPN:** - the channels' "no new segment … ConnectTimeout" failures drop over a game window; - the watch's polls no longer show up as a burst of new connections.
Author
Owner

Closing: merged to develop in PR #123 (merge f520e51), milestone 40. It deploys once nobody is watching.

What changed:

  • One pooled keep-alive client per loop and certificate mode.
  • Each request keeps its own timeout, and a failed connect is tried once more.
  • Callers close the response, never the pool, and the app closes the pool at shutdown.
  • On a loopback server: ten polls open one connection, a burst is reused, and a server that hangs up costs a new connection, not a failure.

Not seen live yet: fewer "no new segment … ConnectTimeout" failures over a game window behind the VPN.

Closing: merged to develop in PR #123 (merge f520e51), milestone 40. It deploys once nobody is watching. **What changed:** - One pooled keep-alive client per loop and certificate mode. - Each request keeps its own timeout, and a failed connect is tried once more. - Callers close the response, never the pool, and the app closes the pool at shutdown. - On a loopback server: ten polls open one connection, a burst is reused, and a server that hangs up costs a new connection, not a failure. **Not seen live yet:** fewer "no new segment … ConnectTimeout" failures over a game window behind the VPN.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/sports-breaker#121
No description provided.