Sites look banned when they are not: the app browser's own challenge backs them off, and a stuck solver challenge reads as an outage #98

Closed
opened 2026-11-07 20:05:28 -05:00 by cmoriarty · 2 comments
Owner

Checked 2026-11-07 (8 h of container logs, 3.5 days of /api/events, a live probe at 00:50 UTC) whether viewing problems come from sites or CDNs throttling or banning the app. They don't. There is no ban page anywhere in the solver log. The red rows on the Admin Sites table are the app's own doing, and two bugs make them look like bans.

What happens

1. The app's own browser's challenge backs good sites off.

  • _read_player_page opens a player page in the app's browser first. On the Cloudflare-gated sites (dudestream1.com, streameast.ga, tvsportslive.fr, mygoodstream.pw, iohit.st) Cloudflare challenges that browser every time, and read_stream_from_page records the challenge (with its 403) against the host.
  • The solver then reads the same page fine (the after_challenge hand-off from PR #82): dudestream1.com 11 of 12, streameast.ga 11 of 11, tvsportslive.fr 20 of 23. All 54 backoffs that day were followed within 3 s by a solver read of the same host.
  • When the page holds no stream the app can read, the backoff stays and doubles, from 60 s up to the 1800 s cap. dudestream1.com reached 12 consecutive failures and streameast.ga 11. From 18:54 UTC on, each try at dudestream1.com earned another 30 min. tvsportslive.fr gets locked out too, though it does deliver streams: Ole Miss vs. Georgia recovered on it at 23:01 and 23:42 UTC.
  • The backoff also refuses the next step of the same read. Same-host player frames were refused 14 times with "source host backed off" (mygoodstream.pw ×6, tvsportslive.fr ×8).
  • The Sites table shows "403", "Cloudflare checks" and "Failed 12", which reads as a ban.

2. A challenge the solver can't clear reads as a solver outage.

  • SOLVE_TIMEOUT_SECONDS (90 s) equals the maxTimeout the app sends the solver (90 000 ms). The solver sends its 500 ("Error solving the challenge. Timeout after 90.0 seconds") at 90.1 s. httpx has given up about 50 ms earlier, so _fetch_clearance raises SolverUnreachable.
  • 16:27–16:59 UTC, right after the 16:26 restart: 22 hub reads in a row (v2.sportsurge.net) got "Just a moment..." and FlareSolverr couldn't clear it. Each time the app logged "solver ... is not answering", kept the stuck session and recorded nothing against the host. It cleared on its own at 16:59.
  • For those 33 min the app couldn't read the hub, so it found no new games and read no match pages. It ended right at the noon ET kickoffs.

What it costs

  • Working sites are skipped for up to 30 min at a time, and their frames are refused.
  • A stuck hub after a restart doesn't show on the Admin page. The app retries the same stuck browser every 90 s.
  • The Sites table points at bans that aren't there.

The idea

  1. Count a read by its outcome, not its first step. When a page is handed to the solver, the app browser's challenge is not a failure on its own. Record the read once, after the solver and any frames have tried. A page the solver reads that holds no stream is "no stream found" and does not back the host off; the link's own cooldown (Ladder.fail, 10 min) already keeps it from being re-read. A challenge the solver can't clear still counts.
  2. Frames of the same read skip the backoff that read set, the way past_backoff lets the solver step through.
  3. Give the solver call a margin over maxTimeout, for example 100 s against 90 s, so the solver's own error arrives. A session that fails to clear a challenge is then discarded, the next read starts a fresh browser, and the challenge counts against the host and shows on the Sites table.
  4. Go straight to the solver on a host whose pages always challenge the app's browser. This halves the reads of those sites and drops a challenge per read. It fits the courtesy policy's aim of fewer conspicuous requests.

Not throttling, for the record

  • CDN 403s are rare: about 25 in roughly 1,000 re-checks over 3.5 days, mostly on links found hours earlier. A fresh read verified within a minute each time, which fits expired signed URLs.
  • Live probe, Ole Miss vs. Georgia, 00:50 UTC 2026-11-08 (edgestream3.pro, segments on TikTok's CDN):
    • Each ~4 s segment downloaded in 0.3–1.7 s, at 9–54 Mbit/s, for a 3.9 Mbit/s stream.
    • The source's playlist added only 44 s of video in 70 s.
    • So the stalls came from the source running at about two-thirds of real time. Several front ends carry it (givemestreams, s2watch, watchsports.online and more), so swapping between them doesn't help (#91).
  • Not proven: whether the two restarts 30 min apart (15:55 and 16:26 UTC) made Cloudflare stricter on the hub. The earlier container's logs are gone.

Related: #78 (the embedsports.me frame those gated pages hold, which gave "no stream found" 17 times on 2026-11-07), PR #82, #91.

Checked 2026-11-07 (8 h of container logs, 3.5 days of `/api/events`, a live probe at 00:50 UTC) whether viewing problems come from sites or CDNs throttling or banning the app. **They don't.** There is no ban page anywhere in the solver log. The red rows on the Admin Sites table are the app's own doing, and two bugs make them look like bans. ## What happens **1. The app's own browser's challenge backs good sites off.** - `_read_player_page` opens a player page in the app's browser first. On the Cloudflare-gated sites (dudestream1.com, streameast.ga, tvsportslive.fr, mygoodstream.pw, iohit.st) Cloudflare challenges that browser every time, and `read_stream_from_page` records the challenge (with its 403) against the host. - The solver then reads the same page fine (the `after_challenge` hand-off from PR #82): dudestream1.com 11 of 12, streameast.ga 11 of 11, tvsportslive.fr 20 of 23. All 54 backoffs that day were followed within 3 s by a solver read of the same host. - When the page holds no stream the app can read, the backoff stays and doubles, from 60 s up to the 1800 s cap. dudestream1.com reached 12 consecutive failures and streameast.ga 11. From 18:54 UTC on, each try at dudestream1.com earned another 30 min. tvsportslive.fr gets locked out too, though it does deliver streams: Ole Miss vs. Georgia recovered on it at 23:01 and 23:42 UTC. - The backoff also refuses the next step of the same read. Same-host player frames were refused 14 times with "source host backed off" (mygoodstream.pw ×6, tvsportslive.fr ×8). - The Sites table shows "403", "Cloudflare checks" and "Failed 12", which reads as a ban. **2. A challenge the solver can't clear reads as a solver outage.** - `SOLVE_TIMEOUT_SECONDS` (90 s) equals the `maxTimeout` the app sends the solver (90 000 ms). The solver sends its 500 ("Error solving the challenge. Timeout after 90.0 seconds") at 90.1 s. httpx has given up about 50 ms earlier, so `_fetch_clearance` raises `SolverUnreachable`. - 16:27–16:59 UTC, right after the 16:26 restart: 22 hub reads in a row (v2.sportsurge.net) got "Just a moment..." and FlareSolverr couldn't clear it. Each time the app logged "solver ... is not answering", kept the stuck session and recorded nothing against the host. It cleared on its own at 16:59. - For those 33 min the app couldn't read the hub, so it found no new games and read no match pages. It ended right at the noon ET kickoffs. ## What it costs - Working sites are skipped for up to 30 min at a time, and their frames are refused. - A stuck hub after a restart doesn't show on the Admin page. The app retries the same stuck browser every 90 s. - The Sites table points at bans that aren't there. ## The idea 1. **Count a read by its outcome, not its first step.** When a page is handed to the solver, the app browser's challenge is not a failure on its own. Record the read once, after the solver and any frames have tried. A page the solver reads that holds no stream is "no stream found" and does not back the host off; the link's own cooldown (`Ladder.fail`, 10 min) already keeps it from being re-read. A challenge the solver can't clear still counts. 2. **Frames of the same read skip the backoff that read set**, the way `past_backoff` lets the solver step through. 3. **Give the solver call a margin over `maxTimeout`**, for example 100 s against 90 s, so the solver's own error arrives. A session that fails to clear a challenge is then discarded, the next read starts a fresh browser, and the challenge counts against the host and shows on the Sites table. 4. **Go straight to the solver on a host whose pages always challenge the app's browser.** This halves the reads of those sites and drops a challenge per read. It fits the courtesy policy's aim of fewer conspicuous requests. ## Not throttling, for the record - **CDN 403s are rare:** about 25 in roughly 1,000 re-checks over 3.5 days, mostly on links found hours earlier. A fresh read verified within a minute each time, which fits expired signed URLs. - **Live probe, Ole Miss vs. Georgia, 00:50 UTC 2026-11-08** (edgestream3.pro, segments on TikTok's CDN): - Each ~4 s segment downloaded in 0.3–1.7 s, at 9–54 Mbit/s, for a 3.9 Mbit/s stream. - The source's playlist added only 44 s of video in 70 s. - So the stalls came from the source running at about two-thirds of real time. Several front ends carry it (givemestreams, s2watch, watchsports.online and more), so swapping between them doesn't help (#91). - **Not proven:** whether the two restarts 30 min apart (15:55 and 16:26 UTC) made Cloudflare stricter on the hub. The earlier container's logs are gone. Related: #78 (the embedsports.me frame those gated pages hold, which gave "no stream found" 17 times on 2026-11-07), PR #82, #91.
Author
Owner

Deployed 2026-11-08 06:11Z in PR #111 (merge a133e29, container 71367c5163eb); OpenSpec change archived in b4d5b94. Left open until a live night shows it: a gated player page (dudestream1, streameast, tvsportslive) read by the solver without backing its host off, a frame on the same host read past a backoff, and a hub challenge the solver can't clear counted against the host (the Sites strip turns amber) instead of logged as "not answering". Hosts that challenged the app's browser in the last 6 h are now read by the solver directly (a shield beside the host on the Sites list).

Deployed 2026-11-08 06:11Z in PR #111 (merge a133e29, container 71367c5163eb); OpenSpec change archived in b4d5b94. Left open until a live night shows it: a gated player page (dudestream1, streameast, tvsportslive) read by the solver without backing its host off, a frame on the same host read past a backoff, and a hub challenge the solver can't clear counted against the host (the Sites strip turns amber) instead of logged as "not answering". Hosts that challenged the app's browser in the last 6 h are now read by the solver directly (a shield beside the host on the Sites list).
Author
Owner

Closed at the owner's word (2026-11-08): the fix is merged to develop in PR #111 (merge a133e29) and deployed. A new issue will be opened if a problem shows up.

Closed at the owner's word (2026-11-08): the fix is merged to develop in PR #111 (merge a133e29) and deployed. A new issue will be opened if a problem shows up.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
cmoriarty/sports-breaker#98
No description provided.