Repository navigation
Retry URL imports as a browser when a site answers 403 (fixes Rumble) - #74
Merged
Merged
Conversation
Rumble now sits behind Cloudflare's bot screen, which refuses yt-dlp's Python client on its TLS fingerprint: "Unable to download webpage: HTTP Error 403: Forbidden". yt-dlp can handshake like Chrome through curl_cffi, which the install did not include. Add the curl-cffi extra (pinned to 0.17.0, abi3 wheels for x86_64 and aarch64), bump yt-dlp to 2026.10.26, and retry a probe or download once with impersonation when it fails with 403. Other sites keep yt-dlp's own client. A 403 that survives the retry, or that cannot be retried because curl_cffi is missing, now says what to do instead of yt-dlp's raw line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1geD4inqJxe1F6HuWV82R
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rumble imports fail with
Unable to download webpage: HTTP Error 403: Forbidden. Rumble is now behind Cloudflare's bot screen, which refuses yt-dlp's Python client based on its TLS handshake. Headers and cookies don't help, and Rumble's extractor is unchanged between our pin and the newest yt-dlp, so a version bump alone won't fix it.Changes
backend/requirements.txt:yt-dlp[default,deno,curl-cffi]==2026.10.26.curl-cffi==0.17.0to the range that release's extra accepts. abi3 manylinux wheels exist for x86_64 and aarch64, so nothing compiles on 3.13.app/ingest/ytdlp.py:_extracthelper. If a probe or download fails with 403, it retries once withimpersonate=ImpersonateTarget(), which presents a Chrome/Safari TLS fingerprint through curl_cffi.readable_errornow turns a 403 that persists into advice: rebuild the image, or useURL_IMPORT_COOKIES_FILE.docs/url-import.md: a short section on 403s and impersonation.Testing
pytest -q: whole backend suite passes.gecko-suite-clientwas installed from a local checkout of the pinned commit.Deploy with
docker compose up --build -d, since the image needs the new dependency.Generated by Claude Code