Skip to content

fix(proxy): move URL credentials out of server and accept local proxy hosts - #1170

Open
alkaz-nodemaven wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
alkaz-nodemaven:fix/proxy-server-credentials-and-hosts
Open

alkaz-nodemaven wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
alkaz-nodemaven:fix/proxy-server-credentials-and-hosts

Conversation

@alkaz-nodemaven

Copy link
Copy Markdown

Two problems with loader_kwargs["proxy"]:

  • Credentials written into the server URL (http://user:pass@host:8080) stay in server, and Playwright ignores them there, so the proxy answers 407. They're now moved to username/password (percent-decoded). If username/password are set explicitly, those win.
  • localhost:8888, a Docker service like http://proxy:8080, or IPv6 fail with unknown proxy server type, because the host check only allows IPv4 or a domain with a TLD. Any host with a port is accepted now; a bare unknown with no port is still rejected, so the existing test still holds.

Error messages no longer include the password either.

Tests added in tests/utils/test_proxy_rotation.py; all fail on pre/beta and pass with the change. test_search_proxy_servers_success hits free-proxy-list.net and fails in my environment both before and after, so it's unrelated.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant