Add --playwright-proxy option for /web
using playwright with chromium
#4184
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi Team,
Thanks a lot for the great work.
It seems that playwright with chromium does not use proxy be default. So this pull request introduces support for specifying a proxy server when using Playwright for web scraping. It adds a new --playwright-proxy command-line argument, allowing users to route Playwright browser traffic through a proxy. This is useful for users who need to access web content behind a firewall or want to anonymize their scraping activities (I guess this feature could be particularly useful for users in China).
Changes
• New Argument: Added --playwright-proxy to the CLI (in aider/args.py and aider/scrape.py) to allow users to specify a proxy URL for Playwright.
• Scraper Update: Updated the Scraper class (aider/scrape.py) to accept and use the proxy setting when launching Playwright browsers. If the argument is not provided, it falls back to the HTTP_PROXY/http_proxy environment variables.
The code changes are done by aider