-
Notifications
You must be signed in to change notification settings - Fork 34
Add support for extra options in the fullDomFetcher #1173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the work on this.
However, look at my comments, but it seems to me that the core issue hasn't been fully addressed.
Could you update the PR to include this, or let me know if I’m misunderstanding something.
Updated logic has been implemented in the latest commits and replicated into htmlonlyfetcher as well. |
Hi @LVerneyEC, I went ahead and refactored some parts of the code and moved the proxy credentials into the browser object instead of keeping them in the module. Since I’m a bit tied up at the moment, it was quicker to implement than to explain. |
I can confirm it works as expected. |
@LVerneyEC, just one more step before we can deliver your contribution. Could you lint the commit messages and add a changelog entry? |
Should be rebased and cleaned. I have no way to test the CHANGELOG locally and tried to adapt to the best I understood. Please feel free to manually edit the CHANGELOG if not matching your practices before merging, as I don't want to be delaying this PR too much and this is becoming a blocking issue on our side. Thanks! |
Add support for proxy, disabling the sandboxing (which is required by some Docker setups) and disabling headless mode in the fullDomFetcher.
As stated in the CONTRIBUTING guidelines, each changelog release must include sponsor information. For example, when a release is developed with the support of the French Ministry for Foreign Affairs, we include:
You can find other examples in the CHANGELOG.md. Could you provide the sponsor information for this release? |
Hey,
This is a follow-up of #1144 as discussed there.
This PR makes the fullDomFetcher configurable through environment variables to support a wider range of setups:
Thanks