"Navigation failed because page was closed!" "navigating to URL "waiting until "load"

[eng3]

I'm trying changedetection for the first time. I'm running the docker on my own local computer.
I keep receiving this error on multiple watches using Playwright. When I try "Browser Steps", it seems to load in less than 5 seconds so it seems like playwright is working. However, when I try a recheck, it seems to timeout with the above error. It seems to happen to randomly on multiple watches but sometimes just works fine. There's only one of my watches that seems to consistently everytime

[B3nihana]

Just wanted to share that I am experiencing this same issue. I run both ChangeDetection (0.45.5) and Playwright via Docker Compose/Docker Desktop as per the official instructions on an x86_64 Windows install.

If just checking for plain text changes, everything is OK, but using Playwright it has consistent issues across different sites. I have tried to change the visual filter browser steps and increasing/decreasing the timeout but to no effect.

[dgtlmoon]

If it's true, then i'll change the docker-file to recommend to leave this off, then create a option/checkbox/setting in the UI to turn it off/on https://www.browserless.io/docs/docker#default-block-ads (on by default perhaps)

[masterwishx]

@esmoyer
and added the line about fs and crypto <- What does it mean ?

Thanks for tip DEFAULT_BLOCK_ADS=false Fixed the issue

[B3nihana]

@masterwishx I think it refers to this:
#1929 (reply in thread)

[masterwishx]

@dgtlmoon
Also found that browserless have v2.0 from july this year but only on github ghcr.io/browserless/chrome
https://github.com/browserless/browserless/blob/main/MIGRATION-2.0.md

[esmoyer]

Correct. Add the env variable mentioned in that reply; i used; FUNCTION_BUILT_INS=["fs","crypto"] in my compose

[jlssmt]

x64 Ubuntu
Docker compose from github root folder

[dgtlmoon]

@jlssmt x64 is not an operating system

[jlssmt]

Already added

[dgtlmoon]

and I also asked for some example URLs to test with

[jlssmt]

It's even with google.com. So I think it's a general problem.

[dgtlmoon]

@eng3

1. I need some example URLs to retry, so i can try to reproduce it
2. You dont mention any technical information that could help, like what OS, how you installed, what version etc

[eng3]

1. Here's one that fails often "https://www.walmart.com/ip/Mercer-Culinary-Renaissance-8-Inch-Forged-Chef-s-Knife/25921309" Note that I have another "https://www.walmart.com/ip/Mercer-Culinary-Genesis-8-Inch-Forged-Short-Bolster-Forged-Chef-s-Knife/157515193" that is very similar that has succeeded every time. Here's another "https://www.presidential.com/Personal-Banking/Checking-Savings/Rates-Yields" It seems somewhat random because if I recheck enough times, it works. But there definitely seems to be some that fail more often than others.
2. I'm on debian 11 using your docker-compose install method. Version 0.45.5

[dgtlmoon]

@eng3 can you say if you have any 'wait for seconds' set?

[eng3]

I have "Wait seconds before extracting text" set to 5 seconds

[dgtlmoon]

It's even with google.com. So I think it's a general problem.

i think the bug might be that the 'wait seconds' is greater than the number of seconds browserless wants to wait for, hence "navigation failed because page was closed"..... try setting jitter to 0 and all 'waits' to 3 seconds... letme know if its more reliable

[masterwishx]

if blank in global waits settings ,is it use some value by default or its =0 ?
changed to 3sec

[dgtlmoon]

also be sure to use browserless/chrome:1.60-chrome-stable in your docker-compose

[masterwishx]

Using Docker in Unraid . changedetection and browserless Both latest version
Also have this issue for some tests randomly ...

[masterwishx]

Also moved to 1-chrome-stable + 0s jitter + 3s wait
but still have same issue for sites :

example to check :

URL:
https://www.amazon.com/gp/product/B08QBJ2YMG/ref=ox_sc_act_title_1?smid=A2XZ7JICGUQ1CX&psc=1
Filter:
/html/body/div[1]/div[2]/div[9]/div[5]/div[4]/div[13]/div[3]/div[1]/span[1]/span[2]

[masterwishx]

Also found many sessions (google) that was no befor :

[dgtlmoon]

@masterwishx when you try to report an issue or discussion, the software version is probably one of the most critical and obvious things to write, please, what version are you on?

[masterwishx]

Sorry but i wrote i have lasted version both :
#1929 (comment)

[dgtlmoon]

@masterwishx and what exactly are you doing on "google hangouts" ? this seems like a really weird page to expect the browser to be able to load, because that page never actually loads, its always feeding some video etc

[masterwishx]

and what exactly are you doing on "google hangouts" ? this seems like a really weird page to expect the browser to be able to load, because that page never actually loads, its always feeding some video etc

Not sure i have it when moved to 1-chrome-stable

[masterwishx]

When i goes back to lasted , no google sessions , checked a couple times .

[dgtlmoon]

@masterwishx pause all watches, restart your chrome container, and then request just the amazon page, does it work?

[masterwishx]

Some sites work when manual check, i will try your tip

[dgtlmoon]

@masterwishx does #1929 (reply in thread) fix it?

[masterwishx]

Thanks, I will check it when will be at home soon

[masterwishx]

Yes , set DEFAULT_BLOCK_ADS=false Fixed the issue

[jucor]

@dgtlmoon Same for me: same problem, and setting DEFAULT_BLOCK_ADS=false fixed it ! Thank you! But will that trigger a false positive whenever the ad changes?

[dgtlmoon]

@jucor well its wont be a false-positive because it will be actually a change :) but I cannot say how all the ad companies in the world present their content... so official answer is "maybe maybe not" :)

[jucor]

Ah, I see. How to monitor for changes except to the ads, please? I.E. changes to the non-ad content only?

[dgtlmoon]

Ah, I see. How to monitor for changes except to the ads, please? I.E. changes to the non-ad content only?

you cant, sorry, that's how ads work, they affect the content.

[esmoyer]

For me, I just use the selector tool and filters, plus all my stuff is behind Adguard Home anyway, so the in-container ad-blocker really isn't needed.

[jucor]

@esmoyer Yeah, makes sense, when you have an adguard home already (I don't). For the selector tool , as for @eng3 suggestion: yes, it's just more work when setting up a task :)

[dgtlmoon]

You're looking for a way to block ads without blocking ads.. answer is no, just use a selector

[jucor]

Oh, I'd love to block ads with DEFAULT_BLOCK_ADS=true! The only reason I'm setting it to false is as a temporary workaround to the random but numerous errors that raises, as reported by the other users in this thread and seeing it myself. Is it normal/expected behaviour that setting it to true causes those errors at random?

[eng3]

That's a good idea too. I'm not sure how "default_block_ads" implements blocking of ads anyways.
You could also setup the container's DNS to point to a DNS adblock if you didn't have it at home.
It is a little confusing why that feature would cause these issues.