Getting EmptyReply on a site even though Visual Selector loads it without issue (v0.40.0.2)

[BryanWall]

The page in question: https://us.switch-bot.com/products/switchbot-blind-tilt

I've tried setting a very long wait time, tried both Basic and Playwright, and anything else I could think of, but this site still reports EmptyReply. The site loads fine in a browser, and the Browser Steps screen also loads it fine. You can also see it in Visual Selector and choose elements to monitor. But when it actually tries to queue the site to be checked it doesn't work.

Any ideas why this site can't be monitored and how to fix it?

[BryanWall]

Running the latest ChangeDetection Docker image on Unraid and latest Playwright Docker image.

When I tested it just now I only added the URL as a basic watch and didn't change anything else or even add filters.

This is the Playwright URL that I'm using: ws://browserless:3000/?stealth=1&--disable-web-security=true

I have 15+ other watches running for quite a while and none of the others have had an issue like this.

Which logs do you need and from where?

I clicked Recheck and watched the console messages and saw the following. Could it be something related to the Playwright config? I haven't changed any of the defaults other than what's changed by the URL parameters.

Processing UUID 470c5465-860e-4c0d-9e3f-15e668e1456f Priority 1 URL https://us.switch-bot.com/products/switchbot-blind-tilt
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt error: Failed to load resource: the server responded with a status of 404 () []
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt error: Refused to execute script from 'https://cdn.shopify.com/s/files/1/0648/1569/4057/t/2/assets/sca.affiliate.refer-customer-data.js?2965' because its MIME type ('text/html') is not executable, and strict MIME type checking is enabled. []
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: Init GrowClient []
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: Timer 53209 isn't allowed on the current page [<JSHandle preview=Timer 53209 isn't allowed on the current page>]
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: subIdTV3FTQ36SRSRF5SUPEXWKJYB []
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: null null null [, , ]
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: by options [<JSHandle preview=>, ]
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: addPopup []
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: addPopup []
Playwright console: Watch URL: https://us.switch-bot.com/products/switchbot-blind-tilt log: Grow Variant Change title Default Title [, ]

[dgtlmoon]

Hmmm yeah, I tried it on our paid instances and it works fine - BUT the site has a lot of JS errors, and it even loads different scripts depending on the country you're requesting from

so i think the paid setup we have gets lucky in this regard

it also works locally for me, but not when the 'auto price follower' is enabled

I have 15+ other watches running for quite a while and none of the others have had an issue like this.

well they arent the same URL, are they? :)

[BryanWall]

That's interesting. It makes me curious what would happen if I sent the traffic through a VPN endpoint in another country.

[dgtlmoon]

I just noticed that - that website is making an extra 'service worker' when I block that, and tweak the 'wait' mechanism it works way more reliably

With this https://github.com/dgtlmoon/changedetection.io/pull/1229/files that URL works perfectly

[BryanWall]

This web site seems to be incredibly over-engineered for just a product page / shopping cart. I wouldn't be surprised if real users sometimes have trouble getting it to load correctly.

I'll try it again when you push this change to the release.

[dgtlmoon]

Can you try the :dev https://hub.docker.com/layers/dgtlmoon/changedetection.io/dev/images/sha256-b1905b2a69448f65380ab134082d1fad50eac41b7eb8d7cbf2358c321b938d99?context=explore docker image ?

[BryanWall]

I tried the :dev image and it was able to check that site successfully for the first time. I was even able to add a class filter and it still worked. Whatever you did seems to be working.

It failed once when I had the "wait seconds" set to 5, but I'm not sure if it failed because of the wait time or because of the flakiness of the site is still occasionally breaking the checking logic. I set it back to 10 seconds and it worked. I'll keep watching it and let you know if it's not working consistently.

[BryanWall]

I switched to the .03 version you released yesterday and the site failed. Today it worked. It's definitely improved, but that site still has some weirdness that breaks the checks sometimes.