-
Notifications
You must be signed in to change notification settings - Fork 626
Description
Summary
Page.navigate via CDP never returns a response for Wikipedia pages (and some other sites like Yahoo Finance). The CLI lightpanda fetch command works perfectly for the same URLs, returning full markdown content. This indicates the HTTP/rendering engine handles these sites correctly, but the CDP protocol path has an issue completing the navigation lifecycle.
Environment
- Lightpanda: Nightly build
1522c902(aarch64-macos) - Connection: Raw WebSocket CDP (not Playwright — using manual
Target.createBrowserContext→Target.createTarget→Target.attachToTargetflow)
Reproduction
1. CLI fetch works (✅)
lightpanda fetch --dump markdown "https://en.wikipedia.org/wiki/Zig_(programming_language)"
# Returns 62,541 chars of markdown content
lightpanda fetch --dump markdown "https://en.wikipedia.org/wiki/List_of_programming_languages"
# Returns 72,988 chars of markdown content2. CDP Page.navigate stalls (❌)
// After connecting via WebSocket and creating a target:
await client.send("Page.enable");
await client.send("Runtime.enable");
// This command never returns a response:
const nav = await client.send("Page.navigate", {
url: "https://en.wikipedia.org/wiki/Zig_(programming_language)"
});
// ^ Times out after 15s — no CDP response received3. CDP events observed
Only two events fire before the timeout:
Page.frameStartedNavigating
Page.frameStartedLoading
Missing events (never received):
Page.frameNavigated
Page.domContentEventFired
Page.loadEventFired
Page.frameStoppedLoading
After Page.navigate times out, Runtime.evaluate also times out — the page is in a broken state.
Scope
Tested across multiple Wikipedia pages (small and large) — all fail identically via CDP. The issue is not page size. Other sites like example.com, news.ycombinator.com, react.dev, httpbin.org all work fine via CDP.
Yahoo Finance (finance.yahoo.com) shows a similar but slightly different pattern: CDP events DO fire (including loadEventFired), but the Page.navigate command response itself never returns.
Expected Behavior
Page.navigate should return a response with frameId and loaderId, and the standard page lifecycle events should fire, matching the behavior of lightpanda fetch for the same URLs.
Workaround
Use lightpanda fetch (CLI mode) or MCP goto tool for sites that stall via CDP.