Skip to content

CDP Page.navigate never completes for Wikipedia (CLI fetch works) #1801

@eddiebelaval

Description

@eddiebelaval

Summary

Page.navigate via CDP never returns a response for Wikipedia pages (and some other sites like Yahoo Finance). The CLI lightpanda fetch command works perfectly for the same URLs, returning full markdown content. This indicates the HTTP/rendering engine handles these sites correctly, but the CDP protocol path has an issue completing the navigation lifecycle.

Environment

  • Lightpanda: Nightly build 1522c902 (aarch64-macos)
  • Connection: Raw WebSocket CDP (not Playwright — using manual Target.createBrowserContextTarget.createTargetTarget.attachToTarget flow)

Reproduction

1. CLI fetch works (✅)

lightpanda fetch --dump markdown "https://en.wikipedia.org/wiki/Zig_(programming_language)"
# Returns 62,541 chars of markdown content

lightpanda fetch --dump markdown "https://en.wikipedia.org/wiki/List_of_programming_languages"
# Returns 72,988 chars of markdown content

2. CDP Page.navigate stalls (❌)

// After connecting via WebSocket and creating a target:
await client.send("Page.enable");
await client.send("Runtime.enable");

// This command never returns a response:
const nav = await client.send("Page.navigate", {
  url: "https://en.wikipedia.org/wiki/Zig_(programming_language)"
});
// ^ Times out after 15s — no CDP response received

3. CDP events observed

Only two events fire before the timeout:

Page.frameStartedNavigating
Page.frameStartedLoading

Missing events (never received):

Page.frameNavigated
Page.domContentEventFired
Page.loadEventFired
Page.frameStoppedLoading

After Page.navigate times out, Runtime.evaluate also times out — the page is in a broken state.

Scope

Tested across multiple Wikipedia pages (small and large) — all fail identically via CDP. The issue is not page size. Other sites like example.com, news.ycombinator.com, react.dev, httpbin.org all work fine via CDP.

Yahoo Finance (finance.yahoo.com) shows a similar but slightly different pattern: CDP events DO fire (including loadEventFired), but the Page.navigate command response itself never returns.

Expected Behavior

Page.navigate should return a response with frameId and loaderId, and the standard page lifecycle events should fire, matching the behavior of lightpanda fetch for the same URLs.

Workaround

Use lightpanda fetch (CLI mode) or MCP goto tool for sites that stall via CDP.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions