feat(headless): add cdp-endpoint option

[dwisiswant0]

Proposed changes

Close #5692

How has this been tested?

# tty1
/path/to/chrome --headless --remote-debugging-port=9222 --ignore-certificate-errors --ignore-ssl-errors

# tty2
go run cmd/nuclei/main.go -headless -t headless-template-1.yaml -u http://scanme.sh -cdp-endpoint "CHROME_DEVTOOLS_ENDPOINT_URL"

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Summary by CodeRabbit

• New Features

  • Added a CLI option to connect to a remote browser via a Chrome DevTools Protocol (CDP) endpoint.

• Refactor

  • Initialization and shutdown now distinguish local vs. remote browser usage, skipping local browser launch and cleanup when a remote CDP endpoint is used.

• Documentation

  • Updated HEADLESS docs across locales to document the new CDP endpoint option.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[dwisiswant0]

But I’m unable to connect to the CDP URL provided by finic:

[FTL] Could not create runner: websocket bad handshake: 400 Bad Request. Failed to open a WebSocket connection: invalid Sec-WebSocket-Key header: nil; Incorrect padding.

Not sure if this is an issue with go-rod or with finic. It needs further investigation, or maybe even a hacky workaround. I’ll try testing it via Playwright - if the issue can’t be reproduced there, I’ll go ahead and raise it as an issue in go-rod.

[dwisiswant0]

--flag=true eq to --flag=1 eq to --flag

[dwisiswant0]

Works with browserless.

Seems like the issue is within finic.

[ehsandeep]

$ ./nuclei -u http://scanme.sh -cdp-endpoint "ws://localhost:3000" -headless -pt headless

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.3.5

		projectdiscovery.io

[INF] Current nuclei version: v3.3.5 (latest)
[INF] Current nuclei-templates version: v10.0.3 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 116
[INF] Templates loaded for current scan: 18
[INF] Executing 18 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[extract-urls] [headless] [info] http://scanme.sh [""]
[piratebay] [headless] [info] https://thepiratebay.org/search.php?q=user:{{user}}
[INF] Screenshot successfully saved at /Users/geekboy/Github/nuclei/screenshots/http:__scanme.sh.png
panic: invalid character 'T' looking for beginning of value

goroutine 72 [running]:
github.com/go-rod/rod/lib/utils.glob..func2({0x103f959c0?, 0x140181cc570?})
	/Users/geekboy/go/pkg/mod/github.com/go-rod/[email protected]/lib/utils/utils.go:68 +0x24
github.com/go-rod/rod/lib/utils.E(...)
	/Users/geekboy/go/pkg/mod/github.com/go-rod/[email protected]/lib/utils/utils.go:74
github.com/go-rod/rod/lib/cdp.(*Client).consumeMessages(0x1400043f680)
	/Users/geekboy/go/pkg/mod/github.com/go-rod/[email protected]/lib/cdp/client.go:148 +0x104
created by github.com/go-rod/rod/lib/cdp.(*Client).Start in goroutine 1
	/Users/geekboy/go/pkg/mod/github.com/go-rod/[email protected]/lib/cdp/client.go:76 +0x88

[dwisiswant0]

@ehsandeep - If we look at the trace, it doesn't seem related to our source code - the underlying issue is in go-rod. Also, this panic wouldn't happen if we were using the system CDP.

$ go run cmd/nuclei/main.go -u http://scanme.sh -cdp-endpoint "ws://*********:9222/devtools/browser/************************************" -headless -pt headless

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.3.5

		projectdiscovery.io

[INF] Current nuclei version: v3.3.5 (latest)
[INF] Current nuclei-templates version: v10.0.3 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 116
[INF] Templates loaded for current scan: 18
[INF] Executing 18 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[piratebay] [headless] [info] https://thepiratebay.org/search.php?q=user:{{user}}
[extract-urls] [headless] [info] http://scanme.sh [""]

[dwisiswant0]

We either need to tweak the Browserless container or there's something on our end that needs to be handled - but we need to figure out the root cause (template) that's triggering that panic.

[github-actions]

This pull request has been automatically marked as stale due to inactivity. It will be closed in 7 days if no further activity occurs. Please update if you wish to keep it open.

[coderabbitai]

Walkthrough

A new cdp-endpoint CLI flag and CDPEndpoint option were added. The headless browser engine now conditionally uses the provided CDP WebSocket URL instead of launching a local Chrome instance; initialization and cleanup are skipped when connecting to a remote CDP endpoint.

Changes

Cohort / File(s)	Change Summary
CLI flag cmd/nuclei/main.go	Added cdp-endpoint (cdpe) CLI flag bound to options.CDPEndpoint.
Options struct pkg/types/types.go	Added CDPEndpoint string field to Options.
Headless engine pkg/protocols/headless/engine/engine.go	Refactored New and Close to: when CDPEndpoint is set, use it as the launcher URL and skip local Chrome launch, temp dir creation, musl detection, launcher setup, and local cleanup; otherwise preserve existing local launch and cleanup logic. Proxy and extra args apply only to local launches.
Documentation (localized) README.md, README_*.md	Added -cdpe, -cdp-endpoint flag documentation under HEADLESS across locales.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant Engine
    participant Browser

    User->>CLI: nuclei --cdp-endpoint ws://host:port
    CLI->>Engine: Pass Options (CDPEndpoint set)
    alt CDP endpoint provided
        Engine->>Browser: Connect to provided CDP WebSocket URL
        Note right of Engine: Skip temp dir creation\nSkip launching local Chrome
        User->>Engine: Perform headless actions
        Engine->>Browser: Send CDP commands over WebSocket
        User->>Engine: Close()
        Engine-->>Browser: Do not terminate remote browser
    else No CDP endpoint
        Engine->>Engine: Create temp dir & configure launcher
        Engine->>Browser: Launch local Chrome process
        User->>Engine: Perform headless actions
        Engine->>Browser: Send CDP commands
        User->>Engine: Close()
        Engine->>Browser: Terminate Chrome process and cleanup
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

I’m a rabbit by the CDP shore,
I hop to ws:// and ask for more. 🐇
No local launch, just websockets spun,
Commands go out — the tabs have fun.
I close my paws, the remote stays on.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(headless): add cdp-endpoint option' clearly and concisely describes the main change: adding a new CDP endpoint option to headless mode.
Linked Issues check	✅ Passed	The PR implements all primary objectives from issue #5692: adds cdp-endpoint flag accepting WebSocket CDP URL, enables connection to existing/remote browser instances, and supports cloud-hosted browsers via CDP endpoint.
Out of Scope Changes check	✅ Passed	All changes are in-scope: CLI flag addition, Options struct field, engine conditional logic for CDP vs local launch, and documentation updates for the feature. No unrelated modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0c8fb50 and 64b4c59.

📒 Files selected for processing (8)

• README.md
• README_CN.md
• README_ES.md
• README_ID.md
• README_KR.md
• README_PT-BR.md
• cmd/nuclei/main.go
• pkg/types/types.go

🚧 Files skipped from review as they are similar to previous changes (6)

• README_KR.md
• cmd/nuclei/main.go
• pkg/types/types.go
• README_ES.md
• README.md
• README_ID.md

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Lint

🔇 Additional comments (2)

README_PT-BR.md (1)

271-271: Documentation change looks good.

The Portuguese translation for the new -cdpe, -cdp-endpoint flag is clear and properly positioned in the HEADLESS section. The description accurately conveys the feature's purpose.

README_CN.md (1)

266-266: Documentation change looks good.

The Chinese translation for the new -cdpe, -cdp-endpoint flag is accurate and consistently positioned within the headless browser section.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[dogancanbakir]

• build fails
• merge conflict

[dwisiswant0]

https://github.com/orgs/projectdiscovery/discussions/6453

[Deamhan]

Very cool and useful functionality, I hope it'll become part of upstream. The only missing thing here is no incognito mode which could be highly useful. Currently upstream has the following from (called from executeRequestWithPayloads)

func (b *Browser) NewInstance() (*Instance, error) {
browser, err := b.engine.Incognito()
if err != nil {
return nil, err
}

and it's not configurable. If we switch to something like

func (b *Browser) NewInstance() (*Instance, error) {
var browser *rod.Browser
if b.options.HeadlessNoIncognito {
// Use the existing browser directly instead of creating an incognito instance
browser = b.engine
} else {
var err error
browser, err = b.engine.Incognito()
if err != nil {
return nil, err
}
}

we could get much more flexible result. Of course it can be added separetely. Correct me if I'm wrong :-)

[dwisiswant0]

[...] The only missing thing here is no incognito mode which could be highly useful. [...]

Since we’re connecting via Chrome DevTools Proto URL (browser’s already live), flags don’t take effect after launch [and are not controllable from the source], right? They’re static, hence any flag changes won’t apply.

I’m not 100% certain. AFK rn, will PoC later to confirm.

[Deamhan]

Since we’re connecting via Chrome DevTools Proto URL (browser’s already live), flags don’t take effect after launch [and are not controllable from the source], right? They’re static, hence any flag changes won’t apply.

I’m not 100% certain. AFK rn, will PoC later to confirm.

I was talking about request.go#L115 - as I see for such requests we create a new instance of browser based on existing one instance.go#L30 no matter if it's provided CDP or not - correct me if I'm wrong. If we want to use the same instance a minor modification is required like I mentioned before - something like avoid of browser, err = b.engine.Incognito(). For production usecase I added this modification and the result works like a charm :-)

[Mzack9999]

lgtm!

$ docker run -p 3000:3000 ghcr.io/browserless/chromium
$ go run . -headless -pt headless -u http://scanme.sh -cdp-endpoint "ws://localhost:3000"
...
[INF] Templates loaded for current scan: 23
[INF] Executing 23 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[INF] Scan completed in 7.841658417s. No results found.