[🐛 Bug]: How to use CDP to capture console network requests in a remote Chrome browser

[jjqi92]

What happened?

I want to grab network request data through cdp, but there has an error that the getDevTools() does not exists.

How can we reproduce the issue?

WebDriver driver = new RemoteWebDriver(new URL(hubUrl), options);
DevTools devTools = driver.getDevTools();

Relevant log output

no

Operating System

windows10

Selenium version

4.23.0

What are the browser(s) and version(s) where you see this issue?

chrome117

What are the browser driver(s) and version(s) where you see this issue?

chromedriver 117

Are you using Selenium Grid?

4.23.0

[github-actions]

@jjqi92, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[pujagani]

Please use the Augmenter. Example usage:
https://github.com/SeleniumHQ/selenium/blob/trunk/java/test/org/openqa/selenium/grid/router/DistributedCdpTest.java#L87

[jjqi92]

@pujagani I tried the below methods, but I still couldn't capture the network layer request data. Could you please help me check again?

RemoteWebDriver driver = new RemoteWebDriver(new URL(hubUrl), options);
driver = new Augmenter().augment(driver);
WebDriver finalDriver = driver;

try (DevTools devTools = ((HasDevTools) driver).getDevTools()) {
devTools.createSessionIfThereIsNotOne(driver.getWindowHandle());
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTools.addListener(Network.requestWillBeSent(), requestWillBeSent -> {
Request request = requestWillBeSent.getRequest();
System.out.println("Request URL: " + request.getUrl());
System.out.println("Request Method: " + request.getMethod());
System.out.println("Request Headers: " + request.getHeaders());
System.out.println("Request Post Data: " + request.getPostData().orElse("No Post Data"));
});

devTools.addListener(Network.responseReceived(), responseReceived -> {
	try {
		TimeUnit.SECONDS.sleep(1);
	} catch (InterruptedException e) {
		logger.error("sleep exception", e);
	}

	Response response = responseReceived.getResponse();
	System.out.println("Response URL: " + response.getUrl());
	System.out.println("Status Code: " + response.getStatus());
	System.out.println("Response Headers: " + response.getHeaders());
	System.out.println("Response Body: " + devTools.send(Network.getResponseBody(responseReceived.getRequestId())).getBody());	

}

driver.get(url);

[pujagani]

What data are you looking to capture about the request?

[jjqi92]

@pujagani request type, request body and response body

[pujagani]

Please use the Augmented driver in the code.

Example code:

public class CDPNetworkCapture {

  public static void main(String[] args) throws MalformedURLException {
    RemoteWebDriver
        driver =
        new RemoteWebDriver(new URL("http://localhost:4444"), new ChromeOptions(), false);
    WebDriver finalDriver = new Augmenter().augment(driver);

    try (DevTools devTools = ((HasDevTools) finalDriver).getDevTools()) {
      devTools.createSessionIfThereIsNotOne(driver.getWindowHandle());
      devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
      devTools.addListener(Network.requestWillBeSent(), requestWillBeSent -> {
        Request request = requestWillBeSent.getRequest();
        System.out.println("Request URL: " + request.getUrl());
        System.out.println("Request Method: " + request.getMethod());
        System.out.println("Request Headers: " + request.getHeaders());
        System.out.println("Request Post Data: " + request.getPostData().orElse("No Post Data"));
      });

      devTools.addListener(Network.responseReceived(), responseReceived -> {
        Response response = responseReceived.getResponse();
        System.out.println("Response URL: " + response.getUrl());
        System.out.println("Status Code: " + response.getStatus());
        System.out.println("Response Headers: " + response.getHeaders());
        System.out.println("Response Body: " + devTools.send(
            Network.getResponseBody(responseReceived.getRequestId())).getBody());
      });

      finalDriver.get("https://www.google.com");
    }
  }
}

[jjqi92]

It worked, thank you very much for your help

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.