[🚀 Feature]: Support file downloads on the nodes

[xZero707]

Feature and motivation

Right now, the suggested solution for downloading files is mounting a volume, but this is not practical when the grid is running as a remote service.

Selenium provides an API for fetching remote files, and this feature had a recent revamp (see: PR SeleniumHQ/selenium#11277 and SeleniumHQ/selenium#11702).

Unfortunately, it seems like nothing of this is implemented in Selenium Docker, the place where this feature is very much needed.

Here are my unsuccessful attempts:

Chrome node

GET /session/:sessionId:/se/files
curl -X GET "http://172.17.53.2:4444/session/98473aa8661fc256bf7c0de98861005e/se/file?filename=100MB.bin"
Result = Unknown command

Firefox node

GET /session/:sessionId:/se/files
curl -X GET "http://172.17.53.2:4444/session/98473aa8661fc256bf7c0de98861005e/se/file?filename=100MB.bin"
Result = Unsupported method - expect POST for file upload

I also tried running this using PHP driver:

$file = $driver->executeCustomCommand('/session/:sessionId/se/file?filename=100MB.bin');

Result:

Facebook\WebDriver\Exception\UnknownCommandException : unknown command: unknown command: session/41d161b598c0fd3f85a4e09b1f5f5a1f/se/file?filename=100MB.bin`

This feature is documented, yet cannot be used.

I defined --downloads-path /usr/downloads using env SE_OPTS and nothing. Chrome is still downloading to /home/seluser/Downloads and ignoring the path I defined.

Version: Selenium 4.8.1 (revision 8ebccac989)

Usage example

Calling API like this would return JSON response with base64 encoded file data.

curl -X GET "http://172.17.53.2:4444/session/98473aa8661fc256bf7c0de98861005e/se/file?filename=100MB.bin"

Expected response:

{
    "filename": "100MB.bin",
    "contents": "ZXhhbXBsZSBiaW5hcnkgZGF0YQ=="
}

[github-actions]

@xZero707, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[github-actions]

This issue is looking for contributors.

Please comment below or reach out to us through our IRC/Slack/Matrix channels if you are interested.

[patilsha]

Can someone please take this issue to include the fix in selenium-docker?
Thanks

[diemol]

This is already working https://www.selenium.dev/documentation/grid/configuration/cli_options/#enabling-managed-downloads-by-the-node

You can pass the flag via SE_OPTS or send us a PR to set the flag via an environment variable.

[xZero707]

@diemol With Selenium 4.8.1, this flag was not recognized by node. As of 4.8.2 it works, but there is no reference to commit that fixed this. Anyway, now that I have it running, I'll test it, and get back with the results.

[xZero707]

I can confirm that this is working as intended, although making it work was challenging. This is what it takes to enable this with php-webdriver/php-webdriver on Chrome.

<?php

use Facebook\WebDriver\Chrome\ChromeOptions;
use Facebook\WebDriver\Remote\DesiredCapabilities;

$desiredCapabilities = DesiredCapabilities::chrome();
$desiredCapabilities->setCapability('se:downloadsEnabled', true);
$desiredCapabilities->setCapability(ChromeOptions::CAPABILITY_W3C, [
    'prefs' => [
        'se:downloadsEnabled' => true
    ]
 ]);

Without ChromeOptions::CAPABILITY_W3C, GET would return empty array of files and POST would fail saying that file doesn't exist.

Without $desiredCapabilities->setCapability('se:downloadsEnabled', true); I would get error:

Cannot find downloads file system for session id: XXXXXXXXXXXX

Documentation doesn't state that ChromeOptions::CAPABILITY_W3C is required too.

---

And this is how to implement it since php-webdriver/php-webdriver has no implemented methods for this:

<?php

/**
* List downloaded files
* Example: {"names":["1MB.bin","5MB.bin"]}
*/
$driver->executeCustomCommand('/session/:sessionId/se/files');

/**
* Download desired file
* Eg. {"filename":"1MB.bin","contents":"UEsDBBQACAgIAEFN..."}
* Eg. {"filename":"5MB.bin","contents":"UEsDBBQACAgIAMpNl..."}
*/
$driver->executeCustomCommand('/session/:sessionId/se/files', 'POST', [
    'name' => '1MB.bin'
]);

Anyhow guys, well done. Very glad to finally have this working.

[patilsha]

I am still getting error as below

*"message":"unknown command: unknown command: session/87c6bf631786eb72a16b3a152d0afa93/se/files*

I tried the same scenario to list downloaded files with below detail
I am using C# code to run our selenium scripts on Selenium Grid on Docker
WebDriver version = 4.8.2
Selenium Grid version = 4.8.3
Chrome version = 104
Below are the settings for Chrome Options (There is no DesiredCapability class available to set the capabilities in C# WebDriver)
ChromeOptions options = new ChromeOptions();
options.AddUserProfilePreference("se:downloadsEnabled", true)

I am making a HTTP Get call to list down the all downloaded files on particular node.
http://{HubIPAddress}:4444/session/87c6bf631786eb72a16b3a152d0afa93/se/files

Can someone please help how can I use this download functionality in C# Selenium Grid on Docker?
I'm suspecting the capability has not been selected correctly.
Thanks

[diemol]

I will close this issue since the main requirement is completed. @patilsha please open a new one with enough information to reproduce the issue.

[patilsha]

Thanks @diemol I've raised a new issue as below
#1831

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.