[🐛 Bug]: Zombie Chrome child processes remain after driver.quit() in containerized EKS environment causing "user data directory is already in use" errors

[Akarshit2211]

Description

In a Python Selenium application using ChromeDriver (running in a Docker container (ubuntu) deployed to EKS), I'm encountering an issue where child Chrome processes are not properly reaped when driver.quit() is called. This eventually causes a failure when attempting to start a new WebDriver session, with the following error:
session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir"
This issue does not happen when running the same container locally (e.g. Docker Desktop), but consistently occurs under load in EKS.

I am processing batches of URLs and at the start of every batch I am initializing the webdriver and at the end I am quitting the webdriver.

I saw the logs and it seems driver.quit(), which should kill the chromedriver as well as the chrome process created by it along with all the child processes (immediate chrome child should reap them), its not killing the child processes of the immediate chrome child of chromedriver, which are then becoming zombie processes.

Steps to reproduce

• Spin up a Docker container running ChromeDriver + Chrome + Selenium in headless mode.
• Process batches of URLs.
• Visit pages and scrap data.
• Call driver.quit() at the end of the batch or if encountered a webdriver exception.
• After a few cycles, Chrome fails to start due to "user data directory in use".
• On inspection, multiple zombie chrome and chrome_crashpad_handler processes remain.

Expected Behavior: All child processes of Chrome/ChromeDriver should terminate cleanly on driver.quit().

Observed Behavior:

• Zombie Chrome processes remain (verified via /proc or ps)
• Eventually Chrome refuses to start due to leftover lock on --user-data-dir

What I've tried:

• Providing a temporary unique --user-data-dir for every webdriver instance.
• Manually killing the chrome processes recursively and cleanly.
• Verified driver.quit() is called.
• Manually reaping orphaned child processes.

Might be a ChromeDriver/Chrome bug rather than Selenium, but reporting here first for triage.

I suspect this is related to the container's PID namespace not cleaning up child processes because of a missing init process (like tini).

Reproducible Code

# options being used when creating webdriver instance
options.add_argument("--headless")
prefs = {"profile.managed_default_content_settings.images": 2}
options.add_experimental_option("prefs", prefs)
options.add_argument("--disable-gpu") 
options.add_argument("--proxy-server='direct://'")
options.add_argument("--proxy-bypass-list=*")
options.add_argument("--start-maximized")
options.add_argument("--disable-plugins")
options.add_argument("--window-size=1920,1080")
options.add_argument("--allow-insecure-localhost")
options.add_argument("--no-sandbox")
options.add_argument(
    "--disable-dev-shm-usage"
)
options.add_argument("--incognito")
options.add_argument("--disk-cache-size=0")
options.add_argument(
    "--disable-extensions"
)
options.add_argument("--disable-popup-blocking")
options.add_argument("--ignore-ssl-errors=yes")
options.add_argument("--disable-application-cache")
options.add_argument("--disable-infobars")
options.add_argument("--ignore-certificate-errors")

Debugging Logs

2025-04-14 22:17:15,016 INFO [root]: restart_driver - Starting to restart driver
2025-04-14 22:17:15,017 INFO [root]: init_webdriver - Initializing WebDriver
2025-04-14 22:17:15,018 INFO [root]: init_webdriver - Webdriver already present, attempting to close it.
2025-04-14 22:17:15,018 INFO [root]: quit_driver_and_reap_children - Starting to close WebDriver
2025-04-14 22:17:15,019 INFO [root]: reap_child_processes - Reaped child: (0, 0)
2025-04-14 22:17:15,020 INFO [root]: check_remaining_processes - Checking for remaining Chrome-related processes
2025-04-14 22:17:16,025 INFO [root]: check_remaining_processes - Remaining Chrome-related processes after kill attempts:
2025-04-14 22:17:16,026 INFO [root]: check_remaining_processes - PID: 35 | Name: chromedriver | Status: sleeping | User: appuser | Memory: 0.1% | CPU: 0.0%
2025-04-14 22:17:16,028 INFO [root]: check_remaining_processes - PID: 41 | Name: chrome | Status: sleeping | User: appuser | Memory: 1.2% | CPU: 1.0%
2025-04-14 22:17:16,029 INFO [root]: check_remaining_processes - PID: 43 | Name: chrome_crashpad_handler | Status: sleeping | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:16,032 INFO [root]: check_remaining_processes - PID: 45 | Name: chrome_crashpad_handler | Status: sleeping | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:16,037 INFO [root]: check_remaining_processes - PID: 50 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:16,038 INFO [root]: check_remaining_processes - PID: 51 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:16,042 INFO [root]: check_remaining_processes - PID: 73 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.7% | CPU: 0.0%
2025-04-14 22:17:16,043 INFO [root]: check_remaining_processes - PID: 79 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.3% | CPU: 0.0%
2025-04-14 22:17:16,043 INFO [root]: check_remaining_processes - PID: 121 | Name: chrome | Status: sleeping | User: appuser | Memory: 1.3% | CPU: 0.0%
2025-04-14 22:17:16,045 INFO [root]: check_remaining_processes - PID: 122 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:16,046 INFO [root]: check_remaining_processes - PID: 148 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.7% | CPU: 0.0%
2025-04-14 22:17:16,047 INFO [root]: check_remaining_processes - PID: 339 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.8% | CPU: 0.0%
2025-04-14 22:17:16,050 INFO [root]: check_remaining_processes - PID: 369 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:16,051 INFO [root]: check_remaining_processes - Finished checking for remaining processes
2025-04-14 22:17:16,125 INFO [root]: init_webdriver - WebDriver closed successfully.
2025-04-14 22:17:16,126 INFO [root]: init_webdriver - Current Driver Value: <selenium.webdriver.chrome.webdriver.WebDriver (session="f4f4f05fdb0f87c85a524ad95d7331eb")>
2025-04-14 22:17:16,126 INFO [root]: init_webdriver - Attempt 1 to start WebDriver
2025-04-14 22:17:16,127 INFO [root]: init_webdriver - Starting WebDriver...
2025-04-14 22:17:16,127 INFO [root]: init_webdriver - Permissions for chromedriver: rrrrrrrrr
2025-04-14 22:17:16,589 INFO [root]: check_remaining_processes - Checking for remaining Chrome-related processes
2025-04-14 22:17:17,596 INFO [root]: check_remaining_processes - Remaining Chrome-related processes after kill attempts:
2025-04-14 22:17:17,597 INFO [root]: check_remaining_processes - PID: 43 | Name: chrome_crashpad | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,598 INFO [root]: check_remaining_processes - PID: 45 | Name: chrome_crashpad | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,598 INFO [root]: check_remaining_processes - PID: 50 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,599 INFO [root]: check_remaining_processes - PID: 51 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.6%
2025-04-14 22:17:17,599 INFO [root]: check_remaining_processes - PID: 73 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.6%
2025-04-14 22:17:17,599 INFO [root]: check_remaining_processes - PID: 79 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,600 INFO [root]: check_remaining_processes - PID: 121 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 1.3%
2025-04-14 22:17:17,600 INFO [root]: check_remaining_processes - PID: 122 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.6%
2025-04-14 22:17:17,601 INFO [root]: check_remaining_processes - PID: 148 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,601 INFO [root]: check_remaining_processes - PID: 339 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.6%
2025-04-14 22:17:17,601 INFO [root]: check_remaining_processes - PID: 369 | Name: chrome | Status: zombie | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,602 INFO [root]: check_remaining_processes - PID: 379 | Name: chromedriver | Status: sleeping | User: appuser | Memory: 0.1% | CPU: 0.0%
2025-04-14 22:17:17,602 INFO [root]: check_remaining_processes - PID: 385 | Name: chrome | Status: sleeping | User: appuser | Memory: 1.1% | CPU: 0.0%
2025-04-14 22:17:17,603 INFO [root]: check_remaining_processes - PID: 387 | Name: chrome_crashpad_handler | Status: sleeping | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,603 INFO [root]: check_remaining_processes - PID: 389 | Name: chrome_crashpad_handler | Status: sleeping | User: appuser | Memory: 0.0% | CPU: 0.0%
2025-04-14 22:17:17,603 INFO [root]: check_remaining_processes - PID: 394 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:17,604 INFO [root]: check_remaining_processes - PID: 395 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:17,604 INFO [root]: check_remaining_processes - PID: 415 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.6% | CPU: 0.0%
2025-04-14 22:17:17,605 INFO [root]: check_remaining_processes - PID: 416 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.3% | CPU: 0.0%
2025-04-14 22:17:17,605 INFO [root]: check_remaining_processes - PID: 441 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.6% | CPU: 0.0%
2025-04-14 22:17:17,605 INFO [root]: check_remaining_processes - PID: 453 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:17,606 INFO [root]: check_remaining_processes - PID: 479 | Name: chrome | Status: sleeping | User: appuser | Memory: 0.4% | CPU: 0.0%
2025-04-14 22:17:17,606 INFO [root]: check_remaining_processes - Finished checking for remaining processes
2025-04-14 22:17:17,607 INFO [root]: init_webdriver - WebDriver started successfully.
2025-04-14 22:17:17,607 INFO [root]: restart_driver - Ending restart driver


2025-04-16 02:08:04 - selenium.webdriver.remote.remote_connection - DEBUG - POST http://localhost:58297/session {'capabilities': {'firstMatch': [{}], 'alwaysMatch': {'browserName': 'chrome', 'pageLoadStrategy': <PageLoadStrategy.normal: 'normal'>, 'goog:loggingPrefs': {'performance': 'ALL'}, 'goog:chromeOptions': {'prefs': {'profile.managed_default_content_settings.images': 2}, 'extensions': [], 'args': ['--headless', '--disable-gpu', "--proxy-server='direct://'", '--proxy-bypass-list=*', '--start-maximized', '--disable-plugins', '--window-size=1920,1080', '--allow-insecure-localhost', '--no-sandbox', '--disable-dev-shm-usage', '--incognito', '--disk-cache-size=0', '--disable-extensions', '--disable-popup-blocking', '--ignore-ssl-errors=yes', '--disable-application-cache', '--disable-infobars', '--ignore-certificate-errors', 'user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.6367.91 Safari/537.3']}}}}
2025-04-16 02:08:04 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): localhost:58297
2025-04-16 02:08:12 - urllib3.connectionpool - DEBUG - http://localhost:58297 "POST /session HTTP/1.1" 500 0
2025-04-16 02:08:12 - selenium.webdriver.remote.remote_connection - DEBUG - Remote response: status=500 | data={"value":{"error":"session not created","message":"session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir","stacktrace":"#0 0x55f485c7053a \u003Cunknown>\n#1 0x55f48576bf00 \u003Cunknown>\n#2 0x55f4857a2c53 \u003Cunknown>\n#3 0x55f48579f4b6 \u003Cunknown>\n#4 0x55f4857ec7b6 \u003Cunknown>\n#5 0x55f4857ebe06 \u003Cunknown>\n#6 0x55f4857e0343 \u003Cunknown>\n#7 0x55f4857ad78a \u003Cunknown>\n#8 0x55f4857ae9de \u003Cunknown>\n#9 0x55f485c3a2cb \u003Cunknown>\n#10 0x55f485c3e242 \u003Cunknown>\n#11 0x55f485c277ac \u003Cunknown>\n#12 0x55f485c3edf7 \u003Cunknown>\n#13 0x55f485c0bb2f \u003Cunknown>\n#14 0x55f485c5f1a8 \u003Cunknown>\n#15 0x55f485c5f370 \u003Cunknown>\n#16 0x55f485c6f3b6 \u003Cunknown>\n#17 0x7f8a7203bea7 start_thread\n"}} | headers=HTTPHeaderDict({'Content-Length': '845', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
2025-04-16 02:08:12 - selenium.webdriver.remote.remote_connection - DEBUG - Finished Request
2025-04-16 02:08:12 - root - ERROR - init_webdriver - Error starting WebDriver: Message: session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Stacktrace:
#0 0x55f485c7053a <unknown>
#1 0x55f48576bf00 <unknown>
#2 0x55f4857a2c53 <unknown>
#3 0x55f48579f4b6 <unknown>
#4 0x55f4857ec7b6 <unknown>
#5 0x55f4857ebe06 <unknown>
#6 0x55f4857e0343 <unknown>
#7 0x55f4857ad78a <unknown>
#8 0x55f4857ae9de <unknown>
#9 0x55f485c3a2cb <unknown>
#10 0x55f485c3e242 <unknown>
#11 0x55f485c277ac <unknown>
#12 0x55f485c3edf7 <unknown>
#13 0x55f485c0bb2f <unknown>
#14 0x55f485c5f1a8 <unknown>
#15 0x55f485c5f370 <unknown>
#16 0x55f485c6f3b6 <unknown>
#17 0x7f8a7203bea7 start_thread

[selenium-ci]

@Akarshit2211, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[selenium-ci]

Hi, @Akarshit2211.
Please follow the issue template; we need more information to reproduce the issue.

Either a complete code snippet and URL/HTML (if more than one file is needed, provide a GitHub repo and instructions to run the code), the specific versions used, or a more detailed description to help us understand the issue.

Note: If you cannot share your code and URL/HTML, any complete code snippet and URL/HTML that reproduces the issue is good enough.

Reply to this issue when all information is provided. Thank you.

[titusfortner]

The issue is that the driver is closed during the quit method, but the failure to start the driver session is preventing a driver object from being instantiated, which means the quit can't be called. I created another issue to track this.

Also, you might see about using xvfb and/or fluxbox. This typo made the difference between when I was getting and when I stopped getting that user-data-dir error:

SeleniumHQ/seleniumhq.github.io@3347521#diff-9c5c607062298096d440cd6b49fc726cc23abc0bbdc87e25e013d312c2ec824bR45-R100

[Akarshit2211]

@titusfortner I have updated the Debugging logs and as you can see, webdriver is closed successfully after driver.quit(), but it should also have killed the child chrome processes which is not being done and they are becoming zombie processes.
Which is eventually leading to that "--user-data-dir already in use" error.
Have logged the processes before and after killing and restarting of webdriver.

[titusfortner]

I do not see where in those logs the quit method is getting sent. Once we merge #15636 and do a release, if that doesn't fix this, then we can reopen.

[cgoldberg]

@Akarshit2211 I just merged the fixes to the Python bindings (#15636) that might help with your issue.

We do a nightly release to TestPyPI... so if you want to test it tomorrow, it will be available at:
https://test.pypi.org/project/selenium

Otherwise, this will be in the 4.32 release.