[🐛 Bug]: "unable to discover open pages" with headless and user-data-dir

[drheld]

What happened?

I recently started getting an error when trying to create a webdriver:
selenium.common.exceptions.WebDriverException: Message: unknown error: unable to discover open pages

It seems like this happens if the two following settings are set:

• headless = True
• adding an argument with a --user-data-dir

If I turn off either one of those options then it works, but if both are set I get the error specified above.

Sample code snippet which is consistently reproducing on my machine attached below as well as full log output.

Any ideas on how to debug this would be appreciated. Thanks!

How can we reproduce the issue?

#!/usr/bin/python3

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService

service = ChromeService(executable_path='/tmp/chromedriver')
options = webdriver.ChromeOptions()
options.headless = True
options.add_argument('--user-data-dir=/tmp/data')
driver = webdriver.Chrome(service=service, options=options)
driver.close()

Relevant log output

14:02 drheld ~ python test.py 
Traceback (most recent call last):
  File "/home/drheld/test.py", line 10, in <module>
    driver = webdriver.Chrome(service=service, options=options)
  File "//home/drheld/.local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__ 
    super().__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/home/drheld/.local/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__
    super().__init__(
  File "/home/drheld/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 277, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/drheld/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/drheld/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/home/drheld/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: unable to discover open pages
Stacktrace:
#0 0x560d60699d93 <unknown>
#1 0x560d604682d7 <unknown>
#2 0x560d604952f4 <unknown>
#3 0x560d6049017b <unknown>
#4 0x560d6048ca3d <unknown>
#5 0x560d604d14f4 <unknown>
#6 0x560d604c8353 <unknown>
#7 0x560d60497e40 <unknown>
#8 0x560d60499038 <unknown>
#9 0x560d606ed8be <unknown>
#10 0x560d606f18f0 <unknown>
#11 0x560d606d1f90 <unknown>
#12 0x560d606f2b7d <unknown>
#13 0x560d606c3578 <unknown>
#14 0x560d60717348 <unknown>
#15 0x560d607174d6 <unknown>
#16 0x560d60731341 <unknown>
#17 0x7f59d5a07fd4 <unknown>

Operating System

Debian rodete 20230126.02.03RD

Selenium version

Python 3.10.9

What are the browser(s) and version(s) where you see this issue?

Chrome 110.0.5481

What are the browser driver(s) and version(s) where you see this issue?

Chromedriver 110.0.5481 google-chrome

Are you using Selenium Grid?

No response

[github-actions]

@drheld, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[titusfortner]

This is a known chromedriver bug.
One suggestion is to try the new headless mode: --headless=new (see https://www.selenium.dev/blog/2023/headless-is-going-away/)

[github-actions]

Hi, @drheld.
This issue has been determined to require fixes in ChromeDriver.

You can see if the feature is passing in the Web Platform Tests.

If it is something new, please create an issue with the ChromeDriver team.
Feel free to comment the issues that you raise back in this issue. Thank you.

[drheld]

Thanks, Titus! That solves it.

[rfrancois]

I had the exact same problem and @titusfortner's solution worked for me too

[williamdes]

Same bug at Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1031114#24
It did show up when they tried to migrate from 109.0.5414.119-1 to chromium 110.0.5481.77-2

[toanutt]

My code got an error and I thought my code was wrong. But it turned out it wasn't.

I had the exact same problem and @titusfortner's solution worked for me too.

[williamdes]

there is more here is what was say on the Debian bug above:

I found a fix has been committed upstream to chromium driver:

https://chromium.googlesource.com/chromium/src/+/b8dc3864aa0ad1048d43353e681bb9d95d7a4a83%5E%21/
https://chromium-review.googlesource.com/c/chromium/src/+/4238878

The bug reports (https://bugs.chromium.org/p/chromedriver/issues/detail?id=4357 and https://bugs.chromium.org/p/chromium/issues/detail?id=1414672) say:

"It is going to land in branch 112.
Appx. 1 - 2 weeks after it (needed for testing) we will back marge it to the branch 111 and, possibly, to the branch 110"

"As a quick workaround you can try passing "--headless=new" instead of "--headless".'

[toanutt]

I run selenium install add extensions.
Initially, running was no problem.
But after a period of time, an error occurs:

selenium.common.exceptions.SessionNotCreatedException: Message: session not created: cannot process extension #1 from unknown error: cannot unzip

Does anyone know how to handle this bug? Many thanks