Forums

Having persistent issues with Chrome

Howdy, I asked this in a different thread but the issue persists and I have more error messages now. Basically every hour I kick off a process to get some data from another web page using Chrome. Here is how I instantiate the browser:

def openWebDriver():

    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--disable-gpu")
    chrome_options.add_argument('--disable-dev-shm-usage')
    web_driver = webdriver.Chrome(options=chrome_options)

    return web_driver

The process continues to fail and the error messages are below. I have updated to the current image and nothing had changed in my code ... so I am sort of at a loss. Any ideas?

Here are the error messages for three consecutive hours:

2023-01-25 14:05:33,316: Exception on /scrape [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2077, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1525, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/dreamdrivesforkids/mysite/dashboard.py", line 425, in scrape
    all_records = processor._bat_get(row['search'] + _bat_limit, row['vehicle'], row['model_year'])
  File "/home/dreamdrivesforkids/mysite/processor.py", line 626, in _bat_get
    web_driver = openWebDriver()
  File "/home/dreamdrivesforkids/mysite/processor.py", line 25, in openWebDriver
    web_driver = webdriver.Chrome(options=chrome_options)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__
    RemoteWebDriver.__init__(
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from unknown error: unhandled inspector error: {"code":-32000,"message":"Target crashed"}
  (Session info: headless chrome=90.0.4430.212)
Stacktrace:
#0 0x562bd4f7de89 <unknown>
**NO MATCH**
2023-01-25 15:05:26,435: Exception on /scrape [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2077, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1525, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/dreamdrivesforkids/mysite/dashboard.py", line 425, in scrape
    all_records = processor._bat_get(row['search'] + _bat_limit, row['vehicle'], row['model_year'])
  File "/home/dreamdrivesforkids/mysite/processor.py", line 626, in _bat_get
    web_driver = openWebDriver()
  File "/home/dreamdrivesforkids/mysite/processor.py", line 25, in openWebDriver
    web_driver = webdriver.Chrome(options=chrome_options)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__
    RemoteWebDriver.__init__(
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: unable to discover open window in chrome
  (Session info: headless chrome=90.0.4430.212)
Stacktrace:
#0 0x55c7071cbe89 <unknown>
**NO MATCH**
2023-01-25 16:05:33,362: Exception on /scrape [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2077, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1525, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/dreamdrivesforkids/mysite/dashboard.py", line 425, in scrape
    all_records = processor._bat_get(row['search'] + _bat_limit, row['vehicle'], row['model_year'])
  File "/home/dreamdrivesforkids/mysite/processor.py", line 626, in _bat_get
    web_driver = openWebDriver()
  File "/home/dreamdrivesforkids/mysite/processor.py", line 25, in openWebDriver
    web_driver = webdriver.Chrome(options=chrome_options)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__
    RemoteWebDriver.__init__(
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from tab crashed
  (Session info: headless chrome=90.0.4430.212)
Stacktrace:
#0 0x56363b84fe89 <unknown>
**NO MATCH**
Beloved premium user dreamdrivesforkids | 6 posts | Jan. 25, 2023, 4:19 p.m. | permalink

Are you sure you're not ending up with many chrome instances running? If you are not cleaning up the chrome processes (including when there is an exception), then you'd get that behaviour.

Staff glenn | 9718 posts | PythonAnywhere staff | Jan. 25, 2023, 6:43 p.m. | permalink

Thanks Glenn. I have web_driver.quit() at the end of every function that opens Chrome but maybe I am missing something. Is there a command I can run from bash to show how many instances of Chrome are running (and to kill. them?)

Please let me know.

Beloved premium user dreamdrivesforkids | 6 posts | Jan. 25, 2023, 8:44 p.m. | permalink