Forums

Selenium headless: "Browser appears to have exited before we could connect"

I'm using python 3.6.

Here's the code I'm using:

display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Firefox()
try:


    browser.get('https://www.api.slack.com')
    print(browser.title)

finally:

    browser.quit()
    display.stop()

Any thoughts? I manually installed selenium 2 onto python3.6.

While we're at it... the eventual use-case I'm going for is using a proxy like BrowserMob with Selenium to capture sites that are accessed by an individual page. If we are unable to use proxies on pythonanywhere, it'd be nice to know before I spend too much time on this.

Which exact version of Selenium did you install? It needs to be 2.53.6 in order to work with the (somewhat old) version of Firefox we have available.

You should be able to use proxies on PythonAnywhere, but only from a paid account: free accounts are limited to accessing websites on our whitelist. Actually, that might also cause problems with your code as posted -- api.slack.com is on the whitelist, but www.api.slack.com isn't.

It's 2.53.6. I fixed the URL but that didn't change anything.

Is the exception coming from the browser.get line, or a different one?

It's from browser = webdriver.Firefox()

2018-02-13 00:27:14,934: Error running WSGI application
2018-02-13 00:27:14,941: selenium.common.exceptions.WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.
2018-02-13 00:27:14,941: 
2018-02-13 00:27:14,942:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1994, in __call__
2018-02-13 00:27:14,942:     return self.wsgi_app(environ, start_response)
2018-02-13 00:27:14,942: 
2018-02-13 00:27:14,942:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1985, in wsgi_app
2018-02-13 00:27:14,942:     response = self.handle_exception(e)
2018-02-13 00:27:14,942: 
2018-02-13 00:27:14,943:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1540, in handle_exception
2018-02-13 00:27:14,943:     reraise(exc_type, exc_value, tb)
2018-02-13 00:27:14,943: 
2018-02-13 00:27:14,943:   File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 33, in reraise
2018-02-13 00:27:14,943:     raise value
2018-02-13 00:27:14,943: 
2018-02-13 00:27:14,944:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1982, in wsgi_app
2018-02-13 00:27:14,944:     response = self.full_dispatch_request()
2018-02-13 00:27:14,944: 
2018-02-13 00:27:14,944:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1614, in full_dispatch_request
2018-02-13 00:27:14,944:     rv = self.handle_user_exception(e)
2018-02-13 00:27:14,944: 
2018-02-13 00:27:14,944:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1517, in handle_user_exception
2018-02-13 00:27:14,944:     reraise(exc_type, exc_value, tb)
2018-02-13 00:27:14,944: 
2018-02-13 00:27:14,945:   File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 33, in reraise
2018-02-13 00:27:14,945:     raise value
2018-02-13 00:27:14,945: 
2018-02-13 00:27:14,945:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1612, in full_dispatch_request
2018-02-13 00:27:14,945:     rv = self.dispatch_request()
2018-02-13 00:27:14,945: 
2018-02-13 00:27:14,945:   File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1598, in dispatch_request
2018-02-13 00:27:14,945:     return self.view_functions[rule.endpoint](**req.view_args)
2018-02-13 00:27:14,946: 
2018-02-13 00:27:14,946:   File "/home/analyticsforyourblog/mysite/flask_app.py", line 43, in get_tasks
2018-02-13 00:27:14,946:     browser = webdriver.Firefox()
2018-02-13 00:27:14,946: 
2018-02-13 00:27:14,946:   File "/home/analyticsforyourblog/.local/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 80, in __init__
2018-02-13 00:27:14,946:     self.binary, timeout)
2018-02-13 00:27:14,946: 
2018-02-13 00:27:14,947:   File "/home/analyticsforyourblog/.local/lib/python3.6/site-packages/selenium/webdriver/firefox/extension_connection.py", line 52, in __init__
2018-02-13 00:27:14,947:     self.binary.launch_browser(self.profile, timeout=timeout)
2018-02-13 00:27:14,947: 
2018-02-13 00:27:14,947:   File "/home/analyticsforyourblog/.local/lib/python3.6/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 68, in launch_browser
2018-02-13 00:27:14,947:     self._wait_until_connectable(timeout=timeout)
2018-02-13 00:27:14,947: 
2018-02-13 00:27:14,948:   File "/home/analyticsforyourblog/.local/lib/python3.6/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 99, in _wait_until_connectable
2018-02-13 00:27:14,948:     "The browser appears to have exited "

Are you running selenium from a console, or a scheduled task? It's not recommended to run it directly from a web app...

Ah, I'm running it just by visiting the web app page.

The end result I'm going for is that I'd like to build an API that gets data about a website that a user inputs, then returns it to my own site (a wordpress site). This has been incredibly difficult for me because I've only ever coded local apps, so all this tangle of servers and databases and web apps is killing me.

Anyway, maybe by telling you the problem I'm trying to solve we can find a good way to do that.

Also, I just tried a curl -i request on it and the same error occurred.

Just to confirm, when you do a curl -i, you see the same error? ie. "the browser appears to have exited"?

Yes, I get the same error

That sounds really weird. When you say that you got the same error, what did curl print out? It would be really strange if it printed out "The browser appears to have exited" -- it doesn't use a browser.

Also -- what do you get if you run which firefox from a bash console?

Oh, the curl command just spits out the standard "something went wrong" page html, it doesn't specify the error. When I check the error log, I see the same error.

22:55 ~ $ which firefox
/usr/local/bin/firefox

Are you sure you're looking at the right errors? The most recent error in the error log is at the bottom, where it's currently showing a last error of

2018-02-14 22:55:52,140: UnboundLocalError: local variable 'browser' referenced before assignment

It's worth noting that this is from two days ago.

Yes, I changed the code a bit to try another approach and it caused that new error. I reverted it and ran it just now. First from the website, and second from the console. You can see the errors now.