Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

@luckydog It's done for you

Thanks! Is there anything I need to do to set it up?

No problem. To use Chrome, you'll need to upgrade Selenium for your account -- for example, if you're using Python 3.7, run this in Bash:

pip3.7 install --user --upgrade selenium

...and then you can run Selenium with Chrome using code like this:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))

finally:
    browser.quit()

I upgraded Selenium and have the code in my program to run it (it runs from my terminal), but it still doesn't start up. Is there anything I'm missing?

How exactly are you running it? Do you get any errors?

There's a button on the home.html page that calls a function in Flask - the first part of the function is running fine, then it skips Selenium and flashes the message that nothing happened.

Add some logging to your code to see what happens.

I get this error - I know the element is present on the page I'm visiting and have tested dozens of times via my terminal. It seems like the browser isn't opening the link? I looked into some other forums on this issue- is it possible the site hasn't been whitelisted for access?

Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="email_address"]"} (Session info: headless chrome=78.0.3904.70) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

If by "my terminal", you mean from your local machine not on PythonAnywhere, then it is entirely likely that the site is not on the whitelist.

Ah ok would it be possible to add the site to the whitelist? It's eversign.com

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

Hi @epireve -- sure, that's done for you.

Thanks! Anything I need to do on my end so that change is registered?

Oh, sorry, @luckydog -- must've overlooked your post. We are happy to whitelist any public API if you provide us with the api endpoint and the official documentation hosted on the same domain.

what should I done regarding this error? if you need to enable something regarding this error, just do it. thank you

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
 (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

hi- you already have our new virtualization stuff enabled.

so if you recently got this enabled, then make sure you are running your code from a new console / reloaded webapp etc.

also, if you simplify everything and follow this, does it run?

my code running on Jupyter notebook PyA, after refresh and reload the kernel, it does not work. any solution?

Currently our new virtualization feature doesn't work in Jupyter notebooks.

virtualisation features just support on console?

currently, I already can run chrome drive in console, however, I cannot found the downloaded file, how to found the file beside can we set the download directory? by default what is the download directory? tq

The new virtualization system works in consoles, scheduled and always-on tasks, and in website code, but not in Jupyter Notebooks or SSH -- when it comes out of beta we will add support for the last two.

The code in this Stack Overflow answer shows how to set Chrome's download path -- you don't need to set the ChromeDriver path, though.

@pafk Sorry for the delay! My endpoint API is 7c75f93c3fbb016a7dfa00e4c5db3116 and the documentation is here: https://eversign.com/api/documentation

I've added api.eversign.com to the whitelist.

Thanks! I'm still getting an error where Selenium doesn't actually open anything - here's the error: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="email_address"]"} (Session info: headless chrome=78.0.3904.70) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

Try using Selenium's screenshot functionality to see what the page looks like just before you try to get that element -- the function is get_screenshot_as_file on your webdriver object, and the one parameter is the full path to a filename (ending in ".png") where it should store the screenshot.

I put the code in two places but it hasn't saved anything. It seems like selenium isn't even launching? Here's my code:

def login(template, email, password):

        chrome_options = webdriver.ChromeOptions()
        chrome_options.add_argument("--headless")
        chrome_options.add_argument("--disable-gpu")
        driver = webdriver.Chrome(options=chrome_options)
        driver.get_screenshot_as_file("/home/luckydog/ldar_contracts/selenium_screenshot.png")
        driver.get(template)
        driver.get_screenshot_as_file("/home/luckydog/ldar_contracts/selenium_screenshot.png")

[edit by admin: formatting]

Does the code print anything out when you run it -- for example, any error messages?

It prints this error message: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="email_address"]"} (Session info: headless chrome=78.0.3904.70) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

That must be coming from different code, as the code that you posted in your previous message does not try to access any field by the ID email_address.

Here is the rest of the code - what I pasted previously was the code initializing the driver and navigating to the login page.

        driver.get_screenshot_as_file("/home/luckydog/ldar_contracts/selenium_screenshot.png")
        driver.set_window_position(0,0)
        driver.set_window_size(500,500)

        emailField = driver.find_element_by_xpath('//*[@id="email_address"]')
        passField = driver.find_element_by_xpath('//*[@id="password"]')
        emailField.send_keys(email)
        passField.send_keys(password)

Can we take a look at your code? We can see it from our admin interface, but we always ask for permission first.

That’d be great, thanks!

I see that you have two functions called login -- are you sure you've put the screenshot code in the correct one?

Whoops sorry about that - good catch, just deleted the other login function and still having the same problem and not getting a screenshot.

Are you sure that the code to take the screenshot is being run? If you're doing this in a web app, have you reloaded it after adding the screenshot code? If you're doing this in a console, you can add prints around the screenshot code to make sure that's the code that you're running, if you're in a web app, prints to stderr will appear in your error log.

Also, I see that you're changing the window position and size, perhaps that's interfering with the screenshots.

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

Hi @hhlimhh73, we're currently in the process of enabling the new virtualization system for all accounts on PythonAnywhere, which unfortunately may take a week or two to complete. We'll let you know when that has been done for your account; I've made a note to make sure that you're in one of the first batches to be moved over to it.

Hi @hhlimhh73 -- we've enabled virtualization feature for you. Here are the next steps:

To use Chrome, you'll need to upgrade Selenium for your account -- for example, if you're using Python 3.7, run this in Bash:

pip3.7 install --user --upgrade selenium

...and then you can run Selenium with Chrome using code like this:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))

finally:
    browser.quit()

hi can i get chromedriver enabled too?

Hi, @vegarian, you've got that already enabled. Do you get any errors? Have you followed the steps mentioned above?

@pafk can i use pyperclip on virtual environment?

Don't think so, there's no clipboard on PythonAnywhere, it's a server environment.

how to enable chromedriver for my account?

@stocktoday You already have everything enabled. You need to upgrade selenium if you want to use it. Take a look at post above

Hi @fjl , could you enable it for my account as well? Thanks!

Hi @the0919 -- you've got that already enabled. Please look at @fjl post above for further instructions.

Hi, could you enable chromedrive for my account?

@aiusko It's already enabled for your account.

Hi! Could you please enable Selenium Chromedriver for my account? Thank you

@tylerbolz31 You already have everything enabled. You need to upgrade selenium if you want to use it. Take a look at post above

Could you enable chrome for my account so that I can run it on headless mode

you are already on the latest system image and have chrome enabled. You need to upgrade selenium if you want to use it.

Hi I am getting the following error for selenium code. Its working fine locally. But it has something issue in pythonanyhwhere. Please help me out .

(unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1029-aws x86_64)

Does it happen always in some circumstances or just from time to time?

Hi Can you please enable selenium and chromedriver for my account. Please Thanks

They are already enabled for your account.

But whats the executable path for the chrome webdriver. Please tell me asap. Thanks

@vedaang -- you can find it using which chromedriver command.

@pafk I runned the command and thanks for the help. It saved my time. Thanks

Glad to hear that you made it!

hi how Can I using selenium>3 please

Good night, hope you are all doing well and staying safe and healthy.

I am trying to run a simple code that checks a web page for an appointment and sends an email with the data collected.

It uses selenium wedriver (Chrome) and the python smtp packages.

When trying to run the code, I am getting the following message:

driver = webdriver.Chrome(options=chrome_options)
TypeError: init() got an unexpected keyword argument 'options'

This is the part of the code to run headless Chrome.

Could you please help me fix this?

Thanks in advance, best regards.

@reza16, @pydevelper -- you need to upgrade Selenium using this command:

pipX.X install --user --upgrade selenium (where X.X is the Python version you're using in your app, e.g. 3.8).

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

It's already installed and enabled -- you just need to follow the steps in the post above.

Hi, every time I try to run selenium with chrome I got the below error. Why would you think it happens?

*(process:255): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
Current firefox version is 17.0
Get LATEST driver version for 17.0
Driver [/home/lancia/.wdm/drivers/geckodriver/linux64/v0.29.1/geckodriver] found in cache
Traceback (most recent call last):
  File "quickstart.py", line 18, in <module>
    session = get_session()
  File "quickstart.py", line 13, in get_session
    session = InstaPy(username=insta_username,
  File "/home/lancia/.local/lib/python3.8/site-packages/instapy/instapy.py", line 325, in __init__
    self.browser, err_msg = set_selenium_local_session(
  File "/home/lancia/.local/lib/python3.8/site-packages/instapy/browser.py", line 142, in set_selenium_local_session
    browser = webdriver.Chrome(options=chrome_options)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
    RemoteWebDriver.__init__(
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1029-aws x86_64)*

[edit by admin: formatting]

Hi, I need to get this site "https://ip4.seeip.org/geoip" in my code but it seems that it's not in whitelist I want you to add this in whitelist if it's possible. thank you

Regarding the first error -- that's a really confusing one, as it appears to be using both Chrome and Firefox (see the start of the error for that one). However, it certainly does seem to be trying to start Chrome, and is failing. Do you have lots of Chrome processes running, perhaps? You can see your running processes in the table at the bottom of the "Consoles" page.

Regarding the whitelist request -- we can whitelist sites if they provide an official public API. Just give us a link to the API documentation.

Thank you for your reply :) I solved above problems but I have a new error now and I don't know why

InstaPy Version: 0.6.13 .. .. .. .. .. .. .. .. Workspace in use: "/home/lancia/InstaPy" OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO INFO [2021-06-06 09:54:49] [*] Session started! oooooooooooooooooooooooooooooooooooooooooooooooooooooo INFO [2021-06-06 09:54:50] [*] -- Connection Checklist [1/2] (Internet Connection Status) WARNING [2021-06-06 09:54:57] [*] - Internet Connection Status: error ...................................................................................................................... CRITICAL [2021-06-06 09:54:57] [*] Unable to login to Instagram! You will find more information in the logs above. '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' INFO [2021-06-06 09:54:59] [**] Sessional Live Report: |> No any statistics to show [Session lasted 20.5 seconds] OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO INFO [2021-06-06 09:54:59] [*] Session ended! ooooooooooooooooooooooooooooooooooooooooooooooooooooo*

You'll need to contact the author of the module that you're using to find out more about what that error means. However, if you're trying to write some kind of bot to interact with Instagram, it might just be that they are blocking incoming connections from cloud environments like PythonAnywhere; in general, they do not want bots on their platform.

Is there any way to run chromedriver but not headlesly. Could you please tell me how to do that.

You could do that by using a virtual display, but I'm a bit confused about why you would want to -- remember, there's no display on PythonAnywhere, so you still wouldn't see anything. The virtual display would be invisible, so the result would be the same as running Chrome headlessly.

hello, can you please enable selenium chrome driver support for my account after pip3.7 install --user --upgrade selenium command in bash I am getting syntax error at 3.7

hi- what is the exact error? and you are getting it on the pip3.7 install command? or on a later command?

Hey Thanks I checked it is already activated for me

Could you please enable Selenium Chromedriver for my account? Error message is " TypeError: init() got an unexpected keyword argument 'options'"

I resolved problem after upgrade selenium. thanks.

You should be also able to fix it by upgrading the system image (see here for details).

import time from flask import Flask from selenium import webdriver from selenium.webdriver.chrome.options import Options

app = Flask(name)

@app.route('/') def hello_world(): return 'Hello from Aryan!'

print('hello') chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--headless") chrome_options.add_argument("start-maximized") chrome_options.add_argument('--no-sandbox') chrome_options.add_argument("disable-infobars") chrome_options.add_argument("--disable-extensions") chrome_options.add_argument('--disable-dev-shm-usage') chrome_options.add_argument("--disable-gpu") driver = webdriver.Chrome(options=chrome_options) driver.maximize_window() chrome_options = webdriver.ChromeOptions()

browser = webdriver.Chrome(options=chrome_options)

try: browser.get("https://www.google.com") print("Page title was '{}'".format(browser.title))

finally: browser.quit() driver.close()

this is my code and when i try to run this code i am getting this error

selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: -5

At what point is it failing?

hello, what if i get this error: i upgraded my account.

    driver.get('**the site im trying to get**')
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from tab crashed
  (Session info: headless chrome=90.0.4430.212)

[formatted by admin]

Do you run multiple instances of that code? That error could mean that you are reaching the limit of processes. Chrome spawns multiple processes.

Hi I am getting below error . Please help to fix it. Traceback (most recent call last): File "/home/shivanshuv/Chartink2.py", line 19, in <module> Todays_stocks = get_R2(url) File "/home/shivanshuv/Chartink2.py", line 15, in get_R2 table=pd.read_html(driver.find_element_by_xpath('//*[@id="DataTables_Table_0"]').get_attribute('outerHTML')) AttributeError: 'Options' object has no attribute 'find_element_by_xpath'

looks like a coding error

hi, could you please enable Selenium chromedriver for me? How do I install chromedriver on this website? Do I just upload webdriver to my file? which os version should I be choosing. Thank you very much. Sorry I'm new to this.

@cpatonn -- it's already enabled for all users. You don't need to install chromedriver. For more details, see this help page.

Hi, I'm running from a virtual environment and I have this ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv. error trying to run pip3.7 install --user --upgrade selenium Will selenium work if I install outside my virtual environment

--user is a flag for installing specifically outside of a virtualenv. If you want to install into the virtualenv, activate it and then do the install without the --user flag.

Can you enable for me also? Thank you in advance!

You're using an older system image. If you want to use Chrome, you can update your system image from your Account page.

Thank you! Just to see clear change "fishnchips" to "glastonbury" and that's all? It means the selenium will work and all saved codes will stay, so for my account everything will stay?

Everything will be left as it is. Be aware of the warnings about re-creating virtualenvs and re-installing packages that is shown when you update. Also, make sure that you use a new console that you create after you have upgraded.

Can you enable for me also? Thank you in advance!

It's enabled for everyone. See https://help.pythonanywhere.com/pages/selenium/

selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED

i have the following error using selenium, what i can do with it?

How do you run it?

.

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
driver = webdriver.Chrome(options=chrome_options)
driver.get('https://google.com')
driver.save_screenshot("s.png")
driver.close()

okay, i have 2 account: one is free and another one no. when i run this code using free account all is fine, but with another one i have error:

driver = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

what about ERR_TUNNEL_CONNECTION_FAILED:

i think the problem is the site i try to access is not in whitelist(https://app.elaisian.com/login), but this platform don't have an official API, so i think its kinda impossible to get access

Is your second account on the most recent system image? The error you get looks like an old version of selenium (check our help page for more info). On free account you will not be able to scrape since free accounts have internet access restricted to the whitelisted domains.

hi! thank you, now it works! last question - what is default download folder if i use chrome to download file with selenium? because i can see that i have less free space but cant find downloaded files

If you haven't specified the download location, I'd suppose the file would be saved in the running script location. You may also scan your directories (and /tmp as well) with command like find to look for the "suspects".

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

It is already enabled for all accounts by default.

Hi, I try to use chromedriver for "aliexpress but I getting this error message :

Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED
 (Session info: headless chrome=90.0.4430.212)

when I try with google website its works....

this is my code:

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = webdriver.Chrome(options=options)

driver.get("https://www.google.com")
print('x')
driver.get("https://m.aliexpress.com/item/1005002272194704.html")
print('check2')
content = driver.page_source  #the html
soup = BeautifulSoup(content, 'html.parser')
link = soup.find('a', attrs={'class': '_9tla3'})
link = link.get('href')
driver.close()
return 0

Free accounts have restricted internet access. If you want to scrape non-whitelisted domains, you need to upgrade your account.

I am having trouble accessing one website. When I run the very simple code below which I copied from your forum I successfully access google, but get the error below when it moves to https://www.ghin.com/login. It works fine when I try to access that site using PyCharm

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))
    browser.get("https://www.ghin.com/login")
    print("Page title was '{}'".format(browser.title))
finally:
    browser.quit()

I get the following error

Page title was 'Google'
Traceback (most recent call last):
 File "/home/hommeswj/Ghin_test.py", line 9, in <module>
 browser.get("https://www.ghin.com/login")
 File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in 
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page 
crash
 from tab crashed
   (Session info: headless chrome=90.0.4430.212)

Does that happen every time you run the code?

Hi, I am trying GET requests with Selenium and WebDriverWait. Most of the time it doesn't work because the WebDriverWait is "timeout" (I tried long time). I think it have some problems with open the chrome driver... how can I fix it and make it more stable?

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
from datetime import datetime
import time
from selenium.webdriver.chrome.options import Options
from couchbase.exceptions import TimeoutException
# from aliexpress_api_client import AliExpress

PRODUCTS_NUM = 3
TRIES_NUM = 500




def find_product_properties(product='iphone x'):
    print('in')
    #first request - find products in the web
    #defind the headless browser
    weight = ''
    options = Options()
    options.add_argument('--headless')
    options.add_argument('--disable-gpu')
    driver = webdriver.Chrome(options=options)
    driver.get("https://he.aliexpress.com/wholesale?&SearchText="+ product) #get request
    print('b')
    try:
        WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.CLASS_NAME, "_9tla3")))
        content = driver.page_source  #the html
        soup = BeautifulSoup(content, 'html.parser')
        link_lst = soup.findAll('a', attrs={'class': '_9tla3'})
        img_lst = soup.findAll('img', attrs={'class': 'A3Q1M'})
    except TimeoutException as exception:
        print('timeout')
        raise exception
    except:
        print('faild ali web reload')
        driver.quit()
        return {}
    print('good')
    driver.quit()

I get the error every time I try to access the GHIN website

@WEIGHTforme -- we've already replied you in a different topic.

@hommeswj what kind of error do you get? And how are you trying to access this website?

I run the code below in PythonAnywhere simply by hitting Run when the file is open.

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))
    browser.get("https://www.ghin.com/login")
    print("Page title was '{}'".format(browser.title))
finally:
    browser.quit()

The complete error message is in my post above, but basically

raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page 
crash

When I check on-line it says the problem is there is not enough memory allocated to the Docker container to fully load a web page. Not sure what this means but is there a way to fix it?

There aren’t a lot of causes for a webpage crash. And if you’re using Docker, Bayesian reasoning should tell you that the described error can probably be solved with the solution below.

Here’s what’s happening: there’s not enough of (shared) memory allocated to the Docker container to fully load a web page. That’s because modern web applications tend to be very RAM-intensive. The default allocated memory of 64MB might simply not be enough.

You can adjust the shared memory when running/composing a Docker container.

There is no docker involved in that, and the RAM limit is 3GB, so it should not be the case. It looks like some problem related to that specific page.

Thanks. It works when I run it on my own computer , just doesn't work with Pythonanywhere, So there appears to be some issue associated with this application. But I'll have to live without it.

I'm using selenium often but my code crashes at random times. So, sometimes it goes well for a while, sometimes I get the error quickly. It seems not to be related to a specific website. When I run the code on my local machine, everything works well.

This is the error: selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

I'm using it as you instructed: chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--headless") chrome_options.add_argument("--disable-gpu") browser = webdriver.Chrome(options=chrome_options)

What can you/I do to prevent those crashes?

Make sure that you're always closing the browser when you're done with it. The exception handling in our example is as important as the arguments to the browser.

I have installed selenium but how do I know if I have access to the chromedriver?

Try the code here https://help.pythonanywhere.com/pages/selenium/ and see if it works.

Thanks! Obviously something else going wrong with my set up, cheers for that!

Hi, just came across this thread. I am attempting the sample code after upgrading selenium through bash console. I getting this error:

Traceback (most recent call last): File "/home/htxshuttersllc/tryit.py", line 5, in <module> browser = webdriver.Chrome(options=chrome_options) TypeError: init() got an unexpected keyword argument 'options'

Any help is appreciated

Consider updating your system image. See https://help.pythonanywhere.com/pages/ChangingSystemImage/

Magic! Thanks :)

Let us know if it solved your problem.

Solved! Sample code works

Glad to hear that!

Hi, I'm new and came to pytonanywhere because I need someting to run my little scraping scrip on. (for now, maybe will use pa for future paid projects)

Am I restricted in websites on my freeaccount? my script is right there bugging where seleniums loads the URL.

How can I achieve that my script runs? Thanks Fabs

@Fabs See https://www.pythonanywhere.com/whitelist/

Hi @PythonAnywhere staff, thanks for answering. Unfortunately its not included. Can I request two for additional whitelisting?

Send us a link to the API documentation for the sites and we'll consider them for the whitelist.

Hey can you please enable selenium for my account?

It's there. No need to enable anything. See https://help.pythonanywhere.com/pages/selenium/

I use Selenium with Chromedriver, I understand that chromedriver is already included and the scripts run well, but it is posible to upload my own version of chromedriver and use it in Chrome_options? (for Linux, obvious, but what version?)

It's probably possible, but why do you want to do that?

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

It is already enabled for all new accounts on PythonAnywhere.

Hi! How can i upgrade chromedriver? i have problem with my Selenium script since few months and probably it's problem with chromedriver i think that because when i run same script with in Google Compute Service it works well with latest chromedriver it works well. Also it works well on my local PC. Maybe you have any other idea what is cause.

We will have an updated chromedriver in our next system image, but we do not know when that will be ready.

Could you enable chromedrivers for my account ? Thank you so much !!

Could you enable chromedrivers for my account ? Thank you so much !!

It is already enabled for all accounts on PythonAnywhere.

Good afternoon, I keep getting "The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed". Once I get this error the only way around it is to close python and start a new session. The only problem is I am using this on a nightly task that runs about 4 hours long, and sometimes has finished sometimes does not. I have added time for waiting the pages to load, I have upgraded/downgraded, reinstalled, added quitting the browser completely after a few pages, though I have found it can stop working after even just one page it does not seem to like. It is random in that it is not the same pages every time. There does not seem to be a remedy I have added try/except handling, but once this exception occurs no more pages will be loadable at any point in the code, sot the task needs to be restarted. Is there anything should do that might help?

How do you run your nightly task? When you try/except, do you create a new instance?

Hello and yes, I do. I figured it out thank you. I cannot believe it took me so long to solve this and one line of code fixes it. For anyone else having the same issue.

I fixed my issue by adding:

import signal

signal.signal(signal.SIGCHLD, signal.SIG_IGN)

Because python-anywhere limits the amount of processes you can have running at once, if you are scraping numerous sites and using driver.quit() or closing chrome through a bash command, python (as the parent process) will retain some defunct processes for every chrome process that was open. If you do this enough time you will have too high a process count and chromedriver will refuse to open.

Running the 2 lines of code will ignore the default setting of retaining child processes until the parent is closed. However, if for some reason you need to retain these defunct processes, then this will cause other problems for you. It works for me, hope it works for you.

Glad to hear that you made it work!

Hello! It seems, that despite api.zoom.us is already in the white list, the registration links for Zoom-meetings are still restricted to open. I have tried one in the next format: "https://[company].zoom.us/webinar/register/[meeting-id]", ended with an error: selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED (Session info: headless chrome=90.0.4430.212)

We whitelist API endpoints, if you want to access non-API urls, you need to upgrade your account.

Hello, I want to run the code below and it works fine on my local machine. But pythonanywhere shows an error.

my code:

import seleniumrequests
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By

options = Options()
options.add_argument('--headless') #Hide window
options.add_argument("--no-sandbox")
options.add_argument('--disable-gpu')
driver = seleniumrequests.Chrome(options=options)
driver.implicitly_wait(10)
driver.get("https://app.lokalise.com/login")
driver.get_screenshot_as_file("./selenium_screenshot.png")
with open("./selenium.html", mode="w", encoding="utf-8") as f:
    f.write(driver.page_source)
driver.find_element(by=By.ID, value='email').send_keys("username")
driver.find_element(by=By.XPATH, value='//*[@id="wrap"]/login-page/div/div/div[2]/form/div[2]/div/input').send_keys("password")
driver.find_element(by=By.XPATH, value='//*[@id="wrap"]/login-page/div/div/div[2]/form/div[3]/button').click()

error:

Traceback (most recent call last):
  File "./champon/classes/sample.py", line 15, in <module>
    driver.find_element(by=By.ID, value='email').send_keys("username")
  File "/home/champon/.virtualenvs/champon.pythonanywhere.com/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/home/champon/.virtualenvs/champon.pythonanywhere.com/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/champon/.virtualenvs/champon.pythonanywhere.com/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"email"}
  (Session info: headless chrome=78.0.3904.70)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.15.0-1017-aws x86_64)

Is there any way to resolve this issue? The screenshot shows only the header image of the site, the JavaScript is written to the html, but the login content is missing from the body.

@champon check this except just click : execute_script("arguments[0].click()")

@champon Maybe you need to wait for page to fully load.

@topeklc Thank you for the answering. The error occurs on the line .find_element(by=By.ID, value='email'), so I don't think it's caused by clicking.

@fjl I tried changing it to driver.implicitly_wait(60), but the error remains the same. Is it because the content loaded by JavaScript is not loaded on ChromeDriver?

Hard to say. You need to inspect what is the state. Maybe try to do it interactively in the console.

Have been having an issue where im using Selenium to help me retreive an item from the network tab of the google dev tools inside chrome. From a script (.py file) it will run fine the first time. If I try to run it again I will get an error, that seems to be Chrome not opening the site, (tries for about 150-200 seconds then returns a Key Error). If I go to the Consoles tab and manually kill the console, I can run the script again however due to this I am unable to utilize the script in a Task (also appears to run once then never run again). Any ideas?

Could you show the full traceback error message?

[code]

def pga_user_tracker():
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--headless")
    chrome_options.add_argument("--disable-gpu")
    chrome_options.add_argument('--disable-dev-shm-usage')
    d = DesiredCapabilities.CHROME
    d['goog:loggingPrefs'] = { 'performance':'ALL' }
    browser = webdriver.Chrome(desired_capabilities=d,options=chrome_options)

    browser.get('https://www.pgatour.com/leaderboard.html')
    logs = browser.get_log("performance")
    browser.quit()
    test = json_normalize(logs)
    test = test[(test['message'].str.contains('userTracking'))& 
               (test['message'].str.contains('statdata'))].copy().reset_index()
    test_json = json.loads(test['message'][0])
    user_tracker = test_json['message']['params']['request']['url']
    user_tracker = user_tracker[user_tracker.find('?user'):]
    with open("user_tracker.txt", mode="w") as file:  
        contents = file.write(user_tracker)
    return user_tracker

[traceback]

Traceback (most recent call last):
  File "execute_inround_showdown_simulation.py", line 23, in <module>
    user_tracker = pga_user_tracker()
  File "leaderboard_functions.py", line 27, in pga_user_tracker
    user_tracker = test_json['message']['params']['request']['url']
KeyError: 'request'

2022-09-16 19:03:14 -- Completed task, took 152.65 seconds, return code was 1.

As noted above, this function will work if i run if I manually close the console it ran in previously. However every attempt after that throws this error, which to me appears that the browswer may not be getting to the website and grabbing the performance logs.

[edit: reformatted by admin]

You could start debugging by running the code in a console (e.g. in a loop) and putting the failing part into a try except clause, and inspecting what is the state of test_json when it fails.

thanks, i did just that. When it fails it the google logs do seem to capture some things, just not what I'm expecting. Have been able to use a try / except section to find what im looking for elsewhere when it can't find it the first time! Thanks!

Great!

Hi, can you please add chromedriver to our account? thank you

Hi, can you please add chromedriver to our account? thank you

chromedriver is already available on your account. See https://help.pythonanywhere.com/pages/selenium/

Hi, can you please add chromedriver to my account ? thank you :)

It's there for all accounts as of a couple of years ago -- you can use it by following the instructions in the link in the post above.

Hi, can you please add chromedriver to our account? thank you

browser = webdriver.Chrome(options=chrome_options) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in init super(WebDriver, self).init(DesiredCapabilities.CHROME['browserName'], "goog", File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in init RemoteWebDriver.init( File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in init self.start_session(capabilities, browser_profile) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace:#0 0x5566f6f58e89 <unknown>

It's there for all accounts as of a couple of years ago -- you can use it by following the instructions in the link in the post above.

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

It's already enabled for all accounts.

Hi. I want chromedriver version 110.0.5481.77 on my account. I've been searching for the cause of unstable run of selenium for months. This has to be it. It's currently using version 90 which is like from ancient times.

Thanks in advance.

Hi Kyazoglu, you can download whatever version of chromedriver that youd like to your account. Then you'd reference that by the executable_path param when you initialize your Chrome object. Of course you'll need to match the chromedriver version to the current version of chrome on the system

Hi Kyazoglu, you can download whatever version of chromedriver that youd like to your account. Then you'd reference that by the executable_path param when you initialize your Chrome object. Of course you'll need to match the chromedriver version to the current version of chrome on the system

Olá, tenho uma conta paga, estou recebendo um erro ao testar a execução do selenium segue código e erro

from selenium.webdriver.chrome.options import Options

from selenium import webdriver

from selenium.webdriver.support.wait import WebDriverWait

from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.common.by import By

from selenium.webdriver.common.keys import Keys

from webdriver_manager.chrome import ChromeDriverManager

import os

def setUpChrome():

try:

    options = Options()

    options.add_argument("--start-maximized")

    prefs = {"profile.default_content_settings.popups": 0,
            "download.default_directory":
                        r"/home/Equillibrium/pilates/navegador",
            "directory_upgrade": True}

    options.add_experimental_option("prefs", prefs)

    options.add_argument("--disable-infobars")

    options.add_argument("--disable-notifications")

    options.add_experimental_option("excludeSwitches", ["enable-automation"])

    options.add_experimental_option('useAutomationExtension', False)

    options.add_argument('--disable-blink-features=AutomationControlled')

    options.add_argument("--headless=new")

    # chrome_options.add_argument("--disable-software-rasterizer")

    options.add_argument("--disable-gpu")

    driver_chrome = webdriver.Chrome(ChromeDriverManager().install(), options=options)

    return driver_chrome

except Exception as ex:

    print(ex)

def inicio():

try:
    driver = setUpChrome()

    driver.get('https://www.google.com/')

    print('Abriu')
except Exception as ex:
    print(ex)

inicio()

ERRO:

"caminho///"/navegador.py:31: DeprecationWarning: executable_path has been deprecated, please pass in a Service object driver_chrome = webdriver.Chrome(ChromeDriverManager().install(), options=options) Message: unknown error: Chrome failed to start: exited abnormally. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

See https://help.pythonanywhere.com/pages/selenium/ for the way that you need to run selenium on PythonAnywhere.

Agora funcionou algumas vezes depois começou a ter esse erro

raise exception_class(message, screen, stacktrace) selenium.common.exceptions.SessionNotCreatedException: Message: session not created from tab crashed (Session info: headless chrome=90.0.4430.212) Stacktrace:

0 0x564d20013e89 <unknown>

é possivel o driver estar aberto em outaras sessões, se sim como encerrar todas?

Estou tentando acessar esse site 'https://web.whatsapp.com/' com selenium chrome driver o código funciona e teoricamente o navegador abre, porem não carrega o site

tirei um print no primeiro teste, e está abrindo diferente de quando abro em outros navegadores vem escrito na imagem "UPDADE GOOGLE CHROME"

caminho do print:

como resolvo isso?

Could you share the screenshot with us at support@pythonanywhere.com?

I am trying to use selenium and chromedriver but i am getting an error. This is what it says in the error log:

2023-05-29 21:13:48,498: Internal Server Error: /CreateZip Traceback (most recent call last): File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, callback_args, *callback_kwargs) File "/home/SteamScreenshotDownload/ScreenshotDownload/Screenshot_Downloader/Screenshot_Downloader/main/views.py", line 16, in CreateZip Filename,ScreenshotCount = GetZipFolder(request.POST['ID']) File "/home/SteamScreenshotDownload/ScreenshotDownload/Screenshot_Downloader/Screenshot_Downloader/main/ScreenshotDownloader.py", line 46, in GetZipFolder def GetZipFolder(ID): File "/home/SteamScreenshotDownload/ScreenshotDownload/Screenshot_Downloader/Screenshot_Downloader/main/ScreenshotDownloader.py", line 22, in GetScreenshotsLinks options.add_argument("--disable-gpu") File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in init super().init( File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in init super().init( File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in init self.start_session(capabilities, browser_profile) File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute self.error_handler.check_response(response) File "/home/SteamScreenshotDownload/.virtualenvs/venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace:

0 0x556d8cf8be89 <unknown>

NO MATCH

Make sure that you are not leaving chrome processes lying around. See https://help.pythonanywhere.com/pages/selenium/

Hello. I have seen similar errors like mine but then no info of how I can solve it. "Message: unknown error: Chrome failed to start: crashed. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /home/uxlivinglab100106/.cache/selenium/chrome/linux64/118.0.5993.70/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace:

0 0x5620357d4e89 <unknown>"

Also info in https://help.pythonanywhere.com/pages/selenium/ is not solving my error.

Kindly Help here.

Try including the options

chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")

Try including the options

chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")