Forums

Chromedriver via Selenium?

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

@luckydog It's done for you

Thanks! Is there anything I need to do to set it up?

No problem. To use Chrome, you'll need to upgrade Selenium for your account -- for example, if you're using Python 3.7, run this in Bash:

pip3.7 install --user --upgrade selenium

...and then you can run Selenium with Chrome using code like this:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))

finally:
    browser.quit()

I upgraded Selenium and have the code in my program to run it (it runs from my terminal), but it still doesn't start up. Is there anything I'm missing?

How exactly are you running it? Do you get any errors?

There's a button on the home.html page that calls a function in Flask - the first part of the function is running fine, then it skips Selenium and flashes the message that nothing happened.

Add some logging to your code to see what happens.

I get this error - I know the element is present on the page I'm visiting and have tested dozens of times via my terminal. It seems like the browser isn't opening the link? I looked into some other forums on this issue- is it possible the site hasn't been whitelisted for access?

Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="email_address"]"} (Session info: headless chrome=78.0.3904.70) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

If by "my terminal", you mean from your local machine not on PythonAnywhere, then it is entirely likely that the site is not on the whitelist.

Ah ok would it be possible to add the site to the whitelist? It's eversign.com

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

Hi @epireve -- sure, that's done for you.

Thanks! Anything I need to do on my end so that change is registered?

Oh, sorry, @luckydog -- must've overlooked your post. We are happy to whitelist any public API if you provide us with the api endpoint and the official documentation hosted on the same domain.

what should I done regarding this error? if you need to enable something regarding this error, just do it. thank you

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
 (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

hi- you already have our new virtualization stuff enabled.

so if you recently got this enabled, then make sure you are running your code from a new console / reloaded webapp etc.

also, if you simplify everything and follow this, does it run?

my code running on Jupyter notebook PyA, after refresh and reload the kernel, it does not work. any solution?

Currently our new virtualization feature doesn't work in Jupyter notebooks.

virtualisation features just support on console?

currently, I already can run chrome drive in console, however, I cannot found the downloaded file, how to found the file beside can we set the download directory? by default what is the download directory? tq

The new virtualization system works in consoles, scheduled and always-on tasks, and in website code, but not in Jupyter Notebooks or SSH -- when it comes out of beta we will add support for the last two.

The code in this Stack Overflow answer shows how to set Chrome's download path -- you don't need to set the ChromeDriver path, though.

@pafk Sorry for the delay! My endpoint API is 7c75f93c3fbb016a7dfa00e4c5db3116 and the documentation is here: https://eversign.com/api/documentation

I've added api.eversign.com to the whitelist.

Thanks! I'm still getting an error where Selenium doesn't actually open anything - here's the error: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="email_address"]"} (Session info: headless chrome=78.0.3904.70) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

Try using Selenium's screenshot functionality to see what the page looks like just before you try to get that element -- the function is get_screenshot_as_file on your webdriver object, and the one parameter is the full path to a filename (ending in ".png") where it should store the screenshot.

I put the code in two places but it hasn't saved anything. It seems like selenium isn't even launching? Here's my code:

def login(template, email, password):

        chrome_options = webdriver.ChromeOptions()
        chrome_options.add_argument("--headless")
        chrome_options.add_argument("--disable-gpu")
        driver = webdriver.Chrome(options=chrome_options)
        driver.get_screenshot_as_file("/home/luckydog/ldar_contracts/selenium_screenshot.png")
        driver.get(template)
        driver.get_screenshot_as_file("/home/luckydog/ldar_contracts/selenium_screenshot.png")

[edit by admin: formatting]

Does the code print anything out when you run it -- for example, any error messages?

It prints this error message: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="email_address"]"} (Session info: headless chrome=78.0.3904.70) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

That must be coming from different code, as the code that you posted in your previous message does not try to access any field by the ID email_address.

Here is the rest of the code - what I pasted previously was the code initializing the driver and navigating to the login page.

        driver.get_screenshot_as_file("/home/luckydog/ldar_contracts/selenium_screenshot.png")
        driver.set_window_position(0,0)
        driver.set_window_size(500,500)

        emailField = driver.find_element_by_xpath('//*[@id="email_address"]')
        passField = driver.find_element_by_xpath('//*[@id="password"]')
        emailField.send_keys(email)
        passField.send_keys(password)

Can we take a look at your code? We can see it from our admin interface, but we always ask for permission first.

That’d be great, thanks!

I see that you have two functions called login -- are you sure you've put the screenshot code in the correct one?

Whoops sorry about that - good catch, just deleted the other login function and still having the same problem and not getting a screenshot.

Are you sure that the code to take the screenshot is being run? If you're doing this in a web app, have you reloaded it after adding the screenshot code? If you're doing this in a console, you can add prints around the screenshot code to make sure that's the code that you're running, if you're in a web app, prints to stderr will appear in your error log.

Also, I see that you're changing the window position and size, perhaps that's interfering with the screenshots.

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

Hi @hhlimhh73, we're currently in the process of enabling the new virtualization system for all accounts on PythonAnywhere, which unfortunately may take a week or two to complete. We'll let you know when that has been done for your account; I've made a note to make sure that you're in one of the first batches to be moved over to it.

Hi @hhlimhh73 -- we've enabled virtualization feature for you. Here are the next steps:

To use Chrome, you'll need to upgrade Selenium for your account -- for example, if you're using Python 3.7, run this in Bash:

pip3.7 install --user --upgrade selenium

...and then you can run Selenium with Chrome using code like this:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))

finally:
    browser.quit()

hi can i get chromedriver enabled too?

Hi, @vegarian, you've got that already enabled. Do you get any errors? Have you followed the steps mentioned above?

@pafk can i use pyperclip on virtual environment?

Don't think so, there's no clipboard on PythonAnywhere, it's a server environment.

how to enable chromedriver for my account?

@stocktoday You already have everything enabled. You need to upgrade selenium if you want to use it. Take a look at post above

Hi @fjl , could you enable it for my account as well? Thanks!

Hi @the0919 -- you've got that already enabled. Please look at @fjl post above for further instructions.

Hi, could you enable chromedrive for my account?

@aiusko It's already enabled for your account.

Hi! Could you please enable Selenium Chromedriver for my account? Thank you

@tylerbolz31 You already have everything enabled. You need to upgrade selenium if you want to use it. Take a look at post above

Could you enable chrome for my account so that I can run it on headless mode

you are already on the latest system image and have chrome enabled. You need to upgrade selenium if you want to use it.

Hi I am getting the following error for selenium code. Its working fine locally. But it has something issue in pythonanyhwhere. Please help me out .

(unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1029-aws x86_64)

Does it happen always in some circumstances or just from time to time?

Hi Can you please enable selenium and chromedriver for my account. Please Thanks

They are already enabled for your account.

But whats the executable path for the chrome webdriver. Please tell me asap. Thanks

@vedaang -- you can find it using which chromedriver command.

@pafk I runned the command and thanks for the help. It saved my time. Thanks

Glad to hear that you made it!

hi how Can I using selenium>3 please

Good night, hope you are all doing well and staying safe and healthy.

I am trying to run a simple code that checks a web page for an appointment and sends an email with the data collected.

It uses selenium wedriver (Chrome) and the python smtp packages.

When trying to run the code, I am getting the following message:

driver = webdriver.Chrome(options=chrome_options)
TypeError: init() got an unexpected keyword argument 'options'

This is the part of the code to run headless Chrome.

Could you please help me fix this?

Thanks in advance, best regards.

@reza16, @pydevelper -- you need to upgrade Selenium using this command:

pipX.X install --user --upgrade selenium (where X.X is the Python version you're using in your app, e.g. 3.8).

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

It's already installed and enabled -- you just need to follow the steps in the post above.

Hi, every time I try to run selenium with chrome I got the below error. Why would you think it happens?

*(process:255): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
Current firefox version is 17.0
Get LATEST driver version for 17.0
Driver [/home/lancia/.wdm/drivers/geckodriver/linux64/v0.29.1/geckodriver] found in cache
Traceback (most recent call last):
  File "quickstart.py", line 18, in <module>
    session = get_session()
  File "quickstart.py", line 13, in get_session
    session = InstaPy(username=insta_username,
  File "/home/lancia/.local/lib/python3.8/site-packages/instapy/instapy.py", line 325, in __init__
    self.browser, err_msg = set_selenium_local_session(
  File "/home/lancia/.local/lib/python3.8/site-packages/instapy/browser.py", line 142, in set_selenium_local_session
    browser = webdriver.Chrome(options=chrome_options)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
    RemoteWebDriver.__init__(
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/lancia/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1029-aws x86_64)*

[edit by admin: formatting]

Hi, I need to get this site "https://ip4.seeip.org/geoip" in my code but it seems that it's not in whitelist I want you to add this in whitelist if it's possible. thank you

Regarding the first error -- that's a really confusing one, as it appears to be using both Chrome and Firefox (see the start of the error for that one). However, it certainly does seem to be trying to start Chrome, and is failing. Do you have lots of Chrome processes running, perhaps? You can see your running processes in the table at the bottom of the "Consoles" page.

Regarding the whitelist request -- we can whitelist sites if they provide an official public API. Just give us a link to the API documentation.

Thank you for your reply :) I solved above problems but I have a new error now and I don't know why

InstaPy Version: 0.6.13 .. .. .. .. .. .. .. .. Workspace in use: "/home/lancia/InstaPy" OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO INFO [2021-06-06 09:54:49] [*] Session started! oooooooooooooooooooooooooooooooooooooooooooooooooooooo INFO [2021-06-06 09:54:50] [*] -- Connection Checklist [1/2] (Internet Connection Status) WARNING [2021-06-06 09:54:57] [*] - Internet Connection Status: error ...................................................................................................................... CRITICAL [2021-06-06 09:54:57] [*] Unable to login to Instagram! You will find more information in the logs above. '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' INFO [2021-06-06 09:54:59] [**] Sessional Live Report: |> No any statistics to show [Session lasted 20.5 seconds] OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO INFO [2021-06-06 09:54:59] [*] Session ended! ooooooooooooooooooooooooooooooooooooooooooooooooooooo*

You'll need to contact the author of the module that you're using to find out more about what that error means. However, if you're trying to write some kind of bot to interact with Instagram, it might just be that they are blocking incoming connections from cloud environments like PythonAnywhere; in general, they do not want bots on their platform.

Is there any way to run chromedriver but not headlesly. Could you please tell me how to do that.

You could do that by using a virtual display, but I'm a bit confused about why you would want to -- remember, there's no display on PythonAnywhere, so you still wouldn't see anything. The virtual display would be invisible, so the result would be the same as running Chrome headlessly.

hello, can you please enable selenium chrome driver support for my account after pip3.7 install --user --upgrade selenium command in bash I am getting syntax error at 3.7

hi- what is the exact error? and you are getting it on the pip3.7 install command? or on a later command?

Hey Thanks I checked it is already activated for me

Could you please enable Selenium Chromedriver for my account? Error message is " TypeError: init() got an unexpected keyword argument 'options'"

I resolved problem after upgrade selenium. thanks.

You should be also able to fix it by upgrading the system image (see here for details).

import time from flask import Flask from selenium import webdriver from selenium.webdriver.chrome.options import Options

app = Flask(name)

@app.route('/') def hello_world(): return 'Hello from Aryan!'

print('hello') chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--headless") chrome_options.add_argument("start-maximized") chrome_options.add_argument('--no-sandbox') chrome_options.add_argument("disable-infobars") chrome_options.add_argument("--disable-extensions") chrome_options.add_argument('--disable-dev-shm-usage') chrome_options.add_argument("--disable-gpu") driver = webdriver.Chrome(options=chrome_options) driver.maximize_window() chrome_options = webdriver.ChromeOptions()

browser = webdriver.Chrome(options=chrome_options)

try: browser.get("https://www.google.com") print("Page title was '{}'".format(browser.title))

finally: browser.quit() driver.close()

this is my code and when i try to run this code i am getting this error

selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: -5

At what point is it failing?

hello, what if i get this error: i upgraded my account.

    driver.get('**the site im trying to get**')
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from tab crashed
  (Session info: headless chrome=90.0.4430.212)

[formatted by admin]

Do you run multiple instances of that code? That error could mean that you are reaching the limit of processes. Chrome spawns multiple processes.

Hi I am getting below error . Please help to fix it. Traceback (most recent call last): File "/home/shivanshuv/Chartink2.py", line 19, in <module> Todays_stocks = get_R2(url) File "/home/shivanshuv/Chartink2.py", line 15, in get_R2 table=pd.read_html(driver.find_element_by_xpath('//*[@id="DataTables_Table_0"]').get_attribute('outerHTML')) AttributeError: 'Options' object has no attribute 'find_element_by_xpath'

looks like a coding error

hi, could you please enable Selenium chromedriver for me? How do I install chromedriver on this website? Do I just upload webdriver to my file? which os version should I be choosing. Thank you very much. Sorry I'm new to this.

@cpatonn -- it's already enabled for all users. You don't need to install chromedriver. For more details, see this help page.

Hi, I'm running from a virtual environment and I have this ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv. error trying to run pip3.7 install --user --upgrade selenium Will selenium work if I install outside my virtual environment

--user is a flag for installing specifically outside of a virtualenv. If you want to install into the virtualenv, activate it and then do the install without the --user flag.

Can you enable for me also? Thank you in advance!

You're using an older system image. If you want to use Chrome, you can update your system image from your Account page.

Thank you! Just to see clear change "fishnchips" to "glastonbury" and that's all? It means the selenium will work and all saved codes will stay, so for my account everything will stay?

Everything will be left as it is. Be aware of the warnings about re-creating virtualenvs and re-installing packages that is shown when you update. Also, make sure that you use a new console that you create after you have upgraded.

Can you enable for me also? Thank you in advance!

It's enabled for everyone. See https://help.pythonanywhere.com/pages/selenium/

selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED

i have the following error using selenium, what i can do with it?

How do you run it?

.

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
driver = webdriver.Chrome(options=chrome_options)
driver.get('https://google.com')
driver.save_screenshot("s.png")
driver.close()

okay, i have 2 account: one is free and another one no. when i run this code using free account all is fine, but with another one i have error:

driver = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

what about ERR_TUNNEL_CONNECTION_FAILED:

i think the problem is the site i try to access is not in whitelist(https://app.elaisian.com/login), but this platform don't have an official API, so i think its kinda impossible to get access

Is your second account on the most recent system image? The error you get looks like an old version of selenium (check our help page for more info). On free account you will not be able to scrape since free accounts have internet access restricted to the whitelisted domains.

hi! thank you, now it works! last question - what is default download folder if i use chrome to download file with selenium? because i can see that i have less free space but cant find downloaded files

If you haven't specified the download location, I'd suppose the file would be saved in the running script location. You may also scan your directories (and /tmp as well) with command like find to look for the "suspects".

Hi! Could you please enable Selenium Chromedriver for my account? Thank you!

It is already enabled for all accounts by default.

Hi, I try to use chromedriver for "aliexpress but I getting this error message :

Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED
 (Session info: headless chrome=90.0.4430.212)

when I try with google website its works....

this is my code:

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = webdriver.Chrome(options=options)

driver.get("https://www.google.com")
print('x')
driver.get("https://m.aliexpress.com/item/1005002272194704.html")
print('check2')
content = driver.page_source  #the html
soup = BeautifulSoup(content, 'html.parser')
link = soup.find('a', attrs={'class': '_9tla3'})
link = link.get('href')
driver.close()
return 0

Free accounts have restricted internet access. If you want to scrape non-whitelisted domains, you need to upgrade your account.

I am having trouble accessing one website. When I run the very simple code below which I copied from your forum I successfully access google, but get the error below when it moves to https://www.ghin.com/login. It works fine when I try to access that site using PyCharm

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))
    browser.get("https://www.ghin.com/login")
    print("Page title was '{}'".format(browser.title))
finally:
    browser.quit()

I get the following error

Page title was 'Google'
Traceback (most recent call last):
 File "/home/hommeswj/Ghin_test.py", line 9, in <module>
 browser.get("https://www.ghin.com/login")
 File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in 
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page 
crash
 from tab crashed
   (Session info: headless chrome=90.0.4430.212)

Does that happen every time you run the code?

Hi, I am trying GET requests with Selenium and WebDriverWait. Most of the time it doesn't work because the WebDriverWait is "timeout" (I tried long time). I think it have some problems with open the chrome driver... how can I fix it and make it more stable?

from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from bs4 import BeautifulSoup from datetime import datetime import time from selenium.webdriver.chrome.options import Options from couchbase.exceptions import TimeoutException

from aliexpress_api_client import AliExpress

PRODUCTS_NUM = 3 TRIES_NUM = 500

def find_product_properties(product='iphone x'): print('in') #first request - find products in the web #defind the headless browser weight = '' options = Options() options.add_argument('--headless') options.add_argument('--disable-gpu') driver = webdriver.Chrome(options=options) driver.get("https://he.aliexpress.com/wholesale?&SearchText="+ product) #get request print('b') try: WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.CLASS_NAME, "_9tla3"))) content = driver.page_source #the html soup = BeautifulSoup(content, 'html.parser') link_lst = soup.findAll('a', attrs={'class': '_9tla3'}) img_lst = soup.findAll('img', attrs={'class': 'A3Q1M'}) except TimeoutException as exception: print('timeout') raise exception except: print('faild ali web reload') driver.quit() return {} print('good') driver.quit()

I get the error every time I try to access the GHIN website

I get the error every time I try to access the GHIN website