Forums

Headless Selenium report - NoSuchElementException

Hi, Beginner here, I am trying to run a daily report which requires logging into a webpage, so I am using selenium to get where I need to go, the script works locally but to get it onto PythonAnywhere I need to use pyvirtualdisplay. I have installed the version for python 3.4 but selenium is hitting the error below:

selenium.common.exceptions.NoSuchElementException: Message: 'Unable to locate element: {"method":"name","selector":"username"}'

As the program works locally I assume I am doing something incorrectly with pyvirtualdisplay, I have added in a fair number of wait_implicitlys and time.sleeps to allow plenty of space for the display to get up and running.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

from pyvirtualdisplay import Display
display = Display(visible=0, size=(1024, 768))
time.sleep(5)
display.start()
time.sleep(5)
driver = webdriver.Firefox()

driver.get('http://www.touchoffice.net/')

driver.implicitly_wait(30)

username = driver.find_element_by_name("username")
password = driver.find_element_by_name("password")

Any advice to get this working?

Free accounts on PythonAnywhere can only make outbound requests to sites on our whitelist, and -- if you look at the body text of the page you're getting back from that driver.get call you'll almost certainly see that it contains an error message regarding that, which is why there's no "username" field on it.

Paid accounts can access any site they like, so if you upgrade it will work. We can also add sites to the whitelist if they have an official public API, though I don't see one on http://www.touchoffice.net/.

Ace cheers, I will set up a paid account then and see if it works.

Thank you for the assistance.

No problem -- and thanks for upgrading!

Hi, I have exactly the same issue but with the website https://www.trainline.eu/signin.

I have a "hacker" account (5$ per month) and by running this code I have the "unable to locate element" error.

 from selenium import webdriver
 import time

 from pyvirtualdisplay import Display
 display = Display(visible=0, size=(1024, 768))
 time.sleep(5)
 display.start()
 time.sleep(5)
 driver = webdriver.Firefox()

 driver.get('https://www.trainline.eu/signin')

 driver.implicitly_wait(5)

 username = driver.find_element_by_name("email")
 password = driver.find_element_by_name("password")

Do you have any advice to make it work ?

Update : by taking a screenshot of Firefox's window, I have the following message : "Error code : ssl_error_no_cypher_overlap" (Secure Connection Failed). I tried this code but it didn't work :

profile = webdriver.FirefoxProfile()
profile.accept_untrusted_certs = True

Hmm, that looks like a problem with the version of Firefox we have installed. It's a pretty old version (newer versions don't work with our virtualization system) and only supports SHA1 ciphers. It looks like the site you're connecting to doesn't support SHA1 cipher suites -- which is not unreasonable, SHA1 has been deprecated in favour of SHA256.

Extending our virtualization system so that we can support newer versions of Firefox is definitely on our to-do list (and this problem bumps it up quite a lot). But unfortunately until we do that you won't be able to access that site using Selenium from PythonAnywhere.

Ok, I'll wait then ! Thank's for your reactivity !

Good afternoon! I'm trying to run a script on your server. The script works with Selenium. And everything was fine at first, but during the program the code crashes and gives the error No Such Element. At the same time, it is strange that the code always throws an error at different stages. Sometimes it throws an error right away, sometimes after several iterations. In general, with the same input data, it works every other time. At the same time, I am sure that the problem is precisely in the use of Selenium. Since with normal get requests everything is fine. Tell me, please, how can I solve this problem? I watched this topic on the forum. Also added all the necessary options “—disable-dev-shm-usage”, “—no-sandbox" and “—disable-gpu". However, the error still occurs

I see that we're discussing this over email too, so let's keep the conversation going there -- if and when we reach a solution, if it's generally applicable, then either you or we can post it here in the forums.

Hello! I have this code lines:

browser.get("https://finance.yahoo.com/quote/LFTS11.SA?p=LFTS11.SA&.tsrc=fin-srch")
lfts = browser.find_element(By.XPATH, '//*[@id="quote-header-info"]/div[3]/div[1]/div/fin-streamer[1]')

And I got this erro message: Unable to locate element. Is the syntax correct? Other times I replaced the line 2 by:

lfts = browser.find_elements(By.XPATH,'//*[@id="quote-header-info"]/div[3]/div[1]/div/fin-streamer[1]')

And the response was a empty bracket "[ ]". The selenium version is 4.8.3. Thanks for help!

Free accounts have restricted internet access to the allowlisted domains, so you will probably not be able to freely scrape.

The site yahoo.com is whitelisted... or I am wrong? Thanks!

I can't reproduce the behavior you get -- on a paid account, a can get the element(s); on a free one I'm getting NoSuchElementException exception.

i have paid account , but still getting nosuch element exception , please check my account if anything is wrong

Look at the HTML that you are getting back to see what is actually in the page. Also, if you have upgraded recently, make sure you are running your tests in a console that you created after upgrading.