Forums

Hi, I am able to create a fresh chrome session using selenium but I need to open other pages in the same session which I am not able to do using selenium on Pythonanywhere. However, when I run the same script on my personal terminal it is doing the intended job of re-using the current browser session. The relevant code snippet is as follows: options.add_argument("--headless") options.add_argument("--disable-gpu") capabilities = options.to_capabilities() browser = webdriver.Remote(command_executor=executor_url, desired_capabilities=capabilities) browser.session_id = session_id

I am getting the following error message: selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed (unknown error: DevToolsActivePort file doesn't exist)

Please note that I am not facing any issue in creating a fresh session. Please guide.

If you're getting a "Chrome failed to start" error message, that suggests that you're not re-using the same browser, but creating a new one. Chrome starts a large number of processes and we have a limit on the number of processes that each user can have on any machine at any time. If you're starting multiple chrome instances, then you'll hit that limit fairly quickly. To use an existing Chrome instance, you shouldn't need to do anything extra, just go the URL that you want to go to.

Hi, I did what you suggested but when I am trying to read the web-page data using beautifulsoup, I am just getting JS (no html content) unlike my own terminal where I get the full html content of the web-page. I am trying to automate certain tasks on my amazon seller account (sellercentral.amazon.in/....). Can you please suggest the way forward?

Apparently Amazon is not fond of being scraped programmaticaly. Developers are supposed to use API, check the link: https://developer.amazonservices.com/ Maybe this could solve your issue?

I am already using the API for things it is meant to be used for. However, there still are several day to day tasks that one has to do on amazon seller central which can't be done using their API. So, I am trying to use this method for the rest of things. Also, another point here is that the same code is working when run from my personal terminal but is not working the way it is supposed to when run here. Really need your inputs to move forward on this.

Amazon is aware that requests made from PythonAnywhere come from the servers from a datacenter and not from a PC used by a human so it's supposedly blocking scraping attempts.