Forums

"Problem loading page" when attempting to web scrape with selenium.

Hello,

I recently started a free account, but then very quickly upgraded it to a paid account. When I attempt to use selenium on the site I intend to use it on, I get the following error "Problem loading page". I'm wondering if I have issues with my code, or if there is a small delay between obtaining a paid subscriptions & web-scraping sites outside of the white-list (see here: https://www.pythonanywhere.com/whitelist/).

Here is my setup: Python 3.8 Selenium 2 Flask 3.8

I have my code based on the following web document: https://help.pythonanywhere.com/pages/selenium/

deleted-user-7025503 | 9 posts | Feb. 26, 2020, 9:11 p.m. | permalink

What's the page that you're trying to connect to?

Staff giles | 12096 posts | PythonAnywhere staff | Feb. 27, 2020, 5:43 p.m. | permalink

Dotabuff.com

deleted-user-7025503 | 9 posts | Feb. 27, 2020, 6:15 p.m. | permalink

The solution is to use our new virtualization system for your account that allows headless Chrome. See an example here https://eu.pythonanywhere.com/forums/topic/16/

We can switch you if you want.

Staff fjl | 4614 posts | PythonAnywhere staff | Feb. 28, 2020, 2:54 p.m. | permalink

That sounds like a great solution. Could you please switch me?

deleted-user-7025503 | 9 posts | Feb. 28, 2020, 4:47 p.m. | permalink

It's done.

Staff fjl | 4614 posts | PythonAnywhere staff | Feb. 28, 2020, 7:06 p.m. | permalink