Forums

Does pythonanywhere is good for web scraping?

It cost 5$/month for 3000sec of cpu, so in simple calculation it I can use 50 mins a day of scraping? am I missing anything?

I don't work at PythonAnywhere, but I should think that'll be fine.

Most of the time spent during scraping is probably waiting for the website to come back with information, which will not count against your cpu time. (ie. your process will be idle while waiting for that)

So you could possibly run a scraper for significanly more than 50min without hitting the limit. Also keep in mind that after exceeding your limit, your processes still run but at a much slower pace.

Where can i read more about scraping possibilities of pythonanywhere? As far as i understand for free plans scraping is not available? Also, do you provide some proxy rotation pool or service?

Hi there, I can answer a couple of those questions -- yes you will need a paid account for most scraping, and no we do not provide a proxy rotation service...

So the 5$ account will fit? Also whant about ip address used. Is it the same ofr account? If not , how and when it is chnaged? Are there any limitations for scrapyng?

Hi there, the $5 should be fine yes. IP addresses change once every 6 weeks or so. The only limitation would be that we can't support you if you're trying to use our service in a way that breaches the Ts & Cs of another site...