Forums

unable to scrape website

Edit: Nevermind, I figured it out.

Hey all,

I have a program that works on my computer which scrapes a particular website. When I try to run it on pythonanywhere it doesn't work. I have a paid account ("Hacker", $5/mo).

The particular module which isn't working is "requests". I'm trying to use requests.get().

Anyone else have this issue?

Edit: Nevermind, I figured it out.

Glad you figured it out! If you have anything you can share about what the problem was, and how you fixed it, it might help other users?

I was trying to scrape Craigslist, and once I figured out how to print the error message I saw a message from CL saying it was blocking the IP (I'm assuming from too many requests from other PA users). I'm just going to run the scraper from my home IP instead.

By the way, I'm loving your book! I bought a hard copy b/c I prefer reading programming books that way. I'm also enjoying your PyCon2014 talk on YouTube. Thank you for teaching the rest of us how to do this stuff!

thanks nathan :) do drop us a line via obeythetestinggoat@gmail.com if you ever need help.