Forums

BeautifulSoup find_all gives 500 Internal Server Error

I have got this code that works on 127.0.0.1 but for some reason not on pythonanywhere.com:

payload = {'username':'user','password':'pass'}
url='http://myurl.com'

session_requests = requests.session()
r = session_requests.post(url, data=payload)


url='myurl.com/page'
r = session_requests.get(url)

bs = BeautifulSoup(r.text, "html.parser")
bs.prettify()

main = bs.find("ul", {"class":"topics"})

sections = main.find_all("div", {"class":"content"})

The link is not myurl.com I just would prefer not to reveal as the problem is probably not related to it as the code works on my laptop... I have imported all modules just didn't pasted it here *I have the the flask normal stuff with app.run, app.route and all that but once again, I think that's not necessary...

It gives this error on the error log:

 File "/home/bio/mysite/flask_app.py", line 22, in data
 sections = main.find_all("div", {"class":"content"})
 AttributeError: 'NoneType' object has no attribute 'find_all'

So it's like it hasn't find the div with class content that, as I have already said, exists when I run it on 127.0.0.1...

What is the status code of the response from the server? You can find it by printing r.status_code -- the output will wind up in your website's server log.

One thing to look out for -- you have a free account, and free accounts are limited to accessing sites on our whitelist.