Forums

Flask web app returns a “403: Forbidden” and a “500 (Internal server error)” error only on production.

Web app description: This is what my web app does.

-1) User types in the name of a Twitter user in an input box

-2) Python file receives the name and makes API calls to Twitter about the said user. (Example, top countries of their followers, frequently used keywords etc) https://apps.twitter.com/

-3) Python receives the data, do some analysis and return a few interesting facts to JavaScript file in a JSON format.

-4) JavaScript file receives the JSON response, interprets it and visualise the data using Plotly CDN. Display it too HTML.

Error desc: Everything works fine in local. When in production, the browser shows this error:

500 (INTERNAL SERVER ERROR)

And the error log in PythonAnywhere has the following error:

urllib.error.HTTPError: HTTP Error 403: Forbidden

But when I look at the server log within PythonAnywhere, I do see some of the results of the analyses returned in python “list” format. (They are just there for my testing purposes.)

I would like help interpreting the error messages and identify what the problem is.

thanks!

Full error log message below:

2020-01-10 16:04:21,566: Exception on /backend_process [POST]
Traceback (most recent call last):
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/socListn/app/app.py", line 256, in backend
    uh = urllib.request.urlopen(geo_url, context=ctx)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
2020-01-10 16:04:21,700: Exception on /backend_process [GET]
Traceback (most recent call last):
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/socListn/.virtualenvs/my-virtualenv/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/socListn/app/app.py", line 314, in backend
    return pyResponse_js
NameError: name 'pyResponse_js' is not defined

Screen grab of the server log page: Note it does show the results of some analyses, which suggests that my API call to Twitter API was indeed successful.

enter image description here

[edit by admin: formatting]

What code do you have in your view function backend? It looks like there must be a code path through it which doesn't initialise the variable pyResponse_js, perhaps when you get an error response from Twitter.

It would also be worth checking what the content of the 403 response you get from Twitter is. A 403 is a general "request not allowed" status code; you could get that on PythonAnywhere if you were accessing a site that is on on our whitelist of sites that free accounts can access, but all subdomains of twitter.com are whitelisted, so if that is the domain you're accessing, I don't think it's that -- instead, it's probably an error from Twitter's site itself.

Thank you for looking into it.

I use the exact same code in the local environment. And I don’t receive an error from Twitter. Also, the server log message in PythonAnywhere suggests API calls are made and I am receiving the answers from there.

What's puzzling me is the same code is returning an error message but only in the production environment.

Again, it's hard to help you without knowing what your code is, and what the error that you're getting from Twitter is.