errorhandler not used on 404 error - Flask

I have a flask app with the following code:

: : : python
def page_not_found(e):
      return render_template('404.html')

thought it works perfectly fine when I spin the server on my localhost. It does not render the template when run from my pythonanywhere server. Instead it renders a different Not Found page. I'm kind of a noob to all of this. Any help is appreciated.

Is the template actually there? Can you render it from a normal view?