"Sorry, there was a problem" when trying to start a cherrypy application through WSGI : Forums : PythonAnywhere

"Sorry, there was a problem" when trying to start a cherrypy application through WSGI

I am trying to start a cherrypy application, but when I hit "reload", I get the following error after a few seconds:

"Sorry, there was a problem. Please try again in a minute, and send us feedback if it keeps happening."

I kept trying for hours but it didn't fix itself. Here is my configuration file:

import sys
sys.path = ['/home/giorgostzampanakis/', '/home/giorgostzampanakis/pylib/'] + sys.path
from cherrypy._cpwsgi import CPWSGIApp
from cherrypy._cptree import Application
from research.sportsmarkets.website.website import Ratings
config_path = '/home/giorgostzampanakis/research/sportsmarkets/website/config'
application = CPWSGIApp(
        Application(Ratings(), '', config = config_path)
)

deleted-user-88859 | 81 posts | May 4, 2013, 6:43 p.m. | permalink

@giorgostzampanakis: Welcome to PA. Having personally never touched or looked at Cherry Py, I won't insult you with any hair brained ideas. I will however ask if you checked out your web server (access & error) logs? They are accessible from the web tab of your account's dashboard.

a2j | 684 posts | May 4, 2013, 9:18 p.m. | permalink

The problem was that I was serving my homepage as "application/xhtml+xml". It appears that the code that runs when clicking "Reload" does not like that content-type. Once I changed it to "text/html" the problem went away.

Any chance of getting the "Reload" code to accept "application/xhtml+xml" properly? I can't change all my markup to HTML.

deleted-user-88859 | 81 posts | May 4, 2013, 11:03 p.m. | permalink

What is Snooker anyway?

a2j | 684 posts | May 5, 2013, 4:53 a.m. | permalink

Snooker is the most fun you can have with balls on a six by twelve foot table. It also happens to be my favourite sport, and today and tomorrow are the final of the World Championship, the most prestigious event in the Snooker calendar - if you have access to BBC channels you can check it out.

If you're familiar with pool, it's a little like that but a lot more tactical and a lot harder. If you do watch any of the final, it's a lot harder than Ronnie O'Sullivan will make it look!

deleted-user-39880 | 669 posts | May 5, 2013, 7:54 a.m. | permalink

I could think of no way that the content type can affect anything in the web app, so I did some experiments. I used this code:

import sys
sys.stdout = sys.stderr

import atexit
import cherrypy

cherrypy.config.update({'environment': 'embedded'})

if cherrypy.__version__.startswith('3.0') and cherrypy.engine.state == 0:
    cherrypy.engine.start(blocking=False)
    atexit.register(cherrypy.engine.stop)

class Root(object):
    def index(self):
        cherrypy.response.headers['Content-Type'] = 'application/xhtml+xml'
        return '<html><body><p>Hello World! app</p></body></html>'
    index.exposed = True

application = cherrypy.Application(Root(), script_name='', config=None)

I tried 'application/xhtml+xml', 'text/plain' and 'text/html' and they all worked fine.

From the server logs I can see that there are errors that your index page does not return a response and the same happened yesterday when I visited your web app in a browser. Could it be that you're not closing tags properly and so CherryPy thinks your response is not done when you're using xhtml, but it thinks it's OK when you're using the looser html?

glenn | 9718 posts | PythonAnywhere staff | May 5, 2013, 1:33 p.m. | permalink

CherryPy does not look inside the content, it simply serves it. So this cannot be the problem. Try running the code above but through WSGI, like I have in my sample above.

Also, I didn't know that you can run cherrypy on pythonanywhere without using wsgi. How exactly does that work?

deleted-user-88859 | 81 posts | May 5, 2013, 4:54 p.m. | permalink

CherryPy doesn't look inside the content, but is it possible that your templating engine is raising an exception which is causing CherryPy to bomb out early? I would have hoped it would have a generic exception handler which at least returned a 5xx error response, however. I think at least some templating engines are specifically designed to error out rather than produce content which isn't well-formed, I think Kid falls under this category (but I may be confusing it with another).

Which templating engine are you using, in case that's relevant?

deleted-user-39880 | 669 posts | May 5, 2013, 8:13 p.m. | permalink

I'm using mako, but mako does not know or care what content-type will be used to serve the content it produces. It just replaces text.

deleted-user-88859 | 81 posts | May 5, 2013, 9:31 p.m. | permalink

giorgostzampanakis said:

The problem was that I was serving my homepage as "application/xhtml+xml". It appears that the code that runs when clicking "Reload" does not like that content-type. Once I changed it to "text/html" the problem went away.

So, my question is have you tried validating your code as "text/html" to see if it passes?

a2j | 684 posts | May 7, 2013, 11:13 a.m. | permalink

Yes, I did try that and my code was validating.

I still don't understand what validation has to do with it though. Invalid markup causes problem to the browser, not the server. The server simply serves bytes.

deleted-user-88859 | 81 posts | May 8, 2013, 12:06 a.m. | permalink

It may be a red herring, but when there's a problem which isn't immediately explicable then the only real way to progress to investigate different areas until you get a good clue as to the real culprit. At first sight it doesn't seem too plausible that changing the content type of the response should fix the problem at all, so presumably there's something subtle going on.

I'm not aware of the web app reloading code paying any attention to the response of your web app at all (although one of the PA devs may correct me if I'm wrong) so changing the content type shouldn't have any effect. I would imagine it just restarts the application wrapper, or perhaps just sends it a signal to reload the source code. Since changing the content type apparently does have an effect then it would be interesting to work out why.

At risk of stating the obvious, have you tried running pyflakes over your code? I can imagine that when the wrapper (re)loads the code then it might hang if there's something odd about it. Do you have a lot of code which is run at start up as opposed to in response to a HTTP request?

EDIT

It appears the reload button does make a request of the web application, presumably to check that it's come up properly - you can see this if you view your access log as suggested by @a2j above (go to the Web tab on your Dashboard and look for the link that ends with access.log). However, I've just written a very quick test page which returns content type application/xhtml+xml and validates correctly as an XHTML 1.0 Transitional document and that reloaded without issue, so there must be something quirky about your case which is triggering the problem.

First, get it back into the state where reloading doesn't seem to work and then try a reload, then examine your access logs and look at the final entry - it should have a user agent starting python-requests/1.2.0 CPython/2.7.3 .... See if there's anything odd about that line - for example, does it have a 200 response? Or if the line is missing, that tells us something too. You can paste it here fairly safely if you wish, it shouldn't contain any sensitive information.

Second, consider what your application does at startup. I wonder if it's just taking awhile to get ready for requests, so the reload function is timing out waiting for a response. Quite why that behaviour would change with content type I don't know, but then again that may be a red herring as I said at the start of my post - it's hard to say at this stage.

Also, I dug up this post which contains this explanation of what the reload button does:

The reload button rewrites the configuration file for the uWSGI worker and recreates the socket files. This is mainly relevant if you have upgraded your account recently or you have added static file mappings.

deleted-user-39880 | 669 posts | May 8, 2013, 10:57 a.m. | permalink

My favourite debugging method is to keep on taking away moving parts until it starts doing something sensible.

@Cartroo, reloading the webapp does care a little bit about the response from the webapp. We hit it once to fully load the code into memory.

hansel | 459 posts | PythonAnywhere staff | May 8, 2013, 11:13 a.m. | permalink

Thanks Hansel - I did spot that myself just now, and my edit above crossed over with your post. I hadn't realised that the reload code made a request of the app itself. I wonder if this might cause problems if someone has a web app that doesn't respond correctly for requests to /?

(Not that there's necessarily much that can be done about that, but just for interest)

deleted-user-39880 | 669 posts | May 8, 2013, 11:18 a.m. | permalink

Maybe I'm missing something, but if the code is validating, I don't get what the problem is. Did I read too fast and miss something?

a2j | 684 posts | May 9, 2013, 11:17 a.m. | permalink

@giorgostzampanakis looking at our own error logs, it looks like that problem has stopped happening now. Did you ever find out what the problem was?

giles | 12074 posts | PythonAnywhere staff | May 9, 2013, 12:24 p.m. | permalink

@a2j: The problem was that apparently using a particular content type would interfere with the app reload button on the web tab. It's not clear why the content type of the response should affect that, but whether the response was valid XHTML also seems fairly unlikely to affect the result either.

deleted-user-39880 | 669 posts | May 9, 2013, 6:02 p.m. | permalink

Right, but I'm wondering if @giorgostzampanakis fixed the problem by changing the content type or by doing something else. All we do on a reload is hit the app and wait for something to come back...

giles | 12074 posts | PythonAnywhere staff | May 9, 2013, 6:38 p.m. | permalink

Yep! I definitely missed that. If I keep this up I'll have to bleach my hair blond...☺

a2j | 684 posts | May 10, 2013, 1:54 a.m. | permalink

I found that the problem goes away only if I use text/html content type. I know it sounds weird, but it's the truth.

deleted-user-88859 | 81 posts | May 16, 2013, 7:58 p.m. | permalink

@giorgostzampanakis, it just occurred to me while looking at the difference between your code and mine that we may be coding to different versions of CherryPy. The reason I think this is that the Application object created by CherryPy is a WSGI application for the version that we use, so we don't need the CPWSGIApp dance that you're doing. Also, the underscores in your imports suggest that those are non-public APIs and are perhaps not supported across versions.

glenn | 9718 posts | PythonAnywhere staff | May 17, 2013, 10:43 a.m. | permalink