Forums

Is there a ballpark estimate of the memory allowed to the applications of free members? My application is very modest, has next to no traffic but still I get MemoryErrors right and left.

The limit is 512MB, which is enough for almost anything... Could you have some kind of a memory leak?

Unlikely, but is there some way to check the memory usage? The "top" command is not installed.

top wouldn't be much help because webapps and consoles run on different machines.

Your webapp is currently running at 511M virtual memory (which ties it with 2 others as the most memory hungry - any more and the workers get recycled) and 75M resident. I have no idea why that might be. I thought it might be CherryPy might be quite large to start off with, but my little test app uses around 200M, so CherryPy is quite hungry, but it's only half of the memory used.

Locally I've never seen it go above 90M.

Any chance there is anything unusual about the scipy installation in pythonanywhere? I know that scipy versions can vary quite a bit from one platform to another.

I wonder if it might be useful to stick something like memory_profiler on the batteries included list at some point. Standard Python makes it well nigh impossible to determine memory usage and that particular profiler is a pure Python module so hopefully should work without modification on PA (it's much more useful with psutil, but that's already included).

That and objgraph are really handy tools for tracing memory usage and tracking down leaks in Python. The latter writes DOT files capable of being parsed by Graphviz to generate pretty diagrams of object references, so it's pretty handy that Graphviz has just been installed, eh? (^_^)

I modified my webapp to log memory usage (using the "resource" module) and I see that it almost immediately threw a MemoryError ('Cannot allocate memory -- malloc: 9127: Cannot allocate memory') but at the same time the maximum resident set size reported by the "resource" module was only 72 MB (this last figure is consistent with the memory usage I observe when I run the webapp locally).

So, is there some explanation on where the excess 440 MB of memory are going?

Well, the 72MB is broadly consistent with the 75MB that Glenn reported in his earlier post. However, the memory limits appear to be imposed on virtual as opposed to resident memory, so presumably your application is allocating some large amount of memory which is then being neither freed nor used, hence is swapped out and stays there. It still counts against your usage, however.

I don't think the resource module gives you any idea of your virtual usage, but I wonder if you could track your memory usage with something crude like this:

import gc
import sys

def get_mem_usage():
    return sum(sys.getsizeof(i) for i in gc.get_objects())

As far as I'm aware the result of gc.get_objects() shouldn't include any duplicates, but there might be some other reason why that figure is inaccurate. Still, it might be interesting to track that over the course of your web app's run and see if it continually rises - if so, some code is leaking objects. If not, somethine else is probably leaking memory at a lower level.

It appears there's something very wrong. My code is giving memory error on startup, even though I changed it to load literally nothing from the database. Can someone check what's wrong please? It appears there are some memory leaks on the "shelve" module that is used by pythonanywhere.

When you call the open() method of shelve (or construct a Shelf instance directly), are you setting writeback to True? That will consume a lot of memory if you end up writing a lot of objects.

No, I'm not using writeback. And I'm not writing anything, I'm only reading.

Speaking of memory errors, we just had one. Back online now...

I changed my app to read zero bytes off the disk and I'm still getting MemoryError. Can someone have a look at my account please? This is getting ridiculous.

Are you sure it's a memory error? I can't tell from looking at your error logs, and your workers seem to be fairly tame, at around 100MB used between uWSGI master and worker...

$ ps --user=giorgostzampanakis -o pid,cmd,rss,size
PID CMD                           RSS  SIZE
7351 giorgostzampanakis uWSGI ma 54196 43540
7355 giorgostzampanakis uWSGI wo 44524 43540

Yes, I am getting the following error (look at the error log):

2013-05-18 19:47:45,119 :Traceback (most recent call last): 2013-05-18 19:47:45,126 :ValueError: ('Config error in section: \'/\', option: \'tools.staticdir.root\', value: "r\'/home/giorgostzampanakis/research/sportsmarkets/website\'". Config values must be valid Python.', 'MemoryError', ())

So the MemoryError occurs while loading the app (reading the config file). There's probably some issue with PA and the memory it allows to my app. This started happening a few hours ago, before that I had the aforementioned less-fatal MemoryErrors.

I'm not familiar with cherrypy, but are you sure that's a memory error? Wouldn't a memory error kill the process?

How does cherrypy load its config? the only way I can see this working (and I'm not expert) is if it's kicking off some kind of subprocess, which is reporting a memory error instead of reading config?

I know you said you already tried to trim the webapp down, but is there any other way you can "start from scratch", as it were, and re-introduce bits of your code one by one? that might pinpoint the problem...

For reference, here's the bit of CherryPy which is generating that slightly odd looking ValueError:

try:
    value = unrepr(value)
except Exception:
    x = sys.exc_info()[1]
    msg = ("Config error in section: %r, option: %r, "
           "value: %r. Config values must be valid Python." %
           (section, option, value))
    raise ValueError(msg, x.__class__.__name__, x.args)

It's in:

/usr/local/lib/python2.7/site-packages/cherrypy/lib/reprconf.py:213

It's quite annoying when frameworks engage in this sort of obscurantism to hide where the true exception is being raised from you - a traceback would be very useful in this case...

EDIT

The odd thing is that a MemoryError is supposed to contain a string indicating which operation failed, but the argument list is empty in this case. I can't help but be a little suspicious that some other library has co-opted MemoryError for its own nefarious purposes. Hard to see how that would be being called in this context, though - the scope of the try...except is pretty limited as you can see above.

That's some bad code by CherryPy there (they should have been catching only relevant exceptions), but it still shows that there is a MemoryError while parsing a 10-line config file . I think it's obvious that there is something seriously wrong with the PA system when this can happen, and so far we have no clue what it is.

I've tried bumping your memory limit to 2GB. The error still occurs. I'm suspicious because it responds with the error very quickly - it feels like, it's too quick to have been able to allocate 2GB, but perhaps I'm wrong.

I've also tried setting up my own minimal cherrypy app - it loads fine, although my setup doesn't use a config file (I followed the examle here)

I'll leave your memory limit at 2GB - it will last until you next hit "reload web app".

If you want to reload your web app without hitting reload web app, you should know that the web app will do a "soft" reload every time you edit or touch you wsgi file in /var/www

Is there some way you can "start from scracth", introducing parts of your app one at a time, or maybe brand new parts? To reconstruct a minimal repro, and pinpoint exactly what is causing the problem?

It appears the error is fixed, and it was caused by my usage of the "resource" python module... I was using that to set the memory limit to 450 MB, but it appears the module does not play well with this environment... Anyway, thanks to all that tried to help, at least we learned something in the process.

Interesting, thanks for posting that. We'll take a look and try to understand what the problem is and see if it's at all fixable.