Forums

I have a pickled file of my class 'Classifier', that I want to use when the user sends a request to my flask webpage. However, the code where the pickled file is loaded gives me an error. The same code works perfectly when run on the console, but gives an error (in the error log) and doesn't work on the webpage. Here is briefly how the code looks:

from flask import Flask
import pickle

class Classifier():
    #class definition

classifier = None
with open('/home/saurabhshirodkar/appname/savefile', 'rb') as savefile:
    classifier = pickle.load(savefile)

app = Flask(__name__)

@app.route('/')
def hello_world():
    return('Hello from flask!')

And this is the error from the error log:

2018-04-27 15:36:54,334: Error running WSGI application
2018-04-27 15:36:54,338: AttributeError: Can't get attribute 'Classifier' on <module '__main__' (built-in)>
2018-04-27 15:36:54,338:   File "/var/www/saurabhshirodkar_pythonanywhere_com_wsgi.py", line 16, in <module>
2018-04-27 15:36:54,339:     from flask_server import app as application  # noqa
2018-04-27 15:36:54,339: 
2018-04-27 15:36:54,339:   File "/home/saurabhshirodkar/instagive/flask_server.py", line 24, in <module>
2018-04-27 15:36:54,339:     classifier = pickle.load(savefile)

Again, the above code works with the console. This error occurs when I open the webpage, and it works when I remove the pickle load code. Why is this the case, and how to solve or work around this problem?

When you have Classifier defined directly in a file you run, it's pickled as coming from the module main. When you unpickle it in the web app, there is no main module, so the code can't find the definition. If you create the classifier in it's own module that you import in both places, then it should work.

@glenn This works. Thank you for your response.

@glenn maybe you can help me?

I can't load a machine learning model (or any object at all) in a django function. The error always look like this (<class 'ImportError'>):

2018-10-08 16:36:39 unable to load configuration from from multiprocessing.semaphore_tracker import main;main(20) 2018-10-08 16:36:41(<class 'ImportError'>)

I can execute code in the bash console that can execute pickle. But when I try to use inside a django function pickle won't load.

Any ideias? In this (https://www.pythonanywhere.com/forums/topic/12738/#id_post_49387) you seem to provide some explanation but I did not understand.

In my local machine is working just fine.

Thanks for your time.

do you know if it is an import error trying to import a module that you have written? or a third party module you have installed?

No conrad, I don't. The server log only informs "ImportError"

Could you post your code? Or, alternatively, if there's private stuff there that you'd prefer not to post on a public forum, can we take a look at your code? We can see if from our admin interface, but we always ask for permission first.

Hi Giles!

I managed to "solve" my problem by implementing a Task that run the code I need. In this format, the code completely works. This task is the "main_script.py" under the project root "/home/LucasGalhardi/django_proj".

Under the same directory, there is the code that is not working: /home/LucasGalhardi/django_proj/proj/s/views.py in line 69, the pickle function.

For now, I will work with the task. But I also hope to solve this. If you think it might help, you are welcome to look at the code.

Thanks.

Interesting! I definitely don't think it's the same problem as the one Glenn was talking about previously. Where is the definition of the type of the object that is stored in those pickled files?

Wow! I did a new test when you said that. I stored a simple python list and worked. The problem is that pickle is not finding the object definition. But I don't know how to tell him (pickle). The dependencies are all installed in the virtual enviroment I'm using. That is why the code works in the bash console and in the Tasks interface.

You need to make sure that the import path for the class that you're trying to unpickle is the same as it was where it was pickled. So, in the file where you pickled it, if you imported it with "from foo.bar import baz", then that import statement must work in the file where you're trying to unpickle it. Also, if it's a class from an installed module, you need to make sure that the versions of the module are the same in both locations.

Ok, I will need to focus in another work priorities. For now, I'll stay with the Task. Later I'll come back and try to get this working following what you said. Thanks for the help! If I need I'll contact you again.