Forums

Keras predict function doesn't work

Hello everyone! I am developing flask application in which I use keras model to predict class of the picture. I have already trained model offline I only load weight in server. Server properly saves image to directory but then it cannot finish evalutaing of predict function.

In Shell I get very quick response from the same program passing the same test picture as argument.

#part of predict_img function
##################################################
test_image = image.load_img(img,target_size=(100, 100)) 
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result  = classifier.predict(test_image) <---------here program stucks
ind = np.where(result[0] !=0)[0][0]
with open('en.json') as json_file:
    classes = load(json_file)

I am currently having the same issue.

My program also hangs when loading a new keras model using:

model.load_weights('{}/model.h5'.format(location))

The location is definitely correct so maybe a problem with Keras running on pythonanywhere?

the thing is that when I run predict function in shell I get response very quickly. While I upload test photo on my web app I cannot see any change in CPU usage although the website is waiting for getting prediction from function. I think that I should notice change in CPU usage. when prediction function is working. Is my way of thinking correct?

This is exactly the same problem I am having. Have you made any progress?

Unfortunately not

Do you know if the classifier is trying to use threads?

This problem is solved by changing the Keras back-end to Theano.

Install Theano using

pip install theano

You can change Keras back-end by modifying the keras.json file in /home/your_name/.keras/keras.json

{
"backend": "theano",
"floatx": "float32",
"image_data_format": "channels_last",
"epsilon": 1e-07
}

Interesting. What backend where you using before?

I got the same problem. I use Tensorflow as my backend

OK. It sounds like we have a problem running Tensorflow inside Flask (and maybe other) websites -- we're looking into a solution, but if you can switch to Theano it sounds like that works as a workaround.

[edit -- I said Keras when I meant Theano]

There may be another fix here:

https://github.com/keras-team/keras/issues/2397#issuecomment-306687500

That sounds promising! We'd love to hear from anyone who tries it out :-)

I'm still facing this same issue, even trying out the suggested fix. I know because I applied that fix on my test webserver from my laptop and it worked, but not on pythonanywhere. Will this be fixed anytime soon?

It's unlikely that we are going to change the things that prevent Tensorflow and Keras from running in a web app any time soon.

So I changed my backend to theano but it still doesn't work. Any suggestions?

What error messages are you getting, if any? There's a link to the error log on the "Web" page, and the most recent error message will be at the bottom of the file.

In keras.json file: Change backend to theano in flask app file: Change tensorflow.keras to only keras everywhere in the file in bash console: pip3 install theano --user (--user is very important otherwise it will not install)

These Changes will make sure your model will start predicting

thanks for helping!

So I've ran into this problem deploying my final year project. I've changed the keras.json file and theano already installed in the virtualenv. Im really not that famillia with threading so I cant really apply the other solutions. the errors I'm getting are :

    2020-04-10 12:40:18 2020-04-10 12:40:18.713118: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-04-10 12:40:18 2020-04-10 12:40:18.713151: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2020-04-10 12:40:18 2020-04-10 12:40:18.713164: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (blue-liveweb19): /proc/driver/nvidia/version does not exist
2020-04-10 12:40:18 2020-04-10 12:40:18.713397: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2020-04-10 12:40:18 2020-04-10 12:40:18.723491: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2300045000 Hz
2020-04-10 12:40:18 2020-04-10 12:40:18.723847: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5711b10 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-04-10 12:40:18 2020-04-10 12:40:18.724541: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-04-10 12:40:18 2020-04-10 12:40:18.728824: F tensorflow/core/platform/env.cc:351] Check failed: -1 != path_length (-1 vs. -1)
2020-04-10 12:40:19 DAMN ! worker 2 (pid: 6) died, killed by signal 6 :( trying respawn ...
2020-04-10 12:40:19 Respawned uWSGI worker 2 (new pid: 53)
2020-04-10 12:40:19 spawned 2 offload threads for uWSGI worker 2

Please let me know If ye find a fix cause I paid money for the months subscription so I really don't want to move to another hosting site.

It appears that you are still loading tensorflow, and not theano. Have a look at https://keras.io/backend/ for how to configure your backend to be theano.

I changed backend to Theano and followed all of the instructions. I can't get past this error now:

ValueError: Unknown initializer: GlorotUniform

I then followed a bunch of other stackoverflow posts with fixes but got new errors instead.

I'm struggling A LOT getting anything to work on pythonanywhere and am also frustrated because I spent money on my membership.

What code are you using to load your model?

Hello again. So I've looked into the Keras site as told by @glenn. And found the following command to check the back-end and it does say I'm using Theano.

keras.backend.backend()

It seems like it cant get past this line of code. Am I doing something wrong here?

model = ts.keras.models.load_model("/home/gedithejedi/indivproj/tsmodels/test.h5")

It also does not seem to output any error.. the response I get on Postman is the PythonAnywhere something went wrong page with error 502.

Thank Ye guys for quick reply and help earlier.

From your logs, I can see this:

2020-04-14 14:26:20 2020-04-14 14:26:20.426770: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-04-14 14:26:20 2020-04-14 14:26:20.426802: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2020-04-14 14:26:20 2020-04-14 14:26:20.426822: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (blue-liveweb19): /proc/driver/nvidia/version does not exist
2020-04-14 14:26:20 2020-04-14 14:26:20.427051: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2020-04-14 14:26:20 2020-04-14 14:26:20.455169: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2300045000 Hz
2020-04-14 14:26:20 2020-04-14 14:26:20.455397: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4b4da30 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-04-14 14:26:20 2020-04-14 14:26:20.455414: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-04-14 14:26:20 2020-04-14 14:26:20.458461: F tensorflow/core/platform/env.cc:351] Check failed: -1 != path_length (-1 vs. -1)
2020-04-14 14:26:21 DAMN ! worker 3 (pid: 9) died, killed by signal 6 :( trying respawn ...

which looks like you're trying to load cuda libraries and it's failing because there is no GPU hardware on PythonAnyhwere. It also looks like your web app is still trying to use tensorflow.

I would suggest trying to move your ML code out of the web app and do the machine learning stuff outside of the web app in an always on task or a sheduled task: http://help.pythonanywhere.com/pages/AsyncInWebApps/

You could also try to extract the machine learning code so that you can try to run it from a console. It may be easier to debug in a console.

I have had the same problem as gedithejedi. I'm confused because I saved my model on Google Colab using a CPU. I got receive these errors even after changing to the Theano backend:

Using Theano backend.
2020-04-24 00:19:23.554136: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror
: libcuda.so.1: cannot open shared object file: No such file or directory
2020-04-24 00:19:23.554173: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2020-04-24 00:19:23.554190: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (b
lue-liveconsole8): /proc/driver/nvidia/version does not exist
2020-04-24 00:19:23.554604: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was no
t compiled to use: AVX2 AVX512F FMA
2020-04-24 00:19:23.562445: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2300045000 Hz
2020-04-24 00:19:23.562633: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5c99210 initialized for platform Host (this does not g
uarantee that XLA will be used). Devices:
2020-04-24 00:19:23.562655: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-04-24 00:19:23.630585: F tensorflow/core/platform/env.cc:351] Check failed: -1 != path_length (-1 vs. -1)

hmm that seems like it is trying to do some sort of cuda/gpu thing? Is that what you are doing?

I am just trying to load an already saved model and its weights from .json and .h5 files; my goal is to use this model to run on a Flask-based website, but I am running these lines in a different .py file. I was able to figure out that the errors were linked to the lines where I started to read the model. Considering I saved these files on Google Colab without any need for GPUs, I was confused why I received these errors.

I just created a model and then loaded it into a web app and it ran fine. What version of keras are you using and what version of Python are you using?

Never mind, I was able to fix this error by switching to Python 3.7 using a shebang/in the main home page of the website. It seems that TensorFlow doesn't work properly on Python 3.8 yet.

That's interesting, thanks for letting us know! Historically TensorFlow didn't work inside website code on PythonAnywhere at all, so it sounds like that might have changed. We're investigating further, because it would be good to get to the bottom of this.

@TheAwesome1568 you are truly awesome sir. Changing the python version to 3.7 was a pain but worked like a charm. Now I can demo my final year project like a boss haha. Thanks

Yup, that's definitely an appropriate username :-) Also, @gedithejedi, thanks for the further confirmation that switching to Python 3.7 helps.

Hi, I am having same problem here.

When I changed from TensorFlow to Theano I got an error: ModuleNotFoundError: No module named 'theano', even with Theano installed.

Also, I am using the Python 3.7 for web app. It's a little bit confusing on what this Python version means, if in terminal if I check the version it shows 3.8 anyway.

I created a virtual environment with Python 3.7.5 and installed all dependencies, but no success.

the module not found error definitely sounds like you have theano installed in the wrong python. (eg: you are running python3.8 but installed to 3.7, or you are running outside/inside of a virtualenv but installed inside/outside of it)

you can specify which python to run- even in the terminal, you could run with python3.8 or python3.7.

for the virtualenv you created, are you also specifying that you want to use it in your webapp / wherever you are running it?

I just deleted the virtual environment and created a new fresh one, in, pyhton 3.7 with Theano backend and it worked well.

@iova0612, I also got this error ValueError: Unknown initializer: GlorotUniform To get rid of it you need to retrain your model without any tensorflow reference with keras. Then load the model with only keras will work even using Theano backend.

@all, FYI, I just tested the same scenario running a composed docker container with 3.7-stretch image and nginx image, running in Linux Ubuntu 19.10, and managed to reproduce the problem. It won't work with tensorflow backend. I'm still investigating, seems tensorflow have problems to run inside a docker container.

that's useful! thanks for letting all of us know! I wonder what's going on with Ubuntu docker container <> tensorflow.