Forums

On my website is an algorithm. In order for the algorithm to run I need two packages:

mkl-fft==1.0.6
mkl-random==1.0.1

pip install doesn't work, so I tried installing from github.

pip install git+https://github.com/IntelPython/mkl_random

I get an error saying I need Cython. So I pip install Cython. Still not working because "failed building wheel for mkl-fft."

I'm not really sure where to go from here.

Also, what can I do to get better coherence between my local environment and my pythonanywhere environment? I thought a pip install from requirements.txt would do the trick but I've spent probably 8+ hours now working through reconciling these environments. My local is conda, is that causing the issues?

It looks like that FFT library requires Intel MKL to be installed and we do not have it. I have raised a ticket, but it could be months before we do another system image update to actually get it installed.

I see. I've got a deadline of tomorrow to get this up and running so in the meantime what is my best course of action to get this working if any? Maybe try reworking it in a non-conda environment?

If you have conda can you not just pull in MKL from the intel channel? I am currently doing that from a different direction to get Intel Distribution for Python tweaked to our needs at work. The EULA for IDP and MKL was liberalized earlier this year.

Not sure I'm understanding. I have no problem getting the MKL packages in the conda environment on my local machine. It's getting them into the pythonanywhere environment that's not working.

I ended up resolving this issue. It was a matter of doing a conda install of the nomkl module and then setting up a non MKL local environment I was then able to successfully push the environment to pythonanywhere and both my local and remote environments are now the same.

Unfortunately this didn't solve my problem and the app still isn't working properly.

The app works by doing a call to the twitter api, and then passing the data through a classifier like so:

def api_call(twitter_handle):
    # load the model
    filename = 'trolls_mnb_classifier.pk'
    with open(filename, 'rb') as f:
        loaded_model = pickle.load(f)
    # pull the tweets and process
    tweets = api.user_timeline(screen_name = twitter_handle, count = 200, tweet_mode = 'extended')
    tweet_list = [tweet.full_text for tweet in tweets]
    tweet_list = [' '.join(tweet_list)]
    # predict
    preds = loaded_model.predict_proba(tweet_list)
    return preds

Flask then takes the data and flashes a result:

    if form.validate_on_submit():
    try:
        preds = api_call(form.twitterhandle.data)[0]
        if preds[0] > 0.5:
            flash("Authentic User", 'success')
        elif preds[0] < 0.5:
            flash("Troll", 'danger')
    except:
        flash("Invalid Twitter Handle", 'danger')

This works on my local machine with the same environment but when deployed on pythonanywhere I get the exception every time.

any ideas?

This is what I'm seeing in my access logs when I try to use the app:

72.196.218.102 - - [13/Dec/2018:01:58:43 +0000] "GET /home HTTP/1.1" 200 1034 "http://www.emotionaldogs.com/troll_classifier" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" "72.196.218.102" response-time=0.003
72.196.218.102 - - [13/Dec/2018:01:58:43 +0000] "GET /favicon.ico HTTP/1.1" 404 233 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" "72.196.218.102" response-time=0.001
72.196.218.102 - - [13/Dec/2018:01:58:45 +0000] "GET /troll_classifier HTTP/1.1" 200 3796 "http://www.emotionaldogs.com/home" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" "72.196.218.102" response-time=0.043
72.196.218.102 - - [13/Dec/2018:01:58:45 +0000] "GET /favicon.ico HTTP/1.1" 404 233 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" "72.196.218.102" response-time=0.002
72.196.218.102 - - [13/Dec/2018:01:58:48 +0000] "POST /troll_classifier HTTP/1.1" 200 3828 "http://www.emotionaldogs.com/troll_classifier" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" "72.196.218.102" response-time=0.035
72.196.218.102 - - [13/Dec/2018:01:58:48 +0000] "GET /favicon.ico HTTP/1.1" 404 233 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0" "72.196.218.102" response-time=0.002

You mention you're getting an exception when it's deployed on PythonAnywhere -- where are you seeing that exception?

I don't know. I tried to break down the above try/except loop into its individual components.

I ran the api_call module in the console. It works, I'm able to define the function, pass a twitter handle to it, and it pulls the data and returns a prediction.

When I try and do this on the website however I get the Invalid twitter handle exception every time. It's supposed to work by getting a string from user input into a flask form, and then passing that string to the api call function. The only thing I can think of is the form isn't properly passing the string to the api call? It's not doing the GET/POST properly? I'm not sure how to verify if that's the case though, and what might be causing it or how to fix it.

I wonder, is this something that my cache could be affecting? I've been trying the algorithm on the same twitter handle. If I didn't clear my cache between the time I fixed the MKL dependencies and now, maybe it's just returning the same results as before?

I'm pretty fuzzy on how caches work to be honest so not sure if this could be possible. Web site is blocked at work so I'll give it a test when I get home.

Never mind, it does not seem to be the cache.

...And it was a directory issue. All fixed now.

Ah, glad you worked it out! So if I understand correctly you had some data in a file somewhere and were using a relative path (like "path/to/my/file.txt") to access it instead of an absolute one?

Correct