Forums

My site is at http://jddunn.pythonanywhere.com/. The app gets responses from files in the dir, but I'm running into an issue with getting the text file data. Here's how I'm pointing to my files (my site needs to grab data from a pickle file and text files in the app folder).

        self.brain_path = "string_of_folder_name"
        dir = os.path.abspath(os.path.dirname(__file__))
        path = os.path.join(dir, 'data', self.brain_path, 'texts_all.txt')
        with open(path) as f:
                texts_all['all'] = [tuple(re.split(r'\t+', line.strip())) for line in f.readlines()]

I have this wrapped up in a try / except statement, and am getting an error.

I have printed a check statement to see if the file was gotten after being opened, and it does print out the information: "<_io.TextIOWrapper name='/home/jddunn/emoter/data/string_of_folder_name/texts_all.txt' mode='r' encoding='UTF-8'>"

This code and app runs just fine running locally on my Windows computer.

Here's the error I get in the access log.

"POST /chat HTTP/1.1" 500 291 "http://jddunn.pythonanywhere.com/" "Mozilla/5.0 (Windows NT 10.0; WOW64)

My server log just ends at the last line in the code sample above. I really don't know what else to try from here; I've tried looking at threads but I haven't gotten to a solution.

Can anyone help me please? I am trying to get this ready for a show today, and will be unable to run it locally on a computer (which is what I planned).

It looks like it's finding the file OK. Not sure what's happening after that, though. If you have that code wrapped in a try/except it may be hiding the exception from you. Remove the try/except and see if you see an exception. Also, try running the code that is giving you trouble in a console and see what it does.

Thanks for the advice, I was trying to do that but the server wasn't updating the cache in time.

I'm getting an error Unicode string decoding. There are no print statements; the error is in the line:

texts_all['all'] = [tuple(re.split(r'\t+', line.strip())) for line in f.readlines()].

This is the error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x85 in position 5838: invalid start byte;

Is there any reason why the file can be read like that locally on my computer with no issues, but not on Pythonanywhere? Are there any other commands I can try to strip or decode the string properly?

You should read the file using the same encoding as you wrote it, or pass either 'ignore' or 'replace' as the second argument to decode when you decode it.

Here is the error I'm getting. Can you see anything wrong with what I'm doing in the commands? Could there be something wrong with readlines()?

File "/home/jddunn/emoter/emoter.py", line 248, in trainDatabase
    texts_all['all'] = [tuple(re.split(r'\t+', line.encode('utf-8').decode('utf-8', 'ignore').strip())) for line in f.readlines()]
File "/home/jddunn/.virtualenvs/virtenv-emote/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x85 in position 5838: invalid start byte

EDIT: I see what I am doing wrong, 'ignore' needs to be in the open() function. Let me try that and update.

This is the new code I have. It works running locally on my computer.

    with open(path, 'r', encoding = 'utf8', errors = 'ignore') as f:
        print("\n\tTEXTS_ALL: ", f)
        for bline in f:
            try:
                line = tuple(re.split(r'\t+', bline.encode('utf-8').decode('utf-8').strip()))
                texts_all['all'].append(line)
            except UnicodeDecodeError:
                continue

But for some reason, Pythonanywhere is still giving me errors. My server log is just saying:

2017-05-15 17:37:26 2017-05-15 17:37:27 DAMN ! worker 2 (pid: 12615) died, killed by signal 9 :( trying respawn ...

And my error log isn't updating with the latest timestamps. Could this be working code, but just not updated in the cache?

Thanks again for the speedy help.

We have a 2G limit on the amount of memory that a web worker can use. If your workers go over that, then they'll get killed. The logs indicate that that is what is happening.

Does that mean Pythonanywhere isn't suitable for my application? Are there any solutions to this?

EDIT: I see now that I can upgrade my web workers amount. Thank you, I think that will be the solution.

Possibly. If you can't keep the memory use below 2G, then you probably can't run it on PythonAnywhere.

Adding web workers will not help. Your workers are being killed because they individually go over 2G. Adding more won't change that.