Forums

Importing pickled file doesn't work on a task, but does work when I run the script

Hi all,

I'm having an issue getting a task to run without kicking up an error message, and the issue stems from a pickled file that's kicking off a FileNotFoundError when it's imported.

However, if I run the file from the console everything works directly, so I'm not sure why it doesn't work when run as a task.

Here's my basic setup:

File Structure:

app.py
db.py
info.pkl

In db.py I import a pickled file that looks like this:

with open('info.pkl', 'rb') as info:
    connect_info = pickle.load(info)

Then in app.py I have lines that look like this:

from db import connect_info

jh_db_df  = connect_to_dbdata(query, connect_info)
if new_jh_rows.shape[0] > 0:
    add_jh_rows(new_jh_rows, connect_info)

If I run app.py from my terminal everything is fine.

But when I schedule it as a task the following error message always turns up:

 Traceback (most recent call last):
 File "/home/jonathanbechtel/covid/app.py", line 1, in <module>
 from db import connect_to_jhdata, connect_to_dbdata, find_new_rows, add_jh_rows, add_states_rows, 
  connect_to_statesdata
File "/home/jonathanbechtel/covid/db.py", line 6, in <module>
  with open('info.pkl', 'rb') as info:
FileNotFoundError: [Errno 2] No such file or directory: 'info.pkl'

Any idea why this might be the case?

Use the full/absolute path instead of the relative path.