Forums

Hello, I have a site using dash/python 3.8 that ingests data from a small csv file and a larger (60M) netcdf file. It runs an update cycle that subsets these two files according to user-selected variables and/or time period, updating a map accordingly. The site is up and updates most of the time, however sometimes it does not respond to user selections. Some more clicking and control manipulation will cause it to update again, and then again it will stop update. The error log indicates a variable from the netcdf file (or the netcdf file itself) is not found, however it will then plot okay if the user continues manipulating the controls, so the file is found and the variable is extracted on some occasions.

It is live now; to have a look at it here is the login info:

https://atkinsonde.pythonanywhere.com/
datkinson
QtPzU2bF

...and here is an error message that is written to the error log (this is for one of the netcdf variables, "shts", but it will also sometimes do it on the other variable in the netcdf file, "shww".)

File "/home/atkinsonde/mysite/app.py", line 254, in update_figure
    ecmwf_sw= pd.DataFrame(nco['shts'][single_date_pick].filled(fill_value=0)).set_index(latf)
  File "src/netCDF4/_netCDF4.pyx", line 4420, in netCDF4._netCDF4.Variable.__getitem__
  File "src/netCDF4/_netCDF4.pyx", line 5363, in netCDF4._netCDF4.Variable._get
  File "src/netCDF4/_netCDF4.pyx", line 1950, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: No such file or directory

I've been thinking file size? But it's well below the 100M limit. Lag? But sometimes it updates quite promptly. File access? No, most of the time it reads the file. My app.run is embedded in main, as:

if __name__ == '__main__':
    app.run()

I can't figure out what it could be. Have I missed something obvious? I'm new to pythonanywhere and could easily have overlooked something, e.g. maybe I'm not launching the app correctly. An earlier iteration of the site, before the netcdf file was incorporated, worked fine . Thanks all.

[edit by admin: formatting]

That does sound very strange. The one thing that comes to mind first is that it could be related to absolute vs relative paths. If you specify a path as (say) "foo.txt" then that path will be resolved relative to the current working directory of the process, which can change over time. That could potentially lead to issues like the ones you describe, if sometimes the working directory is one where that file path can resolve correctly and sometimes is not. You (or one of the libraries you use) would need to be changing directory from time to time for that to be the case, though.

Are you using relative paths like that to specify files? Or are all of your files specified using absolute paths (starting with "/")?

Both data files are absolute path.

Actually I'm wondering if it's a dash issue, rather than pythonanywhere. I'll post the question to their help line as well.