Forums

I discovered just a bit ago that my python application had gone offline. I checked the error log, which is filled nonstop with lines like this:

2013-06-24 17:59:00,764 :Traceback (most recent call last):
2013-06-24 17:59:00,764 :sqlite3.OperationalError: unable to open database file
2013-06-24 17:59:01,813 :Traceback (most recent call last):
2013-06-24 17:59:01,814 :sqlite3.OperationalError: unable to open database file

Digging back further in the error log, I found this:

OperationalError: disk I/O error
2013-06-24 12:09:15,749 :IOError: write error

So, what happened? Did my app suddenly lose write permissions?

Do you know the name of your sqlite file?

Could you check it still exists and hasn't been deleted. If it's still there, could you check the permissions. Also, could you fire up a python console and try opening the database directly and see if that works.

Thanks

Good advice from @rcs1000. Will follow up by email, but here is some general advice, in case anyone else has a similar problem:

• check you haven't maxed out your quota
• make sure you specify an absolute path to your database -- you can never be sure what the working directory for a web app is.
• try reloading your web app,
• as rcs1000 says, try accessing the database from a console -- sqlite3 /path/to/my.db, then .tables or select * from mytable or some insert into commands to check that data is there, and that you can insert new data.
• try (temporarily?) creating a new database file and switching to that.

Thanks for the reply. The problem was fixed by changing to absolute path, as you suggested.

However, I still find this breakage strange because for the past couple of weeks it worked without a problem, and then suddenly ceased working overnight, a few days after the system upgrades.

Definitely a bit strange... You might consider switching to MySQL if you need a more serious database?

The current working directory of a web app might well change after system updates. Remember, your web app isn't run by you directly, it's invoked by the framework that the PA infrastructure provides. It's entirely possible that the PA devs make changes to the infrastructure which, as a side-effect, changes the current running directory in which the script is run. For example, I wonder if the recent upgrade to an Ubuntu-based system might have affected it.

Best policy is to never make assumptions about the working directory and always specify all paths as absolute. This especially applies if you want to invoke external executables or scripts - always build an absolute path to them rather than relying on them being in the shell's PATH list. Executing off the PATH leads to fragile code which can break when you move it to other platforms and can also introduce subtle security flaws.

Sorry if that sounds preachy, it's just a lesson I've learned the hard way over many, many years of things breaking when I move them to another machine, or someone else does an upgrade, or whatever. Now I try very hard to make as few assumptions about the environment in which I'm running as possible - among other things, assumptions that you want to avoid making are:

• The working directory in which you're running.
• The location of any external files you need (including executables).
• Whether particular environment variables are defined (e.g. PATH, HOME, USER and SHELL).
• The user your code will run as.
• "Well known" locations such as "/tmp" and "/dev/null" and your ability to create or write to files there.
• "Well known" utilities such as ls, mkdir, grep, etc.
• The means of constructing a filesystem path and the type of slash required.
• The location of your source files relative to each other or anything else.

That's just a short list off the top of my head - hopefully you get the idea. Of course there are times when it's expedient to make these assumptions, but it should be a conscious choice to limit your portability rather than coming as a nasty surprise later.

Aside from making sure you use Python's own platform-independent functions (e.g. in os.path), the easiest solution is typically to use a simple config file in a well-known place which you can use to define all the other settings in a platform-specific manner. For example, you can get a file from the platform-specific home directory with:

config_filename = os.path.expanduser(os.path.join("~", "my-app.rc"))

That should even do something sensible on Windows.