Forums

Hi, I have multiple hourly tasks, which have not run since around 1am? There are no errors as it looks as though nothing has run.Help please.

When i click 'fetch process list' i get the following message: 'Error fetching list of running scheduled tasks.'

One of the servers had an outage overnight. We've kicked it over, things should be back to normal now. Apologies. Let us know if you see anything else abnormal?

Hi,

My scheduled tasks, which used to work well for months - have stopped running after 8PM yesterday. Now I get a long list of errors in the task log - ending with ErrNo 37: No Locks available" see below.

Is there a problem with PythonAnywhere or my code ? How can I release / increase the number of "available locks" ?

Thanks

web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2015
Version 2.9.12-stable+timestamp.2015.01.17.06.11.03
Database drivers available: sqlite3, pymysql, MySQLdb, mysqlconnector, psycopg2, pg8000, pyodbc, pymongo, imaplib
Traceback (most recent call last):
  File "/home/ICD10/web2py/web2py.py", line 33, in <module>
    gluon.widget.start(cron=True)
  File "/home/ICD10/web2py/gluon/widget.py", line 1128, in start
    cronjob=options.cronjob)
  File "/home/ICD10/web2py/gluon/shell.py", line 237, in run
    _env = env(a, c=c, f=f, import_models=import_models, extra_request=extra_request)
  File "/home/ICD10/web2py/gluon/shell.py", line 158, in env
    environment = build_environment(request, response, session)
  File "/home/ICD10/web2py/gluon/compileapp.py", line 416, in build_environment
    request.env.http_accept_language)
  File "/home/ICD10/web2py/gluon/languages.py", line 481, in __init__
    self.set_current_languages()
  File "/home/ICD10/web2py/gluon/languages.py", line 545, in set_current_languages
    pl_info = self.get_possible_languages_info('default')
  File "/home/ICD10/web2py/gluon/languages.py", line 524, in get_possible_languages_info
    info = read_possible_languages(self.langpath)
  File "/home/ICD10/web2py/gluon/languages.py", line 278, in read_possible_languages
    lambda: read_possible_languages_aux(langpath))
  File "/home/ICD10/web2py/gluon/cfs.py", line 50, in getcfs
    data = filter()
  File "/home/ICD10/web2py/gluon/languages.py", line 278, in <lambda>
    lambda: read_possible_languages_aux(langpath))
  File "/home/ICD10/web2py/gluon/languages.py", line 254, in read_possible_languages_aux
    d = read_dict(fname_with_path)
  File "/home/ICD10/web2py/gluon/languages.py", line 174, in read_dict
    lambda: read_dict_aux(filename))
  File "/home/ICD10/web2py/gluon/cfs.py", line 50, in getcfs
    data = filter()
  File "/home/ICD10/web2py/gluon/languages.py", line 174, in <lambda>
    lambda: read_dict_aux(filename))
  File "/home/ICD10/web2py/gluon/languages.py", line 159, in read_dict_aux
    lang_text = read_locked(filename).replace('\r\n', '\n')
  File "/home/ICD10/web2py/gluon/portalocker.py", line 151, in read_locked
    fp = LockedFile(filename, 'r')
  File "/home/ICD10/web2py/gluon/portalocker.py", line 116, in __init__
    lock(self.file, LOCK_SH)
  File "/home/ICD10/web2py/gluon/portalocker.py", line 86, in lock
    fcntl.flock(file.fileno(), flags)
IOError: [Errno 37] No locks available
2015-10-05 15:09:09 -- Completed task, took 5.00 seconds, return code was 1.

I have the no locks available message as well.

Well, if others experience the same issue with scheduled tasks, then it seems the problem is with PythonAnywhere and not our code...

BTW - reloading the app doesn't help with this issue.

Hi there, I can confirm there does seem to be an issue with locks on one of the servers. Will look into it as soon as possible.

Still no scheduled tasks can be performed.

Any estimate when issue with locks on one of the servers will be taken care of ?

Thanks

We're working on it.

Hi gang. Took us a while to track down the culprit, it turns out one of the disks was full -- an obvious thing in retrospect, it just wasn't the disk the lock was happening on, so that's why we didn't see it at first.

In any case, in our testing, fcntl locks are working again. Let us know if it looks like your scripts are running once more?

Thanks for your patience.

IOError: [Errno 37] No locks available

2015-10-06 13:50:16 -- Completed task, took 7.00 seconds, return code was 0.

Still not running. Thanks, Peter

That's from a couple of hours ago. Have you got a task that's run since 3PM?

2015-10-06 16:01:11 -- Completed task, took 2.00 seconds, return code was 0. Task is running now. Thank you! Peter

You're welcome. Thanks again for your patience!

Hey,

I'm having the same issue as in first post. Same error when fetching the running tasks. My tasks not running since 18:00.

Tested on a free account too, and the scheduled task is on strike as well: "Task has not been run yet."

*Never had my script files as executables but they managed to work. Now I've set them +x, as said in the Scheduled tasks help page, but still no joy.

+1 for same issue as OP and inxflav,

my scheduled processes are not starting up, logs show "Task has not been run yet", i dont think they ever start though refreshing the current process list shows "Error fetching list of running scheduled tasks" message.

my scripts run fine if i run them from bash console

My scripts run fine from PythonAnywhere Schedule...

Ooops ! Checked the logs and the scripts have stopped running (no error codes) Oct 9 around 06:00

Scheduled tasks have failed again!!!

Yep dynamo, there's a problem somewhere, and I think they are nailing it down and we'll be told when things become clearer.

As also points krav, meanwhile we can run the scheduled scripts from the bash console, until the PA crew gets it over.

Looks like the server that your tasks are run on fell over. I'm concerned that our monitoring didn't notice and I'll be investigating why and adding additional monitoring so that we can be alerted to this in future and sort it out.

Looks like this issue is coming up again - I am currently getting "Error fetching list of scheduled tasks" and the log is blank for one of my scheduled tasks. Can someone take a look?

Thanks, DQ

The last time my tasks have run was on 2016-04-30 06:03:13 Checking the logs I realize that the MySQL Backup task is not running for since 2016-04-26 11:00:59

It used to run Ok , but the log now it says.... mysqldump: Got error: 2003: Can't connect to MySQL server on 'mysql.server' (111) when trying to connect 2016-04-26 11:00:59 -- Completed task, took 30.00 seconds, return code was 2.

Are you aware of any issues on PythonAnywhere side ?

Yup. We had an outage on one of the task servers. It's fixed now.

Yes. Scheduled tasks seem to be back on track.

The specific issue with the mysql backup task that has stopped running for me... I've noticed it has stopped working "coincidentally" on the same day I've modified the name of the server (...)

So I've modified the script from the old: mysqldump -u AppName -password -hmysql.server 'AppName$default' 'auth_event'> /home/AppName/tablename.sql;

to the new:

mysqldump -u AppName -password -h-hICD10.mysql.pythonanywhere-services.com 'AppName$default' 'auth_event'> /home/AppName/tablename.sql;

Seems this specific mysql backup task is working now