Forums

Always-on Tasks not starting

Was going to post my issue here when I saw that the forum may have an issue. Hope it is not too bad. Been there.

Anyway here goes.

My always on tasks no longer start. I get no output either from the logs in my code or the task logs.

If I delete a task and make a new identical one, when I start it I get a Task preparing to start message in the task log, but that is it. No processes start.

The state on the website for the tasks says Starting.

I am experiencing the same error. the log just says

2019-11-23 12:35:46 - Task preparing to start

thanks for reporting- let me investigate

Having the same issue here was running fine up until about 4 hours ago and suddenly stopped working so i restarted the task and has been stuck on the “Starting” mode...

I have identified one of our task servers that was having some problems perhaps starting at around 4am UTC. This has now been resolved. Do let us know if you are still seeing any problems! Thanks again for reporting, and sorry for the inconveniences- we are working on how to get better alerting for this.

my task just started at 16:16 , and changed the status to Running

Nov 23 16:16:23 done  2019-11-23 16:16:23.263009

Seems to be working now :) Thanks.

Excellent, thanks for confirming!

I just started having this same problem today

daleofthehill, I see all your tasks in "running" state

Bump, this issue has returned. For me at least.

We didn't get any alerts about always-on task problems, and the logs are clear of any obvious issues. I see that your tasks are both running at the moment. How long had they been in the "starting" state when you posted your message above?

giles, they're still starting. They've been like that for a day or two now. I'll try restarting them.

Never mind, it fixed itself.

can't this issue be fixed once and for all

hi there, what problems are you seeing? I see that you currently don't have any tasks on PythonAnywhere from the account that you are posting from.

hi,

there is a scheduled downtime at 6am UTC today and the always-on-task under my account (“john.kho@outlook.com.au”) is stuck in “preparing to start” state and cannot be deleted.

please help & thanks!!

John Kho

It looks like it's running now. Sometimes, after a deploy, some always on tasks may take a little while to restart correctly.

Hello. Always-on won't work. Have been waiting for 10 hours, no results. I've created a new vitual env - that didn't help as well. Now it is in a state of "Starting" for 15 minutes.

What command are you using to run your tasks?

I am using

workon myvirtenv $$ python /home/PKasyanov1/bot/script.py

Looks like it should be && not $$

Oh yeah, my bad) Please help me with this one, can't use my xlsx file in a script from always on task. It cannot find it

Have a look at https://help.pythonanywhere.com/pages/NoSuchFileOrDirectory/

Good afternoon! My Always-on task doesn't work(((

Neither

python3 /home/Nickname/path/script.py

no

python3.8 /home/Nickname/path/script.py

no

source /home/Nickname/path/venv/bin/activate && python /home/Nickname/path/script.py

don't make my script working. Can't get what is the problem in? Can you help me with that, please?

How is your script not working? I see it in a "running" state.

Yes, it is in a "running" state, but nothing happens and there is no notes in task log, only that:

2020-11-17 08:19:08 - Task preparing to start

Although if i run that script through bash-console it works well.

Could we look into your code? We can but we ask first

Yes, you are welcome.

There is no logging in your script so it won't leave any notes in the task log. If the script is not returning the expected output, I'd suggest adding some logging to it in order to debug it.

Oh, the problem was into my code. Find it - it needs to operate with absolute paths in my case. Thanks for your help!

Glad to see that you figured it out!

Greetings! Thanks for the helpful and understandable service. I am a beginner programmer and am trying to deploy my first project on Pythonanywhere. This is a Telegram bot. Everything works well on my PC and in the console mode of your service. But as you know, consoles are short-lived, so I became interested in always running tasks mode. Unfortunately, I am haunted by the same problem - the application starts, but stops responding after a short time of work. While remaining in the "Running" state, just doesn't react. Perhaps I have not mastered virtualization (if it is needed) or am I launching with an incorrect command? I just do "python3.5 /home/MaktoumNorsaphi/my_project_env/db.py". The situation is repeated after a manual restart. Please, tell me what my mistake is.

Depending on how your bot works, could you give it a heartbeat? ie. something that prints to a log every 30 seconds etc.

For an always on task, if it dies we restart it for you. Do you see messages about it stopping and being restarted etc?

Hello. I'm trying to start my telegram bot in always-on tasks, but when I start, I get errors that I cannot find the files necessary for the bot to work (these files are in the bot folder). From the console, the bot works fine. P.S. I am writing this message through a translator, so I apologize if something is written strangely.

I think the difference is the working directory. If you run this in Bash:

cd /home/PocketBrain/myscripts
python bot.py

...then the working directory will be /home/PocketBrain/myscripts, so a reference to a file like config.cfg would expand to /home/PocketBrain/myscripts/config.cfg

By comparison, if you do this:

python /home/PocketBrain/myscripts/bot.py

...then the working directory will be the default, /home/PocketBrain/, so a reference to a file like config.cfg would expand to /home/PocketBrain/config.cfg

This help page has some suggested solutions. I recommend in particular the "file for cross-platform scripts" section.

Thanks for help

No problem!

Hi, this issue faced me,, the status "Task preparing to start " not changed and there are no tasks running

Your task is starting and then immediately failing with an error. Look in the log file for the task to see the error so you can fix it.

I'm having this same issue, if I run my loader.py in a console it works fine, but as an always on task it just hangs on starting and won't ever actually run

What is the task doing? If it finishes quickly and is not printing anything it will stop and start again, but it may happen fast enough so from the point of view of our API it will be always in a strting state.

Thanks for the reply pafk, that shouldn’t be the case though, the script attached to the always on task kicks off two files and each runs a bot that sits in a listening state for messages etc

Edit: I also know they’re not running because I have a heartbeat listener and it doesn’t fire, so I can’t get responses from the bots and the heartbeat doesn’t print

Could you add some logging to the always-on task script, so we could determine where (and possibly how) it fails?

I can put some additional logging in but it doesn’t even hit the “booting up” print I have in there, is there some specific way I’ll need to write the code for printing to the log? I’m on day 3 of my membership so I could be missing something, but what’s interesting is the same stack of code/files has been running via one of my two consoles for a couple days, just worried for the future as consoles seem to get cut more often

EDIT: Even if I put print('test') as my first action it doesn't show in my log

EDIT EDIT: I think I have isolated the issue down to subprocess.Popen(["python3.9", "py file here"]) VS subprocess.call(["python3.9", "py file here"])

If I try to set the loader.py file to run two tasks using the Popen command it will never start/run, if I choose to use the "call" command instead it will run but with the problem being "call" is process blocking whereas "Popen" is not, so the Popen would allow me to run consecutive subproccesses as I do in the console and "call" will only allow the first subproccess to run, I assume it would run the next one when the first completes but they don't "complete" by design they listen, so I need the Popen for the non blocking design

Make sure you are flushing the output when you print:

print("something", flush=True)

Okay, so by adding flush=true I can get a print to work in the log

it hits the initial print but doesn't hit any inner prints for I have in the subprocesses for logging, UNLESS I switch from subprocess.Popen -> subprocess.call, why should this matter? If the always on task is unable to run non blocking subprocesses I would think that limitation would be documented.


NOT WORKING:

import sys;
import subprocess;

print("test", flush=True)

bot_files = [
    "/pathtobot/bot1.py"
    ,"/pathtobot/bot2.py"
]

for botFile in bot_files:
    subprocess.Popen(["python3.9", botFile])

WORKING (but only kicks off bot1):

import sys;
import subprocess;

print("test", flush=True)

bot_files = [
    "/pathtobot/bot1.py"
    ,"/pathtobot/bot2.py"
]

for botFile in bot_files:
    subprocess.call(["python3.9", botFile])

Call waits for subproces to finish. There is no limitation of that kind in always-on tasks. You can use threading for example.

Okay... I want to believe what I'm being told guys but I'm proving otherwise with every test you have me perform...

The new code:

import sys;
import subprocess;
import threading;

# Edit this to add each python file you wish to use

#print(sys.path)
def spinup_Thread(myMethod):
    thr = threading.Thread(target=myMethod, args=(), kwargs={})
    thr.start()

def start_Bot(botFile):
    subprocess.call(["python3.9", botFile])

bot_files = [
    "/pathToBot/Bot1.py"
    ,"/pathToBot/Bot2.py"
]

for botFile in bot_files:
    spinup_Thread(start_Bot(botFile))

# Do not edit below this line

Now this RUNS but guess what? Still only bot 1 will run...

Output: 2022-03-29 13:23:44 - Task preparing to start Mar 29 13:29:53 We have logged in as Bot1#XXXX

Now THIS Code doesn't even run/start:

import sys;
import subprocess;
import threading;

# Edit this to add each python file you wish to use

#print(sys.path)
def spinup_Thread(myMethod):
    thr = threading.Thread(target=myMethod, args=(), kwargs={})
    thr.start()

def start_Bot(botFile):
    subprocess.**Popen**(["python3.9", botFile])

bot_files = [
    "/pathToBot/Bot1.py"
    ,"/pathToBot/Bot2.py"
]

for botFile in bot_files:
    spinup_Thread(start_Bot(botFile))

But here is the kicker to all of this still... ALL of the sample code I have posted to this thread runs FINE if done via A CONSOLE, to be clear my code is only proving to be an issue when ran as a TASK...

I have tried threading, I have tried subprocesses both call & Popen (and yes I know the difference, Popen would be preferred as it doesn't care about process completion) but none of it works, so if you're telling me there are no limitations I'm telling you there are problems

I can't stress enough how I want this service to work out for me, your consoles do exactly what I need and all of the code I have uploaded runs via the consoles (runs extremely well) so I'm able to limp by currently but need this always on task to work so I don't have to kick the jukebox every couple days so to speak...

Can we take a look at your code, and a running instance of the always-on task showing the problem?

@giles can we set this up via email? Not keen on sharing my code publicly but can certainly remove tokens and show it to staff

And unfortunately I won't be able to provide you with a running instance of the always-on task, I can provide you with the perpetually starting task, its possible you'll be able to see logs I cannot but in my attempts to print caught errors or anything outside of a heartbeat I've came up short

Do you have a specific email you'd like me to share to?

EDIT: if it makes life easier we could setup a screenshare or if you have the capacity you have my consent to just go into my dashboard to explore this particular situation

If you would like to switch this to email, you can send your email to support@pythonanywhere.com

[Always on Task] suddenly stop working since around 1am UTC. I tried to restart same task then now 'Task preparing to start'. [State] is 'Starting'. Same Task had been working fine for several months, never had a this problem. Please help!

FYI, I tried changing from the "Always on Task" to a "Scheduled Task" that runs hourly and it seems to be working as a temporary solution.

We did have an issue with networking on our always-on system at around 1am, which would explain the restart, but it should have come back up again afterwards. Was it logging anything when it appeared to be stuck in the "Starting" state? If it task is "Starting" for a long time, it normally means that it's starting up, the exiting immediately (or almost immediately), then starting again, and so on -- you could potentially see it in a "Running" state if you got the timing exactly right, but if those windows of opportunity are small enough you'll essentially just see the "Starting" state. So the log files are a good place to look, just in case the problem is that the task is crashing on startup.

Hi, I have a similar situation as @BlazdRiteNow, an "Always on Task", running and/or closing bots, was his issue solved? I have a basic question, if I use subprocess.Popen or os.system(f'nohup python -u {file_name} >> out.txt 2>&1 &') to start my processes, when I want to check if they are running I use psutil, when I tried to do it from console I realized that every console had its own process list independents so I think with "Always on task" will work fine. The question is, no matter if my process crashes or the "Always on task" crashes, will I always be able to find the process with psutil if I run it from the "Always on task" script?

No, you would not be able to see it form the other alwaysontask as it is all isolated.

I'll try to explain better (sorry about my English) I have a script that is looking for updates in a file or DB that has the processes that I have to run or stop, so the script I something like: 1. Look for updates. 2. If processes are in the list, the script start the process, if its already running does nothing. Or 3. If processes are not in the list, the script stop it if its running. 4. Go step 1 This is the script I want to run as alwaysontask, since I start each process using os.system (f'nohup python -u {file_name} >> out.txt 2>&1 &') , where will be my processes be running? each one in one different console? the same console as the alwaysontask? the same as the web server? Because I need also to eventually check if they are running by the psutil library. Thank you. Another question, do you think Im using the rights tools? (os.system, psutil) thank you.

All always-on task processes run in their own sandbox, so from within a task, you'll see all of the subprocesses that it has spawned if you use psutil (or similar tools). However, if the "root" process of the task -- that is, the one that is launched when it starts up, to run the command that you specify on the "Tasks" page when you create the task -- crashes or otherwise exits, then all of its subprocesses will be killed, and the top-level command will be started up again.

Thank you, understood.

Hello guys I would like to have X always on tasks do you know if it's possible? cause looking at plans they are all limited to 1, incase you know any python script that can do this for me?

You can use the "Customise your plan" button to add always on tasks to your account.

Hello, I have this problem. "Always-on tasks" state "running", but log gives: 2024-02-17 22:51:35 - Task preparing to start. What to do about that. It is happening from friday, but earlier script worked perfectly.

We had some issues with our logging system related always-on tasks -- could you try restarting the task?

I restarted for few times, same situation

You might see an error after a while

Also running into an error - it's stuck in "Task preparing to start" even though I've double checked the directory is correct. Runs fine in Bash

It's a bit ridiculous that this thread was started in 2019... Could the PA team please at least make this error in log "Task preparing to start" a bit less ambiguous?

Look in the log file of your always on task to see what it happening to it.