Forums

Hi. I've got some code (basically web scrape + store in MySQL db) that can take upwards of 10-20 minutes to run. Because of this time I'm finding that it times out when run in a Flask web app. I am therefore thinking an always-on task might be appropriate.

If I run my py script as an always on task, will it only truly be 'always on' if there's an infinite loop? Or, once the script has finished executing, will it just re-start from the top?

Basically, I want my script to run (10-20+ minutes) without timing out, and then stop/exit. I don't want it to repeat.

Is this a reasonable use case for an always on task? Can I execute the code once in this manner, without it repeating?

Unless there's a better suggestion, I'm thinking a workaround might be to just run my code as an always-on task, and then at the end of the code I can run an infinite while loop to prevent starting from the top and duplicating what has already been done. To avoid duplication of effort in the case the script crashes and therefore needs to restart, I can build checks into the code to determine if it had already been successfully executed, and therefore jump straight back to the infinite while loop.

How often do you want your script to be run? Did you check regular scheduled tasks? I guess a daily schedule might be a good solution for you.

I did consider scheduled tasks, though It’ll be running on a bit of an ad hoc schedule. Less frequent than daily, but probably more than Monthly. Eventually I’ll probably get to a point where consistent monthly is good, but for now I need a bit more flexibility.

Maybe another option for now is to set up scheduled tasks starting on the specific days I want to run it, and then just kill the schedule after one run (until I decide to re-create).

Yes, I'd suggest you just schedule a daily task which runs your script on specific day(s) of the week. No infinite loop or killing and re-creating tasks needed then.

Thanks for the help, appreciated!

Thinking of other possible approaches, I just saw that the Pythonanywhere API (https://help.pythonanywhere.com/pages/API/) has a DELETE method for always-on tasks. Is a feasible option to call that API once the script has completed successfully?

You could do it that way if you want to.

Great, thanks for the input!