Forums

I have a "data refresh" process that I need to run every ~5 minutes or so (and it takes ~60 seconds to complete). I currently implement this with 12 separate scheduled tasks (one for every 5 minute increment). This works fine. However, it can obviously be implemented using a single always-on task with a "while True" loop and a "sleep(60*5)" statement. Which approach would you recommend I take from the perspective of reliability and cost?

I don't think there's any objective reason to choose one over the other. Choose the one that you find the easiest to manage and reason about.

Got it. But how would the pricing differ in terms of required CPU minutes? 12 scheduled tasks that each take 60 seconds equates to 17,280 minutes per day. In contrast, won't a single always-on task require 86,400 CPU minutes? It seems like that would equate to a difference of ~$60 per month. Am I doing the math correctly?

time.sleep does not consume CPU time. There may be a difference in how much CPU each of the methods uses, but it will be negligible.

Got it. I didn't know that about sleep(). The @huey.periodic_task decorator (https://huey.readthedocs.io/en/latest/guide.html#periodic-tasks) seems to be another option. The huey_consumer script can presumably be run using an always-on task. Do you have any experience with this? Is there any reason this might be a bad idea e.g., impact on CPU minutes? (Also, since it supports storing tasks in a sqlite flat file, this also seems like a good way to handle PA's restriction on using Celery/Redis.)

Hmm I personally don't have any experience with huey.

However, do take a look at cpu seconds- unless huey somehow waits busily etc, I don't think it would impact CPU seconds.

Huey is apparently not single threaded (https://huey.readthedocs.io/en/latest/consumer.html#worker-types). I know PA doesn't support multi-threading for web apps. Does multi-threading work for always-on tasks?

Multi-threading should work without problem in always-on tasks.