Forums

Usually when I have a large list of email addresses to send to after a user POSTs, I start a thread like so: :::t = threading.Thread(target= sendoff_emails_thread, args=[request], kwargs = {}) t.setDaemon(True) t.start()

I just tested my code locally, pushed to my repo, pulled into PA, and reloaded my webapp. But the emails aren't getting sent when testing on PA.

I made a thread in iPython in a console and it works fine. Is there something I'm missing? I have a "hacker" account if that matters. (I read <a href="https://www.pythonanywhere.com/forums/topic/432/">here</a> that threads are blocked, but I am not getting any error messages)

Thanks

We currently don't run web apps in an environment where new threads can be started. We're considering the possibility of allowing threads for web apps, but we'd have to be sure that it's a good idea for everyone.

Personally I would really recommend staying away from threads in web apps, and especially staying well away from processes (i.e. os.fork()). Webservers tend to use both threads and processes for their own purposes, and creating instances of your own for any other purpose can introduce subtle problems.

Also, HTTP in general isn't really designed for long-running requests - whilst they work, they tend not to be very robust. The chances of something going wrong in the middle of your request are much higher.

I suggest having a background work queue where your request submits items and then uses Javascript in the browser to periodically poll the progress. This has the boon that if something goes wrong with a request, the Javascript can simply retry again a few seconds later. Also, the user could in principle close their browser and come back later to see the progress. This approach is probably more effort, but if you care much about scalability and robustness then I really wouldn't suggest doing too much back-end work in the context of a single HTTP request.

This sort of thing is easier with beanstalkd but until the PA devs have chance to get background daemons and the like working I wouldn't suggest that at present. You could do something with a background task and a scheduled job to restart it as necessary, but presently there isn't really a way to make it bullet-proof - the PA devs have spoken about support for background daemons, but they seem pretty busy right now so I wouldn't expect it for a little while.

Sounds to me like you're just doing IO in the thread (i.e. talking to remote email servers), so for now maybe you can just use non-blocking IO in a single thread instead of spawning multiple threads? In general non-blocking IO is a better approach than threading anyway - lighter on system resources and much more predictable. See the select module for examples.

You could add all the emails to some kind of tasks list in your database, and then use a scheduled task which runs every hour and deals with all the emails from the task list?

Yes, that would work, although I was rather assuming that the user would want some immediate feedback on whether the emails had been sent. Still, that's an assumption on my part, and you know what they say about assumption being the mother of all creatively designated features...