Forums

Parallel long running tasks? Socket lock that does not lock leading to multiple instances?

Hi,

I am trying to follow your code at Long Running Tasks, but the code still allows multiple spawns of my program. Rather than pasting my entire code here, the following trivial code exemplifies what I mean. This code simply prints 'Hello" on the screen every 5 seconds until a keyboard interrupt.

Running this code for first time, works as expected. But running it again launches another instance. The so-called socket lock fails to lock, so it will run as many times I execute the code. What is going on? I have checked this forum for similar cases, but no one seems to be able to give me a clear, straightforward answer.

I appreciate any help. Thank you.

import logging
import socket
import sys


lock_socket = None


def is_lock_free():
    global lock_socket
    lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
    try:
        lock_id = "myname.testing123"
        lock_socket.bind('\0' + lock_id)
        logging.debug("Acquired lock %r" % (lock_id,))
        return True
    except socket.error:
        # socket already locked, task must already be running
        logging.info("Failed to acquire lock %r" % (lock_id,))
        return False


if not is_lock_free():
    sys.exit()


import time


while True:
    print('Hello...')
    time.sleep(5)

That code only works when the 2 processes are running on the same machine. If you're running it in different consoles, then they're probably on different machines.

So, if this code was only launched from the PA scheduler (e.g., at every one hour) it is guaranteed to work, because it is always run from the same machine?

Yes.

I "append myself" to this conversation because I think my question is relevant to this. Given that if I launch manually the task on two bash consoles the lock does not work because they are on two different machines, how can I test that my lock is effectively working then?

Consider that:

  • I have modified the example code quite a bit, thus I need to test it
  • I am just playing with PythonAnywhere since a few days for the first time and thus I am still on a free account (but soon I will be a paying user, I am enjoying it so much :)). Therefore it is not really feasible (i.e.: clever) to create a long running task that lasts for 2 days just for testing if the lock is working
  • I develop on a MAC and I get "[Errno 2] No such file or directory" when trying to get lock just for testing, but I am not expert enough with sockets to understand why that happens...

Thank you for the wonderful support.

You can use tmux, which will allow you to essentially have 2 bash prompts in one console, so they will be on the same machine.

Nice idea, thank you, I will try it!