Forums

504-loadbalancer error. Website hangs when ML model is run by only one user.

My webapp is a natural language playlist generator. It takes in a description of a playlist, like:

"midwest emo songs to cry to in the shower because my girlfriend broke up with me"

and converts it into an embedding generated by a NLP transformer model and does recommender system stuff to return songs in a playlist for a user.

My website hangs after the user has submitted their description, and I get a 504 load balancer error after 5 minutes. After tracing where the code hangs, it seems to stop during model.encode(text), which runs the user's query through the ML model to get the embedding.

This code runs no problem on my local machine, and when I run it in the console it also has no problem processing the text through the ML model.

What should I do? Add more workers? Free up space in the program? Let me know.

Below are my server logs after model.encode() is run.

2022-11-26 07:53:26 entered the get embedding function

2022-11-26 07:53:27 announcing my loyalty to the Emperor...

2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - *** HARAKIRI ON WORKER 4 (pid: 18, try: 1) ***

2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - HARAKIRI !!! worker 4 status !!!

2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - HARAKIRI [core 0] 10.0.0.75 - POST / since 1669448649

2022-11-26 07:54:11 Sat Nov 26 07:54:10 2022 - HARAKIRI !!! end of worker 4 status !!!

2022-11-26 07:54:11 DAMN ! worker 4 (pid: 18) died, killed by signal 9 :( trying respawn ...

2022-11-26 07:54:11 Respawned uWSGI worker 4 (new pid: 33)

2022-11-26 07:54:11 spawned 2 offload threads for uWSGI worker 4

2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - *** HARAKIRI ON WORKER 3 (pid: 15, try: 1) ***

2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - HARAKIRI !!! worker 3 status !!!

2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - HARAKIRI [core 0] 10.0.0.75 - POST / since 1669449206

2022-11-26 08:03:28 Sat Nov 26 08:03:27 2022 - HARAKIRI !!! end of worker 3 status !!!

2022-11-26 08:03:28 DAMN ! worker 3 (pid: 15) died, killed by signal 9 :( trying respawn ...

2022-11-26 08:03:28 Respawned uWSGI worker 3 (new pid: 36)

2022-11-26 08:03:28 spawned 2 offload threads for uWSGI worker 3

NEVERMIND I FIGURED IT OUT ALL BY MYSELF

I used an always on task to run the queries through the model and spit them back into the main script.

Works like a dream!

Awesome, glad to hear you got it working :-D