7.WorkerManagement.md

Documentation - Worker Management

Workers are spawned by the command line but will also quit themselves automaticaly. This introduces the need to manage these workers in an automated way.

For Linux there is a system called supervisord. Supervisord controls processes, will start them automatically and restarts them when they are stopped. This part of the SlmQueue documentation does not fully explain all features of supervisord, but it gives a kickstart for users who are in need of worker management.

Basic configuration

Supervisord has a configuration file under /etc/supervisord/supervisord.conf. There you define some basic settings, for example logging. Under [unix_http_server] and [supervisorctl] you have to define some required fields as well:

[supervisord]
logfile          = /var/log/supervisord/supervisord.log
logfile_maxbytes = 50MB
logfile_backups  = 10
loglevel         = error

[unix_http_server]
file             = /tmp/supervisor.sock

[supervisorctl]
serverurl        = unix:///tmp/supervisor.sock

Worker configuration

In supervisord, every process "group" it manages is sectioned under a [program:x] key. Here, x is the name of the program you want to manage. For using SlmQueue in your application, you might want to choose my-app as an appropriate name.

[program:my-app]
command     = php /var/www/mysite/vendor/bin/laminas slm-queue:start default
user        = www-data
autorestart = true

For every program, at least the command line must be set, as supervisord must know which process it should manage. Secondly, there are options to set e.g. the user under which the process will run, but there much more options available. All options can be found at the manual of supervisord.

When a process is killed, for example because the number of maximum runs is reached, it will exit. By default, the exit code 0 is used. In this case, supervisord registers the killed process and starts a new process. Every time the PHP script is stopped, supervisord makes sure a new process spawns again.

In case of an error or exception, the worker will probably be killed very soon. Supervisord checks for stop conditions which occur within 1 second after the process started. If the process is killed within 1 second for 3 consequetive times, supervisord stops respwaning the process. This event will be registered in the log of supervisord.

Multiple workers

When a large number of jobs are inserted into the queue, you might want to spin up more than one worker. Supervisord is capable of managing more processes of one program, under the key numprocs. Because be default, the process name is the program name, you have to define the process_name as well, to distinguish the different processes.

[program:my-app]
user         = www-data
command      = php /var/www/mysite/vendor/bin/laminas slm-queue:start default
numprocs     = 3
process_name = my-app-worker-%(process_num)
autorestart  = true

Navigation

Previous page: Events

1. Introduction
2. Configuration
3. Jobs
4. QueueAware
5. Workers
6. Events
7. Worker management