Using Celery

Using Celery

Retrieving Activity Metadata

It is stated in the xAPI spec that, "If an Activity IRI is an IRL, an LRS SHOULD attempt to GET that IRL, and include in HTTP headers: "Accept: application/json, /". This SHOULD be done as soon as practical after the LRS first encounters the Activity id. Upon loading JSON which is a valid Activity Definition from an IRL used as an Activity id, an LRS SHOULD incorporate the loaded definition into its internal definition for that Activity, while preserving names or definitions not included in the loaded definition."

Because of the SHOULD clause, the ADL LRS will now make retrieving activity metadata optional for LRS setups since this process requires a bit more setup time and isn't too convenient if you want an extremely lightweight LRS system. This will also speed up the processing of incoming statements, especially those that are in large batches.

Voiding Statements

In case batch statements don't come in the correct order when voiding statements, the LRS will wait until all statements from that batch are stored, then will start worker processes to attempt to void the statements that were specified instead of returning a 404 error when trying to void statements sequentially.

Setup

The LRS will be using Celery as an asynchronous task queue and RabbitMQ as its message broker to retrieve the activity metadata. These libraries should already be installed in your environment (amqp and celery) from the requirements.txt document.

RabbitMQ Setup

For details and more in-depth documentation, visit the celery docs.

1. sudo apt-get install rabbitmq-server
2. sudo rabbitmqctl add_user <username> <password>
3. sudo rabbitmqctl add_vhost <vhost_name>
4. sudo rabbitmqctl set_permissions -p <vhost_name> <username> ".*" ".*" ".*"

Celery Setup

For details and more in-depth documentation, visit the celery docs.

1. Configure /path/to/ADL_LRS/lrs/celery.py

   app = Celery('lrs',
            broker='amqp://<username>:<password>@localhost:5672/<vhost_name>',
            include=['lrs.tasks'])
   

2. Configure /path/to/ADL_LRS/celeryd.conf

   command=/path/to/env/bin/celery worker -A lrs --loglevel=INFO
   
   directory=/path/to/ADL_LRS
   

3. Create upstart script in /etc/init (our file will be called supervisorlrs.conf) Also make sure both of these directories already exist - /var/log/supervisord and /var/log/celery. Both directories should be owned by the system user running the LRS.

   description    "supervisor for lrs-celery"
   start on runlevel    [2345]
   stop on runlevel    [!2345]
   
   respawn
   
   setuid <Name of system user LRS is running under>
   chdir /path/to/ADL_LRS
   exec /path/to/env/bin/supervisord --nodaemon
   

   You can stop start/stop the celery tasks by typing sudo {start|stop} supervisorlrs