Restarting Tango

Restart Tango

Do this if Tango is not stable or if you need to change Tango configurations

1. Login to armstrong
2. Bounce into the tango_deployment container
3. Check the status of the running services

supervisorctl status

You should see something like the following output

nginx                            RUNNING    pid 23246, uptime 26 days, 21:48:20
redis_server                     RUNNING    pid 23150, uptime 26 days, 21:49:15
tango:0                          RUNNING    pid 23217, uptime 26 days, 21:48:20
tango:1                          RUNNING    pid 23203, uptime 26 days, 21:48:20
tangoJobManager                  RUNNING    pid 23158, uptime 26 days, 21:48:20
wrapdocker                       RUNNING    pid 23159, uptime 26 days, 21:48:20

4. Stop all services

supervisorctl stop all

5. Ensure all services are stoped

ps aux

The following three processes should not appear

... python /opt/TangoService/Tango/jobManager.py
... python /opt/TangoService/Tango/restful-tango/server.py 8611
... python /opt/TangoService/Tango/restful-tango/server.py 8610

If they are not stopped, kill the pids

kill -9 {pid}

6. Start the Redis service

supervisorctl start redis_server

7. Blow away the Redis cache to clear the job queue/etc.

redis-cli
> FLUSHALL
> ctrl+C

8. Start the services

supervisorctl start all

9. Wait. Startup has been known to take 10+ minutes
10. Navigate to AutoLab and check the Tango status to verify that it is running
11. Send Tango a job and verify that it is successful