Troubleshooting Production

See the Architectural Overview to get an understanding of the parts of RapidFTR. This document assumes that your production setup is using our chef repo or is very similar to the setup from the chef repo. (The chef repo stuff is very new, but it's been used in our only production deployment so far.)

If no pages are getting served, then see whether Nginx is running.

If the login page is getting served but users get an error page after entering their username and password, then it's likely CouchDB is not running. From the server, try this.

curl http://localhost:5984/

You should get something like this if couch is up and running.

{"couchdb":"Welcome","version":"1.0.1"}

(Your version may vary.) If it's down, try starting it with this.

sudo /etc/init.d/couchdb start

If CouchDB is up, but things are still not working correctly, you can look at the Couch logs with this command on the server.

sudo less /var/log/couchdb/couch.log

If you can view and register new children but get an error page when you try to search, then it's likely Solr is not running. From the server, try this.

curl http://localhost:8902/

You should get a 404 page that provides a link to /solr. If Solr is down, try starting it with this.

sudo /etc/init.d/solr start

If there's some data issue requiring you to browse the CouchDB data, you won't be able to access Futon (the CouchDB UI) from your browser due to firewall restrictions. You can access it using ssh port forwarding. From your workstation, log into the server via ssh, adding the -L (dash-capital-L argument).

ssh [email protected] -L 5999:localhost:5984

After logging in, open your browser to http://localhost:5999/_utils and you should see the remote server's Futon.