Skip to content

Web server for pulling from github and updating our static content websites

License

Notifications You must be signed in to change notification settings

igsr/gca_website_updater

 
 

Repository files navigation

gca_website_builder

Web server for pulling from github and updating static content

Ian's advice: I recommend you never do any more development on the static website udpater. Use something like Bamboo instead. Better to use a tool designed exactly for this purpose instead of this custom code. And our friends in the SPOT team already use bamboo.

The only endpoint is /update_project/{project_name}

e.g. curl -XPOST http://localhost:8001/update_project/hipsci

The process is basically this:

  1. Queries the rate limiter to find out if the project is already updating
  • Queues a new job and exits if a job is already queued
  1. Uses the GitUpdater module to pull updates from git
  2. Uses the NPMBuilder module to build the static content from the git repo
  3. Uses the Rsyncer module to copy the jekyll _site directory to the webserver directories
  4. Uses the ElasticSitemapIndexer to put the static content into elasticsearch. This enables site search.
  5. Uses the PubSubHubBub module to announce updates to the rss feeds. This triggers automatic tweets.
  6. Repeat 2-6 if a new job has been queued since the update began

Steps 2-6 get executed in a forked process because they are blocking.

Errors get emailed to users, set in the config file

Install

./etc/init.d/website_updater install

This installs the following to the ./local directory:

  1. cpan modules
  2. Ensembl git tools
  3. git (because it's not installed on our VMs by default)

Run the installation command from a login node with the same linux distribution as our VM, i.e. from a rh-6 login machine

About

Web server for pulling from github and updating our static content websites

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Perl 91.1%
  • Shell 8.9%