AMI testing using ServerSpec

[cb372]

AMIgo should automatically test all AMIs that it bakes.

Tests would be per-role, defined inside the Ansible role directory. There's an example here

[cb372]

This could be useful: https://github.com/envato/ami-spec

[cb372]

I want to work on this as soon as I've finished implementing scheduled bakes.

[cb372]

Here's an email I wrote to Simon ages ago containing a vague implementation plan.

---

I think it makes sense to put the specs inside a 'spec' directory inside the corresponding role's directory. That's where I put my one and only sample spec. The idea is for AMIgo to gather all the specs for the roles it's just installed onto an image, spin up an instance and run the specs (possibly using ami-spec, as we discussed).

At some point we may want to run some kind of whole-system specs as well as per-role ones, but I'm not worrying about that for now.

As for the changes to AMIgo to make this happen, I guess we need at least the following:

• A way of running the specs and collecting and parsing the output
• A new testStatus (Untested, Testing, Passed, Failed) field in the Bake model
• A new Dynamo table in which to store the output
• Extension to the events model to broadcast test logs and/or test results to the event bus
• A way of viewing the test output (either on the bake logs page or a new page)
• Code to add a tag to the AMI once the tests complete

[sihil]

Here are some reflections on how to run serverspec during the build rather than having to spin up the instance a second time: http://annaken.github.io/testing-packer-builds-with-serverspec
h/t: @adamnfish