Adding support for Additional Hooks Paths

[CerealBoy]

Description

We would like to provide the ability to place hooks in multiple locations, meaning there can be multiple hooks of the same type, and they can be provided under different circumstances. These additional hooks are comma separated list, and ordered as they are provided in terms of how multiple hooks are executed. Those in hooks-path are always first, and then sequentially through additional-hooks-paths.

There are 2 places where we pull in these hooks:

1. During the agent startup phase
2. Through the job execution phases

We don't currently support the pre-bootstrap hook with additional paths.

Note

There is no change to any default behaviour included, nor is there any change in existing functionality when configuration is not changed. The only way to have any impact from this change is to set the new option with some value (which should be a comma separated list of paths).

A test is included here to validate the ordering of the hooks is predictable and preserved, however there is a weird timing issue during the test that prevents the additional hooks from being executed successfully. All manual usage of the changes has yielded expected results, so the test is left with t.SkipNow() to highlight that it should be possible to test this, and perhaps there's something I've overlooked here.

Screenshots

Before any additional hook paths are defined:

Example trace showing additional hooks being run for a job:

Build output of the above within Buildkite

Changes

• Adding BUILDKITE_ADDITIONAL_HOOKS_PATHS configuration option
• Find and run any hooks found after hooks-path
  • For the agent-startup hook
  • For hooks within job execution

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go fmt ./...)
• Additional tests covering new functionality (although t.SkipNow() is used)

[wolfeidau]

Looks great 🚀 ready for some testing.