Feature: Job post-run hook

[mathieumd]

Scope

Service (Job Scheduling, Job Preparation)

Description

We use (and love) Debops Ansible collection, which use git repo to store data, like TLS certificates (see secret for details).

I would like to use AW to yearly schedule the regeneration of private TLS certificates. But I need a way to git commit && git push the AW repo after the job completion. Otherwise, modification of the PKI database would be lost.

As a proof of concept (though unsatisfactory, as I need to commit just after the run has succeed, not before the next run, which would be one year later!), I tried to set the Git Override Update-Command to if ! git diff-index --quiet HEAD --; then git add . && git commit -m "Modification from $USER on $HOSTNAME" && git push; else; git pull; fi, but I could not save:

Got error: error (400) - Provided repository data is not valid: '{'non_field_errors': [ErrorDetail(string="Found illegal characters in field 'git_override_update'", code='invalid')]}'

Anyway: we would love an hook-script, defined in the repository configuration, after job executions.

[superstes]

Makes sense.
Will take a look at the post run hooks.

The input seems to be blocked by the XSS filter.. will fix that 👍

BYW: Interesting to hear about debops

[ansibleguy]

Regarding the post-run hook - in my opinion it would only make sense to configure it on a job-basis.
The current-working-directory for those hooks would have to be the playbook-directory, as we are in the job-context.
Would this be an issue if you have to perform git-action?

Additionally I could add a repository-based post-run hook. But I think this might not have many practical use-cases. Or what do you think?

[ansibleguy]

The XSS-filter for these command-fields should now allow for such commands:

COMMAND: git reset --hard
HEAD is now at dbb7999 allow single-quotes in hook-commands (#61)
COMMAND: git pull
Already up to date.
COMMAND: ABC=test && echo "$ABC"
test

If you are interested in testing the latest version - you can do so like this:

cd /tmp
git clone https://github.com/ansibleguy/webui.git
python3 -m virtualenv /tmp/abc
source /tmp/abc/bin/activate
python3 -m pip install -e /tmp/webui
AW_DB=/tmp/abc.db python3 -m ansibleguy-webui

[mathieumd]

I tried (by applying both commits) and I can now save the repo config, and the commit && push is well done at each start of the job. (Well, if you fix the command, because I left a defunct else; git in my first message)

if ! git diff-index --quiet HEAD --; then git add . && git commit -m "Modification from $USER on $HOSTNAME" && git push; else git pull; fi

Now, it would be perfect if this command could be run after job execution, not before ;-)

I'm in favor of a repository-based post-run hook, because you don't want to forget to add the post-hook on each and every job you may create based on the same repo.

[ansibleguy]

Alright. Hope I find time to implement it tomorrow evening

[mathieumd]

Just thinking loud: I wonder how will be handled several job executions in parallel? Due to scheduling, I guess two different jobs from the same repo could run at the same time:

Start----------------------------------------------------->time
Job1------(running longer)-------+(create local files)------(still running)-------End: Nothing to commit
Job2------(running shorter)------|----------End: Commit Job1's files

Job2 would commit files created by Job1 before Job1 could. So I guess it's confirming the hook should be at repo level (not job). But it's also showing that we should not have job's info (name, etc.) available in the hook, as it may actually be from another job.

[ansibleguy]

I guess those race-conditions could be tackled by using the git-repo-isolation (separated repo-clone per run)
Of course this would add some wait-time and the repo should be small (no multi-GB monorep (; )

[ansibleguy]

The cleanup-hook seems to work:

COMMAND: git reset --hard
HEAD is now at 18c16df add log-live-view refresh-toggle and made buttons visible (fix #63)
COMMAND: git pull
Already up to date.
COMMAND: echo 'THIS IS A CLEANUP'
THIS IS A CLEANUP

[ansibleguy]

Feel free to test it.
If you do not need a job-specific hook, I will not (yet) implement it as it would be a little more complicated.

[mathieumd]

For my use case, it's just perfect!

Thank you very much!! 🙇

Are you going to publish it to PyPI?

[superstes]

Sure. Just wanted to wait for your response before releasing it (: