Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate spack_packages tagging utility #36

Open
CodeGat opened this issue Aug 16, 2023 · 2 comments
Open

Investigate spack_packages tagging utility #36

CodeGat opened this issue Aug 16, 2023 · 2 comments

Comments

@CodeGat
Copy link
Contributor

CodeGat commented Aug 16, 2023

Background
This repository uses tags in order to both mark significant events in it's history (namely, when model(-component)s are updated) for devs; and for CI pipelines to effectively name dependency images (for example, appending the version of spack_packages that the dependency images is built on - [email protected] <- the end bit).

A simple date-based tag (eg. 2023.08.15) isn't enough to specify what exactly has changed. So we can add an optional message after the tag (git tag 2023.08.15 -m "libaccessom2 1.2.3, mom5 1.3.4, ...") to note the versions explicitly at this point in history.

There are some things to consider below.

Tasks
Investigate whether:

  • tags should be automagically generated by the CI pipeline, based on what parts of spack_packages are updated.
    • Pros: Less dev overhead
    • Cons: More complex CI pipeline
  • tags with messages should be a release instead - allows better formatting than just a single message
  • we should split tagging across access-om2 and access-om3, allowing building models conditionally based on what has changed in spack_packages. Examples of both follow:

If we rebuild everything even when only one model changes...

* tag: 2023.08.16 (access-om2: 1.3, <other accessom2 deps>, access-om3: 1.34, <other accessom3 deps>)
|\
| * commit: some change to access-om2 that requires a version bump
|/
* tag: 2023.08.13 (access-om2: 1.2, <other accessom2 deps>, access-om3: 1.34, <other accessom3 deps>)

We will eventually get these two dependency images:

build-accessom2-...-2023.08.16 (same spack_packages version)
build-accessom3-...-2023.08.16 (same here too)
# and the old ones
build-accessom2-...-2023.08.13
build-accessom3-...-2023.08.13

Even if nothing has changed for access-om3.

If we break them down model lines:

* tag 2023.08.20 (access-om2: 1.3, <other accessom3 deps>)
|\
| * commit: access-om2 bump
|/
* tag 2023.08.19 (access-om3: 1.35, <other accessom3 deps>)
|\
| * commit: access-om3 bump
|/
* tag 2023.08.13 (access-om2: 1.2, <other accessom2 deps>, access-om3, <other accessom3 deps>)

We would eventually have the following dependency images:

build-accessom2-...-2023.08.20
build-accessom3-...-2023.08.19
# and the old ones
build-accessom2-...-2023.08.13
build-accessom3-...-2023.08.13

Both options have pros/cons. Thanks to @harshula and @aidanheerdegen for the conversations surrounding this issue.

@CodeGat CodeGat self-assigned this Aug 16, 2023
@aidanheerdegen
Copy link
Member

I don't have anything substantive to add at this point, but I was interested to see how your git diagram would render with mermaid ...

gitGraph
       commit tag: "2023.08.13"
       branch "access-om3-bump"
       commit
       checkout main
       merge access-om3-bump tag: "2023.08.19"
       branch "access-om2-bump"
       commit
       checkout main
       merge access-om2-bump tag: "2023.08.20"
Loading

@CodeGat
Copy link
Contributor Author

CodeGat commented Oct 4, 2023

Unassigning myself for now as I'm working on Nirin runner stuff

@CodeGat CodeGat removed their assignment Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants