understanding the error message from test-bot when using the --keep-old flag

[ipatch]

Output of brew config

na

Output of brew doctor

na

Description of issue

hello again 👋, active maintainer of homebrew-freecad tap. i've been working with the default tests.yml action for building bottles with the homebrew-freecad tap. Something down the line I'd like to implement is being able to label different runners within the tap using some form of yaml filtering provided by the github actions syntax, or possibly using labels and groups to trigger certain runners when a PR is made. I'd like this feature because sometimes I only need to rebuild a bottle for particular OS ie. say big_sur (self-hosted runner). and I do not need to rebuild the bottles for other 4 runners associated with the homebrew-freecad tap. so i did a little reading of the various flags with the test-bot command provided by homebrew, and noticed there is a --keep-old. I think is beneficial if I submit a PR, and the PR is only for a specific version of macos. However after updating my tests.yml file to use such flag provided by test-bot i get the lovely error messages seen below,

Error: 1 failed step!
brew bottle --merge --write --no-commit --no-all-checks ./[email protected]_2.ventura.bottle.json --keep-old
Error: Process completed with exit code 1.

==> brew bottle --merge --write --no-commit --no-all-checks ./[email protected]_2.ventura.bottle.json --keep-old
==> FAILED
Full bottle --merge --write --no-commit --no-all-checks ./[email protected]_2.ventura.bottle.json --keep-old output
  ==> freecad/freecad/[email protected]
  Error: `--keep-old` was passed but there are changes in:
  sha256 ventura: old: "73461fb88af969d9a6e6cfc06930c0831d6c336386fea732244e884c0f20ee25", new: "0b750ac8d15ce5199d6d09789e177c5f20b74402f8526df946d1a854ef394ef6"

even though the below github action run / job shows/reports an error the bottles are still built.

would make since to update the test-bot command to rather show a warning instead of error, because generally when i see error it means something did not complete but in this case there is an error, but the bottles are still built.

[Bo98]

--keep-old is supported for adding additional bottles, but you can't replace existing bottles.

The reason for this is because replacing a bottle requires incrementing rebuild (since a released bottle is immutable). Rebuild numbering however is currently scoped globally, e.g.

  bottle do
    root_url "https://ghcr.io/v2/freecad/freecad"
    rebuild 1 # <--- global rebuild
    sha256 cellar: :any, big_sur:  "5d63c496f3a0682a414d31207d290fbde724a9b8b2ba8cf8aaa195bf19a27179"
    sha256 cellar: :any, catalina: "a28dbd0a545c76ea8caef7aa5533622bfec130a4985724702bc67d0eb031e239"
    sha256 cellar: :any, mojave:   "45f987db03e2a0fa996c855b5f3987ac904fde6a41ecb64f8ffbbdca28a073a6"
  end

so you can't rebuild specific bottles without rebuilding all bottles.

If we had per-bottle rebuilds, what you describe could work. This currently isn't supported.