Replies: 14 comments 20 replies
-
|
A bunch of random initial thoughts on the subject: BuildID is already overloaded in rpm, any such thing would need a different name. Appending more and more stuff into Release is ugly and problematic for other reasons, but of course adding any new fields to rpm version comparison is a long, painful road. Just for the purposes of giving build artifacts different filenames you don't need any code changes, just define The only monotonically increasing value available to rpm is the clock though (which is not as monotonic as time itself, unfortunately), anything else would somehow require storing a previous value somewhere "central" and rpm has no such place available. So it goes back to some outside thing setting a macro that rpm slaps into a tag and forgets. Adding tags is cheap, whether it actually solves an issue is a different question. And then proper uniqueness for eg build environment requires hashes (eg hash of all installed packages, macros and env variables during build), which in turn are not version comparable. So there seem to be multiple, conflicting goals here, and reproducible builds are another, pre-existing and conflicting goal. |
Beta Was this translation helpful? Give feedback.
-
|
Is something like Nix the answer? |
Beta Was this translation helpful? Give feedback.
-
That already exists - it's the build time! A mode where RPM completely ignores the EVR and simply compares build times would help unlock a whole bunch of things, including making it easier to revert to a previous version.
Why do you say that? That's how ostree works for example today - the version number is just for humans, and the code instead just cares about (by default) a monotonically increasing timestamp: https://github.com/ostreedev/ostree/blob/98587a72db9b52eee63b4bfa9c47a77d2e327501/src/libostree/ostree-repo-pull.c#L1668 |
Beta Was this translation helpful? Give feedback.
-
|
On Sat, 2022-04-23 at 05:14 -0700, Colin Walters wrote:
> numerical build id - a monotonically increasing build number,
That already exists - it's the build time!
A mode where RPM completely ignores the EVR and simply compares build
times would help unlock a whole bunch of things, including making it
easier to [revert to a previous
version](https://lwn.net/Articles/513346/).
Time could be used, but we need a the same build Id for all the rpms
generated by the build for all arches.
So we can't just use the build time of the individual RPM.
> and buildtime is hardly a solution for that either.
Why do you say that?
That's how ostree works for example today - the version number is
just for humans, and the code instead just cares about (by default) a
monotonically increasing timestamp:
https://github.com/ostreedev/ostree/blob/98587a72db9b52eee63b4bfa9c47a77d2e327501/src/libostree/ostree-repo-pull.c#L1668
This could cause issues if two builders have a time offset or package
building races and any number of other things.
Time *can* be used, when set in stone at SRPM creation time perhaps,
but it is not the only or best option, especially for build systems
that do not forcibly re-create the SRPM.
Simo.
…--
Simo Sorce
RHEL Crypto Team
Red Hat, Inc
|
Beta Was this translation helpful? Give feedback.
-
Sure, but that's basically having koji capture "time of build creation" (which already exists actually of course - the toplevel koji task has a "Start time") and propagate that via the equivalent of |
Beta Was this translation helpful? Give feedback.
-
|
I'd say when you apply the logic like this it stops being the Build Time and it becomes "some counter which is provided by the buildsystem". Timestamp can be used as input to set that counter, but how exactly it happens is an implementation detail of that specific buildsystem. And real build time is a thing with strict definition, which build system must not override. |
Beta Was this translation helpful? Give feedback.
-
|
Generally speaking what kind of entities we have in play?
From the point of a build system each build needs to have a unique key. The meaning of that key is not important, it is like a primary key field in the database, which simply needs to be unique and allows to store the rest of the data. For example in Jenkins the unique key for a build is a BUILD_TAG = string of And I agree with Simo that while build time can be used as an implementation of some part of a build tag, it is not necessarily the best one, and it very much depends on the way how the build system operates. From the point of reproducible builds it is more interesting to track meaningful build information, thus one needs to encode enough data for the comparison of two builds taken in isolation from their original build systems. While reproducible build is an interesting topic on its own I think it is not exactly related to the set of problems I listed in the top post. Now I would maybe reformulate my original question in a way: can we provide some generic approach how build system can set the build version at build time, so that it will be used in the calculation of the upgrade path. And what form that build version should have. And I would leave it to the build system to decide what this version should be within the restrictions of its form, same way how we leave it to maintainer to decide what Release version of a package should be. RPM format itself provides a place for this version, and some constraints, but it doesn't set it. For example if we agree on the form as number, then some build systems may use build time, some - build id and some will set a custom number which user enters via GUI. It should all be allowed as long as the format and upgrade path rules are set on the rpm side. |
Beta Was this translation helpful? Give feedback.
-
|
Random thoughts from me:
|
Beta Was this translation helpful? Give feedback.
-
I agree that Mock is a reference implementation for this. But I'd say we need to have build system in mind here. We can not forget about build system, we just need to be build-system-agnostic.
Do I understand correctly that rpm, dnf, mock and other tooling operate on rpm metadata and don't use filename for anything? So filename only matters for the external user, like a sysadmin who tries to do something with the list of filenames without proper metadata access?
I think reproducible builds is all about comparison rules, but to start comparing things you need to build and store them under unique names anyway.
I think this topic is in scope of a build system, not RPM metadata. Things like Koji Build Id or "timestamp of the src.rpm build" are self explanatory for that system, but RPM knows nothing about them.
This is interesting topic indeed. We want to build things without changing the dist-git sources, but it doesn't mean we shouldn't track the reasons for the rebuild. Should it be tracked on a build system side or we need a "binary changelog" next to the "spec changelog" in the rpm itself? |
Beta Was this translation helpful? Give feedback.
-
Indeed. Such a message is essentially property of that build, so we could add cli switches to communicate the "build message", which just gets tacked on top of the binary changelog. Such messages would of course disappear in future rebuilds, but from the package's POV they are quite irrelevant anyhow. |
Beta Was this translation helpful? Give feedback.
-
|
I like the idea. I think that even without BUILD tag, one can add default changelog message like We would need to investigate though if we have some comparison tools which rely on changelog when calculating the upgrade info. Things like Satellite, LEAPP, Insights.. if they compare changelogs of two builds and expect older changelog to be included in the new one, then disappearing line of a build changelog could become an issue. |
Beta Was this translation helpful? Give feedback.
-
Right
Right. And if you overload them, there is a risk that they will ignore the whole thing.
Personally, I would prefer dual changelog. But I have no idea how to implement it :) |
Beta Was this translation helpful? Give feedback.
-
|
So thinking more about it and adding a compatibility consideration (we want build-versioned rpms to work alongside with the non versioned builds and we want to provide a possibility for iterative development for both rpm and build system developers) I would come with the following proposal:
This way we can let build-system and rpm to be developed and updated in a non-blocking way: the step 3 can take as much time as it needs while buildsystem will already implement the BUILDTAG and then can be switched to the new functionality when it is ready. Basically it encodes the workarounds we already do in Packit or in Fedora CI for scratch builds, but it adds the path how to get to the point where it stops being a workaround and becomes a proper setup. |
Beta Was this translation helpful? Give feedback.
-
|
It seems like we're heading towards a solution that makes it actively harder for users to understand what is going on by hiding the |
Beta Was this translation helpful? Give feedback.
-
|
I am opposed to invisible build id and epoch is not replacement at all. |
Beta Was this translation helpful? Give feedback.
-
|
Oh good, Epoch still evokes strong feelings ;) My suggestion on Epoch was simply to illicit response but also to highlight that hiding things seems like an anti-pattern. I don't really want to see Epoch overloaded, but I also don't want to see |
Beta Was this translation helpful? Give feedback.
-
|
@jwboyer Sorry, I missed your comment.
I think there is a misunderstanding: I do not propose to hide the build id in the end. Let's say in the transition stage we have Name: smth And filename = {name}-{version}-{release}.rpm Then in "After" stage we would set: Name: smth Filename = {name}-{version}-{release}.{build}.rpm So the outcome - visibility for the user - would be exactly the same. We would just treat Build tag differently from the tooling perspective, and advanced tools and libraries will be able to use Build tag instead of parsing release tag and guessing the value from it. |
Beta Was this translation helpful? Give feedback.
-
|
Updated version:
|
Beta Was this translation helpful? Give feedback.
-
|
Is there a reason we couldn't generate the Buildtag value and put it in the file name via a macro until RPM grows support for the tag? Then there's no transition other than moving away from the macro. The concept becomes visible to the users ASAP and you get feedback on it before it's encoded in the RPM format "forever". |
Beta Was this translation helpful? Give feedback.
-
|
I might be wrong, but I think Filename doesn't participate in the rpm version comparison. If we only adjust a filename, RPM won't be able to calculate the upgrade path between |
Beta Was this translation helpful? Give feedback.
-
|
Apologies, I was unclear. I mean use a macro and temporarily put it in Release so that it shows up in the filename. |
Beta Was this translation helpful? Give feedback.
-
|
That is in fact exactly the short-term solution that we have proposed. So I'm glad we're aligned here. 😄 |
Beta Was this translation helpful? Give feedback.
-
|
@sgallagher @jwboyer It is possible to do a local version of this on build-system level without any changes to RPM. But I would prefer if we get the initial setup confirmed, which would for me include the definition of Build tag and the "build reason". This will give us some confirmation that temporary local solution would indeed be temporary, and that it is aligned to where upstream is going. |
Beta Was this translation helpful? Give feedback.
-
How do you define unique identifier? It is commonplace to change RPMs after build-time via signing. Just because two packages have the same NEVRA and build time does not mean they are the same package. If you want something truly unique, you probably want a checksum. |
Beta Was this translation helpful? Give feedback.
-
|
FWIW as automatic rebuilds are a natural part of openSUSE maybe there's some inspiration from how it's done there. Basically the build system maintains it's own release value by tracking the so called checkin counter and rebuild counters. OBS' build system edits the spec file it passes to rpmbuild to set the release string. So the first build of a package That makes life for packagers really easy as one has to basically never care about the release value. A spec file can easily be copied around and reused in different projects. The binaries end up having independent release values. |
Beta Was this translation helpful? Give feedback.
-
|
Do I understand correctly that both check-in counter and build-counter are tracked by the build system? So there is no way to recover that information for a local build without access to the centralized database? If I am rebuilding |
Beta Was this translation helpful? Give feedback.
-
|
If my understanding is correct, in the local From @mcatanzaro on [email protected]: "[on SUSE] the Release is always set to 0 in the specfile". |
Beta Was this translation helpful? Give feedback.
-
If the local build tool doesn't track the release itself you'd get whatever is in the spec file. So if you use a checkout from OBS it's most likely 0 or 1, as that's what packagers usually put. If you rebuild the src.rpm of your example you'd get 1.3. |
Beta Was this translation helpful? Give feedback.
-
|
I think another part missing from this thread is that the build system "burns in" the value of Release into the spec file before running rpmbuild, so it's part of the shipped artifact. |
Beta Was this translation helpful? Give feedback.
-
|
On Tue, 2022-04-26 at 05:24 -0700, Aleksandra Fedorova wrote:
> Forget about Koji, Copr. Or OBS. Solve the task on a local level
> first. I.e. in plain Mock on the command line.
I agree that Mock is a reference implementation for this. But I'd say
we need to have build system in mind here. We can not forget about
build system, we just need to be build-system-agnostic.
> The filename is already long. And 3rd party users (look into
> Stackoverflow questions) already use foo.rpm removing the VRA from
> the filename. Not even mention the dist tag.
Do I understand correctly that rpm, dnf, mock and other tooling
operate on rpm metadata and don't use filename for anything? So
filename only matters for the external user, like a sysadmin who
tries to do something with the list of filenames without proper
metadata access?
> On the other hand, make sure that the outcome does not go against
> idea of reproducible builds.
I think reproducible builds is all about comparison rules, but to
start comparing things you need to build and store them under unique
names anyway.
> The number should be self-explanatory. I.e. when we use disttag
> fc36 people know what it is. Compared to ID number from the row
> from PDC.
I think this topic is in scope of a build system, not RPM metadata.
Things like Koji Build Id or "timestamp of the src.rpm build" are
self explanatory for that system, but RPM knows nothing about them.
> Even for automatic rebuilds you may want to know the reason:
> Rebuild because of new Python. Mass rebuild of all packages (just
> because). Rebuild because of a new definition of macro X...
This is interesting topic indeed. We want to build things without
changing the dist-git sources, but it doesn't mean we shouldn't track
the reasons for the rebuild. Should it be tracked on a build system
side or we need a "binary changelog" next to the "spec changelog" in
the rpm itself?
I think this is going too far and should be explicitly be considered
out of scope.
For many build systems the reason why you are doing a rebuild is fully
confined within the builds production, there is no need to note it at
the rpm level.
For those cases where noting it in the RPM is needed, people should
explicitly go and bump the NVR and write a changelog entry.
Simo.
…--
Simo Sorce
RHEL Crypto Team
Red Hat, Inc
|
Beta Was this translation helpful? Give feedback.
-
Hi,
I'd like to start a conversation about build-time versioning in RPM
Problem
There are several use cases, like mass-rebuilds, soname bumps, Packit, Fedora ELN, CentOS Remixes, RHEL rebuilds.. where we need to rebuild RPMs without changing their content, neither sources nor spec files. The only change in such case is a change of a buildroot environment which changes the outcome of the build process.
Currently to address that we use some workarounds:
smth-1.2.3-5.fc36->smth-1.2.3-6.fc36,smth-1.2.3-5.eln101->smth-1.2.3-5.eln102), which allows us exactly one rebuild of any package,The issues with these workarounds are:
Strictly speaking the problem comes not from the RPM format itself, but from a build system which enforces the policy that every build must have a unique identifier, and therefore not allowing two builds with the same filename(Name-Version-Release combination).
But since this is a reasonable policy for any build system, we get to the question: can we give an rpm package a unique identifier, which would not only depend on the Version of the sources and Release field of the spec file, but which would also uniquely identify the build environment used to build it?
And while we could resolve it on the level of each individual build infrastructure, it would be much better to have a standard interface shared between different build infra implementations and supported by the format specification.
Possibilities
While we were brainstorming the options the most obvious solution which we came up with was to introduce a numerical build id - a monotonically increasing build number, and then use a macro to inject it in the Release field in the build time. (Similar to Packit approach above)
Basically we set
Release: 5%{disttag}.%{build-id}in a spec file, and on rebuild we get different build-ids in the filename:smth-1.2.3-5.el9.15172316orsmth-1.2.3-5.el9.15172321and so on.In case of Koji that build id would come from the Koji build id. But other build systems could define it differently.
But this kind of solution feels incomplete without proper metadata support in the RPM.
Without clear specification to compare rpms people would have to parse the Release field of an rpm with fragile regexps and custom scripts on the client side.
Questions
CC @sgallagher
Beta Was this translation helpful? Give feedback.
All reactions