Reflinkable content format

[lnussel]

#1470 by @malmond77 introduced a format for reflinkable rpm payload. It adds an extra metadata block at the end of the payload. That doesn't seem to be needed actually. I wrote this tool to produce content without extra metadata: https://github.com/lnussel/rpm2extents

Even if support for this kind of package won't make it into rpm any time soon, I wonder whether we could agree on the potential payload format already?

WDYT?

CC @chantra

[chantra]

Thanks @lnussel . I am currently in holidays, but will get back into RPM work sometime in August and get back to this issue then.

[chantra]

Also quickly cross referencing #2057

[pmatilai]

Couple of remarks based on a quick look:

Breaking the header+payload hashes and signatures is unavoidable, but then those are on their way out anyway. You can't modify or remove the payload digests from the tool, those are signed and modification would (by design) break the header signature(s).
The payload digest supports an alternative version though. In vanilla rpm that's calculated based on the uncompressed payload (for the benefit of deltarpm, supposedly), but technically you could make rpm calculate the alt version based on the reflinkable content instead (at extra cost + complexity at build-time of course, but no harm/cost to normal use), in which case this conversion wouldn't break the payload digest.

FWIW, rpm's own Python bindings export the rpmio facilities as a file-like object (rpm.fd) which you could use instead of wrestling with separate libraries for each compression type (f = rpm.fd(path, 'r', compressor') where compressor is what you get from RPMTAG_PAYLOADCOMPRESSOR). (except of course in this case, you'd be reopening the fd from the package file rather than path, but that's also supported). It also supports iterating over the payload regardless of the format (oldstyle cpio or new, large packages), so you don't need to roll your own.