[Buildroot] [PATCH] Revert "support/download: generate even more reproducible tarballs"
Peter Korsgaard
peter at korsgaard.com
Sat Jan 13 20:26:13 UTC 2024
>>>>> "Yann" == Yann E MORIN <yann.morin.1998 at free.fr> writes:
> Commit 768f9f80f62c (support/download: generate even more reproducible
> tarballs) causes non-reproducibility in tarballs we previousy
> generated, especially the archives for two cargo-vendored packages,
> ripgrep and sentry-cli.
> The cause is that those two pakcages eventually vendor a file that has
> the u+x bit set, but is otehrwise go-x. With 768f9f80f62c, the files are
> now go+x, so the hash for those generated archives has changed.
> Besides, that commit was wrong: it did not account for the 'r' bit for
> go part, leaving some non-reproducibility still unaccounted for.
> So, to generate really reproducible archives, we would need to fix that
> read bit as well, and that has the potential to affect all the archives
> we generated so far. If we wanted to do so, we'd need a way to version
> all generated archives, like we do for git and svn, but now for all the
> different CVSes, as well as for all the vendoring post-processes.
> For 768f9f80f62c, all that was of conern was the working copies of CVSes
> (i.e. git, svn, cvs...) that we cache in the Buildroot download dir, not
> the temporary files during post-processing. Indeed, in that latter case,
> the user has virtually no way to mangle with the mode of the
> intermediate extract before repack.
> And we do have a big fat warning that users should not attempt to meddle
> with the git tree that Buildroot caches.
> As 768f9f80f62c however demonstrates, is that it took quite a long time
> between the introduction of the git caching, and the time someone
> eventually discovered they could meddle in there. This shows that the
> issue it not actually critical in most setups.
> Also, the tar manual [0] hints at a better solution to handle
> reproducibility, which even avoids touching the files on disk which is
> even nicer:
> ‘--mode='go+u,go-w'’
> Omit irrelevant information about file permissions.
> If we were to actually handle the mode bit for reproducibility, we'd
> need to:
> - introduce archive versioning for all download backends and
> prost-processing
> - use the tar officially suggested method
> So, revert that change, as it was incomplete, was not really fixing much
> issues, and causes actual issues.
> This reverts commit 768f9f80f62c1da6e298c680f0f4bfa887f38c78.
> [0] https://www.gnu.org/software/tar/manual/tar.html#Reproducibility
> Thanks to Vincent and Arnout for pointing at the tar manual.
> Reported-by: Antoine Coutant <antoine.coutant at smile.fr>
> Reported-by: Thomas Petazzoni <thomas.petazzoni at bootlin.com>
> Signed-off-by: Yann E. MORIN <yann.morin.1998 at free.fr>
> Cc: Vincent Fazio <vfazio at xes-inc.com>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout at mind.be>
Committed to 2023.02.x and 2023.11.x, thanks.
--
Bye, Peter Korsgaard
More information about the buildroot
mailing list