[Buildroot] [PATCH v3 1/3] Revert "support/download: generate even more reproducible tarballs"

[Antoine Coutant]

From: "Yann E. MORIN" <yann.morin.1998 at free.fr>

Commit 768f9f80f62c (support/download: generate even more reproducible
tarballs) causes non-reproducibility in tarballs we previousy
generated, especially the archives for two cargo-vendored packages,
ripgrep and sentry-cli.

The cause is that those two pakcages eventually vendor a file that has
the u+x bit set, but is otehrwise go-x. With 768f9f80f62c, the files are
now go+x, so the hash for those generated archives has changed.

Besides, that commit was wrong: it did not account for the 'r' bit for
go part, leaving some non-reproducibility still unaccounted for.

So, to generate really reproducible archives, we would need to fix that
read bit as well, and that has the potential to affect all the archives
we generated so far. If we wanted to do so, we'd need a way to version
all generated archives, like we do for git and svn, but now for all the
different CVSes, as well as for all the vendoring post-processes.

For 768f9f80f62c, all that was of conern was the working copies of CVSes
(i.e. git, svn, cvs...) that we cache in the Buildroot download dir, not
the temporary files during post-processing. Indeed, in that latter case,
the user has virtually no way to mangle with the mode of the
intermediate extract before repack.

And we do have a big fat warning that users should not attempt to meddle
with the git tree that Buildroot caches.

As 768f9f80f62c however demonstrates, is that it took quite a long time
between the introduction of the git caching, and the time someone
eventually discovered they could meddle in there. This shows that the
issue it not actually critical in most setups.

Also, the tar manual [0] hints at a better solution to handle
reproducibility, which even avoids touching the files on disk which is
even nicer:

    ‘--mode='go+u,go-w'’
        Omit irrelevant information about file permissions.

If we were to actually handle the mode bit for reproducibility, we'd
need to:
  - introduce archive versioning for all download backends and
    prost-processing
  - use the tar officially suggested method

So, revert that change, as it was incomplete, was not really fixing much
issues, and causes actual issues.

This reverts commit 768f9f80f62c1da6e298c680f0f4bfa887f38c78.

[0] https://www.gnu.org/software/tar/manual/tar.html#Reproducibility

Thanks to Vincent and Arnout for pointing at the tar manual.

Reported-by: Antoine Coutant <antoine.coutant at smile.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni at bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998 at free.fr>
Cc: Vincent Fazio <vfazio at xes-inc.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout at mind.be>
Tested-by: Romain Naour <romain.naour at smile.fr>
Tested-by: Antoine Coutant <antoine.coutant at smile.fr>
---
 support/download/helpers | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/support/download/helpers b/support/download/helpers
index 265685eff5..90a7d6c1ec 100755
--- a/support/download/helpers
+++ b/support/download/helpers
@@ -53,9 +53,6 @@ mk_tar_gz() {
     tmp="$(mktemp --tmpdir="$(pwd)")"
     pushd "${in_dir}" >/dev/null
 
-    # Enforce group/others mode bits
-    chmod -R go-wx+X .
-
     # Establish list
     find . -not -type d -and -not \( -false "${find_opts[@]}" \) >"${tmp}.list"
     # Sort list for reproducibility
-- 
2.25.1