#721472 dpkg-source: new field Tarsum with a tree hash in .dsc to possibly skip reunpacking the tarballs on build

#721472#5
Date:
2013-09-01 01:04:40 UTC
From:
To:
Package: dpkg
Version: 1.17.2
Severity: normal

It would have the output of: find <sourcedir> -type f -print0 | sort -z | xargs -0 sha1sum | sha1sum

This way the very slow step of verifying that the dir tree hasn't changed can be done very fast,
without re-unpacking the source tarball, and just comparing the value, to the same run on the build tree.

Should I write up a patch?

#721472#10
Date:
2013-09-01 14:04:54 UTC
From:
To:
Control: reassign -1 dpkg-dev
Control: severity -1 wishlist
Control: retitle -1 dpkg-source: new field Tarsum with a tree hash in .dsc to possibly skip reunpacking the tarballs on build
Control: user dpkg@packages.debian.org
Control: usertag -1 new-field dsc-format dpkg-source

Hmm, that implies that to build a new version the previous .dsc file
needs to be available, I'd rather not make the code having to poke on
previous .dsc files.

Also currently before doing a diff, the trees are compared until a path
does not match, as a speed up. Unpacking a new tree is going to be
somewhat slow, but hashsing the entire tree is going to be slow too,
and with that it cannot bail out early. So for large trees I'm not
sure if the performance difference is going to be so significant. More
so with an operation (building) which is supposedly done seldomly
(once, while extracting is done many times), and while optimizing it
would be nice, I don't think this specific proposal is the right
direction to follow.

Thanks,
Guillem

#721472#23
Date:
2013-09-11 18:50:03 UTC
From:
To:
I ran into the when building chromium, a 500+MB source tarball, which
started by habit of always using -b except when building for upload.

maybe dpkg could unpack into a tmpfs, removing files after it has checked
they are the same, to avoid the fs/io churn?
--- Shawn Landden +1 360 389 3001 (SMS preferred)