Dear Maintainer,

mk-origtargz uses tar --delete to remove files listed in Files-Excluded.
However, when there are huge folders with many files in it, then it calls tar
--delete a huge amount of times.

This is because the size of the tar --delete command cannot contain all files
to exclude and then calls it as many times as needed.

When working on big archives this can take a very long time. For example I was
working on a 600MB archive (compressed) from which I wanted to removed ~2GB of
uncompressed data. This should lead to ~ 300MB archive in the end.
Since it takes forever with mk-origtargz I finally repacked it with a script
that uncompresses the tar, removes the files/folder listed in Files-Excluded,
and then creates a new archive.

In the documentation of --delete [1] it is already mentioned that "The
‘--delete’ operation can run very slowly"

Would it be possible to improve mk-origtargz so that it doesn't use tar
--delete but a faster alternative?

BTW, this also affects uscan which calls mk-origtargz.

[1] https://www.gnu.org/software/tar/manual/html_node/delete.html
--- /etc/devscripts.conf ---
Empty.

--- ~/.devscripts ---
Not present

Hi Yadd,

Would you mind giving a little explanation on why? I'm considerably slowed down
by this when packaging rustc, a fairly large package, and frustrated by the fact
that it's single-threadly doing: read archive, update archive, reconstructing
the entirety of the file on disk due to the update, repeat.

Hi Yadd,

Would you kindly reconsider this? Our fellow rustc maintainer Fabian Grünbichler still suffers from this for hours on each rustc release, which needs many files excluded. It would immensely ease their work. Thank you!

 Hi Blair, Fabian,

You may want to have a look at how I workaround this issue on Qt-Android.

It requires a version 3 d/watch with personal script `debian bash debian/scripts/repack.sh`

https://salsa.debian.org/bastif/qt-android/-/raw/qt6.9.1+ds-1/debian/watch?ref_type=heads

The required scripts which use the Files-Excluded field of d/copyright are these:

https://salsa.debian.org/bastif/qt-android/-/raw/qt6.9.1+ds-1/debian/scripts/repack.sh?ref_type=heads
https://salsa.debian.org/bastif/qt-android/-/raw/qt6.9.1+ds-1/debian/scripts/remove-excluded-files.sh?ref_type=heads

I try to be as conservative as possible wrt to upstream's origtar

Regards
Fab

Hi,

Le lundi 14 juillet 2025, 09:48:11 CEST Fab Stz a écrit :

Note that the scripts have the advantage of producing a reproducible orig.tar

Le lundi 14 juillet 2025, 09:54:42 CEST Fab Stz a écrit :

BTW, you have to invoke uscan with `--no-exclusion` since the script will manage this instead of mk-origtargz.