Dear Maintainer,

I'm doing some experiments with dpkg on filesystems which support
reflinks, like XFS and BtrFS

The idea is to uncompress and align the content of the .deb files
during the download phase, so during the installation phase files
can be copied by reflinking file content from the archive files
to the destination paths.

This is implemented with the FICLONERANGE ioctl, as `cp --reflink=always`
does since a few time.
If reflinks are not supported, the code falls back to the copy transparently.

With reflinks the installation of a big package like linux-firmware takes
rougly a sixth of the time needed with a regular file copy:

Stock archive:
# time dpkg -i linux-firmware_20240318.git3b128b60-0ubuntu2.4_amd64.deb
(Reading database ... 214246 files and directories currently installed.)
Preparing to unpack linux-firmware_20240318.git3b128b60-0ubuntu2.4_amd64.deb ...
Unpacking linux-firmware (20240318.git3b128b60-0ubuntu2.4) over (20240318.git3b128b60-0ubuntu2.4) ...
Setting up linux-firmware (20240318.git3b128b60-0ubuntu2.4) ...

real    0m39,264s
user    0m3,141s
sys     0m4,720s

Flattened and aligned .deb package:
# time dpkg -i linux-firmware_20240318.git3b128b60-0ubuntu2.4_amd64.debc
(Reading database ... 214246 files and directories currently installed.)
Preparing to unpack linux-firmware_20240318.git3b128b60-0ubuntu2.4_amd64.debc ...
Unpacking linux-firmware (20240318.git3b128b60-0ubuntu2.4) over (20240318.git3b128b60-0ubuntu2.4) ...
Setting up linux-firmware (20240318.git3b128b60-0ubuntu2.4) ...

real    0m6,531s
user    0m0,306s
sys     0m1,471s

On a system with a 2 TB magnetic disk, a full upgrade with 1450 packages
took 44 minutes instead of 90, the extraction phase alone went from 78
minutes to just 31.

Another advantage is that the extraction doesn't use extra space,
so you don't need to account both the compressed and uncompressed
size of the files at the same time.

The code is available at:
https://github.com/teknoraver/dpkg/tree/cow

and consists of three patches:
1. reworks dpkg to avoid using pipes when data.tar is not compressed
2. add PAX header support to dpkg
3. the actual reflink support

It's just a proof of concept yet, I just want to share the idea so
don't focus too much on the code.

Regards,
Matteo Croce

Hi,

I've seen on the debian-devel mailing list that the optimization of
removing the fsync with cow raised some concerns[1].
Due to the nature of cow the performances doesn't change at all with
or without fsync(), so I've refreshed the patches keeping the sync as
before.
This avoids the ambiguity where the performance benefit comes from not
syncing, which is not true.

Regards,

[1] https://lists.debian.org/debian-devel/2024/12/msg00602.html