I'm getting this if I use symlinks for tarballs in unshare mode: Warning : /home/anarcat/.cache/sbuild/UNRELEASED-amd64.tar.zst is a symbolic link, ignoring I'm not sure why this is happening, but it's quite inconvenient as I need to build a new chroot just for that UNRELEASED thing... This is related to #1089035 in the sense that I wouldn't need those symlinks if the auto-guessing would Just Work.
Hi, Quoting Antoine Beaupre (2024-12-22 21:34:14) can you show me a more complete log? I wonder what is producing this error message. I grepped for it in the sbuild source and cannot find what emits it. This is in my ~/.cache/sbuild: lrwxrwxrwx 1 josch josch 18 Dec 22 22:17 UNRELEASED-arm64.tar -> unstable-arm64.tar -rw-r--r-- 1 josch josch 450M Dec 22 22:17 unstable-arm64.tar And when I run sbuild in a package with UNRELEASED in d/changelog I get: I: Applied base distribution name mangle rule s/(?^:^(experimental|rc-buggy|UNRELEASED|UNRELEASED.*)$)/unstable/ turning "UNRELEASED" into "unstable" I: Unpacking /home/josch/.cache/sbuild/UNRELEASED-arm64.tar to /home/josch/tmp/tmp.sbuild.yNFwjl6TAA... I: Setting up the chroot... I: Creating chroot session... I: Setting up log color... I: Setting up apt archive... How can I reproduce the issue you see? I'm also on sbuild 0.88.1. Sorry, I failed to follow up on that one. Let me fix this in a bit. Thanks! cheers, josch
Quoting Antoine Beaupré (2024-12-22 22:39:12)
can you give me the output of this:
$ file /home/anarcat/.cache/sbuild/UNRELEASED-amd64.tar.zst
Thanks!
cheers, josch
/home/anarcat/.cache/sbuild/UNRELEASED-amd64.tar.zst: symbolic link to unstable-amd64.tar.zst Or do you mean: $ file -L /home/anarcat/.cache/sbuild/UNRELEASED-amd64.tar.zst /home/anarcat/.cache/sbuild/UNRELEASED-amd64.tar.zst: Zstandard compressed data (v0.8+), Dictionary ID: None
Hi, Quoting Antoine Beaupré (2024-12-23 02:32:25) thank you for both outputs. My hunch was that the symlink was broken and that's why it failed. I am still wondering where the message you see is from. This made me find this here: https://sources.debian.org/src/libzstd/1.5.6+dfsg-1/programs/zstdcli.c/?hl=1328#L1328 Could you try switching the compression of your tarball (or use no compression) and check if the problem persists? If not, maybe something special has to be done with zstd compressed tarballs? Thanks! cheers, josch
Hi, Looks like zstd ignores symlinks unless --force is passed [1]. Editing the command in ChrootUnshare.pm [2] to add it makes it work for me, but it may be too lenient with the file? From the zstd man page: [1] https://github.com/facebook/zstd/commit/680e4e0953bf7c895165e41098a37c3279870c1f [2] https://salsa.debian.org/debian/sbuild/-/blob/main/lib/Sbuild/ChrootUnshare.pm#L520 Thanks for all the work on sbuild!
Hi, Quoting Alper Nebi Yasak (2024-12-23 10:32:39) commits without any rationale behind them are the best </scarcasm> This also reminds me of #1089105 which comes down to zstd not accepting compressed data on stdin and writing the uncompressed result to stdout without also passing --force. Seems entirely unintuitive to me... Thank you for having found this. Maybe we should just call zstd with '--force' and call it a dway... Thanks! cheers, josch
Ugh, wtf. Uh. So it looks like this is a feature of zstd that it won't follow symlinks when reading compressed files!! So i guess this is not a bug in sbuild after all, but specifically about zstd tarballs. Sigh. Is it worth filing this against zstd and affecting sbuild maybe? me like a security feature, albeit poorly documented... If I read this right, there's even a TOCTOU bug in there, because we're checking symlinks before use, and and an attacker could replace a file with a symlink later. Anyway. Not sure what to do about this, I think the best might possibly be to move this to the zstd package... Sorry for all the trouble! :) a.
Hi, Quoting Antoine Beaupré (2024-12-29 16:52:26) this is not the hill I want to die upon. But feel free to make an argument with the developers. I found a different solution. I was wondering how GNU tar does it and perused its source code a bit. And then I just copied their solution. :) If you like, can you try this patch:--- /usr/share/perl5/Sbuild/ChrootUnshare.pm +++ /usr/share/perl5/Sbuild/ChrootUnshare.pm @@ -533,14 +533,15 @@ sub begin_session { my $pid_decompress = fork(); if ($pid_decompress == 0) { open(STDOUT, '>&', $decompress_writer); + open(STDIN, '<', $tarball); close $filter_reader; close $tar_reader; close $filter_writer; if ($self->get_conf('DEBUG')) { printf STDERR ( - "running $decompress[0] --decompress --stdout $tarball\n"); + "running $decompress[0] --decompress --stdout\n"); } - exec @decompress, $tarball; + exec @decompress; } my $pid_filter = fork(); if ($pid_filter == 0) { Essentially, we do not pass a path to zstd anymore but we let sbuild open the path and then pass the filedescriptor to what we opened to zstd via its standard input. Thanks! cheers, josch
Patch works for me, thanks! live well, vagrant
On 2024-12-30 06:46:27, Johannes Schauer Marin Rodrigues wrote: [...] Ah yes, that would work of course! Probably harmless in terms of security too... riiight? :) a.
Hi, Quoting Antoine Beaupré (2025-01-01 03:52:54) yes. Do you have any suspicions why it would not be harmless? Thanks! cheers, josch
For reading files? Not really. And especially in this context, where the cache directory is owned by the user, I can't really think of an attack vector there that wouldn't already otherwise give the attacker RCE access (ie. if i can write to your ~/.cache i can write to your ~/.bashrc). a.
Hello, Bug #1091169 in sbuild reported by you has been fixed in the Git repository and is awaiting an upload. You can see the commit message below and you can check the diff of the fix at: https://salsa.debian.org/debian/sbuild/-/commit/9b906e0b579b3bca9dc45cd3e84579a36145887d ------------------------------------------------------------------------ lib/Sbuild/ChrootUnshare.pm: instead of passing the tarball path to the decompressor, let sbuild open it and feed it to the decompressor program via standard input Not all decompression programs behave the same. Specifically, zstd refuses to work on paths that are symlinks without the -f option. To hide the symlink from zstd, let sbuild open the file and pass its contents to zstd via its standard input. To make the behaviour uniform among all decompressors, do the same thing for the others as well. This is also how GNU tar handles this. Closes: #1091169 ------------------------------------------------------------------------ (this message was generated automatically) -- Greetings https://bugs.debian.org/1091169
We believe that the bug you reported is fixed in the latest version of
sbuild, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 1091169@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Johannes Schauer Marin Rodrigues <josch@debian.org> (supplier of updated sbuild package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Thu, 16 Jan 2025 07:46:28 +0100
Source: sbuild
Architecture: source
Version: 0.88.2
Distribution: unstable
Urgency: medium
Maintainer: sbuild maintainers <sbuild@packages.debian.org>
Changed-By: Johannes Schauer Marin Rodrigues <josch@debian.org>
Closes: 1089035 1089105 1090336 1090358 1091169 1091170 1091370 1092091 1092557
Changes:
sbuild (0.88.2) unstable; urgency=medium
.
[ Johannes Schauer Marin Rodrigues ]
* lib/Sbuild/Conf.pm:
- fix examples for ENVIRONMENT_FILTER (Closes: #1090336)
- do not ignore an empty DEB_BUILD_PROFILES environment variable
- Add CHROOT_ALIASES configuration option, allowing one to look up chroots
under a different name (Closes: #1089035)
* lib/Sbuild/ConfBase.pm:
- run check after setting all values
- preserve order of configuration options
- Set variables to their default values in ~/.config/sbuild/config.pl
(Closes: #1090358)
* lib/Sbuild/ChrootUnshare.pm:
- check if mmdebstrap is installed only when needed
- instead of passing the tarball path to the decompressor, let sbuild open
it and feed it to the decompressor program via standard input (Closes:
#1091169)
* debian/tests/unshare:
- install ca-certificates for https debci hosts. Thanks to Paul Gevers
- run dpkg-deb with --root-owner-group (Closes: #1092091)
* lib/Sbuild/Build.pm: fail if dpkg --print-foreign-architectures failed
* lib/Sbuild/ResolverBase.pm: build metapackage with dpkg-deb
--root-owner-group (Closes: #1092557)
* Use dpkg-buildtree to figure out if fakeroot is required
* die if exec fails because it cannot find the program
.
[ Simon McVittie ]
* man: Cross-reference the --build-dir option in BUILD ARTIFACTS
* man: Document how and where log files are saved
.
[ Guillem Jover ]
* lib/Sbuild/Conf.pm: Do not use deprecated Dpkg::Build::Info module
.
[ Chris Hofstaedtler ]
* Remove deborphan from Suggests:, package was removed
* sbuild-update: abort if chroot-mode=unshare (Closes: #1089105, #1091370)
.
[ Santiago Ruano Rincón ]
* Add Recommends on iproute2
* Fix mmdebstrap+unshare setup-hook when handling *-security suites
.
[ Jakub Wilk ]
* man: Remove stray quotation mark
.
[ Richard Lewis ]
* man/sbuild.1.in: remove stray quotation mark
* Document the need to retain package lists in the chroot if $apt_update is 0
(Closes: #1091170)
.
[ Jochen Sprickerhof ]
* unshare: Use $BUILD_USER as the default user
* unshare: Use 'sbuild' as the default user
* Use BUILD_USER for architecture and disk space check
Checksums-Sha1:
0f0099edcbf367fae51ea461298ed81e97f28812 2643 sbuild_0.88.2.dsc
7c8afb029abcb8524160a037aee55cf1ff601080 255988 sbuild_0.88.2.tar.xz
Checksums-Sha256:
43147313e02531d784b5fed0b81ad45481bc89e0b17f13c89cdc268d91a3af47 2643 sbuild_0.88.2.dsc
569736857d4e906faf50ce25baa3e10d45d4be983237b70ce4e1e00871557116 255988 sbuild_0.88.2.tar.xz
Files:
47556984b7a862d35ab78354c7c3a68c 2643 devel optional sbuild_0.88.2.dsc
1b63edbc9459af6f09986dfc82e34a74 255988 devel optional sbuild_0.88.2.tar.xz
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmeIq2YACgkQ8sulx4+9
g+ESFQ//Y/wgtNcLRqfw/sIWtUd1OC1a/o2U26BHY9KZjW5Rqp01B4FepDwoH3c/
r+brucA3j2/0nvx6BpcF2EupcMlgPpY4J4EYmisyZrHgYBPlHyc8LyKVm2gDEnPV
LapflaCbdTWprYQoijxdjQ9lGJiz1XQCS+xhwI8EcIK+KXxx18pbB3i3DSLhEkX/
Ud5qHb0Y/tnj+xtdKVZED+Z6AmlGiT0+GEjWw1N7bC+uLy2J8IbdUB37ANopvVtE
l3u54HIh8M/+hDW7B4X/QeB+houFPXzRNZx+QlccWJvf19MiPPKpbOa213ieG5IH
raijuMEwNzH2PevH8nBKA1REcIAyYXlcdk4OaLm3rsiXLk2rPtXaVbFazxMiqP8m
3UyOMPmBfZSjOMjkbVtYHiw6YuA7uYCejyW5h//jW1v4NNBVQZKbHAddLjp4m6+A
oBSbuGLVamTvqcUJG+n94V+U1pRHCtvvvtYHi4h991k4Cph/klzYF3zHmZsxcET6
WBaOv4erPMP1BJkD7B2O5oxGr8tJ/jprEKtbJa1LsNakjDu7pBYumL+mmj4OKlhO
oIjw3ujGfVkXE+9Glmtj5WRTZTzRJ+cYz6zTQfwJRZ3WJxLtmJCUazK+uXi/VL01
M3bNHlwwCAyLDZ9yGm45YwneY7EiMfZytL6S8czYs+FTwXRC8og=
=H3Yw
-----END PGP SIGNATURE-----