* Package name : libcurlfs Version : 0 Upstream Contact: наб <nabijaczleweli@nabijaczleweli.xyz> * URL : https://sr.ht/~nabijaczleweli/libcurlfs * License : 0BSD Programming Lang: C++ Description : mounts remote HTTP/HTTPS URLs as a FUSE filesystem This program allows $ libcurlfs -o kernel_cache https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.7.0-amd64-DVD-1.iso mnt $ qemu-system-x86_64 -cdrom mnt/debian-12.7.0-amd64-DVD-1.iso ... and the DVD is pulled on demand, and only the bits actually used. Multiple URLs may be given, and all appear as their basenames; the manual bares all (not that there's much more): https://srhtcdn.githack.com/~nabijaczleweli/libcurlfs/blob/man/libcurlfs.pdf This was written as a direct replacement for httpfs2, which, after popping up on bug of the day, was evaluated as simply not functional, with no prospects, and removed in #1079931. The interface is compatible with httpfs2. This program doesn't suffer from any of its issues (FUSE 3, libcurl instead of hand-rolled HTTP implementation (that doesn't work), libcurl instead of hand-rolled TLS implementation (that doesn't work)) and I (and my associates) have used it extensively to fuse the download and use steps for medium-to-large files, during my daily work (and their whatever-they're-doing) since. The existence of httpfs2 proves the general demand for this functionality. I am not aware of any other similar packages. I have prepared preliminary packaging in git-buildpackage format at https://git.sr.ht/~nabijaczleweli/libcurlfs.deb I would need a sponsor.
Hi наб, Hi debian-mentors (package replacement question below), this package looks useful. I'll sponsor it. Thanks for working on this. Looking at d/control, you don't seem to declare the replacement properly. You need Conflicts+Replaces, not +Breaks, https://www.debian.org/doc/debian-policy/ch-relationships.html#replacing-whole-packages-forcing-their-removal Quoting policy: Since you include an overlapping httpfs2 symlink in your package Breaks is inappropriate as even just unpacking would cause a file conflict with httpfs2. I get this when trying to install libcurlfs over httpfs2: dpkg: regarding libcurlfs_0-1_amd64.deb containing libcurlfs: libcurlfs breaks httpfs2 httpfs2 (version 0.1.4-1.1) is present and installed. dpkg: error processing archive libcurlfs_0-1_amd64.deb (--install): installing libcurlfs would break httpfs2, and deconfiguration is not permitted (--auto-deconfigure might help) Further Provides: httpfs2 (= 1) doesn't look right. I don't think you need a versioned provides here since no other packages depend on httpfs2, but I don't have much experience with replaceing packages. I think it would also be useful to just take over httpfs2 on upgrade from bookworm, but I can't find any relevant advice on how one might do that. I thought perhaps the provides with a higher version number than httpfs2 in stable would encourage apt to upgrade from httpfs2 to libcurlfs on it's own but I didn't see that happen when upgrade testing in my local repo setup. One idea I had was to just to build a httpfs2 pseudopackage that depends on libcurlfs as part of src:libcurlfs. Thoughts? Thanks, --Daniel
Hi! thanks :) Oh, it's definitely wrong; ideally I wanted to have the upgrade happen httpfs2 -> libcurlfs without a transitional package (hence also the Provides:); having now tested this, I don't think that's possible. This gave me lintian conflicts-with-version, which points out that and https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces agrees on this point; I think this best describes what we're trying to do here, since... My testing agrees, so I think the canonical way of doing this is having a transitional package, as you say. ...so httpfs2 Arch=all transitional package Depends: libcurlfs libcurlfs Breaks+Replaces: httpfs2 (<< 0.1.5) since we're moving /bin/httpfs2 &c. from httpfs2=0.1.4 to libcurlfs=0-1. This also means we need to start this package at epoch 1 (since 0-1 < 0.1.4-1.1+b1, and we want a httpfs2 that sorts newer). This configuration fresh-installs and upgrades fine for me on bookworm. I've updated the gits thus. On the topic of the gits ‒ do I really need Restrictions: isolation-machine on the autopkgtest? it works fine as a regular user in a chroot or container, /if/ fuse.ko is loaded (indeed, I patch it out on builds.sr.ht CI https://git.sr.ht/~nabijaczleweli/libcurlfs.deb/commit/de6a8f0d58534cd66c81ec63302f6d5e78a8b0b2) but the Salsa autopkgtest CI step naturally doesn't have fuse.ko loaded, and it can't auto-load from the chroot/container/whatever, but it also can't provide isolation-machine so it skips it. So I don't really know if it works (or if I need an explicit load step?). Best,
Hi!
Reviewing my open debbugs, this came up.
That was a big mail and like a month ago,
so the infelicities of this transition are summarised below.
The httpfs2 in the archive is from src:httpfs2 0.1.4-1.1;
src:libcurlfs would be version 1:0-1 and provide
libcurlfs containing /usr/bin/libcurlfs
and /usr/bin/httpfs2 -> libcurlfs (same for the manual)
and an empty httpfs2 (Depends: libcurlfs).
This ensures a clean
httpfs2 0.1.4-1.1 -> httpfs2 1:0-1 -> libcurlfs
upgrade path. This seems to be SOP.
httpfs2 will be able to be dropped in... forky? forky+1?
The autopkgtests are Restrictions: isolation-machine
because they need to modprobe fuse.
(If fuse is already loaded then that restriction is pointless,
but this can't really be expressed.)
The current state of the packaging can be found at:
https://git.sr.ht/~nabijaczleweli/libcurlfs.deb
https://salsa.debian.org/nabijaczleweli/libcurlfs
(CI passes (except test-crossbuild-arm64 but that's just broken(?)):
https://builds.sr.ht/~nabijaczleweli/job/1323121
https://salsa.debian.org/nabijaczleweli/libcurlfs/-/pipelines/728612)
Best,
наб
Hi наб,
Hmm, yeah. Now that I think about it again the httpfs2 transitional package
would depend on libcurlfs and it having a Conflicts: httpfs2 would probably
make httpfs2 uninstallble :)
The only nit I have is we may want to add a ~ at the end of the
replaces+breaks constraints.
See https://lists.debian.org/debian-devel/2024/10/msg00094.html for a
similar discussionbut it's probably not critical in this case.
I'm afraid so.
simply bumped it to be larger than httpfs2's instead of using an epoch
here.
If you do want to go with the epoch per d-policy (5.6.12 `epoch`) you
should consult d-devel before doing so. While this is a new package we are
squatting on the httpfs2 name with the transitional package so I think it
would still be required.
Come to think of it: why aren't you packaging this as a native package if
you're upstream anyway? Means you only need one git repo and only one
branch. No need for gbp's upstream/debian branch distinction either. Just
makes things simpler.
Looks fine to me too. Autoremove even picks up httpfs2 for removal. Neat. I
didn't know that handles transitional packages!
$ apt-get autoremove
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
fuse httpfs2 libfuse2
https://salsa.debian.org/salsa-ci-team/pipeline#support-for-salsa-ci-use
Feel free to CC me if you end up sending an email.
Hi! Hm. As I read it, the ~ is only really meaningful with a Debian version (except for, like, +really versions? maybe?), and I have "httpfs2 (<< 0.1.5)". This indicates to me that this should be "httpfs2 (<< 1-1~)", since this package supersedes all httpfs2es older than 1, even if someone has a newer "real" httpfs2 installed locally. Sure, I can justify this as "this is httpfs2 1, too". Yeah, I foolishly hoped this would be sufficiently run-of-the-mill to not warrant this, but this is clearly not the case. On principle, I never mix downstream packaging with the upstream, and maybe I'm too gbppilled, but the standard gbp setup I find much easier. Plus, this makes it impervious to #986320! (if it's ruled as "don't native unless it's Debian infra"; unlikely, given we have 822 native packages, but). I remember reading some manual that said to format transitionals like this, for "special handling", didn't know that was this. Cool! Made libcurlfs 1, updated the gits to have 1-1 (and << 1-1~). Doesn't build on CI due to the perl transition, but it works locally. Best, наб
It does now! and ci.debian.net apparently actually runs tests with Restrictions: isolation-machine so they will probably get exercised :)
Hi наб, Thanks for the update, I have some more review notes: Development documentation nit: I would add a note to d/control on when Package: httpfs2 can be removed (trixie+1 I suppose) and that a Provides: httpfs2 should(?) be added to libcurlfs when that happens. Sponsorship process with gbp: when gbp is involved the convention is for sponsorees to leave UNRELEASED in d/changelog though that's more important with teams to make sure it's clear to all DDs whether an upload happend yet or not. In any case Sponsors should push the final debian/$version tag (and sometimes minor packaging tweaks to expedite things) so it's usually least error prone to have repo access. Easiest way to do that is to move the packaging repo into the debian/ salsa namespace where any DD has the appropriate access. Please add me as Owner to the repo (@dxld on Salsa) so I can do the move if you agree. Collaborative packaging in git: you pushed your updated version as an unrelated history. That's very out of the ordinary. Simply adding another upstream release in the usual way would have been preferred. I've forcefully merged your new history into my repo with $ git merge origin/debian --allow-unrelated-histories --no-ff -Xtheirs Please don't do that in the future unless absolutely necessary even during initial review it makes it harder for your sponsor to review your changes between rounds -- which is like the whole point of using a VSC for packaging and the sponsorship process in the first place :-) Right, this shouldn't be in your upstream version. I don't think it's likeley it will matter either way in our case, just trying to keep to best practices where possible. Yeah thats what I was thinking. I did some more upgrade experiments to understand whats going on here and why the advice is the way it is from first principles: First off unversioned Breaks+Replaces doesn't work because installing our httpfs2 1-1 transitional package would be blocked completely: libcurlfs : Breaks: httpfs2 but 1-1 is to be installed Alright so we want to include 0.1.4-1.1 and exclude 1-1 at the very least, so the tightest bound to achive that is (<= 0.1.4-1.1). So why is the advice to use << instead of <=? I believe this is to cover the +local1 and similar cases Peter mentions where higher version numbers may be yet to come. For example if we declare (<= 0.1.4-1.1) but the user has a 0.1.4-1.1+local1 installed that that would prevent the upgrade to our transition package. Alright so we don't want to be that tight let's increment the Debian revision and consider (<< 0.1.4-2), what's wrong with that? In our case nothing that I can see, but in the framework of Peter's advice a -2 upload is a real possibility and if someone were to backport it you'd end up with 0.1.4-2~bpo12 satisfying Replaces: (<< 0.1.4-2). I'm not sure that's really a problem but it's not really desired and thinking about the consequences seems harder than just excluding the backport version from the relation by appending ~. So I'm still torn on which of 0.1.4-2~ and 1-1~ is technically better, but I'm pretty sure either will be fine in this case :) I am a bit worried we may be subverting policy here since a new httpfs2 upstream release and subsequent re-introduction would now have to employ an epoch to override our transition package, but given that httpfs2 has been removed I reckon it's version namespace is up for grabs. Eh, I don't see that bug reaching consensus ;) Your call. Nice.
Hi! Already had this in my calendar $ cat ~/.ratrun/Remove\ bin\:httpfs2\ from\ sid\=forky 2025-06-01 but added note to d/control, which seems best if I pass before then. I usually upload to mentors.d.o so this is obviously a little different, and (last time I checked) wasn't really outlined there. Agree; likewise. I'd say 1-1~ because it's the version we define and have control over, whereas 0.1.4-2~ is based on some archive state somewhere at some point. But for our case it's a wash. We'd only be hijacking if bin:httpfs2 were still in sid, as you say; it isn't, so AFAICT this is perfectly-well-sanctioned, and a replacement like this is quite common (well, in the decroded-ass salvage packages I usually deal with, anyway). In the umview removal (#1085454 #1085456 &c.), we've replaced src:fuse-umfuse-iso9660's fuseiso9660 with src:fuseiso's fuseiso (and fuseiso9660 transitional Depends: fuseiso) in #1085688, pretty much exactly like src:httpfs2/src:libcurlfs. Actually this /was/ a binary package hijack (0.3-2 -> 20070708-6), because the RMs are yet to clear. Victimless crime. The same thing is planned/staged for src:fuse-umfuse-ext2/src:e2fsprogs. What I could've done instead for this is salvage src:httpfs2 and keep the name, but replace the guts with libcurlfs. Same difference, realistically? (This is what happened with src:fuse-umfuse-fat in #1085458, and half of my DDPO.) If someone wants to reintroduce src:httpfs2 (somehow & god forbid), and have it make bin:httpfs2, it'd need to be version >1 or epoch >0, yeah. But that's their version to bumpo and epoch to justify. Thanks for all your help, наб
Hi, That's just a special case of running out of time for Debian work. Best to always be prepared for this common case ;-) Yeah the docs and perhaps even workflow consensus are lacking, no worries. Its not really important here I just decided to mention it since it's relevant if you ever do any team git work. Seems like a reasonable tie breaker :) Right if we'd reintroduced src:httpfs2 we could have also bumped the version is whatever way we like so indeed there's no difference. Yeah and that's what got me thinking about what the intention of policy is here. AFAIK the "ask d-devel about epochs" policy is just so people don't use them gratuitously when there's other options (mainly +really I guess) but considering whether there may be other reasons I'm not aware of got me thinking.