Dear Maintainer,

The Rust package manager Cargo, makes it possible to declare two-sided version
constraint ranges such as X (>= 6, << 7). In dpkg, historically this has been
expressed by declaring two one-sided version constraint ranges for the same
package like X (>= 6), X (<< 7), however this is not sufficient for Cargo due
to its other feature of allowing transitive dependencies on multiple versions
of the same package.

Currently to support this we are creating packages called X-$version, and
declaring Provides: X (= $version). This breaks in the following situation:

mdbook 0.1.7 transitively depends on slab 0.1.3, 0.3.0, and 0.4.0, via mio 0.5.1,
mio 0.6.14, and ws-0.7.6. These dependencies are declared as:

mio-0.5.1/debian/control: librust-slab+default-dev (>= 0.1.0~~),
mio-0.5.1/debian/control: librust-slab+default-dev (<< 0.2~~),
ws-0.7.6/debian/control: librust-slab+default-dev (>= 0.3~~),
ws-0.7.6/debian/control: librust-slab+default-dev (<< 0.4~~),
mio-0.6.14/debian/control: librust-slab+default-dev (>= 0.4.0~~),
mio-0.6.14/debian/control: librust-slab+default-dev (<< 0.5~~),

I have:

librust-slab-0.1-dev_0.1.3-1_amd64.deb Provides: librust-slab+default-dev (= 0.1.3-1), librust-slab-0.1+default-dev (= 0.1.3-1), librust-slab-dev (= 0.1.3-1)
librust-slab-0.3-dev_0.3.0-1_amd64.deb Provides: librust-slab+default-dev (= 0.3.0-1), librust-slab-0.3+default-dev (= 0.3.0-1), librust-slab-dev (= 0.3.0-1)
librust-slab-dev_0.4.0-1_amd64.deb

Unfortunately, APT chooses to resolve the full set by only installing slab
0.1.3 and 0.4.0, because these two are enough to satisfy the full set,
including the two separate (>= 0.3~~) (<< 0.4~~) constraints. From dpkg's
viewpoint these are allowed to be satisfied by two separate packages.

RPM has resolved this by implementing "rich dependencies" which is a general
solution to express "the same package must satisfy both A and B". However, it
is apparently more difficult to do this in dpkg.

What I am suggesting is smaller: we only need to support two-sided version
ranges like (>= 0.3~~, << 0.4~~), which does not seem to be such a major
addition to the existing one-sided version range syntax.

In the Debian Rust team, we previously experimented with e.g. converting:

- Cargo dependencies X (>= 6, << 7) into dpkg dependencies X-6 | X-7
- Cargo dependencies X (>= 6)       into dpkg dependencies X-6 | X-7 | X-8 | X-9 | X-10

but this causes other problems (unwieldy dependency conversion logic) and does
not really solve the problem described within this bug report, since it is
perfectly legal for separate Cargo crates to declare (= 6.1) (= 6.2) (= 6.3)
dependencies and Cargo will require all three to be installed even though they
are (supposed to be) semantically compatible. In this case, we would have to
create new Debian packages called X-6.i (Provides: X-6 = 6.i) for i={1,2,3}
which puts us right back where we started. (Although granted it is unlikely,
assuming that everyone in the Rust ecosystem is sane and avoids this forever.)

So the cleanest solution to this problem would be to support two-sided version
ranges. This would be beneficial across the whole archive as well - I have
needed to do the Depends: X (>= 4), X (<< 5) hack before when what I really
meant was (>= 4, << 5), and I remember this in others' packages too.

X

Ximin Luo:

X

While I *would* like to see support for version intervals, this
particular issue is a bug introduced through changes to debcargo. The
original debcargo intentionally generated versioned dependencies on
packages like librust-slab-0.3+default-dev rather than
librust-slab+default-dev. That would prevent this issue.

Quoting the relevant commit message:

That change evidently got dropped as part of further changes to debcargo
(and the Rust packaging policy). Re-adding that will eliminate this
issue.

(Two-sided version constraints will make it possible, in the future, to
avoid the "unrepresentable dependencies" mentioned above; however, those
don't occur often.)

- Josh Triplett

Josh Triplett:

Josh, I'm well aware of what the previous debcargo behaviour was and it doesn't eliminate this issue, as I tried to explain in my previous comment:

I wrote:
You haven't been following the latest developments which are plentiful; please take some time to get up-to-date before confusing the discussion with irrelevant information.

I made the decision to do it the current way and diverge from the previous approach, where we generate dependencies based on versionless-package-names, based on a request from Sylvestre Ledru about the long-term maintenance of rust packages. I've been auto-building hundreds of variations of these Rust packages for the past few months; I'm well aware of the repercussions of both approaches and don't need to be reminded. After implementing Sylvestre's request I do think it's much cleaner; and the old way doesn't solve the problem it just hides it under the carpet and we will have to deal with it eventually anyway.

Let's please get back on topic.

X

+pkg-rust-maintainers, -901827 bug report

Josh Triplett:

How not?

It was on IRC and that's where much of the discussion happens, you're missing a lot by not being on it. Also on https://salsa.debian.org/rust-team/debcargo/issues/6 where you can see I was initially against the proposal until I implemented it.

The old code was complex and took me many hours to get right, e.g. see all the commits from June 09 that I did. The previous version was buggy and would not work for some packages; that includes both your old code and Vasudev's work on top of it. The bug-free version has pretty complex logic that I don't think is easy for a newbie to grasp either.

By contrast the new code took me about 2 hours to write, and only fails for this mdbook case out of all of the transitive dependencies of {mdbook, exa, ripgrep, debcargo}, and only because dpkg cannot properly express what is needed for Cargo crates.

It is possible to revert to the previous behaviour but Sylvestre's concerns in issue #6 remain and I am starting to agree with him. I actually would just prefer to add an override to mdbook for the time being in order to forcibly add an extra dependency on slab 0.3.0, and link to this issue in the comments explaining that override.

X

I very carefully read your comment before replying.

This is not equivalent to what I was referring to.

I've checked for that exact problem throughout the Rust crate ecosystem,
and found very few instances of it (some of which were quite fixable).

I have not seen the proposal or rationale to diverge from the previous
approach discussed on pkg-rust-maintainers at all; do you have a pointer
to where that was proposed and discussed?

I also very specifically said that I *do* want to see the proposed
improvement to dpkg regardless.  But a change to dpkg's version handling
won't be usable until after a subsequent stable release.

On the contrary, the previous approach works for the majority of version
dependencies, as long as you don't want multiple compatible versions
simultaneously installed. As long as you don't hit that case, which the
vast majority of Cargo crates don't hit (I scanned for such dependencies
throughout the entire ecosystem at the time and found very few), then it
handles cases such as the one you originally posted just fine. And we
could always switch to the other approach after version range support
appears in dpkg.

By contrast, this approach to Provides and Depends seems to fail in the
most *common* cases of Cargo dependencies, including simple "compatible"
dependencies.

I'm not proposing to hide a problem; I'm proposing that, given that
*both* approaches would benefit from dpkg support for version range
dependencies, we should in the meantime attempt to accommodate the most
common dependencies without that dpkg support.

Hi!

We discussed this briefly yesterday on IRC before Ximin filed the
report. I don't see much difference with what was mentioned and I'll
repeat what David Kalnischkies and me proposed on the spot, as a
potentiall working solution.

The problem is that you want to depend on independent "major" versions
of these packages, while making using the minor versions available.

I see multiple ways to go about this, but these are based on
assumptions about the versions and the relationships that I'm not sure
hold within Cargo.

The key is, when having an X.Y.Z version in Cargo, not to use that
either as part of the real or virtual package names. Because I
understand you/Cargo care only about the "major" part of that version.

So you could have package slab-X.Y and then depend on just that, or if
for some reason you need to have coinstallability down to the minor
version, then slab-X.Y.Z, in which case that package would provide
slab-X.Y (= X.Y.Z). In addition, all of these would also provide
slab-X (= X.Y.Z) and slab (= X.Y.Z) and probably also slab, so that
you can represent all the range types.

  Cargo deps (A)		dpkg deps (A-X.Y || A-X.Y.Z)
  A				A
  A (>= 6)			A (>= 6)
  A (>= 6.1)			A (>= 6.1)
  A (>= 6.1.3)			A (>= 6.1.3)
  A (>= 6, << 7)		A-6
  A (>= 6.1, << 6.2)		A-6.1
  A (= 6)			A-6
  A (= 6.1)			A-6.1

I've not checked how difficult it would be in dpkg. The main problem
here is that the dependency resolution logic is not concentrated just
within dpkg itself, but spread all over the place, and changing/extending
the semantics of these tend to be very painful, and long-winded
processes. So there's general reluctance to do that when there's no
apparent need.

Check for example versioned provides, some people are still afraid to
use them!

It's still a change in the current semantics, and implies modifying a
ton of projects, I'm afraid.

That seems indeed wrong. At least the first should have been just X-6,
and yeah the second should have used more granular versioned provides
as shown above.

Depends on how you define cleanest. :) In this case that means you
cannot use this anyway until a stable+1 (assuming just dpkg and apt
implemented this right away), and it would not be usable anyway until
most major dependency parsers/satisfiers would support it too, which
might be stable+2, perhaps.

Of course the counter-proposal that David and me provided assumes the
versions have sane semantics (similar to semver), and that their
format is uniform all over Cargo. If that's not the case then that
might be a problem, if you are converting the dependencies
automatically. :)

Thanks,
Guillem

Guillem Jover:

To be fair, neither of you were very specific and the words you said could just have as easily described what we were already doing previously, which is how I interpreted it at the time.

Thanks for being specific here. After thinking after it for a bit, I think this might work. The key is to only generate a single item within in the comma-separated list of AND-clauses in the dpkg dependency, where the item itself is a "|"-separated OR-clause. It would be pretty ugly in some circumstances:

Cargo deps                      dpkg deps
A (>= 6.1, << 9.5) ---->        A-6 (>= 6.1) | A-7 | A-8 | A-9 (<< 9.5)

but I think I have a decently-simple way of achieving this in debcargo.

The first was a typo, and as for the second I did not realise by "granular" you meant Provides both X (= 6) and X-6 (= 6) etc, I thought you meant just the latter which is what we were already doing.

If you guys implement it, I'd be happy to switch to it after 2 stable releases, that's only 4 years and I'm sure Rust will be around for a long time. The fact that it's so hard means there's a load of technical debt there that is worth trying to pay back anyways.

X

Sorry for the inconvenience. For context, the current Debian rust situation is a workaround for dpkg bug #901827.

In general though, arbitrary buffer sizes should be a thing of the past. Have you considered rewriting that code in Rust? :) Or perhaps less effort, just reusing whatever code dpkg itself uses to parse it.

X

Ximin Luo:

In the abstract example above, since A-5 (= 5.1.1) replaces files from a previously-uploaded A (= 5.1.1), it must declare Replaces+Breaks: A (= 5). But in reality, there may be multiple Debian uploads of 5.1.1 including security uploads and backports. So we really need to declare:

Breaks: A (>= 5.1.1~~, << 5.1.2~~)
Replaces: A (>= 5.1.1~~, << 5.1.2~~)

but this is not possible in Debian today. Note that this:

Breaks: A (>= 5.1.1~~), A (<< 5.1.2~~)
Replaces: A (>= 5.1.1~~), A (<< 5.1.2~~)

won't work as it is equivalent to Breaks: A, Replaces: A which is not what we want.

X

Control: severity 901827 wishlist

As has been mentioned elsewhere, I don't think we really need to do
this strict range coverage, because in general we do not support
downgrades, and only upgrading forward.

In any case, this is still a whishlist bug, reset accordingly.

Thanks,
Guillem