devscripts currently depends on sopv | gpgv, where sopv is a purely
virtual package. This means that an unpredictable implementation of the
sopv interface gets chosen. When I tried

$ podman run --rm -it debian:trixie-slim
# apt update
# apt upgrade
# apt install --no-install-recommends devscripts

it chose to install sopv-gpgv and gpgv, but if I instead end with

# apt install devscripts

it chooses to install both gpgv and sq (possibly because dpkg-dev
Recommends those).

Similarly, on test systems for a Debian 13 derivative, I've variously
ended up with sopv-gpgv or rsopv, depending whether I'm upgrading an
existing installation or creating a new one.

Compare with dpkg-dev, which Recommends concrete packages (sqop, sqopv)
in preference to a similar virtual package (sopv), so that users who
don't know what they want will predictably receive the implementation
that is recommended by the dpkg-dev maintainer.

I'm unsure of the benefits and drawbacks of rsopv vs. sqopv. I know that
gpgv has some semantics that are unfortunate for the authenticity check
use-case, like assuming that an unverifiable signature is a serious
problem even if it's accompanied by a valid, verifiable signature.

Can someone (perhaps Daniel) recommend a sopv implementation that is a
good choice for someone who doesn't know much about the finer points of
PGP verification, and just wants something not too large that will check
signatures "correctly"?

I think it would be better for system reproducibility if either
devscripts had a Depends on some-concrete-package | sopv | gpgv, or
devscripts had a Depends on default-sopv | sopv | gpgv, where
default-sopv is either a metapackage that Depends on a recommended sopv
implementation (like default-jdk), or a virtual package that is provided
by exactly one binary package per architecture (like
default-mta and default-logind). The approach where each virtual package
FOO has an accompanying virtual package default-FOO with at most one
implementation seems to be a popular one in recently-implemented
alternatives sets, and it doesn't require trips through NEW.

    smcv

control: severity -1 normal
thanks

Hi Simon,

thanks for filing this bug report. I agree, reliable behavior is
desirable, hence raising the severity.

I'm not sure I want to introduce a new virtual package at this stage
of the release. I'd rather change the depends from the current
"sopv | gpgv" to
"sqv | sqopv | rsopv | sopv | gosop | pgpainless-cli | gpgv-sq | gpgv"
taken from dpkg-dev.

More feedback definitly much welcome.

Control: tags 1107155 + patch

I don't think this is quite right, holger, since devscripts doesn't know
how to handle either "sqv" or "gpgv-sq".

Ideally, devscripts wouldn't be the package to make this decision, but
for the sake of predictability the patch below might be useful.

If devscripts only knows how to operate a sopv implementation or, as a
special case, gpgv, then yes it should only list sopv implementations
and gpgv in this list of alternatives.

Sure, but unfortunately we don't live in an ideal world, and we're
unlikely to arrive in one before the trixie release!

What is "the system default OpenPGP verifier"? Is that a
yet-to-be-designed mechanism, or some sort of default-sopv virtual
package or metapackage, or what?
...

Thanks, this answers the question I wanted to ask on behalf of my Debian
derivative: "which sopv implementation should we prefer to work around
this?" -> sqopv.

But, is there a reason why the less-preferred (non-sqopv)
implementations need to be listed explicitly? If they can be dropped
from the or-group, then that would mitigate your maintenance concerns.

sqopv seems to exist on all release architectures, and on all -ports
architectures that have a working rustc/cargo. Even if we're assuming
that non-Rust -ports architectures are important, the the minimum to get
a predictable installation would be to list some real package that
exists on those architectures as higher-preference than (i.e. before)
the sopv virtual package, like maybe:

     sqopv     # real package, but not on all architectures
     | gpgv    # real package, portable fallback
     | sopv    # virtual package

which I believe would result in this result:

- if the user already has either a sopv implementation or gpgv installed,
   then the dependency is satisfied, so do nothing;
- else if this is a Rusty architecture, install sqopv (most-preferred
   alternative);
- else install gpgv (less-preferred alternative)
- and in principle if gpgv was somehow unavailable, fall back to
   installing any sopv implementation (least-preferred alternative), but
   in practice gpgv seems to have architecture coverage equal to or wider
   than any sopv implementation

sqopv has wider architecture coverage than rsopv, so listing rsopv
doesn't seem like it adds value: on any architecture where rsopv is
available, you're asking apt to prefer sqopv anyway.

Similarly sqopv has wider architecture coverage than gosop (which FTBFS
on most -ports architectures) so listing gosop doesn't seem like it adds
value.
won't have a practical effect on what gets installed.

If I understand apt correctly, what I'm asking for is effectively: on
every architecture, there is a real (non-virtual) package listed as
more-preferred than the sopv virtual package. That makes the choice of
package to install (mostly) predictable.

If we can cut the or-group down to "sqopv | gpgv | sopv", then I think the
only maintenance that will be needed is: if our most-preferred sopv
implementation changes, such that instead of sqopv you would prefer new
installations to get my-new-sopv, then devscripts will need changing to
replace the or-group with "my-new-sopv | gpgv | sopv".

(Or, perhaps more realistically, if we get a default-sopv metapackage or
virtual package in forky, then that would replace sqopv.)

     smcv

agreed, hence the patch ;) I'd love to hear suggestions for what next
steps you think we need to get closer to the ideal world post release!
Provides: sopv, and is pointed to by sopv in the default $PATH.  At the
moment, that's an update-alternatives question.  Guillem and I have had
a lengthy conversation [0] about possible other ways of doing it, which
haven't really resolved into anything beyond what's in place today.

[0] https://gitlab.com/dkg/openpgp-stateless-cli/-/issues/42

I'd love to get more feedback on what you (and others) think a stable
and simple way to have that be a stock choice for debian.

i don't personally have a strong preference among the three listed
implementations.  I've tested them all personally and i found them
reasonable both in terms of performance and robustness; and they all
perform well on the OpenPGP Interoperability Test Suite [0]. I chose the
order here following Holger's listing, really.

[0] https://tests.sequoia-pgp.org/

I agree we could remove them, probably without any functional
difference.  Similarly, we could choose either of the other two as that
primary placement and it would also be fine on all release
architectures.

I think this order would be a mistake.  We should prefer sopv over gpgv,
because the sopv semantics and interface are deliberately minimalist,
stable, and well-understood.  gpgv has corner cases that make it
challenging to use in situations where OpenPGP might evolve (e.g., it
fails when it sees a signature that it doesn't understand, even if that
signature is coupled with a signature it *does* understand; and, its
ability to select which OpenPGP certificate to trust can't cope with
armored certificates; and, the way that you identify certificates has
idiosyncratic rules that don't follow standard unix conventions; and, it
deliberately doesn't implement the latest version of OpenPGP).

Please list gpgv *after* sopv.  I say this as one of the co-maintainers
of the GnuPG suite in debian.  If anything, i would recommend dropping
support for gpgv from devscripts.  It will still be usable in normalized
form with sopv-gpgv for those people who really want that to be their
system OpenPGP verifier.  And there will be less code to maintain in
devscripts.

That's an interesting proposal, and i'd be happy to work through the
details there and make it happen after the trixie release.

That answers the question for this one specific installation ("the
system" in the sense of "this system I'm typing my email into"), but it
doesn't answer the question for "the Debian system". The system default
MTA is the package that provides /usr/sbin/sendmail, but deciding that
wasn't enough: we also needed to choose Exim. That isn't the only choice
we could have reasonably made - Ubuntu chose Postfix instead, and in
fact I use the non-default Postfix on my own Debian systems - but it's
necessary that each distro did choose one, so that sysadmins can predict
the behaviour of a new installation.

And, similarly, if Debian decided to switch our default to Postfix, or
if Ubuntu decided to switch their default to Exim, that would be
completely fine (both are entirely reasonable MTAs), as long as it's
someone's deliberate decision rather than something that happened by
mistake.

I think the design that's used for MTAs and the D-Bus message bus could
make a lot of sense for sopv:

- packages that want to verify signatures via this interface and don't
   particularly care about the specifics have Depends: default-sopv | sopv

- at most one package per architecture Provides the default-sopv virtual
   package at any one time, so that there is no ambiguity about what it
   means

D-Bus is perhaps a more interesting example here than MTAs, because the
default-dbus-session-bus virtual package is genuinely implemented by
different real packages on Linux (dbus-user-session) and non-Linux
(dbus-x11), as a result of dbus-user-session requiring systemd which is
non-portable.

Or default-sopv could be a real metapackage/dependency package instead
of a virtual package, more like default-jdk; that would be fine too.

For historical reasons default-mta and mail-transport-agent use
different names, as do default-jdk and java-sdk; but the precedent I've
tried to set with the D-Bus virtual packages is that Debian's default
implementation of the virtual package FOO should be named default-FOO,
and I think that's a good pattern to follow in future. For D-Bus, the
values of FOO are dbus-session-bus and dbus-system-bus. For the PGP
ecosystem, I imagine the values of FOO would be sop and sopv, and
possibly others.

Either of those designs means that if/when we want to change the default
implementation, we can change the O(1) packages involved in providing
the implementation of the interface, and all consumers of that interface
can continue to depend on "default-FOO | FOO" and will automatically
pick up the new default.

OK, in that case:

     sqopv | sopv-gpgv | sopv

or perhaps

     sqopv | sopv-gpgv | sopv | gpgv

The desirable property here is that on every architecture, there is an
existing, "real" package listed before the first virtual package:

- on architectures with Rust, apt will prefer sqopv unless told otherwise,
   giving us predictability; it won't install a randomly chosen one of
   { sqopv, rsopv, sopv-gpgv } based on the phase of the moon

- and similarly on architectures without Rust, apt will prever
   sopv-gpgv, rather than choosing a random implementation of sopv

What I'm trying to avoid is that when I bootstrap a container with
Essential + apt, either in Debian or in a Debian derivative that I'm
responsible for, I get sqopv one week and rsopv the next, without anyone
having made a conscious decision to change the default. I don't really
care which one I get, and I'm happy to defer to your much-more-expert
opinion on which one should be preferred, as long as what I get is
predictable and mostly stable, only changing if the relevant facts or
opinions have changed.

     smcv

hi,

so for devscripts in trixie,
dkg suggests:
	sqopv | rsopv | gosop | sopv | gpgv
while Simon suggests:
	sqopv | sopv-gpgv | sopv | gpgv

I agree we want predictability (on all archs), so I think/believe Simon's
suggestion is the better one here and now?

Hi Simon--

Thanks for the extensive and thoughtful reasoning about the longer-term
strategy here.

Regarding the immediate question:

As i understand it, the point of this recommendation is to have a
predictable sopv choice (sopv-gpgv) for platforms that can't build
sqopv (e.g., due to lack of Rust)

The other nice thing about this approach is that we could drop drop all
code that invokes gpgv directly from devscripts, simplifying both the
codebase and the types of OpenPGP certificates that can be used
(e.g., sopv accepts both armored and unarmored certificates; gpgv only
accepts unarmored certificates)

This one i'm not as convinced by.  If gpgv is already installed,
then installing devscripts (on a platform with or without rust) won't
bother installing any of the sopv variants, right?

To be clear, we're talking here about devscripts, which isn't involved
in either Essential or apt, right?

All the best,

Sorry, yes, I was conflating two things here.

When I bootstrap an Essential + apt container, you're right that
devscripts isn't involved, but I want apt to pull in a predictable
implementation of whatever interface apt uses to verify signatures, and
not flap between two or more different implementations with each
rebuild. At the moment, apt has a hard dependency on sqv rather than
participating in alternatives, which is great for predictability
(although less so for flexibility).

And, when I bootstrap an Essential + apt container and then install a
stack of additional packages to make a runtime platform or SDK (which
might reasonably include devscripts), I want that container to pull in a
predictable implementation of whatever interface devscripts uses to
verify signatures, and not flap between two or different implementations
with each rebuild. This is the use-case that devscripts' dependencies
contribute to.

A concrete example is that the Steam Runtime SDK that I help to maintain
for Valve does pull in devscripts - albeit without most of its
Recommends, because we only want parts of its functionality.

     smcv

Yes. On release architectures and other rusty platforms, sqopv is the
preferred choice, it gets installed, success. On non-rusty ports, sqopv
is unsatisfiable, so apt tries sopv-gpgv, which gets installed, success.
The non-default sopv option (which is non-deterministic because it has
multiple implementations) is only reached if an implementation happens
to be installed already.

(If you only care about predictability on release architectures and other
rusty platforms, then the sopv-gpgv option can be omitted. I am,
personally, not very interested in -ports, but I try to be nice to them
when it's this easy to do so.)

As a change for forky, great, please do that. As a change for trixie, in
the release team's place I'm not sure that I'd accept that code-deletion
during hard freeze (but I'm not the release team and perhaps my guess at
their preference is wrong).

Yes, I think so. This is the more conservative of my two suggestions
quoted in this message, matching current behaviour more closely than my
other suggestion; but if you're unhappy with the quality of the
(non-sopv-) gpgv code path, then you should prefer my other suggestion
over this one.

     smcv

Soo, trixie was released some time ago. :-)

Simon pointed me at this bug because I asked on IRC about this
dependency, and because someone else noted that a grml-full build
shipped with `gosop`. Apparently `gosop` also `Provides: sopv`, and
apt picked that.

Is there something that needs to be unblocked? Or should someone
just push a change into devscripts.git?

Best,
Chris