Greetings,

At the moment, NoCeM messages generated using News::Article::NoCeM
declare a hard-coded signature hash algorithm SHA1:

|  $self->add_body("-----BEGIN PGP SIGNED MESSAGE-----");
|  $self->add_body("Hash: SHA1");
[ll.202]

This broke NoCeM processing here as the actual algorithm used was
SHA512, and the verification in INN2's perl-nocem fails then (took a
while to find out as the only way to debug perl-nocem is strace, and
there was only a warning about a hash algorithm mismatch. But changing
the above line made the problem go away).

Following the robustness principle, the fix should be here, in
News::Article::NoCeM. That however will be a bit delicate as I doubt
this will be easy.

Some *bad* ideas:

* In my setup, signing is done using gpg (i.e. gpg2). Perhaps enforcing
  gpg1 resolves the issue - I haven't checked - but that's not the
  direction we should go.

* Enforcing SHA1 in the signing might be doable (but not easy), still,
  no. That algorithm should be phased out.

* Hard-coding a different algorithm is a no-go, for bad style to start
  with.

* News::Article::NoCeM could inspect the generated signature and
  act accordingly. This adds a lot of code and feels pretty wrong.

* Omitting the hash declaration is not an option either, perl-nocem
  fails then.

So I guess this will require some co-ordination with PGP::Sign (Cc'd).

It seems the latter does not provide an option to define the hash
algorithm, unless perhaps via GnuPG's configuration file. That's
somehting News::Article::NoCeM could do by switching to the OO interface
of PGP::Sign. But at the same time, this voids a nonoutspoken goal of
making GPG::Sign backend-agnostic (having more than gpg1 and gpg2 was
desireable but that's something for another day).

Another solution I can think of: PGP::Sign could, as an option, already
provide the marker lines with a correct pseudo header about the hash
algorithm, so News::Article::NoCeM does not have to do this. Or at
least signalize the proper value. But there's nothing of that kind as
far as I can see.


But these are just thoughts - I'm looking for a robust way to generate
NoCeMs and if that goal is met, I care little how this is actually done.

    Christoph

Christoph Biedl <debian.axhn@manchmal.in-ulm.de> writes:

I'm somewhat surprised by this, as my impression was that these Hash lines
are optional and GnuPG did the right thing if they were omitted entirely
(although you do still need a blank line).  I have not looked into this in
detail, but I thought the hash algorithm was also present in metadata
inside the signature itself.  This is essentially required for the main
use case of PGP::Sign, Usenet control messages, since the syntax of the
X-PGP-Sig header has nowhere to put this metadata and thus it is always
lost.

perl-nocem itself doesn't seem to care and just copies the whole input
into a temporary file for GnuPG.  What's the nature of the failure?  Is
GnuPG failing to validate the resulting file if the hash algorithm is
omitted?

Russ Allbery wrote...
habe been a quick solution.

It seems that pseudo-header is mandatory but I haven't checked further:
https://sources.debian.org/src/gnupg2/2.2.40-1.1/g10/sig-check.c/?hl=188#L188

So, a blank line doesn't help. The message by gpgv is

| gpgv: Signature made Fri Jan  5 18:21:01 2024 UTC
| gpgv:                using RSA key 87FB8F9D33883045A832B4FFD90D76CC97A7B20D
| gpgv: WARNING: signature digest conflict in message
| gpgv: Can't check signature: General error

and this leads to an error message from perl-nocem:

| Article <redacted>: unknown error (ID D90D76CC97A7B20D)

where "WARNING: signature digest conflict in message" is the same as
I had seen in the first place, when there was the hardcoded "SHA1".

For completeness, this is gpgv 2.2.40-1.1, from Debian 12 ("bookworm").
Also, neither the NoCeM message nor the key are publicly available.

It is indeed present there, I used pgpdump to reveal the hash algorithm
is actually SHA512. So this is a design decision I don't quite follow,
but possibly there is or was a need to do things that way.

(...)

See above.

    Christoph

I think the critical thing I missed in the original message is that
News::Article::NoCeM is constructing an inline signature by calling
pgp_sign.  The Hash header here is before the signed body, not before the
signature, which is obvious in your original message but which I failed to
pay proper attention to.

Christoph Biedl <debian.axhn@manchmal.in-ulm.de> writes:

I think this is a bug in News::Article::NoCeM.  It is constructing an
inline signed document using PGP::Sign's pgp_sign function, but pgp_sign
creates detached signatures.  Detached and inline signatures are subtly
different, which has historically been the cause of all sorts of pain and
suffering trying to deal with OpenPGP signatures.

This is explicitly called out in the PGP::Sign manual page, although it
should be clearer since it implies the only issues are with whitespace
munging, but it seems like there are more issues than just that.

    The whitespace munging support addresses the most common difference
    between cleartext and detached signatures, but does not deal with all
    of the escaping issues that are different between those two
    modes. It's likely that extracting a cleartext signature and verifying
    it with this module or using a signature from this module as a
    cleartext signature will not work in all cases.

The other use cases for PGP::Sign (control message signatures and
PGPMoose) both use detached signatures, and it does try to document that
it only deals with detached signatures:

    This module supports only two OpenPGP operations: Generate and check
    detached PGP signatures for arbitrary text data.

Again, though, I should make this clearer.

I'm not sure where that leaves this bug, though.  It's quite
understandable that News::Article::NoCeM doesn't want to implement the
annoying logic of figuring out the correct flags to call GnuPG, but if the
expectation for NoCeM messages is that they use inline signatures (which I
believe is the case, although ideally they should use multipart/signed and
application/pgp-signature), PGP::Sign doesn't do that.  I do have other
use cases for inline signatures currently, so I am not completely opposed
to adding that support, although the more correct thing for me to do with
those other use cases would be to switch to multipart/signed instead.  At
least the last time I looked, inline signatures were very poorly
documented and standardized.

It's possible that this specific bug could be fixed if there were a way to
pass the desired hash algorithm into the sign() method of PGP::Sign so
that News::Article::NoCeM can force SHA-1 as a hash algorithm, thus making
the signature match the headers.  You suggested that in your original
message.  That's a bit more within the remit of PGP::Sign and I feel more
comfortable supporting that.  But I fear that may not be a full fix, since
there's still the detached versus inline signature mismatch that I think
is quite likely to produce more problems in the future.  (And of course
there's also the problem that News::Article::NoCeM really should be using
SHA-256, but that raises backwards compatibility issues.  There are a lot
of ancient PGPs out there in Usenet world.)

This is ringing a vague bell.  I think the issue with inline signatures is
that since the document is stream-processed, the hash function that should
be used for the text has to be specified *before* the signed body text.
By the time the signature is read, it's too late; the body has already
been consumed and hashed with the default hash algorithm, and the correct
hash is no longer available.

I believe what hash algorithm GnuPG uses by default is controlled by local
GnuPG configuration, and it may well default to SHA-256 these days.

Also, all of these modules should switch to a sane interface to OpenPGP
signing and verification, like sup, but that's a whole other discussion.

This topic is getting a bit bigger, perhaps let me sort first what
issues we're talking about here. So to summarize the longer text that
follows:

1. News::Article::NoCeM may embed an invalid hash algorithm declaration,
   depending on the gpg program used by PGP::Sign, and possibly other
   reasons.
2. News::Article::NoCeM uses a detached signature as an inline signature.
   This is at least bad style, and possibly fragile.
3. There are issues with NoCeMs messages in general:
   * For various reasons, they should better be multipart messages.
   * The specification is rather vague on signing.
   * There might be need to define good and acceptable hash algorithms.
4. Last, but certainly not least: The problem of very old PGP programs.

This here should mostly about the first item as this is real issue, and
something I'd like to see fixed in Debian 13 ("trixie") - I doubt it's
that urgent somebody would want to handle this in a point release.

The second is something of robustness although I'm not aware of any
actual problems caused by this. If it's easy to do, sure thing.

The rest should better be discussed elsewhere (news.admin.*,
inn-workers or whatever fits).


Russ Allbery wrote...

#TIL

Now I'm wondering how many people know these signatures are different
things. I did not so I will not blame anyone for mixing them.

Well, there's (emphasis mine)

| PGP::Sign - Create *detached* PGP signatures for data, securely

... but this doesn't help if people aren't even aware there is a
difference.

All I know about a specification of NoCeM messages is Tim Skirvin's FAQ
at [1]. Which does not go into detail about the actual signature
format. So we're left with perl-nocem and derive something like a
specification from there. And yes, perl-nocem expects an inline
signature. Enhancing it to process multipart message is not impossible
but it would take years until installations out there have followed.

Of course I will not object if you would enhance PGP::Sign to properly
create inline signatures :-)  But I read between the lines you'd prefer
to move into a different direction.

So perhaps doing this inside News::Article::NoCeM might be an option as
well, none that I like, though. But that for upstream to decide, I've
invited them to join this discussion.

As I'm already taking a lot of your time, mind to explain what problems
you are seeing here? If that might break NoCeM processing in general,
we should take precautions.
other implementations that handle NoCeMs? My impression is news admins
who want to process NoCeMs must install GnuPG.

And the NoCeMs I generate, with SHA512, were - once the Hash: header
was correct - handled without a problem using GnuPG 1.4.10 from Debian
squeeze(sic!). So I cannot see a reason to limit hashing to SHA-256.

And if it's really about these decades-old PGP programs: As they are
not supported by perl-nocem, I cannot see they have relevance when it
comes to NoCeM processing. And, aside, for control message, I think
we're getting closer to the point where we no longer have to care:
While I don't claim to be an expert on (quantum) cryptography, it's
obvious those very old algorithms and short keys are the first to
become breakable with the advance of that technology, something that
might even happen within the next five years. And once the first forged
but cryptographically valid control messages float around, there is no
alternative to disable anything that is still based on that.


[ On the mandatory "Hash:" pseudo header ]

Makes sense, explains why a mismatch is an error and not just a warning.
Thanks for shedding some light on this.

Regards,

    Christoph

[1] http://www.cm.org/faq.html