Dear release team,

this is a summary of a conversation which I've just had with Helmut
Grohne on the phone. We've discussed the several options which are
available to resolve the security problems within the golang ecosystem.

First of all, I've just uploaded the golang compilers 1.25 and 1.26 to
unstable. Both contain the minimal patch which had been suggested by
Helmut, in order to support a new environment variable DEB_GOMINCOMPAT.
With this variable, we are able to set the golang compiler compatibility
version during package builds.

Currently, this is a no-op, because the variable DEB_GOMINCOMPAT does
not exist yet in any d/rules file.

The next step would be to update dh-golang to actually set
DEB_GOMINCOMPAT, which this bug report is about.

Helmut and I discussed the "baseline" compatibility version which should
be used for unstable. While I initially had been in favor to use the
same version the available compiler provides (which would be 1.26), I
can also see some advantages in downgrading the compatibility to 1.24 in
unstable.

Most of the problematic settings (regarding security issues) are already
resolved in 1.24. From what I can tell, the only security related
difference between 1.24 and 1.26 is the setting "tlssha1=1". This flag
re-enables SHA-1 signature algorithms in TLS 1.2, which had been dropped
in Go 1.25.

In order to actually use the new compatibility setting for our packages,
we would need to schedule binNMUs for 502 golang packages, which build
an executable binary. Fortunately, the majority of golang packages just
contain source code and are therefore not affected.

During a test rebuild, I've found 10 packages which currently FTBFS in
unstable, even without changes to the compiler or dh-golang. Not all of
them have RC bugs filed yet, but none of those build failures are
related to the proposed changes.

Some packages *do* fail due to this change in dh-golang, though.
However, their number is really low, just 7 packages.

golang-github-anacrolix-missinggo
golang-github-d5-tengo
golang-github-mendersoftware-mender-artifact
golang-github-traefik-yaegi
golang-github-xenolf-lego
secsipidx
snapd

From a quick glance at their build logs, it seems that they might just
need a small patch to build with a compatibility setting of 1.24.

Helmut and I also thought about the new script in dh-golang, which
checks for problematic settings in the compiled binaries. The initial
approach is not sensible, because it would show an error (and fail the
build) if some specific settings are detected. However, for some legacy
packages, it might be the only option to use a compatibility version
lower than 1.24 -- the build should then still succeed, of course.

A better approach (with more visibility) is to integrate such a check
into Lintian. It could be of severity info or warning. If the latter, it
would also show up on tracker.d.o.

Therefore, the required changes to dh-golang are now even smaller. I've
attached the proposed patch for dh-golang.

Please note that there are also unrelated minor changes in the dh-golang
git repository, due to some package maintenance. IMHO, those changes do
not harm the intended transition, which is why I've left them in.
However, if you prefer a minimal patch, I'll create another one without
the unrelated changes.

Regards,
Tobias

[ I am not a member of the release team ]

For people reading your email it would be helpful if you could provide a
summary of what "the security problems" are.

Carrying a distribution patch for an actively developed compiler forever
is not good. There is also a related issue that whatever problems and
solution Debian wants are likely also desirable by other distributions.

What is the actual problem and how could that be fixed upstream?

Is my understanding correct that the version in the mandatory go
directive in go.mod is documented as a minimum version, but it is
also used as default setting for godebug with no way for go.mod
to tell godebug "use the latest version"?

If such semantics is really desired, the ability to overwrite default in
the GODEBUG environment variable should be possible.

Is there a reason against setting a default GODEBUG in dh-golang
(that debian/rules might override) containing only the relevant
security settings?

This would be a smaller change, and also avoid patching the compiler.

Do these 502 golang packages have the compiler in Built-Using or Static-Built-Using?

If yes, then any changes will anyway get picked up by the next round of
"Rebuild for outdated Built-Using" binNMUs the release team regularly
does in unstable.

If no, then this is a security problem within the golang ecosystem that
should be be resolved - this information is needed for enabling binNMUs
of statically linked Go binaries when a Go module package compiled into
a binary got a security update.

cu
Adrian

Am 21.05.26 um 16:59 schrieb Adrian Bunk:

Hi Adrian,

true -- sorry for that. We've had many mails with the release and
security team during the time when this had not been disclosed, so they
do have enough context. But people reading this the first time do not.

A very short summary is that the golang compiler enables compatibility
settings for legacy packages via DefaultGODEBUG flags. Some of those
flags allow creating RSA keys < 1024 bits, TLS 1.0, SHA1 etc., basically
discarding modern security standards. Right now, due to the way Debian
builds its golang packages, every executable binary is allowed to use
insecure settings during execution. See the talk of Andrew Lee at
MiniDebconf Hamburg for more details.

https://meetings-archive.debian.net/pub/debian-meetings/2026/MiniDebConf-Hamburg/hamburg2026-85-rescue-forky-we-have-go-back-to-2011-building-go-like-its-2011-is-broken.lq.webm
which did work out really well during our test rebuilds. It's not
planned to keep this patch forever. Quite the contrary, our hopes are
that we might get rid of it before Forky is released.

Regarding your second point: as it turns out, this is a problem only
specific to Debian, due to the old (and by today's standards in the
golang language obsolete) way of building golang packages. We completely
ignore go.mod files and turn GO111MODULE=off during build. Due to this,
the compiler basically falls back to a compatibility level of
golang-1.0, released in 2012. From what I understand, all other major
distributions do not use this approach, but do use the go.mod files for
building their packages.

See above, it's not an upstream problem, this is specific only for
Debian builds.

Yes, that's how I understand it as well. Note that the version in the
go.mod file of the "main" package is used, it's not the minimal go
version from all dependencies combined.

I don't quite understand what you mean here. Could you elaborate?

I don't know -- we've calculated the list of packages based on their use
of golang-compilers in Build-Depends. I guess that most (all?) packages
do have the compiler in Static-Built-Using, unless they are *really*
old. The use of Static-Built-Using has been implemented in dh-golang in
April 2022, released as version 1.55. Even oldstable has 1.59.

Regards,
Tobias

Am 21.05.26 um 21:14 schrieb Dr. Tobias Quathamer:

I've just realized that I forgot to write why we hope to get rid of the
patch. :-)

The plan is to enable GO111MODULE=on for Debian package builds, so using
the version from go.mod. This is overdue since a few years already, but
so far nobody found the time/energy to tackle this.

Regards,
Tobias

Hi Tobias,

are you sure that using go.mod would get rid of the patch?

That's what I was talking about:

secsipidx-1.3.2$ cat go.mod
module github.com/asipto/secsipidx

go 1.16

require (
        github.com/gomagedon/expectate v1.1.0
        github.com/google/uuid v1.4.0
)
secsipidx-1.3.2$ go list -f '{{.DefaultGODEBUG}}' github.com/asipto/secsipidx
asynctimerchan=1,containermaxprocs=0,cryptocustomrand=1,decoratemappings=0,gotestjsonbuildtext=1,gotypesalias=0,httpcookiemaxnum=0,httplaxcontentlength=1,httpmuxgo121=1,httpservecontentkeepheaders=1,multipathtcp=0,netedns0=0,panicnil=1,randseednop=0,rsa1024min=0,tls10server=1,tls3des=1,tlsmlkem=0,tlsrsakex=1,tlssecpmlkem=0,tlssha1=1,tlsunsafeekm=1,updatemaxprocs=0,urlmaxqueryparams=0,urlstrictcolons=0,winreadlinkvolume=0,winsymlink=0,x509keypairleaf=0,x509negativeserial=1,x509rsacrt=0,x509sha256skid=0,x509usepolicies=0
secsipidx-1.3.2$ sed -i "s/go 1.16/go 1.24/g" go.mod
secsipidx-1.3.2$ go list -f '{{.DefaultGODEBUG}}' github.com/asipto/secsipidx
containermaxprocs=0,cryptocustomrand=1,decoratemappings=0,tlssecpmlkem=0,tlssha1=1,updatemaxprocs=0,urlstrictcolons=0,x509sha256skid=0
secsipidx-1.3.2$ sed -i "s/go 1.24/go 1.26/g" go.mod
secsipidx-1.3.2$ go list -f '{{.DefaultGODEBUG}}' github.com/asipto/secsipidx
secsipidx-1.3.2$

I can reproduce the secsipidx FTBFS from your list by adding
  export GODEBUG=rsa1024min=1
to debian/rules.[1]

That's the generation of the misc:Static-Built-Using value,
I was thinking of missing usage in debian/control.

cu
Adrian

[1] below/instead of the obsolete GODEBUG=x509sha1=1

Hi Adrian,

sorry for the delay, I've mostly been AFK.

Am 22.05.26 um 00:00 schrieb Adrian Bunk:

Yes, I think that when dh_golang is rewritten to make use of the go.mod
files from upstream, the patch should be dropped.

However:

I did some more experiments in the last days, to make sense of how the
golang compiler determines the compatibility settings. When a main
module is compiled (resulting in an executable), the language features
of the golang version specified in that module's go.mod file are used.

So in your example with secsipidx, indeed the compiler will use
compatibility settings for golang 1.16, therefore enabling lots of
insecure flags.

I'm not sure how to avoid this. Maybe the line with "go 1.16" in the
go.mod file would need to be patched by dh_golang to "1.24" or higher.

Ok, I see. You're right, this approach doesn't need a patch for the go
compiler. However, it's not trivial to come up with a list of needed
flags for the GODEBUG environment variable. Go has introduced several
flags in the last years, some of which have been removed again in the
meantime. Also, every time a new go compiler version get uploaded (1.27
etc.), this list needs to be revisited.

I would be in favor of patching the golang compiler for now, until the
Debian golang ecosystem has made the switch to using go.mod files. This
way, we get all flags which are currently applicable with a single
setting for the golang compatibility settings.

After rewriting dh_golang to make use of the go.mod files, the building
of golang packages within Debian should then be very similar to the way
other distributions build their golang packages.

Okay. I didn't look at all ~3000 golang packages, obviously. But I doubt
that Build-Using is in wide use. Why would the generated list in
${misc:Static-Built-Using} not be enough to determine which packages
need a rebuild?

Regards,
Tobias