Dear Policymakers,

With the increasing amount of programs in Debian that Build-Depend and
statically link with Golang and Rust libraries, it's important that the
Debian Policy clearly sets out the requirements for declaring these
statically-linked libraries.

Currently, section 7.8 of the policy is a bit vague regarding stating static
libraries. It begins with saying:

  Some binary packages incorporate parts of other packages when built but do
  not have to depend on those packages. Examples include linking with static
  libraries or incorporating source code from another package during the build.

It then states:

  [The Built-Using field] should be used only when there are license or DFSG
  requirements to retain the referenced source packages. It should not be added
  solely as a way to locate packages that need to be rebuilt against newer
  versions of their build dependencies.

This makes it seem that Built-Using shouldn't be used to declare
statically-linked dependencies, though that is not the case[1].

In early 2022, Guillem added support for a new Static-Built-Using field to
dpkg, encouraging packagers to use it over Built-Using to specify
statically-linked dependencies [2]. The commit message states the following:

  This field mimics the previous Built-Using field semantics, but is
  specifically intended for shadow dependencies stemming from static builds.

  In Debian, the Rust and Go teams agreed to use this language agnostic field,
  instead of one for each of the languages. This means it can be easily
  supported by dpkg, and can be used by other languages and run-times.

This was also added to the deb-control(5) manpage, specifically differentiating
it from Built-Using as a field to be used "for static building purposes (for
example linking against static libraries, builds for source-centered languages
such as Go or Rust[...])".

The proposed change is to clarify and formally mandate the requirement to
declare statically-linked libraries used to build packages containing binaries
in Static-Built-Using. Attached is a draft patch of the proposed change.
Feedback is welcome!

Kind regards,
Maytham

[1]: The Go team requires that Built-Using is specified for packages containing
     binary programs, and this is evident in the templates outputted by
     dh-make-golang. (This is being migrated over to Static-Built-Using, the
     correct field for this purpose.)

     Similarly, the Rust team also requires that
     Static-Built-Using are specified, as evident in debcargo's output when
     generating d/control files. (X-Cargo-Built-Using is currently being used
     but will soon be replaced by Static-Built-Using with the next upload.)

     The Rust team also uses Built-Using when required, where dh-cargo/cargo
     will check for and collate a list of any packages with copyleft licenses
     in the dependency tree that require accompanying source.

[2]: https://git.dpkg.org/cgit/dpkg/dpkg.git/commit/?id=16c412439c5eac5f32930946df9006dfc13efc02
https://lists.debian.org/debian-devel-changes/2022/03/msg03007.html

Hello Go and Rust packagers,

I think Maytham is right that updating Policy for this is a priority.
It has been bothering me that misunderstandings of Built-Using have been
proliferating somewhat among newer contributors.  See also #999785.

Could you take a look at his proposed changes to Policy in #1069256,
please, and confirm they successfully reconcile Debian Policy with your
team policies?

I think that we should also include an explanation of why we have chosen
to do this using a new field in d/control like Static-Built-Using.
Policy is meant to be a record of our lessons learned in building a
distribution, and the lessons learned in trying to handle these new
hyper-statically linked language ecosystems seem important.

My immediate question is why the Haskell and OCaml team's approaches,
were not copied.  They seem to work well and don't require a new field.
That seems worth writing down.

Thank you Maytham for your work so far.

Maytham Alsudany <maytha8thedev@gmail.com> writes:

I think there is a gap between "statically linked libraries" and "builds
for source-centered languages" -- could it be clarified if
Static-Built-Using is intended to cover situations where package A
incorporate source code from package B and source code from B affects
whatever goes into the binary package of A?  That is definitely true for
statically linked libraries.

I'm thinking of how gnulib C code is included in many packages, which
could be set up to work in a way similar to how Go packages work.  I
just made 'libntlm' use the 'gnulib' package this way, to reduce
xz-related risks with vendored gnulib code.  Should libntlm's
debian/control now include a 'Static-Built-Using: gnulib'?

/Simon

IMO this should not only state when it is to be used, but also what
it is used for and by whom. IOW who is the intended receiver. What
will they do with the info provided in this field?

Should this also include non-libraries that were included?
publicsuffix is a package providing *data* that is often translated
into binary data during package builds and thus included. But it is
not a "library" in the traditional way.

Chris

Hi!

I think this (and the policy proposal) is omitting several important
parts for the intended use for the field, as it was considered on its
addition. Quoting from deb-control(5) for context, which I think
covers the concerns from the mails from Simon and Chris:

   Built-Using: <package-list>
       This dependency field lists extra source packages that were used
       during the build of this binary package, for license compliance
       purposes.  This is an indication to the archive maintenance
       software that these extra source packages must be kept whilst this
       binary package is maintained.  This field must be a comma-separated
       list of source package names with strict ‘=’ version relationships
       enclosed within parenthesis.  Note that the archive maintenance
       software is likely to refuse to accept an upload which declares a
       Built-Using relationship which cannot be satisfied within the
       archive.

   Static-Built-Using: <package-list>
       This dependency field lists extra source packages that were used
       during the build of this binary package, for static building
       purposes (for example linking against static libraries, builds for
       source-centered languages such as Go or Rust, usage of header-only
       C/C++ libraries, injecting data blobs into code, etc.).  This is
       useful to track whether this package might need to be rebuilt when
       source packages listed here have been updated, for example due to
       security updates.  This field must be a comma-separated list of
       source package names with strict ‘=’ version relationships enclosed
       within parenthesis.

       Supported since dpkg 1.21.3.

Thanks for starting this!

As mentioned before I think the currently proposed description is too
restrictive and specific to Go and Rust, and it should be expanded. The
example usage seems also too specific, and might lead people to think
this is the only valid use, and their placement feels a bit odd, perhaps
these should be given as a footnote? (But this is really for the Policy
editors to decide.)

Thanks,
Guillem

Speaking for the Rust side - I'd say they match our expected usage of
the field. A slight rephrasing of "linked to libraries in other
packages" might match it even better, since in Rust's case, we are
usually talking about linking to libraries compiled from sources shipped
in other packages (librust-X-dev only contains the sources and possibly
helper scripts needed to build them, but not the compiled library).

But it also covers static linking of native libraries, so the current
phrasing matches that. Might be bikeshedding/nitpicky though ;)

I am not sure about the details of their approach... are those available
somewhere?

Thanks to both of you! :)

Regarding ;-

"(for example linking against static libraries, builds for
  source-centered languages such as Go or Rust, usage of header-only
  C/C++ libraries, injecting data blobs into code, etc.)"

Perhaps Pascal & Lazarus could be added to that list for clarity? [1]


Regards,
Peter

[1]  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=997948

Hi everyone,

Thanks for your input and suggestions. I've attached an updated patch with
several changes, including improving making the description of the field more
specific, adding another example that is not Go/Rust related, and improving the
Rust example to show the simultaneous use of Static-Built-Using and Built-Using.

I would greatly appreciate any more feedback for this new patch. If you believe
that it is complete (and you are a DD), it would be very helpful if you could
second this consensus and proposal.

The relevant part of the new patch has been included at the bottom of this
email.

Yes, that is the intention, and the updated description should reflect that.

I believe so. There are a lot of packages that would need cleaning up after
this proposal is implemented.

The updated description should be better now, thanks for your feedback.

I've expanded the example usage by also adding a sans-Go/Rust example. I've
placed the Static-Built-Using examples in the same way as the examples for the
Built-Using field.

On Tue, 23 Apr 2024 at 21:02 +0200, Fabian Grünbichler <debian@fabian.gruenbichler.email> wrote:

Updated phrasing should address this.

I am also unsure about this, and wonder whether this is necessary since Haskell
and OCaml are completely different languages with their own approaches.

I'm unfamiliar with how the buildsystems for these languages work, so I would
appreciate it if you could suggest a specific phrase to add to the list of
scenarios. Even better if you could come up with an example to add at the bottom
:)

Below is the relevant part of the updated patch, to save you from downloading
the attachment:

``Static-Built-Using``
~~~~~~~~~~~~~~~~~~~~~~

This ``Static-Built-Using`` field must list source packages who's
contents (like source code or data) were incorporated into the binary
package during the build, including an "exactly equal" ("=") version
relation on the version that was used to build that version of the
incorporating binary package.

Cases where this field may be used include (but are not limited to)
linking against static libraries in other packages, builds for
source-centered languages such as Go and Rust, usage of header-only
C/C++ libraries and injecting data blobs into code.

This is useful to track whether the package might need to be rebuilt
when source packages listed here have been updated. This is important
to stay ahead of the package failing to build from source (FTBFS) with
the updated versions of the listed source packages, or security
updates in the listed source packages.

Unlike Built-Using, the Debian archive will **not** retain the
versions of the source packages listed in the Static-Built-Using
field. This means that any package listed in Static-Built-Using who's
license requires its source code to be available must also
simultaneously be listed in the Built-Using field.

A package that needs domain name suffix data from the publicsuffix
binary package would list it in the ``Static-Built-Using`` field like
so:

::

    Static-Built-Using: publicsuffix (= 20231001.0357-0.1)

A package statically linked with a library from the
golang-github-mattn-go-xmpp-dev binary package would have this field
in its control file:

::

    Static-Built-Using: golang-github-mattn-go-xmpp (= 0.2.0-1)

A package statically linked with the libraries contained in the
librust-gtk4-dev and librust-pulsectl-rs-dev binary packages, where
the latter is licensed under GPL-3+ (a license that requires full
source code to be available), would have these fields in its control
file:

::

    Built-Using: rust-pulsectl-rs (= 0.3.2-1+b1)
    Static-Built-Using: rust-gtk4 (= 0.7.3-3), rust-pulsectl-rs (= 0.3.2-1+b1)

[..]

LGTM, consider this Seconded :) (even though it is not currently tagged as
"Wording Proposed ;))

Fabian

Hi all,

I would greatly appreciate it if you could have a look at the proposal attached
below to add a new section to the Debian Policy detailing the use of the Static-
Built-Using and differentiating it from the Built-Using field.

Could you please ensure that it aligns with your goals in the Golang/Rust teams
and that it matches your expectations? If there is anything that is surprising
to you or something that doesn't make sense please let me know.

Otherwise, I encourage you to send a signed statement endorsing the patch as-is.

The bug log and full patch file can be found at https://bugs.debian.org/1069256
(Note that the patch has been revised, the patch attached to the first message
is not the one currently being proposed.)

Thanks,
Maytham

PS: I'm around on IRC in #debian-(rust|golang|devel) if you want to discuss.
[..]

Hi,

Ping for further feedback or seconds for proposed policy change to
clarify and document the use of the Static-Built-Using field.
[..]

On Thu, 20 Jun 2024 13:40:11 +0800 Maytham Alsudany  <maytha8thedev@gmail.com> wrote:
 > Hi,
 >
 > Ping for further feedback or seconds for proposed policy change to
 > clarify and document the use of the Static-Built-Using field.


Hi Maytham,


thanks for your work. LGTM too, consider this seconded.


best,


werdahias

Hi,

Ping for further feedback or seconds for proposed policy change to
clarify and document the use of the Static-Built-Using field.
[..]

Hi Maytham,

could also mention that this field would be useful for fpc & lazarus
packages.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=997948


Regards,
Peter

Any DDs on the Pascal team able and willing to second this?

-------- Forwarded Message --------
Subject: 	Re: Bug#1069256: debian-policy: clarify requirement for use of
Static-Built-Using
Resent-Date: 	Sun, 14 Jul 2024 16:11:11 +0000 (UTC)
Resent-From: 	debian-devel@lists.debian.org
Date: 	Sun, 14 Jul 2024 17:03:19 +0100
From: 	Peter B <peter@pblackman.plus.com>
Reply-To: 	peter@pblackman.plus.com
To: 	1069256@bugs.debian.org
CC: 	debian-devel@lists.debian.org, Maytham Alsudany
<maytha8thedev@gmail.com>

Hi Maytham,

could also mention that this field would be useful for fpc & lazarus
packages.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=997948


Regards,
Peter

Seconded.

Thanks!

Chris

Hi!

Thanks for the update!

I'm not sure “who” would be idiomatic to refer to packages contents?

I find the placement/phrasing after the “, including …” a bit
confusing, because I read it as a continuation from the “were
incorporated into the binary package during build“, instead of
”must list source packages”. It is also perhaps not clear that the
exact version restriction is mandatory.

Same comment about “who“.

Perhaps reword these or preface with some text to make them more clear
they are just some examples for how the field could be used? I mean, I
guess this is implicit with the “would“, but perhaps making this
explicit is preferable in a document like this?

(I'm also always a bit conflicted with examples that are based on real
current package data, because while this are then extremely clear right
now, they can quickly become obsolete or seem stale or odd after some
time has passed. :/ But I'm not sure what would be an alternative, and
I think this is something for the policy editors to weight in if at all.)

Beside these comments, which to me seem more on the editorial side,
I'm now happy with the essence of the text to second it, once it has
seen some wordsmithing. :)

Thanks,
Guillem

With this interpretation of the relationship between the two fields,
is there any situation where a source package would validly appear in
Built-Using but not in Static-Built-Using? (Other than the obvious
"this package has not yet been updated to implement Static-Built-Using")

Thanks,
    smcv

Hi,

Most of the problems with wording have been fixed, except for the one I
note below.

The wording used in the examples and their nature is based on how they have
been written in the Built-Using section already in the policy, which I quote below:

It would be beneficial for the policy editors to weigh in on this.

Please review the revised patch that is attached when you can.

Thanks!

I don't believe so (at least not in any scenario I can think of).


Thanks,

Hello,

I think that Guillem makes a good point.  If there's no alternative,
though, an example that might get out-of-date is probably better than no
example.

Hi Peter,
Yes I do.

We will recommend using this new field and
close https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=997948 as it becomes now
the responsibility of packages maintainers to add it to their packages.

Maytham Alsudany <maytha8thedev@gmail.com> writes:
Static-Built-Using and I find the above explanation ambiguous wrt if the
transitive closure of packages should be included or not.

(A side-note is that referring to source package names in
'Static-Built-Using' makes it hard to know which binary package from
that source package actually ship the embedded code.  I suspect it is
too late to change this though.)

My naive reading of the text above suggests that ALL source packages
with that property ought to be included.  I'm not certain that is a good
idea or even if this is intended.

My situation is that 'tkey-ssh-agent' embeds a RISCV firmware app blob
which comes from the 'tillitis-tkey-device-signer' package which embeds
RISCV object code from the 'tillitis-tkey-libs' package.

In an upcoming upload, I plan to add

Static-Built-Using: tillitis-tkey-libs (= 0.1.2-2)

to the 'tillitis-tkey-device-signer' package.

However, what is the right header for the 'tkey-ssh-agent' package?
Naively interpreted, I think the above policy would suggest this:

Static-Built-Using: tillitis-tkey-libs (= 0.1.2-2), tillitis-tkey-device-signer (= 0.1.2-2)

However is that useful?  Wouldn't the following be sufficient:

Static-Built-Using: tillitis-tkey-device-signer (= 0.1.2-2)

Code that goes looking at the 'tillitis-tkey-device-signer' source
package ought to be able to find that one of its binary packages, namely
'tillitis-tkey-device-signer', has a Static-Built-Using on another
package.

The problem with interpreting the text above as indicating the
transitive closure is that there may be no end to the list.  For
example, 'tillitis-tkey-libs' most likely embeds static code coming from
the compiler, in the 'clang-19' package (or some other dependent package
of 'clang-19').  Should 'clang-19' be added to 'Static-Built-Using'?
Recursively, I suspect that the 'clang-19' binary package in Debian of
version X either embeds static code coming from gcc version Y or clang
version X-1.  Should 'clang-19' have a Static-Built-Using on its
compiler?  I'm not sure this is useful or intended.

Maybe adding an example for embedded static C object code like this
would help clarify the intention.

/Simon

Hi Simon,

On Mon, 2025-02-03 at 13:32 +0100, Simon Josefsson wrote:
[...]

Would adding the following after the first paragraph be sufficient in
your view?

  Note that you do not need to list source packages recursively. For example,
  consider the following situation: A package named "foo" declares
  Static-Built-Using on "bar". "bar" declares Static-Built-Using on "baz". As
  long as "foo" does not directly incorporate the contents of the "baz" package
  during its build, then "foo" must not list "baz" in its Static-Built-Using
  field.

Maytham Alsudany <maytham@debian.org> writes:

Thanks also for agreeing that this is the reasonable interpretation for
this situation, I wasn't 100% certain on that.

I believe the phrase

   "does not directly incorporate the contents"

still leaves room for ambiguity though.  Certaintly to some way of
reading "foo" WILL incorporate the particular bits coming from "baz"
directly, since those bits are provided identically within the "bar"
package.

Can we find some better articulation of the concept?

If package "bar" has "Static-Built-Using: baz", and could even have a
"Depends: baz" so that "baz" is actually installed when "bar" is
installed, I believe what we want is that a package "foo" that
Build-Depends on "bar" and statically uses files from "bar" to have
"Static-Build-Using: bar" when it refers ONLY to files in "bar".  If
"foo" also refers to and uses files provided by "baz" then it MUST also
have "Static-Build-Using: bar, baz".

I think the property we care about for distinguishing the recursive case
is more about the exposed interface (normally filenames) rather than the
actual content of the files.  The content WILL be included recursively,
that's the point of statically linking.  But mirroring that in
Static-Built-Using is not useful, as it will chain back to include all
compiler versions used to build them (and potentially the compilers that
built those, etc).  So we only mirror the filename interfaces needed.

I hope someone else can find better terminology and description of the
desired behaviour.

I'm also find with using your improved explanation as-is, and leave my
comment describing this detail in the bug tracker for future word
tweaks, in case there is ever any real problem due to this ambiguity.

/Simon

Hi Simon,

Perhaps a footnote on what "directly incorporating" means?

  Note that you do not need to list source packages recursively. For example,
  consider the following situation: A package named "foo" declares
  Static-Built-Using on "bar". "bar" declares Static-Built-Using on "baz". As
  long as "foo" does not directly incorporate the contents [#]_ of the "baz"
  package during its build, then "foo" must not list "baz" in its
  Static-Built-Using field.

  [...]

  .. [#]
     A good indication is whether "baz" needs to be listed the Build-Depends of
     "foo". In other words, if upstream's buildsystem in "foo" explicitly looks
     for "baz" or its source code imports something provided by the "baz", then
     "baz" should be listed in the Static-Built-Using field of "foo".

Hi Maytham,
thanks for your work.
Consider this seconded by me, too.

best,

werdahias

Hi!

There seems to be an issue with this patch, and potentially with the
whole Static-Built-Using field syntax/semantics:

Note that these fields refer to version 0.3.2-1+b1, which is a binNMU
version, not a source package version (though it may be the source
package name).  So this would presumably not be acceptable.  But as
the primary purpose of the Static-Built-Using field is to record the
versions of packages used during the build, surely we should be using
*binary* packages and versions in this field, not the *source*
packages and versions?

Here's where this would be relevant:

Binary packages pkg1, pkg2, pkg3 are built from pkg1-source,
pkg2-source, pkg3-source respectively.

pkg1 statically depends on pkg2
pkg2 statically depends on pkg3

pkg1 is at version 0.1.0-1, Static-Built-Using: pkg2-source (= 0.2.0-1)
pkg2 is at version 0.2.0-1, Static-Built-Using: pkg3-source (= 0.3.0-1)
pkg3 is at version 0.3.0-1

pkg3-source is then updated to version 0.3.1-1

This requires pkg2 to be rebuilt against the new version, and a binNMU
is adequate for this purpose, so we get pkg2 version 0.2.0-1+b1, but
we still have pkg2-source version 0.2.0-1.

But now a rebuild of pkg1 is not triggered, even though that may well
be required: pkg2-source is still at version 0.2.0-1, which pkg1 was
built against, but pkg2 has been updated.  (And in the case of libjs-*
packages, I believe that this rebuild is potentially needed: pkg1 may
well incorporate bundled code from pkg2, which in turn incorporates
code from pkg3.)

The only way to get round this in the current setup would be to do a
source upload of pkg2-source with a new version number 0.2.0-2, and
that somewhat defeats the idea of being able to do updates in an
easily automated fashion.


I would therefore propose changing the Static-Built-Using field to use
*binary* packages and versions rather than *source* packages and
versions to fix this.

At present, there are only about 250 packages in testing that use the
Static-Built-Using field, so the impact would be manageable.


But there may be a really good reason to use source package versions
instead; it would be helpful to hear those.

Best wishes,

   Julian

[..]

Thanks for noticing. At least I've overlooked this in the past, but
I always understood S-B-U to list binary package names and binary
package versions.

Let's see what everyone else thinks, but I think dealing here with source
package names is not a good idea.

Chris

Hi Chris!

I'm glad I'm not going mad ;-)

To quote the current deb-control(5):

       Static-Built-Using: package-list
           This  dependency  field  lists  extra source packages that were used
           during the  build  of  this  binary  package,  for  static  building
           purposes  [...]
           This field must  be  a  comma-separated  list  of
           source  package names with strict ‘=’ version relationships enclosed
           within parenthesis.

So that would need changing too.

I don't know whether any tools are yet using this field, but if they
are, they would need to be updated similarly.

Best wishes,

   Julian

I think for rust-*, it should make no practical difference, except for corner
cases where binary packages/versions would be more correct anyhow.

The tooling  emitting this (and possibly consuming it already?) would
need to be updated of course..

Indeed.

Looking at the packages listed in Static-Built-Using (SBU) fields in
Debian testing main, we find the following:

* 2215 distinct packages are listed in SBU fields.
* 38 of these are also the name of a binary package
* 19 of these 38 have a different Source package version number
* Of the remaining source packages:
  - 742 are called golang-* (excluding golang-1.18 to golang-1.24)
  - 1409 are called rust-*
  Modifying the tooling in these two ecosystems would presumably solve
  all or almost all of the affected packages.
* This leaves 21 other packages listed (excluding golang-1.*) that
  would need modifications to their "users"; there are 43 distinct
  "users" in total:

    castle-game-engine
    Package: castle-model-viewer

    continuity
    Package: distrobuilder
    Package: docker-buildx
    Package: docker-compose

    etcd
    Package: minio-client

    gobgp
    Package: incus
    Package: incus-agent
    Package: incus-base
    Package: incus-client
    Package: incus-extra
    Package: lxd
    Package: lxd-agent
    Package: lxd-client
    Package: lxd-migrate
    Package: lxd-tools

    go-containerregistry
    Package: cosign
    Package: gitsign
    Package: gittuf
    Package: rekor
    Package: sigstore-go

    go-md2man-v2
    Package: dasel
    Package: didder
    Package: distrobuilder
    Package: docker-buildx
    Package: docker-compose
    Package: etcd-client
    Package: etcd-server
    Package: gh
    Package: gitsign
    Package: glab
    Package: lego
    Package: hugo
    Package: incus
    Package: incus-agent
    Package: incus-base
    Package: incus-client
    Package: incus-extra
    Package: lxd
    Package: lxd-agent
    Package: lxd-client
    Package: lxd-migrate
    Package: lxd-tools
    Package: rclone
    Package: syncthing
    Package: syncthing-discosrv
    Package: syncthing-relaysrv
    Package: victoria-metrics

    gopacket
    Package: coredhcp-client
    Package: coredhcp-server
    Package: incus
    Package: incus-agent
    Package: incus-base
    Package: incus-client
    Package: incus-extra
    Package: lxd
    Package: lxd-agent
    Package: lxd-client
    Package: lxd-migrate
    Package: lxd-tools

    klibc
    Package: mksh

    nanosvg
    Package: fuzzel

    relic
    Package: cosign
    Package: gitsign
    Package: gittuf
    Package: rekor
    Package: sigstore-go

    tree-sitter
    Package: sdml
    Package: turtlefmt

    tree-sitter-c
    Package: neovim

    tree-sitter-lua
    Package: neovim

    tree-sitter-markdown
    Package: neovim

    tree-sitter-query
    Package: neovim

    tree-sitter-sdml
    Package: sdml

    tree-sitter-vim
    Package: neovim

    tree-sitter-vimdoc
    Package: neovim

    trillian
    Package: gitsign
    Package: gittuf
    Package: rekor

    universal-detector
    Package: unar

    zlib
    Package: gpgv-static
    Package: sash


Best wishes,

   Julian

Hi!

[…]

To give some context, the new field came about when concerns were
raised about reuse of Built-Using for the non-license cases. So the
field naturally inherited the previous semantics, just in a different
field. (But I'm not saying this to argue that this is thus correct. :)

As has been mentioned elsethread I think currently the main users of
the field involve source based binary packages (similar in nature to
C/C++ header-only -dev packages), where I don't think this would make
much of a practical difference (because pkg1 would list both pkg2-source
and pkg3-source in its Static-Built-Using).

For the static library cases (libfoo.a), I'm not sure this might make
(in general) much of a difference either. As I'd expect static library
packages to only include objects generated from the same source
package (and not from all transitive dependencies), and where the final
executable or shared library (if everything else has been built with
PIC/PIE), then embeds all the static libraries (and where the package
should also list the equivalent of all source packages that provide
libraries from say «pkgconf --static --libs libfoo»).

_But_, in contrast to the license-relevant Built-Using field, where the
important part is the source package (to be kept around for compliance
reasons), if we do really have packages that are to be embedded by
others and that are currently embedding (or might do so in the future),
other transitive static content, then this would be a problem, indeed.
And where using binary names and versions would be better, while this
would not really affect/disrupt every other case listed above.

I'm also thinking that while in general the most important part is going
to be explicit source changes (in a new package revision), that cascade
into whoever ends up embedding that code, there can also be generated
source changes (via tools that generate source from source, f.ex. yacc),
or binary object changes (via tools that generate objects from source,
f.ex. gcc). Where the former would be closer to the scenario that
Julian was presenting, where a package that ends up with compiled
output from yacc in its libfoo.a could declare itself
«Static-Build-Using: yacc (= ...)», but then it would not make much
sense to transitively add that into all users of libfoo.a. But for the
latter I don't think this is a problem, because this would arguably
affect all/most/many binary packages in the archive, including those
not statically linking/embedding, so we'd need some other way to track
what to rebuild anyway (?).

Although we have some tooling generating the fields (mainly in the Go
and Rust teams), I don't think there's currently tooling in Debian
consuming the fields yet. But then also, the fields have been documented
with those semantics since dpkg 1.21.3 (as present in Debian bookworm),
so there could be local reliance on those. :/


If there is consensus that we'd want the more precise (although in
many cases possibly irrelevant) information tracking, I'd be fine
with the changed semantics, and I'd be happy to update the
deb-control(5) man pages. Taking into account that while it feels
early enough in Debian terms (given that we currently have no
consumers that I'm aware of), this has the unknown of potential local
users.

Thanks,
Guillem

[...]

Hi Guillem!

Thanks for this really thoughtful and helpful email.

Yes, indeed; that was obvious!  :-)

OK.  But I can see the JavaScript team moving in this direction over
the forky development cycle, and there it would make a difference.

In this case, I presume that the binary libfoo.a would not list any
Static-Built-Using packages, as these would not be relevant; there
would not be anything embedded in libfoo.a that would require libfoo.a
to be rebuilt when they are updated.

Agreed.
presume that an upgrade to yacc is not likely to change the behaviour
of packages that use it during the build, and the yacc code itself is
not embedded into the resulting binary package (as opposed to the code
generated by running yacc).  If we're interested in knowing long-term
which versions of packages were used to build a specific binary
package, that's a separate issue, and we could perhaps start keeping
the buildinfo files in the archive rather than throwing them away.

Yes; let's say that a critical bug/exploit was discovered in yacc.  We
would then need to rebuild all packages that Build-Depends on yacc,
and those are easy to identify.

That's good to know.  I also doubt there's much local reliance on
those, given how little they've been used within the main archive, and
I expect that anyone who is relying on them locally will be competent
enough to cope with this proposed (breaking) change.  I would also
expect that any local users relying on this field would appreciate the
benefit of the changes proposed.

If there is consensus on this, then given what you've pointed out
above, we should probably tighten up the wording of the proposal to
indicate that Static-Built-Using should only list packages whose
upgrading should trigger a rebuild of the package, for example
packages whose content is embedded in the resulting binary package.
But the compiler used should not generally be included in this field.

Best wishes,

   Julian

CCing Stefan, who currently schedules binNMUs based on Built-Using and
Extra-Source-Only, IIRC. not sure whether RT already acts on S-B-U
anywhere as well, in addition to that?

note that for Rust, this is for the most part not true - except for
niche use cases (nostd, like when building embedded things or Linux
kernel stuff), the standard library which is part of the toolchain
packages *is* statically linked into any Rust executable. and even for
nostd, the same applies to a smaller standard library (libcore).

so while Static-Built-using rustc could go away following this line of
reasoning (which I do agree with - grave compiler codegen bugs warrant
rebuilding the world anyway), libstd-rust-dev would remain.

Dear Fabian,

Thanks!  I don't know about Extra-Source-Only.

Ah, interesting.  Would one want to/need to automatically rebuild
every Rust package every time libstd-rust-dev is updated?  (I don't
know Rust anywhere near well enough to know the answer to this
question.)

Best wishes,

   Julian

yes, that is already happening in unstable (either naturally by incoming
uploads, or with a slight delay via binNMUs scheduled by the release
team/Sebastian :)).

in unstable/main, there's currently three versions of src:rustc kept
around:

1.70.0+dfsg1-9 (not sure why?)
1.85.0+dfsg2-3 (proton-caller in contrib which was not yet rebuilt
it seems)
1.85.0+dfsg3-1 (the current one)

proton-caller on riscv64 has a Built-Using rustc (= 1.70+dfsg1-9).

Cheers

Not yet. We could use it to rebuild packages that are affected by a
security issue in, say, a rust crate that is referenced via
Static-Built-Usig. We never had the use case so far, though.

Cheers

Hi!

I've been thinking a bit more about this field, and wondered if we
could tighten up its purpose?  The current wording in deb-control
says:

  This is useful to track whether this package might need to be
  rebuilt when source packages listed here have been updated, for
  example due to security updates.

Could we say instead something like this?

  When packages listed here are updated, this package should also be
  updated, for example because of changed functionality potentially
  impacting the package behaviour, or for security updates.

Then this opens the door to potentially using it for automatic
rebuilding, rather than just a convenient place to list packages that
were used during the build process.  (The buildinfo files already do
that, though I don't believe we currently retain them, which is a
shame.)

Best wishes,

   Julian

Hi!

See below <js>.

That would be my expectation, yes.

<js>

I've been thinking about the case of packages embedding their
transitive dependencies, and that seems like something we might not
want to support anyway? It feels like it can/will create nightmarish
support scenarios.

Perhaps I did not understand correctly how the direction the JavaScript
team is planning on taking, but if it implies that say:

  app-js-z depends on libjs-a and embeds its contents
  libjs-a depends on libjs-b and libjs-c and embeds their contents
  libjs-b depends on libjs-d and libjs-e and embeds their contents
  libjs-c depends on libjs-f and libjs-g and embeds their contents

Then both app-js-z and libjs-a would end up carrying the contents for
libjs-[a-g]. And when libjs-g or libjs-e get updated the whole
dependency tree (not only the leaf packages, such as app-js-z) would
need to be rebuilt! And done in the correct order, otherwise you might
end up with outdated embedded copies.

This would be compounded with libjs packages being arch:all, which
means (currently) normal binNMUs cannot be triggered, and would require
a mass sourceful transition!

In cases of security issues, this also seems like a bad place to be.

So, given that this seemed like the only case where this was initially
a concern (?), and given the above, I'm not sure we might even want to
support this, and thus the current semantics are fine?

(But perhaps I'm missing something or dismissing an important aspect
of this. Perhaps the JavaScript team instead is going into embedding
all transitive libjs-* dependencies in leaf app packages, if so, then
that looks like a similar scenario to what we currently do with C/C++
static linking?)

Ah, right, sorry, this is indeed probably still like a normal build tool
case than embedding actual sources from the tool itself.

Right.

Probably a good idea yes.

Thanks,
Guillem

Hi!

I guess my concern with at least turning this into a "should" is that
it feels it would potential imply a burden to whoever might be on the
receiving end of this, where I don't think we are currently ready to
honor that commitment (due to missing tooling, processes, etc).

I think the rest of the clarification looks good, and I'd have no
problem with incorporating that in into the man page, let's say
perhaps for now with this?

  This is useful to track whether this package might need to be
  rebuilt when source packages listed here have been updated, for
  example because of changed functionality potentially impacting
  the package behavior, or for security updates.

And we can apply the "should" part later on, once/if we are ready
to handle that.

Thanks,
Guillem

This sounds like a very reasonable, non-committal description.  Nice
wording!  The only change I'd make is "source packages" -> "packages",
as we're suggesting recording the binary package used in the build,
not the source package.  So the resulting paragraph would read:

  This is useful to track whether this package might need to be
  rebuilt when packages listed here have been updated, for
  example because of changed functionality potentially impacting
  the package behavior, or for security updates.

Best wishes,

   Julian

Hi!

I should say first that I am not a core member of the JavaScript team,
nor am I even an expert in JavaScript - only a mere novice.  But some
of the Python packages I am involved in have significant JavaScript
dependencies, so I have been involved with the JS team over the last
year or so.  My coments above are based on my reading of the
situation, though I am not speaking on behalf of the team.

That might be the case, but having binaries not working because their
dependencies are not in sync is also pretty bad.

That would seem to me to be the correct course of action.

Indeed, and that is the current state of affairs.  It would be ideal
of binNMUs could be performed for arch:all packages, and I can imagine
other situations where having this facility might be useful.  I
imagine that it would not be too difficult to modify the appropriate
scripts to support such binNMUs if it is decided that they are useful.

I'm not sure how it would be worse than any other setup.  For example,
if every package that required libjs-f embedded a copy of it in the
source code, then resolving a security issue in libjs-f would mean
identifying every occurrence of it in the sources in the whole
archive.  Or if, instead, we were to insist that app-js-z explicitly
depend on all of its recursive libjs-* dependencies, one would still
need to rebuild them all when building app-js-z and also keep the
recursive dependencies up to date.  With increasingly complex and deep
dependency trees, that becomes unmanageable.

The current semantics don't allow for the potential for this future
development, and don't seem to offer any other advantage over the
proposed change to listing the binary packages used (except for not
changing the current semantics, and given how few packages outside of
Rust and Go currently use this field, I don't think that's a
significant problem).

I'm sorry, I don't understand this - please could you elaborate?  How
does C/C++ static linking work in Debian?  (And please forgive my
ignorance!)

Best wishes,

   Julian