Hi

When using `dgit` on a Debian package where there is a `.gitattributes`
file, I get the warning:

"""
dgit: .gitattributes not (fully) defused.  Recommended: dgit setup-new-tree.
"""

For me, this is a "Debian-ism" or a "delta to how everyone else uses
git"[1] and therefore I was hoping we could work towards getting rid of
this workflow delta.

I see this delta as a problem because people wanting to use `dgit` have
to change how they use git compared to everyone else. If I am upstream,
I can use `.gitattributes` to ensure `git` enforces certain properties
for me.
   Concretely, I can have `git` enforce that I do not get Windows
newlines into my scripts just because the contributor wrote the patch on
a Windows machine in a less than ideal configured editor[2].

In a standard git workflow, I can edit the `.gitattributes` as needed
and push it. From there on, branches based on that commit will now
respect the delta. This is a nice property for me as an (upstream)
developer.

If I am *also* a Debian packager for the same git repo, I now have to
run a special magic `dgit` rune to "defuse" my `.gitattributes`. Based
on my understanding, I would have to perform this step every time
`.gitattributes` is updated. And this solution does not follow the
standard branch logic, so I can now observe different behavior on a
"defused" check compared to a vanilla one when I change branches.

I think that is an anti-feature for reducing the "Debian specific git
workflow". I am not blind to the possibility that you probably consider
`.gitattributes` an anti-feature for `dgit` given the write up you made
for it.
   Nevertheless, I think `dgit` should change its behavior here, since
we are making a Debian specific git workflow and it makes Debian
contributors that are also upstream developers a second class citizen.

Best regards,
Niels

PS: I do not claim to have answers to the problems at hand nor do I
expect it to be easy. :)

[1]: https://lists.debian.org/debian-devel/2024/08/msg00350.html

[2]: Technically, CI would be better at enforcing this case because that
is too much effort to write and test. The `.gitattributes` wins a lot on
being extreme lightweight on implementation given the Windows is a minor
annoyance in most cases.

I also do not seem to be the only one with this mindset:

https://dev.to/deadlybyte/please-add-gitattributes-to-your-git-repository-1jld

https://github.com/gitattributes/gitattributes

Niels Thykier writes ("Bug#1079434: dgit: Can we avoid Debian specific rules for gitattributes?"):

Did you read the section GITATTRIBUTES in dgit(7) ?

I can see how this is useful.  I think a better approach to this kind
of thing might be to use git's hook arrangements.

I think maybe you have misunderstood what `dgit setup-new-tree` does.
It does not manipulate the git tree object, or your branch.

It manipulates the per-tree *configuration* (.git/info/attributes) to
arrange that the in-tree .gitattributes don't cause discrepancies
between your working tree and the git history.  (Such discrepancies can
cause `dgit push-source` to fail.)

When I developed this aspect of dgit, I was thinking of upstreams
who put all manner of surprising things in .gitattributes.  For
example, upstream Xen git has a .gitattributes file which encodes
version information in working tree files.

This isn't compatible with dgit's core invariant, which is that the
git tree object is precisely the same as the content of the source
package.  (The alternative invariant would be that source package is
identical to content of the working tree *as transformed by
gitattributes* - but the gitattributes are typically
context-sensitive, lossy, and very complex, so that isn't workable.)

I agree that this whole situation is not optimal.  To be honest,
I think the whole gitattributes system in git is a mistake.
(See also git subtrees which are an even more badly broken thing[1]
that dgit doesn't support.)

But the situation is not as bad as I think you imagine.

If your gitattributes don't in fact transform files, in practice,
in a way that makes your source packages different from your git tree
object, then:

 * You can safely ignore the warning, since even un-defused,
   the attributes won't cause discrepancies that cause dgit to fail.

 * You can suppress the warning by providing a defuse line
   that affects no files (see `dgit setup-gitattributes` in dgit(1))
   (Possibly there could be a nicer way to do this.)

 * Users who heed dgit's advice (or use `dgit clone`) will not
   experience lossage either.  (Assuming they don't also apply
   patches with Windows line endings, or something.)

I'm open to suggestions for how this could all be better.  But the
situation is certainly not straightforward.

Ian.

[1] See my blog post "Never use git submodules"
https://diziet.dreamwidth.org/14666.html

Ian Jackson:

I did find it before filing this bug. Though I had to hop via the `dgit
setup-new-tree` -> `dgit setup-gitattributes` to find it, which I did
not find desirable in terms of documentation.

As for the section itself, I understand the gist of it but it was a bit
too abstract to my liking.

Whether it is `git`'s hooks, a CI job or a `pre-commit` hook is that it
is effort to write and test the code works as intended. Compare with
"echo '*.py eol=lf' > .gittaributes", which has almost no investment.

For me, the trade-off in simplicity strongly favors `.gitattributes`. My
goal (as an upstream developer) is to develop on the upstream project,
not write and maintain CI/pre-commit hooks. So I can spend 5s on the
`.gitattributes` and then go back to what I am here to solve (rather
than spend an hour creating and debugging a CI pipeline or pre-commit
hook, where the implementation might differ depending on which git
hosting platform/CI platform I use).

As I understand git, the `.git/info/attributes` is global across all
branches where as `.gitattributes` is subject to the which branch
(commit) you are on.

So are you saying this can "defuse" the `.gitattributes` without
affecting how they work in general? Because that is not at all what I
read out of `dgit(7)`.

What I read is that it will disable the "transforming" gitattributes
(not sure what that is, but rewriting newlines do sound "transforming"
to me). And if it disables a gitattribute in regular git, then there is
a delta in the behavior of git for "upstream-only-developer" vs.
"upstream-and-debian-developer", which is what I want to avoid.
https://salsa.debian.org/dgit-team/dgit/-/issues/7 to discuss build-time
generation (or enrichment) of `debian/control`, which I suspect feature
interacts with this invariant due to some other constraints.

I would like the dgit maintainer's feedback on that too, since I do not
like the status quo very much and I hope we can find a solution that
solves my goal that is also supported by `dgit` to the extent possible.

End side-bar.

It would be even better if the user did not get a warning when the file
will not cause problems. Though, given your remark below, newline
transformations seems to be in scope for the warning.

I think I would need an example for this in the manpage. I am not even
sure what the `dgit-defuse-attrs` macro does.

So, I assume the problem is that `dgit` and `dpkg-source` sees to
different things here during `dgit push-source`, so the generated
dpkg-source does not need the attribute and `dgit` in its commit does?

For a native package (where I am coming from but the smallest possible
user-base for you) or the `debian/` part of a non-native package, having
git normalize the newlines (the `eol` attribute) as requested before
calling dpkg-source should work.

But then, the majority of the `.gitattributes` that `dgit` will see are
from the upstream part of a non-native package, and that is considerably
more difficult to deal with. Since the `.orig`-tarball is actually the
source of truth (and not git). Would this happen to be correct by virtue
of `gbp import-orig` (or will a `.gitattributes` not affect that case)
for users of `gbp import-orig`?

I get this only covers the `eol` attribute (and then also the `text`
attribute due to their inter-relationship). But I think there is a lot
to be said for supporting that those without a warning if possible as I
think those are the most common one.

Perhaps also a special-case for merge drivers (like
`dpkg-mergechangelogs`), which should not affect the committed form.

I think my best bet is to cover the most common attributes and avoid
warnings for those when possible. At least as a starting point this
would be a partial solution that from a user perspective would be better
than the status quo if possible to do reliably. But this all boils down
to whether it is feasible to deal with the `.orig` where `git` is
not/might not be the source of truth.

It would make the warning less likely to occur for many projects meaning
fewer people in dual roles would need to worry about this problem.

Best regards,
Niels

This seems to be best discussed as part of #1079434,
so moving the discussion there:

Simon Josefsson writes ("Bug#1111548: tag2upload service makes .origs with un-defused gitattributes"):
...

Firstly, dgit has suppressed .gitiattributes since January 2017.  That
was eight and a half years ago, and 3 and a half years into dgit's
existence.  I made that change as soon as I encountered the problem -
in a package of which I was the maintainer.

As I write earlier in this bug:

  When I developed this aspect of dgit, I was thinking of upstreams
  who put all manner of surprising things in .gitattributes.  For
  example, upstream Xen git has a .gitattributes file which encodes
  version information in working tree files.

This is the use case you're discussing in your email.

  This isn't compatible with dgit's core invariant, which is that the
  git tree object is precisely the same as the content of the source
  package.  (The alternative invariant would be that source package is
  identical to content of the working tree *as transformed by
  gitattributes* - but the gitattributes are typically
  context-sensitive, lossy, and very complex, so that isn't workable.)

That there has to be such an invariant follows from dgit's design
goal, which is to be a bidirectional gateway.  A bidirectional gateway
is required for any kind of sane transition.

I feel I should re-explain some parts of the relationship between the
various git branches, source package imports, etc.

For an package whose maintainer git view is afflicted by
gitattributes, a tarballs-only upload with gbp and dput will apply the
gitattribute transformation.  Then, a dgit user will see a .dsc
import.

Obviously it would be better for the maintainer to use dgit
push, so that the dgit user gets real history.

I think the only sane way to achieve that, if the upstream can't be
persuaded not to use this broken git feature, is for the maintainer
git tree also to not expect gitattributes to be applied.  Probably the
best approach is for the maintainer to apply a patch which contains
(i) the effect of the gitattributes (ii) deletion of the
.gitattributes file, as I suggested in #1111548.

(I feel justified in saying "broken" because these .gitattributes can
cause all manner of weird behaviour.  For example, I think your
version-substituting package will contain different contents in its
working tree, when you check it out, depending on what git tags happen
to exist in your local ref namespace.  No reliable software can be
built on top of such shifting sands.)


In theory it might be possible to regard this is a
"maintainer-to-dgit-view" transformation.  We do have such
transformations.  But there are two significant objections to that:

Firstly, tree transformations are currently *only* for "3.0 (quilt)"
source packages.  I think this is quite a helpful rule.  "3.0 (quilt)"
is very odd, but the underlying task (maintaining a delta, long term)
is an open problem - there are multiple ways to do it, but none of
them are perfect.

Secondly, the transformations are capricious: they do different things
in different context.  So what you get when you run it locally
wouldn't necessarily be the same thing as the tag2upload conversion
service sees.  In your versioning example, the tag2upload service
might not have the tag that was expected to be used to substitute the
version information.  This means that the meaning of a tag2upload tag
(the instruction to upload) is no longer clear simply by examining the
tag and the git objevts it references.


So I hope this explains why I think changing dgit and tag2upload to
honour gitattributes is out of the question.

That's not to say that there might not be things we can do better to
paper over these cracks and/or help the user avoid lossage.

Ian.

tis 2025-08-19 klockan 15:33 +0100 skrev Ian Jackson:

Oh!  My mistake.

The reason things work for libidn (and other projects that use
.gitattributes export-subst for .tarball-version-git to set the
./configure version number) is that I import a 'git-archive' tarball
(done WITH support for .gitattributes) into Salsa, and that was used to
upload into dgit.  I realize now that if I would change and use a git-
to-git mechanism pulling in upstream git content into Salsa, things
will break, since then the .gitattributes conversion is lost, and I
would need a debian/patches/ workaround to fix things.

I'm happy with this setup (it gives me PGP authentication of the
upstream git-archive tarballs), and definitely agree you should not
respect .gitattributes in dgit/tag2upload because that path leads to
madness.

I think this complicate moving towards fully git-based workflows where
upstream git is pulled into dgit and then built for Debian.  Many
upstream rely on export-subst-like mechanism to set the version number,
and this is a nice git feature, and I don't think we are in a position
to say they are wrong to use it.

Re-reading the original report for #1111548 make me believe that the
proper solution is to document the following somewhere:

   If upstream uses .gitattributes to achieve something magical, you
   need to replicate all necessary magic via debian/rules and/or via
   debian/patches/.  You cannot rely on .gitattributes doing the right
   thing in the Debian dgit/tag2upload workflow, in fact you can and
   should only rely on it doing nothing.

/Simon

Earlier I wrote:

The way this fits into the git transition plan is (now) explained in
my blog post
  Debian's git transition
https://diziet.dreamwidth.org/20436.html

As I write there, this is indeed an annoying but unavoidable
consequence of necessary design principles.  I don't think we are
going to change this aspect of the design, but this bug ought to stay
open to document the problem.


An aside about the bug title:

I don't think it is fair to describe this as "Debian specific rules
for gitattributes".  Any system that needs a reliable, stable, and
bidirectional relationship between (a) git objects and (b) working
trees (or tarballs), will encounter similar problems.

This is just yet another one of those situations where we in Debian
try to solve problems more comprehensively, more faithfully, and more
reliably, than is often the case elswhere.  Or to put it another way:
transforming .gitattributes are inherently hazardous (IMO,
misdesigned), and here we're just doing what *everyone* ought to do.

If there is a Debian-specific rule, it is that we in Debian are
demanding sanity.  Here, as elsewhere, that can indeed look like a
silly Debianism, from a distance.

Ian.

Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie
Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran,
Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun.

Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie
für weitere Details.

Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie
Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran,
Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun.

Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie
für weitere Details.

Hi.  I got this failure for golang-github-smallstep-certificates that
uses a .gitattribute for export-subst version handling:

 .VERSION | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
git-debpush: check failed: the upstream source in tag v0.29.0 is not identical to the upstream source in refs/heads/debian/latest ('upstream-nonidentical' check)

I don't pin to upstream git here, but instead imports the tarball, which
differs for this file.

I've learned that your --quilt=gbp mode ignores differences in the
upstream top-level .gitignore file.  I really dislike upstream source
differences, and have been using --quilt=unapplied to detect this
situation and then revert all such changes in my Debian packages.

However this got me thinking about an improvement here:

Couldn't you extend --quilt=gbp (or add another quilt mode) that behave
the same for .gitignore but for all files in .gitattributes marked with
export-subst?

You'd need a .gitattributes parser, but it is fairly simple.  Any file
marked with 'export-subst' would then be subject to the same "ignore"
handling as the .gitignore file.

What do you think?

Of course, this opens up for supply-chain vulnerabilities planted in
differences in those files, but you already have that for .gitignore,
and even extending the set further severely limit the scope of such
attacks compared to having the same problem for all upstream source code
-- while at the same allows a possibly important and growing use-case
for version-related export-subst files.

/Simon

Simon Josefsson writes ("Idea how to improve .gitattributes handling"):

I'm afraid we've had this conversation before.  This possibility is
mentionioned in

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1079434#20

where I wrote:

  In theory it might be possible to regard this is a
  "maintainer-to-dgit-view" transformation.  We do have such
  transformations.  But there are two significant objections to that:

  Firstly, tree transformations are currently *only* for "3.0 (quilt)"
  source packages.  I think this is quite a helpful rule.  "3.0 (quilt)"
  is very odd, but the underlying task (maintaining a delta, long term)
  is an open problem - there are multiple ways to do it, but none of
  them are perfect.

  Secondly, the transformations are capricious: they do different things
  in different context.  So what you get when you run it locally
  wouldn't necessarily be the same thing as the tag2upload conversion
  service sees.  In your versioning example, the tag2upload service
  might not have the tag that was expected to be used to substitute the
  version information.  This means that the meaning of a tag2upload tag
  (the instruction to upload) is no longer clear simply by examining the
  tag and the git objevts it references.

In case it's not obvious, people might want .gitattributes mutations
applied for other source formats too, even 3.0 (native).  I don't
think source-format-specific gitattributes handling is a good idea.

With .gitignore it is possible to reliably determine what the file
contents is supposed to be.  Even if we restrict ourselves to
export-subst, that's not possible with gitattributes, because we might
have the wrong set of tags available (at the point where the
export-subst is being processed).

Regards,
Ian.