Hi!

Yesterday I've finally given tag2upload a try with the cubeb package.
It's wonderful!

Still, I've encountered a surprising issue related to git submodules.

cubeb upstream uses git submodules, but they are not needed in Debian.
In my tarball-based workflow, I've been able to ignore them, as the
tarball generated by Git(Hub) creates empty directories where submodules
should be. Fine.

When switching to git-debpush, I've also switched how I get upstream
code. Since I package Git snapshot, I followed the instructions in
dgit-maint-debrebase and now fetch the code from the upstream Git.

With this change, though, submodules are now properly represented in the
source. I still ignored them.

When pushing with git-debpush I then received an email with the
following error:

    Format `3.0 (quilt)', need to check/update patch stack
    canonical suite name for experimental is rc-buggy
    dgit (build host): found orig(s) in .. missing from build-products-dir, transferring:
    dgit (build host): hardlinked orig from ..: cubeb_0.0~git20250529.78ee5f0+ds.orig.tar.xz
    dgit: split brain (separate dgit view) may be needed (--quilt=gbp).
    examining quilt state (multiple patches, gbp mode)
    dpkg-source: warning: cubeb-0.0~git20250529.78ee5f0+ds/debian/rules does not exist
    dpkg-source: warning: unknown information field 'Build-Driver' in input data in general section of control info file
    dgit (build host): base trees orig=25938cea9885d0da060b o+d/p=cf711e3c67958fc2bf9b
    dgit (build host): quilt differences: src:  ## orig ##     gitignores:  == orig ==
    dgit (build host): quilt differences:      HEAD ## o+d/p               HEAD == o+d/p
    dgit:  cannot represent change: not a plain file (000000->160000): cmake/sanitizers-cmake
    dgit:  cannot represent change: not a plain file (000000->160000): googletest
    dgit:  cannot represent change: not a plain file (000000->160000): src/cubeb-coreaudio-rs
    dgit:  cannot represent change: not a plain file (000000->160000): src/cubeb-pulse-rs

    dgit (build host): error: HEAD has changes to .orig[s] which are not representable by `3.0 (quilt)'

Ok, I just repacked the source removing the submodules with `git rm`,
but I found this error kinda odd, especially as I had been able to build
the package locally just fine.

Is this expected? How am I supposed to deal with git submodules?

Bye!

Note:
  I'm cloning this bug a couple of times for clearly defined
  action items.  Let us leave this original bug #1107219 for user
  support, and discussion of further options.

Andrea Pappacoda writes ("Bug#1107219: git-debpush: upload failing when source has submodules"):

I'm glad you're enjoying it :-).

Sadly, git submodules are a complete disaster.  I wrote a blog post
with a rant:  https://diziet.dreamwidth.org/14666.html

But, I think our tooling could have served you better.  Let me try to
give you some advice, and discuss possible improvement options.

Aha.

Yes, that is the workflow we would normally recommend.  It is more
faithful and more traceable to upstream.  Unfortunately the
faithfulness also exposes you to the submodules.

This is a reasonable thing for you to have attempted, even though it
was never going to work.

Our tooling ought to have detected this problem much sooner; and,
I think there are opportunities for making the workaround more
convenient.

tag2upload uses dgit to do a lot of the work, and this error is
basically #726953 "dgit fails with submodules".

I don't know your local building workflow, but many approaches for
local builds omit certain important consistency checks.

One of the key properties of our git transition tools, like tag2upload
and dgit, is that the source package (.dsc) is *completely identical*
to the (canonical) git view.  That makes it possible to replace .dsc
with git without changing what source code you get, to convert in both
directions, and so on.

For a git submodule it's not quite clear what "identical" means here,
but it probably means the submodule is populated.  This is discussed
at much greater length in another bug:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=726953#64

With our current tooling, I suggest treating the submodule the same
way we would treat nonfree materials.  Ie, this part of the workflow
manpage:

https://manpages.debian.org/bookworm/dgit/dgit-maint-debrebase.7.en.html#DFSG-non-free:_When_upstream_tags_releases_in_git

It's not quite clear to me what you mean by "repacked".  With
tag2upload you do not, in fact, ever need to have a terball on your
system.  So you don't need to run git deborig.

As for our tooling and docs:

 * git-debpush should definitely reject a submodule.
   That would have detected the problem locally.  (clone -2)

 * git-debrebase should detect upstream trees that have
   submodules and call that situation a snag.  (clone -3)

 * #726953 discusses more comprehensive submodule support
   for dgit (which, if implemented, would extend to tag2upload too).

 * Is there good tooling for doing DFSG-filtering entirely in git?
   Eg is there some git tooling that will read uscan config?

 * We need workflow manpages for tag2upload.  Separately we want more
   opinionated documentation of a git-first flow - that does not
   involve gbp import-orig.

 * I'm tempted to suggest an interim version of #726953 that makes it
   possible to simply filter out submodules during canonicalisation.

   That would improve convenience when upstream has submodules but the
   Debian package wants to ignore all of them completely.  How common
   do we think this situation is?

   (Note that it's easy to speak of doing this "as a quilt mode" or
   "in quilt fixup" but it's not a "3.0 (quilt)" thing - it might
   apply to native format source packages.)

Ian.

Sean Whitton writes ("[PATCH fyi 2/2] git-debpush: Check for presence of git submodules"):
dangling commit reference instead of a tree reference in the git
objects.  .gitmodules is just some metadata used by the
`git submodule` command.

But after doing some experiments, I think this test is good in
git-debpush.  But thinking about this makes me want dgit to be more
picky.

I don't think this is urgent since the anomalous states seem to be
rejected by most git submodule tooling so probably don't exist in the
wild very much, but you could perhaps make them by manually deleting
.gitmodules.

Ian.

Using 3cbfc9ed2d0e32f1d0556b2541087d1bb79f4a19
from https://github.com/mozilla/cubeb

zealot:cubeb> git checkout upstream/main
Previous HEAD position was b29e340 Update changelog for 0.0~git20240524.dabff20+ds-1 release
HEAD is now at 832fcf3 wasapi: Fix formatting.
zealot:cubeb> cat .gitmodules
[submodule "googletest"]
        path = googletest
        url = https://github.com/google/googletest
[submodule "cmake/sanitizers-cmake"]
        path = cmake/sanitizers-cmake
        url = https://github.com/arsenm/sanitizers-cmake
[submodule "src/cubeb-coreaudio-rs"]
        path = src/cubeb-coreaudio-rs
        url = https://github.com/mozilla/cubeb-coreaudio-rs
        branch = trailblazer
[submodule "src/cubeb-pulse-rs"]
        path = src/cubeb-pulse-rs
        url = https://github.com/mozilla/cubeb-pulse-rs
        branch = dev
zealot:cubeb> git submodule status
-aab6948fa863bc1cbe5d0850bc46b9ef02ed4c1a cmake/sanitizers-cmake
-40412d85124f7c6f3d88454583c4633e5e10fc8c googletest
-8f39e9fc3ad4868e6ff7188c52575087a1a02777 src/cubeb-coreaudio-rs
-6bac666467e4a37cf057f0e17e8c9e8a024b060b src/cubeb-pulse-rs
zealot:cubeb> git-rm -f .gitmodules
rm '.gitmodules'
zealot:cubeb> git commit -m 'rm .gitmodules'
[detached HEAD 6954ccd] rm .gitmodules
 1 file changed, 14 deletions(-)
 delete mode 100644 .gitmodules
zealot:cubeb> git submodule status
fatal: no submodule mapping found in .gitmodules for path 'cmake/sanitizers-cmake'
zealot:cubeb> ll googletest/
total 8
drwxrwsr-x  2 ian ian 4096 Jun  6 17:39 ./
drwxrwsr-x 12 ian ian 4096 Jun  6 17:40 ../
zealot:cubeb> git log googletest/
zealot:cubeb> git-ls-tree googletest/
fatal: Not a valid object name googletest/
zealot:cubeb> git-ls-tree googletest
fatal: Not a valid object name googletest
zealot:cubeb> git-ls-tree HEAD:googletest
fatal: not a tree object
zealot:cubeb> ls
AUTHORS  CMakeLists.txt   cubeb.supp  googletest/  INSTALL.md  README.md               src/          test/
cmake/   Config.cmake.in  docs/       include/     LICENSE     scan-build-install.sh*  subprojects/  tools/
zealot:cubeb> git-ls-tree HEAD:docs
100644 blob 8ec863be8ea00ebdc3818535f09193442033059e    Doxyfile.in
zealot:cubeb> git-ls-tree HEAD:googletest
fatal: not a tree object
zealot:cubeb> git-ls-tree HEAD:
100644 blob b8b0df0bf257a507ac981593af45ce5bf4dc8585    .clang-format
040000 tree 94006555eb06dbd4ea8c5fc570a784da8f1e0f59    .github
100644 blob e524d792f4cbf97b0d78d50636ebbd6b3192b88a    .gitignore
100644 blob f0f9595227f27af11dd640dd86c3ce18dad50478    AUTHORS
100644 blob 633653d7444639518baf71de4472ac4f18ebc66c    CMakeLists.txt
100644 blob be464aa4923e214ddcc73c2d41695fbc6b5c91e6    Config.cmake.in
100644 blob 4f83a652001802630e2febfaebca08bed13097b3    INSTALL.md
100644 blob fffc9dc4053647ff5895bd3a2e43c8c8f9d14c50    LICENSE
100644 blob e4e1658824394ab6b885af71f565a9625e5c340c    README.md
040000 tree bcbcbe97f08d1f410a40a25490d4f33feb07cea7    cmake
100644 blob 0012ea51e6543a169018104cd795fe74a3f131bb    cubeb.supp
040000 tree 2b9892f6a668f7899be9c27950b376baf66152ac    docs
160000 commit 40412d85124f7c6f3d88454583c4633e5e10fc8c  googletest
040000 tree 09c5f19e330a6041ae4534bc51c8f276b7e0b67b    include
100755 blob 3e1b4969e7f459674d2d7c045c74e988a1e2040e    scan-build-install.sh
040000 tree 6b2c118ffe666921072071aa94b35f407fdeb0e8    src
040000 tree 758603bae5c9c8eeeab0ab9655483c2fb5dc0712    subprojects
040000 tree 66d44b4e2b9aafd8ebaab51094f211b9872a31ec    test
040000 tree cabedf4c07fc654ecb70b8e886e1fec468acc9f3    tools
zealot:cubeb> git submodule foreach echo
fatal: No url found for submodule path 'cmake/sanitizers-cmake' in .gitmodules
zealot:cubeb> git submodule status
zealot:cubeb>

Hello,

Yes, I went through a few iterations before settling on looking for
.gitmodules.

Sounds reasonable.

We believe that the bug you reported is fixed in the latest version of
dgit, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1107226@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Ian Jackson <ijackson@chiark.greenend.org.uk> (supplier of updated dgit package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Fri, 13 Jun 2025 15:07:04 +0100
Source: dgit
Architecture: source
Version: 13.1
Distribution: experimental
Urgency: medium
Maintainer: Debian tag2upload Delegates <dgit-owner@debian.org>
Changed-By: Ian Jackson <ijackson@chiark.greenend.org.uk>
Closes: 1105756 1105815 1105862 1105878 1106078 1106481 1107226 1107550
Changes:
 dgit (13.1) experimental; urgency=medium
 .
   git-debpush behavioural improvements [Sean Whitton]:
   * Detect apparent intent to use pristine-tar (which is not supported).
     Closes: #1105862.  [Reports from Simon Josefsson, Anton Gladky]
   * Detect presence of git submodules, and fail early.
     Closes: #1107226.  [Report from Andrea Pappacoda]  See also #726953.
   * When checks fail, prompt interactively, asking whether to proceed.
     Closes: #1105815, #1107550.
   * Rename hazardous --quilt=auto, to --quilt=try-linear.
     [Prompted by report from Anton Gladky]
 .
   git-debpush(1) improvements [Sean Whitton]:
   * Document that we never do a git force push.
     Closes: #1105756.  [Report from Simon Josefsson]
   * --quilt=gbp: Mention quilt(1) explicitly.  [Report from Anton Gladky]
   * Fix erroneous dedent location for end of list of options.
 .
   tag2upload infrastructure:
   * t2u-b-rebuild: Improve logging and error handling.
   * tag2upload-oracled: Predeclare some function prototypes.
     Closes: #1105878.  [Sean Whitton]
 .
   Documentation:
   * Update to non-ancient distribution names in many docs.  [Marc Haber]
   * tag2upload(t5); Fix and clarify semantics of unknown tags,
     multiplicity, ordering, and syntax of keyword=value.
     Closes: #1106078.  [Report from Matthias Urlichs]
   * i18n: Updated message translation for nl.
     Closes: #1106481.  [Frans Spiesschaert]
Checksums-Sha1:
 181aea32ab01d93753bf6743db24aeebceb441f8 2050 dgit_13.1.dsc
 93df9c97bff90d5fefd58b98adb6913ef2363ba9 761885 dgit_13.1.tar.gz
Checksums-Sha256:
 2163074b54233784aae995e0388506c3cfd94683dcdf5348b1da83babcf3363b 2050 dgit_13.1.dsc
 5d4a7a997eee5bf447396510df3d800fa03523746dd41c6716710e5c7d252d19 761885 dgit_13.1.tar.gz
Files:
 bebb4d2e1437a947eb978f83c64df91c 2050 devel optional dgit_13.1.dsc
 62454e70ccfe131e7ad6891e2bab0dfc 761885 devel optional dgit_13.1.tar.gz
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAmhMOZQACgkQ4+M5I0i1
DTkHMwf+OJ/FCw1nhaPRsRTKZIXtVCqRjdWTNtqgP2l1YMxUifDE4aaLMwGjm9qn
mCQZlR6TM7Gw7/hIW07CMotKudOEnP6DLVImO++dbfLryd9pXy2h/IB4k74fzMwb
RKoLV91qKAi8P83QKoEuWvWQao6T9coVB6mb8xyQcvaLyB23PuxaC97B23sxWSOf
dPupUr7JdcZgYGtjYOm3oC+QUzjSrk/VJDTsMAcpqiT7EKZUgUkhnNmpPCYr6qwP
GjU4XeM7nVLEBzaSHgxJTKhff3DzLrX+of2e47P7MURS8SCzotvGzcNnbhH7JSMu
4qMVLCOw3ruWyELPnMmSP+jGl4Taew==
=EGP9
-----END PGP SIGNATURE-----