#1145549 fails to download submodules if run from git checkout and upstream is configured as remote #1145549
- Package:
- devscripts
- Source:
- devscripts
- Description:
- scripts to make the life of a Debian Package maintainer easier
- Submitter:
- Simon Richter
- Date:
- 2026-11-12 03:35:02 UTC
- Severity:
- normal
Hi,
I have upstream's git repository configured as a remote in my work tree.
Upstream uses submodules, which get squashed into a single archive by
uscan, and that archive is imported into the git repo for the packages.
As a result, when uscan is run, the checked out branch (with the
debian/watch file) does not have submodules. uscan detects that a remote
for the upstream git exists in the current repository, and tries to reuse
that, which is commendable, but fails to do so:
$ git clone https://salsa.debian.org/electronics-team/nextpnr.git
$ cd nextpnr
$ git remote add yosys https://github.com/YosysHQ/nextpnr.git
$ git fetch yosys
$ uscan
Newest version of nextpnr on remote site is 0.11.1, local version is 0.10
=> Newer package available from:
=> https://github.com/YosysHQ/nextpnr.git refs/tags/nextpnr-0.11.1
fatal: Use '--all' if you really want to deinitialize all submodules
Successfully symlinked ../nextpnr-0.11.1.tar.gz to ../nextpnr_0.11.1.orig.tar.gz.
$ tar -tzvvf ../nextpnr-0.11.1.tar.gz nextpnr-0.11.1/tests | wc -l
1
The "tests" submodule is missing from the generated archive.
If the remote for upstream is not configured, uscan performs a normal clone
in a separate directory, which works.
Simon
- -- Package-specific info:
- --- /etc/devscripts.conf ---
Empty.
- --- ~/.devscripts ---
Not present
- -- System Information:
Debian Release: 13.6
APT prefers stable-security
APT policy: (990, 'stable-security'), (990, 'stable'), (500, 'stable-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, riscv64
Kernel: Linux 7.1.8+deb13-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled
Versions of packages devscripts depends on:
ii dpkg-dev 1.22.22
ii file 1:5.46-5
ii gpg 2.4.7-21+deb13u1+b4
ii gpg-agent 2.4.7-21+deb13u1+b4
ii gpgv 2.4.7-21+deb13u1+b4
ii libconfig-inifiles-perl 3.000003-3+deb13u1
ii libdpkg-perl 1.22.22
ii libfile-dirlist-perl 0.05-3
ii libfile-homedir-perl 1.006-2
ii libfile-touch-perl 0.12-2
ii libio-string-perl 1.08-4
ii libmoo-perl 2.005005-1
ii libwww-perl 6.78-1
ii libyaml-libyaml-perl 0.903.0+ds-1
ii patchutils 0.4.2-1
ii perl 5.40.1-6
ii python3 3.13.5-1
ii sensible-utils 0.0.25
ii wdiff 1.2.2-9
Versions of packages devscripts recommends:
ii apt 3.0.3
ii curl 8.14.1-2+deb13u4
ii dctrl-tools 2.24-3+b1
ii debian-keyring 2025.07.26
ii debian-tag2upload-keyring 1.1
ii diffstat 1.67-1
ii dput-ng [dput] 1.44
ii equivs 2.3.2
ii git-debpush 16.7~bpo13+1
ii libdistro-info-perl 1.13
ii libencode-locale-perl 1.05-3
ii libgitlab-api-v4-perl 0.27-1
ii libjson-perl 4.10000-1
ii liblwp-protocol-https-perl 6.14-1
ii libmetacpan-client-perl 2.033000-1
ii libsoap-lite-perl 1.27-3
ii libstring-shellquote-perl 1.04-3
ii liburi-perl 5.30-1
ii licensecheck 3.3.9-1
ii lintian 2.122.0
ii lzip 1.25-3
ii man-db 2.13.1-1
ii mmdebstrap 1.5.7-1+deb13u1
ii patch 2.8-2
ii pristine-tar 1.50+nmu2
ii python3-apt 3.0.0
ii python3-debian 1.0.1
ii python3-magic 2:0.4.27-3
ii python3-requests 2.32.3+dfsg-5+deb13u1
ii python3-unidiff 0.7.5-2
ii python3-xdg 0.28-2
ii sbuild 0.89.3+deb13u4
ii strace 6.13+ds-1
ii unzip 6.0-29+deb13u1
ii wget 1.25.0-2
ii xz-utils 5.8.1-1+deb13u1
Versions of packages devscripts suggests:
pn adequate <none>
ii at 3.2.5-2.2
ii autopkgtest 5.49
pn bls-standalone <none>
pn bsd-mailx | mailx <none>
ii build-essential 12.12
pn check-all-the-things <none>
ii debhelper 13.24.2
pn diffoscope <none>
pn disorderfs <none>
ii docker.io 26.1.5+dfsg1-9+deb13u1
pn dose-extra <none>
pn duck <none>
pn elpa-devscripts <none>
pn faketime <none>
ii gnuplot-qt [gnuplot] 6.0.2+dfsg1-1
pn how-can-i-help <none>
ii libauthen-sasl-perl 2.1700-1
pn libdbd-pg-perl <none>
pn libterm-size-perl <none>
ii libtimedate-perl 2.3300-2
ii msmtp-mta [mail-transport-agent] 1.8.28-3
ii mutt 2.2.13-1+deb13u1
ii openssh-client [ssh-client] 1:10.0p1-7+deb13u4
pn piuparts <none>
pn postgresql-client <none>
pn pristine-lfs <none>
ii python3-debianbts 4.1.1
ii python3-pycurl 7.45.6-1
ii quilt 0.68-1
pn ratt <none>
pn reprotest <none>
pn svn-buildpackage <none>
ii w3m 0.5.3+git20230121-2.1
- -- no debconf information
-----BEGIN PGP SIGNATURE-----
iQFDBAEBCgAtFiEEtjuqOJSXmNjSiX3Tfr04e7CZCBEFAmqNs80PHHNqckBkZWJp
YW4ub3JnAAoJEH69OHuwmQgR9GUIAIX1h2jbtPwuqykKr2vK2Peo6rG20RcXkiPe
mz4KXfON5wwgZeEL/w/pz3HlfjN1YaulH7h8Bd/DTfH+uIc2RWaKkW0Xow0qgGa2
0oyxPzC785s5j0WiZ/rMtPU6D4IGoUUNPwOiKbZLZ47PExjNG2rygZTxonBuJ2/K
OVE1ReU5tORhjPr+Tc1UyOMKhVAGjXmWEsDPXKzi+6Kylp86E1XdqsH/DWi+7xzg
DxKX7iWDaYworqludexGyBTKeHgAwgtnrmAxzSvsK0mJqJnMVfHzAjQZ979eT5gR
epWcYRwWkIS1C/FXuBz6K+4cOaQmsUzvqSrhlqajQuJ1xFvhpO4=
=vgkn
-----END PGP SIGNATURE-----
Hi, Simon I tried to manually run what uscan is actually doing in this case (seen using uscan -vv) after above three commands are issued, only with commands needed to get result (no tags searching, I already know the result). $ git fetch --recurse-submodules yosys $ git archive --format=tar --prefix=nextpnr-0.11.1/ --output=/home/myuser/src/nextpnr-0.11.1.tar refs/tags/nextpnr-0.11.1 $ git submodule --quiet foreach 'echo $name' git archive creates the base tarball, without modules, but "git submodule --quiet foreach ..." above gives no result. "git submodule" also gives no result. submodules are defined in .gitmodules, but repo seems to not know about them. $ git submodule --quiet deinit -- Above error seems caused when deinit command is called on empty @sm_deinit. While that can be easily hidden, may be that would hide an actual problem. After this, git submodule foreach --recursive 'git archive ...' on submodules does nothing, so nothing is added to tarball. When no upstream remote is configured, upstream repo is cloned and all the work is done in that cloned repo, apparently working as expected. Not a fix, but I hope this may help to understand what is happening. Regards,
Hi, Cc'ing Hugh McMaster, who did a lot of work in this section in case he can add something. I kept trying to understand when this happens, and noticed that submodules are not initialized when cloning from salsa Debian repo, but they are when cloning from upstream repo. # Does not work with salsa repo. Submodules not initialized $ git clone https://salsa.debian.org/electronics-team/nextpnr.git $ cd nextpnr $ git submodule update --init --recursive -- 3rdparty/corrosion himbaechel/uarch/xilinx/meta tests # Works with upstream repo. Submodules are initialized. $ git clone https://github.com/YosysHQ/nextpnr.git $ cd nextpnr $ git submodule update --init --recursive -- 3rdparty/corrosion himbaechel/uarch/xilinx/meta tests This seems to be the origin of the problem. When a remote is defined, local repo is used and submodules are not used since they are not initialized. When no remote is defined, upstream repo is used and submodules are initialized and used. Do not understand why this is happening. Hope this helps in some way. Regards,--- Agustin
Hi, I have tested this a bit more and I tend to think that this is not a problem in uscan but in nextpnr salsa git repo. $ git clone https://salsa.debian.org/electronics-team/nextpnr.git --recurse-submodules Cloning into 'nextpnr'... ... $ cd nextpnr $ git submodule update --init --recursive -- 3rdparty/corrosion himbaechel/uarch/xilinx/meta tests $ git submodule add https://github.com/YosysHQ/nextpnr-tests tests fatal: 'tests' already exists in the index $ git submodule add -f https://github.com/YosysHQ/nextpnr-tests tests fatal: 'tests' already exists in the index and is not a submodule Then tried to see what happens after removing tests contents for git $ git rm -r tests/ rm 'tests/COPYING' ... $ git commit tests -m 'Removing tests submodule contents' [master 8e1cd0e] Removing tests submodule contents ... and adding it again, now really as a submodule, $ git submodule add https://github.com/YosysHQ/nextpnr-tests tests Cloning into '/home/unsafe/.nobackup/src/kk/nextpnr/tests'... ... $ git fetch --recurse-submodules Fetching submodule tests $ git submodule aea0b4bac59e07e0c5088a0966af73abdd8ab217 tests (heads/master) So, seems that intially submodules were not declared as submodules, but present in the repo. This did not happen when using upstream git repo (what happens when there is no remote defined). If I now add the remote and run "uscan --download-current-version" created upstream tarball contains tests/ contents, but not the contents for other submodules. Regards,
Hi,
That does not set up submodules, because none of the existing git
mappings for Debian source packages has submodule support, so the
"upstream" branch merges them (and omits the excluded files). It should
perhaps omit the .gitmodules file, but that's a different problem.
The watch file points to another git server though, which *does* have
submodules.
Now uscan, when run from a checkout, will check if we already have a
remote configured for the server listed in the watch file, to avoid a
clone if we already have one. That is good.
The problem is that in order to also get the submodule contents from the
existing clone, the submodule's base URL must also be configured as a
remote.
So, after
You can run uscan there with no problem, but it will do a full clone
into a temporary directory:
$ uscan -ddd
[some warnings]
Successfully repacked ../nextpnr-0.11.1.tar.xz as
../nextpnr_0.11.1+dfsg.orig.tar.xz, deleting 461 files from it.
$ tar -tJvvf ../nextpnr_0.11.1+dfsg.orig.tar.xz nextpnr-0.11.1/tests/ |
wc -l
260
If you add
$ git remote add yosyshq https://github.com/YosysHQ/nextpnr.git
running uscan will detect that the "yosyshq" remote can be used, and no
fresh clone is necessary. It will even fetch from the remote first:
$ uscan -ddd
[fetch output]
Could not access submodule '3rdparty/abseil-cpp' at commit 0afa0da1
Could not access submodule '3rdparty/corrosion' at commit 06992bda
Could not access submodule '3rdparty/googletest' at commit 64db93e3
Could not access submodule '3rdparty/fpga-interchange-schema' at commit
ba69b355
Could not access submodule 'himbaechel/uarch/xilinx/meta' at commit 0cd6e72d
Could not access submodule 'tests' at commit cd7f7c12
Could not access submodule 'torc' at commit 36edbdcb
fatal: Use '--all' if you really want to deinitialize all submodules
Successfully repacked ../nextpnr-0.11.1.tar.xz as
../nextpnr_0.11.1+dfsg.orig.tar.xz, deleting 461 files from it.
but tests/ is empty:
$ tar -tJvvf ../nextpnr_0.11.1+dfsg.orig.tar.xz nextpnr-0.11.1/tests/ |
wc -l
1
The problem seems to be that uscan tries to avoid using the working
directory, except during submodule handling, where it uses whatever is
currently checked out (so if upstream changes the list of submodules,
that is ignored as well).
Yes, because now tests/ is a submodule in the current working directory
-- but that is local state influencing the output of uscan, when all we
want is to use the cached objects -- and what's probably worse, uscan
also changes local state by attempting to deinitialize and reinitialize
the submodules.
Simon