#1145549 fails to download submodules if run from git checkout and upstream is configured as remote

Package:
devscripts
Source:
devscripts
Description:
scripts to make the life of a Debian Package maintainer easier
Submitter:
Simon Richter
Date:
2026-11-12 03:35:02 UTC
Severity:
normal
#1145549#5
Date:
2026-08-25 15:25:01 UTC
From:
To:
Hi,

I have upstream's git repository configured as a remote in my work tree.
Upstream uses submodules, which get squashed into a single archive by
uscan, and that archive is imported into the git repo for the packages.

As a result, when uscan is run, the checked out branch (with the
debian/watch file) does not have submodules. uscan detects that a remote
for the upstream git exists in the current repository, and tries to reuse
that, which is commendable, but fails to do so:

$ git clone https://salsa.debian.org/electronics-team/nextpnr.git
$ cd nextpnr
$ git remote add yosys https://github.com/YosysHQ/nextpnr.git
$ git fetch yosys
$ uscan
Newest version of nextpnr on remote site is 0.11.1, local version is 0.10
 => Newer package available from:
        => https://github.com/YosysHQ/nextpnr.git refs/tags/nextpnr-0.11.1
fatal: Use '--all' if you really want to deinitialize all submodules
Successfully symlinked ../nextpnr-0.11.1.tar.gz to ../nextpnr_0.11.1.orig.tar.gz.
$ tar -tzvvf ../nextpnr-0.11.1.tar.gz nextpnr-0.11.1/tests | wc -l
1

The "tests" submodule is missing from the generated archive.

If the remote for upstream is not configured, uscan performs a normal clone
in a separate directory, which works.

   Simon

- -- Package-specific info:

- --- /etc/devscripts.conf ---
Empty.

- --- ~/.devscripts ---
Not present

- -- System Information:
Debian Release: 13.6
  APT prefers stable-security
  APT policy: (990, 'stable-security'), (990, 'stable'), (500, 'stable-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, riscv64

Kernel: Linux 7.1.8+deb13-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled

Versions of packages devscripts depends on:
ii  dpkg-dev                 1.22.22
ii  file                     1:5.46-5
ii  gpg                      2.4.7-21+deb13u1+b4
ii  gpg-agent                2.4.7-21+deb13u1+b4
ii  gpgv                     2.4.7-21+deb13u1+b4
ii  libconfig-inifiles-perl  3.000003-3+deb13u1
ii  libdpkg-perl             1.22.22
ii  libfile-dirlist-perl     0.05-3
ii  libfile-homedir-perl     1.006-2
ii  libfile-touch-perl       0.12-2
ii  libio-string-perl        1.08-4
ii  libmoo-perl              2.005005-1
ii  libwww-perl              6.78-1
ii  libyaml-libyaml-perl     0.903.0+ds-1
ii  patchutils               0.4.2-1
ii  perl                     5.40.1-6
ii  python3                  3.13.5-1
ii  sensible-utils           0.0.25
ii  wdiff                    1.2.2-9

Versions of packages devscripts recommends:
ii  apt                         3.0.3
ii  curl                        8.14.1-2+deb13u4
ii  dctrl-tools                 2.24-3+b1
ii  debian-keyring              2025.07.26
ii  debian-tag2upload-keyring   1.1
ii  diffstat                    1.67-1
ii  dput-ng [dput]              1.44
ii  equivs                      2.3.2
ii  git-debpush                 16.7~bpo13+1
ii  libdistro-info-perl         1.13
ii  libencode-locale-perl       1.05-3
ii  libgitlab-api-v4-perl       0.27-1
ii  libjson-perl                4.10000-1
ii  liblwp-protocol-https-perl  6.14-1
ii  libmetacpan-client-perl     2.033000-1
ii  libsoap-lite-perl           1.27-3
ii  libstring-shellquote-perl   1.04-3
ii  liburi-perl                 5.30-1
ii  licensecheck                3.3.9-1
ii  lintian                     2.122.0
ii  lzip                        1.25-3
ii  man-db                      2.13.1-1
ii  mmdebstrap                  1.5.7-1+deb13u1
ii  patch                       2.8-2
ii  pristine-tar                1.50+nmu2
ii  python3-apt                 3.0.0
ii  python3-debian              1.0.1
ii  python3-magic               2:0.4.27-3
ii  python3-requests            2.32.3+dfsg-5+deb13u1
ii  python3-unidiff             0.7.5-2
ii  python3-xdg                 0.28-2
ii  sbuild                      0.89.3+deb13u4
ii  strace                      6.13+ds-1
ii  unzip                       6.0-29+deb13u1
ii  wget                        1.25.0-2
ii  xz-utils                    5.8.1-1+deb13u1

Versions of packages devscripts suggests:
pn  adequate                          <none>
ii  at                                3.2.5-2.2
ii  autopkgtest                       5.49
pn  bls-standalone                    <none>
pn  bsd-mailx | mailx                 <none>
ii  build-essential                   12.12
pn  check-all-the-things              <none>
ii  debhelper                         13.24.2
pn  diffoscope                        <none>
pn  disorderfs                        <none>
ii  docker.io                         26.1.5+dfsg1-9+deb13u1
pn  dose-extra                        <none>
pn  duck                              <none>
pn  elpa-devscripts                   <none>
pn  faketime                          <none>
ii  gnuplot-qt [gnuplot]              6.0.2+dfsg1-1
pn  how-can-i-help                    <none>
ii  libauthen-sasl-perl               2.1700-1
pn  libdbd-pg-perl                    <none>
pn  libterm-size-perl                 <none>
ii  libtimedate-perl                  2.3300-2
ii  msmtp-mta [mail-transport-agent]  1.8.28-3
ii  mutt                              2.2.13-1+deb13u1
ii  openssh-client [ssh-client]       1:10.0p1-7+deb13u4
pn  piuparts                          <none>
pn  postgresql-client                 <none>
pn  pristine-lfs                      <none>
ii  python3-debianbts                 4.1.1
ii  python3-pycurl                    7.45.6-1
ii  quilt                             0.68-1
pn  ratt                              <none>
pn  reprotest                         <none>
pn  svn-buildpackage                  <none>
ii  w3m                               0.5.3+git20230121-2.1

- -- no debconf information
-----BEGIN PGP SIGNATURE-----

iQFDBAEBCgAtFiEEtjuqOJSXmNjSiX3Tfr04e7CZCBEFAmqNs80PHHNqckBkZWJp
YW4ub3JnAAoJEH69OHuwmQgR9GUIAIX1h2jbtPwuqykKr2vK2Peo6rG20RcXkiPe
mz4KXfON5wwgZeEL/w/pz3HlfjN1YaulH7h8Bd/DTfH+uIc2RWaKkW0Xow0qgGa2
0oyxPzC785s5j0WiZ/rMtPU6D4IGoUUNPwOiKbZLZ47PExjNG2rygZTxonBuJ2/K
OVE1ReU5tORhjPr+Tc1UyOMKhVAGjXmWEsDPXKzi+6Kylp86E1XdqsH/DWi+7xzg
DxKX7iWDaYworqludexGyBTKeHgAwgtnrmAxzSvsK0mJqJnMVfHzAjQZ979eT5gR
epWcYRwWkIS1C/FXuBz6K+4cOaQmsUzvqSrhlqajQuJ1xFvhpO4=
=vgkn
-----END PGP SIGNATURE-----

#1145549#10
Date:
2026-09-05 17:55:27 UTC
From:
To:
Hi, Simon

I tried to manually run what uscan is actually doing in this case (seen
using uscan -vv) after above three commands are issued, only with commands
needed to get result (no tags searching, I already know the result).

$ git fetch --recurse-submodules yosys
$ git archive --format=tar --prefix=nextpnr-0.11.1/ --output=/home/myuser/src/nextpnr-0.11.1.tar refs/tags/nextpnr-0.11.1
$ git submodule --quiet foreach 'echo $name'

git archive creates the base tarball, without modules, but "git submodule
--quiet foreach ..." above gives no result. "git submodule" also gives no
result. submodules are defined in .gitmodules, but repo seems to not know
about them.

$ git submodule --quiet deinit --

Above error seems caused when deinit command is called on empty
@sm_deinit. While that can be easily hidden, may be that would hide an
actual problem.

After this, git submodule foreach --recursive 'git archive ...' on
submodules does nothing, so nothing is added to tarball.

When no upstream remote is configured, upstream repo is cloned and all the
work is done in that cloned repo, apparently working as expected.

Not a fix, but I hope this may help to understand what is happening.

Regards,

#1145549#15
Date:
2026-10-14 10:19:30 UTC
From:
To:
Hi,

Cc'ing Hugh McMaster, who did a lot of work in this section in case he can
add something.

I kept trying to understand when this happens, and noticed that submodules
are not initialized when cloning from salsa Debian repo, but they are when
cloning from upstream repo.

# Does not work with salsa repo. Submodules not initialized
$ git clone   https://salsa.debian.org/electronics-team/nextpnr.git
$ cd nextpnr
$ git submodule update --init --recursive  -- 3rdparty/corrosion himbaechel/uarch/xilinx/meta tests

# Works with upstream repo. Submodules are initialized.
$ git clone   https://github.com/YosysHQ/nextpnr.git
$ cd nextpnr
$ git submodule update --init --recursive  -- 3rdparty/corrosion himbaechel/uarch/xilinx/meta tests

This seems to be the origin of the problem. When a remote is defined, local
repo is used and submodules are not used since they are not initialized.
When no remote is defined, upstream repo is used and submodules are
initialized and used.

Do not understand why this is happening. Hope this helps in some way.

Regards,
--- Agustin
#1145549#20
Date:
2026-11-11 23:12:36 UTC
From:
To:
Hi,

I have tested this a bit more and I tend to think that this is not a
problem in uscan but in nextpnr salsa git repo.

$ git clone https://salsa.debian.org/electronics-team/nextpnr.git --recurse-submodules
Cloning into 'nextpnr'...
...
$ cd nextpnr
$ git submodule update --init --recursive  -- 3rdparty/corrosion himbaechel/uarch/xilinx/meta tests
$ git submodule add https://github.com/YosysHQ/nextpnr-tests tests
fatal: 'tests' already exists in the index
$ git submodule add -f https://github.com/YosysHQ/nextpnr-tests tests
fatal: 'tests' already exists in the index and is not a submodule

Then tried to see what happens after removing tests contents for git

$ git rm -r tests/
rm 'tests/COPYING'
...
$ git commit tests -m 'Removing tests submodule contents'
[master 8e1cd0e] Removing tests submodule contents
...

and adding it again, now really as a submodule,

$ git submodule add https://github.com/YosysHQ/nextpnr-tests tests
Cloning into '/home/unsafe/.nobackup/src/kk/nextpnr/tests'...
...
$ git fetch --recurse-submodules
Fetching submodule tests
$ git submodule
 aea0b4bac59e07e0c5088a0966af73abdd8ab217 tests (heads/master)

So, seems that intially submodules were not declared as submodules, but
present in the repo. This did not happen when using upstream git repo (what
happens when there is no remote defined). If I now add the remote and run
"uscan --download-current-version" created upstream tarball contains
tests/ contents, but not the contents for other submodules.

Regards,

#1145549#25
Date:
2026-11-12 03:32:44 UTC
From:
To:
Hi,

That does not set up submodules, because none of the existing git
mappings for Debian source packages has submodule support, so the
"upstream" branch merges them (and omits the excluded files). It should
perhaps omit the .gitmodules file, but that's a different problem.

The watch file points to another git server though, which *does* have
submodules.

Now uscan, when run from a checkout, will check if we already have a
remote configured for the server listed in the watch file, to avoid a
clone if we already have one. That is good.

The problem is that in order to also get the submodule contents from the
existing clone, the submodule's base URL must also be configured as a
remote.

So, after

You can run uscan there with no problem, but it will do a full clone
into a temporary directory:

$ uscan -ddd
[some warnings]
Successfully repacked ../nextpnr-0.11.1.tar.xz as
../nextpnr_0.11.1+dfsg.orig.tar.xz, deleting 461 files from it.

$ tar -tJvvf ../nextpnr_0.11.1+dfsg.orig.tar.xz nextpnr-0.11.1/tests/ |
wc -l
260

If you add

$ git remote add yosyshq https://github.com/YosysHQ/nextpnr.git

running uscan will detect that the "yosyshq" remote can be used, and no
fresh clone is necessary. It will even fetch from the remote first:

$ uscan -ddd
[fetch output]
Could not access submodule '3rdparty/abseil-cpp' at commit 0afa0da1
Could not access submodule '3rdparty/corrosion' at commit 06992bda
Could not access submodule '3rdparty/googletest' at commit 64db93e3
Could not access submodule '3rdparty/fpga-interchange-schema' at commit
ba69b355
Could not access submodule 'himbaechel/uarch/xilinx/meta' at commit 0cd6e72d
Could not access submodule 'tests' at commit cd7f7c12
Could not access submodule 'torc' at commit 36edbdcb
fatal: Use '--all' if you really want to deinitialize all submodules
Successfully repacked ../nextpnr-0.11.1.tar.xz as
../nextpnr_0.11.1+dfsg.orig.tar.xz, deleting 461 files from it.

but tests/ is empty:

$ tar -tJvvf ../nextpnr_0.11.1+dfsg.orig.tar.xz nextpnr-0.11.1/tests/ |
wc -l
1

The problem seems to be that uscan tries to avoid using the working
directory, except during submodule handling, where it uses whatever is
currently checked out (so if upstream changes the list of submodules,
that is ignored as well).
Yes, because now tests/ is a submodule in the current working directory
-- but that is local state influencing the output of uscan, when all we 
want is to use the cached objects -- and what's probably worse, uscan
also changes local state by attempting to deinitialize and reinitialize
the submodules.

    Simon