Dear Maintainer,

Using Debian's singularity to pull a Docker container somehow removes the
environment variables that are set in the Dockerfile (using ENV) from the
container. This bug is not present in 3.9.9 upstream, so it has to do with how
it is packaged in Debian.

The issue is also discussed here, and there is concluded it is a Debian issue:
https://github.com/sylabs/singularity/issues/852

An example run that displays the problems:

$ singularity pull --name astronrd_linc.sif docker://astronrd/linc
$ singularity exec --contain --cleanenv astronrd_linc.sif bash
Singularity> echo $PYTHONPATH

Singularity> echo $LINC_DATA_ROOT

Singularity>

Both these variables are set in the Docker container, but are no longer set
when the docker container is pulled using Debian's singularity. The correct
output should be:

Singularity> echo $PYTHONPATH
/usr/local/bin
Singularity> echo $LINC_DATA_ROOT
/usr/local/share/linc

Singularity's own package for 3.9.9 does this correctly (and various other
versions were tested as well, see link to github issue above)

Kind regards,
André

Dave did some analysis of the cause of this bug, and he wrote the following:

I did a little bit more digging, as I was curious how a dependency
version change would cause this particular error. I now don't think it's
directly from a dependency version difference, but it's indirect through
a patch instead.

Debian has this patch:
https://salsa.debian.org/hpc-team/singularity-container/-/blob/master/debian/patches/conveyorPacker_oci.patch

... which makes that function return an empty imgspecv1.ImageConfig{}
always... instead of the ImageConfig from the Docker/OCI image.

Later on we retrieve the ENV values out of the config that this function
returns. But with the Debian patch the config is going to be empty, so
no ENV is seen.

I think this patch was done because of an incompatibility with the
version of a dependency they have switched to, versus our pinned
version. However it's the bad patch causing the error with generating
Docker -> Singularity containers, not the dependency change itself.

Thanks for the report.

I see.
Does Dave has some time to propose a proper patch for this, then?
(Unfortunately I am out of time myself)

CC'ed Benda who wrote the patch originally, if he could chime in here w/ a better solution.

André,

I have done a few uploads on top of it. Could you please try out with
"3.10.0+ds2-3" please?
It works fine locally for me with the same.

Might take some time to propagate to the mirrors from the time of me writing this email
though :)

We believe that the bug you reported is fixed in the latest version of
singularity-container, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1012469@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Nilesh Patra <nilesh@debian.org> (supplier of updated singularity-container package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Fri, 17 Jun 2022 11:03:52 +0530
Source: singularity-container
Architecture: source
Version: 3.10.0+ds2-3
Distribution: unstable
Urgency: medium
Maintainer: Debian HPC Team <debian-hpc@lists.debian.org>
Changed-By: Nilesh Patra <nilesh@debian.org>
Closes: 1012469
Changes:
 singularity-container (3.10.0+ds2-3) unstable; urgency=medium
 .
   * Team upload.
   * Modify oci patch to return right imgspec
     (Closes: #1012469)
   * Add desc to patches
   * d/u/metadata: Make yamllint happy
Checksums-Sha1:
 1d23bd72281ce702d1f3b132ffd87f54c264ed5a 4480 singularity-container_3.10.0+ds2-3.dsc
 86b2d76edf9d4f343e45105298f4989ad9375a7e 18348 singularity-container_3.10.0+ds2-3.debian.tar.xz
 ad8e4e71c8406842dbef715324762250e5f0870e 26084 singularity-container_3.10.0+ds2-3_amd64.buildinfo
Checksums-Sha256:
 d061b74cbc9a07611275648652e7fb95c460bf42afe0aa057de0c84d52730c67 4480 singularity-container_3.10.0+ds2-3.dsc
 d00f0c29972cac1f03bc583db4bac6f84c835a3951cd2f777df7c896f0352315 18348 singularity-container_3.10.0+ds2-3.debian.tar.xz
 795cc249aa953a7185ebf77d7545292f66ff7ef0f9e43e731c198676c911a26b 26084 singularity-container_3.10.0+ds2-3_amd64.buildinfo
Files:
 47162c31437cfa1cca762b6d16d5120e 4480 admin optional singularity-container_3.10.0+ds2-3.dsc
 3572d91ce63380309af5de2a9da1c758 18348 admin optional singularity-container_3.10.0+ds2-3.debian.tar.xz
 9a9d7e932af53b4703979a6490e953bd 26084 admin optional singularity-container_3.10.0+ds2-3_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----

iQJGBAEBCgAwFiEEPpmlJvXcwMu/HO6mALrnSzQzafEFAmKsGGASHG5pbGVzaEBk
ZWJpYW4ub3JnAAoJEAC650s0M2nxXesQALLG875KSSt8Fe8U+ENM6REMgLmMTHRJ
cIkZaunUBz2PtiAG8wxYQVsSqkrYeOlu/yEL/QC6rD++tSmqFQUwnJUZLSmWgBHA
IA2wArX6/nhTAsVDf8xSLV/6oWXE9dVzMNNuEU+suMy+C4b8jooaUr/GbMy9EheA
NCkj4Ur6tTsefmXkTXrWEvp3CaM+vGDC9NpXOwFk8hvIaabgeWgYqLLKb20tvZHp
TLEaeINMH3+0m325e0+LcDa+HY3APGNUmJRe0assejuDFWeJX/1UpTVFoFjlpo/Y
fl9rnVdbSw8KATQPmPMgzxBNNSMfom/NtdDG+u90W0MtO7YOtH+xt/GjGSZ5OSii
yVXAuqAcjr1CvF+HIxDQ/lej3Qd4VK+Oiw/z5Ov35CoJqLxDilJUH8ZFNNU7hXmQ
RKznZon6YVQmuXVHLIs/wvNEyVQJAbTdLTtwpH131K/zM4uAegCVxa+Q8YsSYvuD
tUEk1L0bvzZpCW37JcA9a+i2hyinpkTT8zYba6VhY+W2ERvrM6jVD7DUzp25rsYG
hEVth8TU5blL/LzwoKCvZJSDltP9epgqQqP28vbuVq07KvTEb9u6jLbHZViaGqqR
/senLqh9GXqNenUdrT7WAIHHFhoPOjYm8kkAvzZAxSGKTNKeFhYrvhcKXiAUb5o/
5Fthb8ep8eP4
=ddAs
-----END PGP SIGNATURE-----

On Fri, 17 Jun 2022 11:30:46 +0530 Nilesh Patra <nilesh@debian.org> wrote:
 > I have done a few uploads on top of it. Could you please try out with
 > "3.10.0+ds2-3" please?
 > It works fine locally for me with the same.

Dear Nilesh,

With 3.10.0+ds2-3 the issue is indeed solved. Thanks!

Regards,
André