#1126440 libsoup-3.0-tests: not reproducible according to reproduce.debian.net

#1126440#5
Date:
2026-01-26 13:19:23 UTC
From:
To:
libsoup-3.0-tests is not reproducible according to
reproduce.debian.net. The Debian Release Team have said that failures
there will block migration to Testing, likely later this year.

This failure can be seen with Salsa CI as the default pipeline now has
a debrebuild task.
https://salsa.debian.org/gnome-team/libsoup3/-/merge_requests/5

From looking at the diffoscope output and at meson.build, it looked
like the have_apache checks were failing so the build doesn't run some
tests or install those test files in the -tests binary package.

I was also able to reproduce it locally, which is good since the logs
at Salsa CI and at reproduce.debian.net were not verbose enough.

Approximate steps to reproduce:
1. sudo apt install equivs devscripts sbuild
2. Download the libsoup3 source. I like using pull-debian-source from
ubuntu-dev-tools (which can be installed with
--no-install-recommends).
3. Download the corresponding buildinfo file from
https://buildinfos.debian.net/buildinfo-pool/libs/libsoup3/
4. cd libsoup3-3.6.5/
5. debrebuild --builder=sbuild+unshare ../libsoup3_3.6.5-7_amd64.buildinfo

debrebuild output excerpt
===================
Program apachectl found: YES (/usr/sbin/apachectl)
Program apache2 found: YES (/usr/sbin/apache2)
Message: Failed to locate necessary Apache modules for full test coverage
Message: stdout:
Message: stderr: [core:warn] [pid 260:tid 260] AH00111: Config
variable ${APACHE_RUN_DIR} is not defined
apache2: Syntax error on line 80 of /etc/apache2/apache2.conf:
DefaultRuntimeDir must be a valid directory, absolute or relative to
ServerRoot
Something went wrong when calling /usr/sbin/apache2!

…

    Tests requiring Apache: false

comparable output from buildd.debian.org or from running sbuild+unshare locally
=========================
Program apachectl found: YES (/usr/sbin/apachectl)
Program apache2 found: YES (/usr/sbin/apache2)
Message: Apache modules directory: /usr/lib/apache2/modules
Message: Apache SSL module directory: /usr/lib/apache2/modules
Message: Apache mod_unixd module directory: (none)
Message: Apache HTTP/2 module directory: /usr/lib/apache2/modules

…

    Tests requiring Apache: true


At this point, I don't know why the debrebuild environment is
different than the regular sbuild+unshare environment.

Thank you,
Jeremy Bícha

#1126440#10
Date:
2026-01-26 17:07:54 UTC
From:
To:
I forgot to include this link:

https://reproduce.debian.net/excuses.html#libsoup-3.0-tests

Thank you,
Jeremy Bícha

#1126440#25
Date:
2026-10-25 07:38:33 UTC
From:
To:
Hi Jeremy,

* Jeremy Bícha <jbicha@debian.org> [2026-10-23 18:57]:

The problem is that apache2 depends on /run/apache2 being present. You
can get the same result with:

sbuild -d unstable libsoup --starting-build-commands='rm -r /run/apache2'

The list is here:

https://release.debian.org/reproducible-builds-host-requirements.html

And it states that builds should not rely on the content of /run.

There is already a similar bug for gnome-settings-daemon with more
discussion:

https://bugs.debian.org/1150166

Cheers Jochen

#1126440#30
Date:
2026-10-26 22:09:34 UTC
From:
To:
So, I don't have a soluation, but figured I would share what I found...

I was able to reproduce the error message from the build log with this:

  ./get_apache_modules_dirs.py /usr/sbin/apache2
  [Mon Oct 26 21:54:09.862824 2026] [core:warn] [pid 57:tid 57] AH00111: Config variable ${APACHE_RUN_DIR} is not defined
  apache2: Syntax error on line 80 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot
  Something went wrong when calling /usr/sbin/apache2!

I suspect it detects that apachectl is present, but it cannot be run:

  ./get_apache_modules_dirs.py /usr/sbin/apachectl
  mkdir: cannot create directory ‘/run/apache2’: Permission denied
  Something went wrong when calling /usr/sbin/apachectl!

So meson.build follows the codepath that leads to the original error in
the build log, by first trying apachectl and then falling back to
calling apache2 directly:

  if have_apache
    apache_modules_dirs_out = run_command('get_apache_modules_dirs.py', apachectl.full_path(), check: false)
    have_apache = (apache_modules_dirs_out.returncode() == 0)
    # Same as above, using apachectl might fail, try apache directly.
    if not have_apache
      apache_modules_dirs_out = run_command('get_apache_modules_dirs.py', apache_httpd2.full_path(), check: false)
      have_apache = (apache_modules_dirs_out.returncode() == 0)
    endif

/usr/sbin/apachectl contains these calls:

  [ ! -d ${APACHE_RUN_DIR:-/run/apache2}/socks ] && mkdir_chown ${APACHE_RUN_USER:-www-data} ${APACHE_RUN_DIR:-/run/apache2}/socks
  [ ! -d ${APACHE_LOCK_DIR:-/run/lock/apache2} ] && mkdir_chown ${APACHE_RUN_USER:-www-data} ${APACHE_LOCK_DIR:-/run/lock/apache2}

Which would fail for a non-root user... so the attempt to call apachectl
is always going to fail if /run/apache2 is not already set up and
owned by www-data.

And calling apache2 (a.k.a. httpd) directly without setting up the
environment variables is also going to fail...

I vaguely was aware of some quirks with /run, but nice to have that
specific list, thanks!

I do not see any more discussion there, just the initial report.


live well,
  vagrant