#1145118 adios4dolfinx: FTBFS: E ValueError: [Sat Aug 22 10:11:38 2026] [ADIOS2 EXCEPTION] <Core> <IOHDF5> <MakeEngineHDF5> : A serial HDF5 engine cannot be used with a communicator that is MPI-based.

Package:
src:adios4dolfinx
Source:
src:adios4dolfinx
Submitter:
Santiago Vila
Date:
2026-09-12 16:41:00 UTC
Severity:
normal
Tags:
#1145118#5
Date:
2026-08-22 23:24:58 UTC
From:
To:
Dear maintainer:

During a rebuild of all packages in unstable, this package failed to build.

Below you will find the last part of the build log (probably the most
relevant part, but not necessarily). If required, the full build log
is available here:

https://people.debian.org/~sanvila/build-logs/202608/

About the archive rebuild: The build was made on virtual machines from AWS,
using sbuild and a reduced chroot with only build-essential packages.

If you cannot reproduce the bug please contact me privately, as I
am willing to provide ssh access to a virtual machine where the bug is
fully reproducible.

If this is really a bug in one of the build-depends, please use
reassign and add an affects on src:adios4dolfinx, so that this is still
visible in the BTS web page for this package.

Thanks.
--------------------------------------------------------------------------------
[...]

=================================== FAILURES ===================================


/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/tests/test_mesh_writer.py:25:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/checkpointing.py:806: in write_mesh
    _internal_mesh_writer(
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/writers.py:45: in write_mesh
    with ADIOSFile(
/usr/lib/python3.14/contextlib.py:141: in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

adios = <adios2.bindings.adios2_bindings_mpi.ADIOS object at 0x7fecba2553b0>
filename = PosixPath('/tmp/pytest-of-sbuild/pytest-1/test_mesh_read_writer_True_Gho1/adios_mesh_HDF5_True.h5')
engine = 'HDF5', mode = Mode.Write, io_name = 'MeshWriter'
comm = <mpi4py.MPI.Intracomm object at 0x7fecb8d3b030>

    @contextmanager
    def ADIOSFile(
        adios: adios2.ADIOS,
        filename: Union[Path, str],
        engine: str,
        mode: adios2.Mode,
        io_name: str,
        comm: MPI.Intracomm | None = None,
    ):
        io = adios.DeclareIO(io_name)
        io.SetEngine(engine)
        # ADIOS2 sometimes struggles with existing files/folders it should overwrite
        if mode == adios2.Mode.Write:
            filename = Path(filename)
            if filename.exists() and comm is not None and comm.rank == 0:
                if filename.is_dir():
                    shutil.rmtree(filename)
                else:
                    filename.unlink()
            if comm is not None:
                comm.Barrier()
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/adios2_helpers.py:70: ValueError

/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/tests/test_mesh_writer.py:25:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/checkpointing.py:806: in write_mesh
    _internal_mesh_writer(
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/writers.py:45: in write_mesh
    with ADIOSFile(
/usr/lib/python3.14/contextlib.py:141: in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

adios = <adios2.bindings.adios2_bindings_mpi.ADIOS object at 0x7fecb896dcb0>
filename = PosixPath('/tmp/pytest-of-sbuild/pytest-1/test_mesh_read_writer_True_Gho4/adios_mesh_HDF5_True.h5')
engine = 'HDF5', mode = Mode.Write, io_name = 'MeshWriter'
comm = <mpi4py.MPI.Intracomm object at 0x7fecb896e100>

    @contextmanager
    def ADIOSFile(
        adios: adios2.ADIOS,
        filename: Union[Path, str],
        engine: str,
        mode: adios2.Mode,
        io_name: str,
        comm: MPI.Intracomm | None = None,
    ):
        io = adios.DeclareIO(io_name)
        io.SetEngine(engine)
        # ADIOS2 sometimes struggles with existing files/folders it should overwrite
        if mode == adios2.Mode.Write:
            filename = Path(filename)
            if filename.exists() and comm is not None and comm.rank == 0:
                if filename.is_dir():
                    shutil.rmtree(filename)
                else:
                    filename.unlink()
            if comm is not None:
                comm.Barrier()
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/adios2_helpers.py:70: ValueError

/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/tests/test_mesh_writer.py:25:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/checkpointing.py:806: in write_mesh
    _internal_mesh_writer(
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/writers.py:45: in write_mesh
    with ADIOSFile(
/usr/lib/python3.14/contextlib.py:141: in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

adios = <adios2.bindings.adios2_bindings_mpi.ADIOS object at 0x7fecb8931f80>
filename = PosixPath('/tmp/pytest-of-sbuild/pytest-1/test_mesh_read_writer_False_Gh1/adios_mesh_HDF5_False.h5')
engine = 'HDF5', mode = Mode.Write, io_name = 'MeshWriter'
comm = <mpi4py.MPI.Intracomm object at 0x7fecba3f57a0>

    @contextmanager
    def ADIOSFile(
        adios: adios2.ADIOS,
        filename: Union[Path, str],
        engine: str,
        mode: adios2.Mode,
        io_name: str,
        comm: MPI.Intracomm | None = None,
    ):
        io = adios.DeclareIO(io_name)
        io.SetEngine(engine)
        # ADIOS2 sometimes struggles with existing files/folders it should overwrite
        if mode == adios2.Mode.Write:
            filename = Path(filename)
            if filename.exists() and comm is not None and comm.rank == 0:
                if filename.is_dir():
                    shutil.rmtree(filename)
                else:
                    filename.unlink()
            if comm is not None:
                comm.Barrier()
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/adios2_helpers.py:70: ValueError

/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/tests/test_mesh_writer.py:25:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/checkpointing.py:806: in write_mesh
    _internal_mesh_writer(
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/writers.py:45: in write_mesh
    with ADIOSFile(
/usr/lib/python3.14/contextlib.py:141: in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

adios = <adios2.bindings.adios2_bindings_mpi.ADIOS object at 0x7fecb896dec0>
filename = PosixPath('/tmp/pytest-of-sbuild/pytest-1/test_mesh_read_writer_False_Gh4/adios_mesh_HDF5_False.h5')
engine = 'HDF5', mode = Mode.Write, io_name = 'MeshWriter'
comm = <mpi4py.MPI.Intracomm object at 0x7fecb896d410>

    @contextmanager
    def ADIOSFile(
        adios: adios2.ADIOS,
        filename: Union[Path, str],
        engine: str,
        mode: adios2.Mode,
        io_name: str,
        comm: MPI.Intracomm | None = None,
    ):
        io = adios.DeclareIO(io_name)
        io.SetEngine(engine)
        # ADIOS2 sometimes struggles with existing files/folders it should overwrite
        if mode == adios2.Mode.Write:
            filename = Path(filename)
            if filename.exists() and comm is not None and comm.rank == 0:
                if filename.is_dir():
                    shutil.rmtree(filename)
                else:
                    filename.unlink()
            if comm is not None:
                comm.Barrier()
/<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build/adios4dolfinx/adios2_helpers.py:70: ValueError
=========================== short test summary info ============================
FAILED tests/test_mesh_writer.py::test_mesh_read_writer[True-GhostMode.shared_facet-HDF5-.h5]
FAILED tests/test_mesh_writer.py::test_mesh_read_writer[True-GhostMode.none-HDF5-.h5]
FAILED tests/test_mesh_writer.py::test_mesh_read_writer[False-GhostMode.shared_facet-HDF5-.h5]
FAILED tests/test_mesh_writer.py::test_mesh_read_writer[False-GhostMode.none-HDF5-.h5]
========== 4 failed, 998 passed, 5 skipped, 180 deselected in 16.84s ===========
E: pybuild pybuild:485: test: plugin pyproject failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_adios4dolfinx/build; python3.14 -m pytest -k "not test_original_checkpoint"
dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p "3.13 3.14" --parallel=2 returned exit code 13
make: *** [debian/rules:13: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess failed with exit status 2
--------------------------------------------------------------------------------
#1145118#10
Date:
2026-08-31 09:22:49 UTC
From:
To:
This error makes no sense.  It's saying that adios2 is using serial
HDF5, when MPI parallel HDF5 is needed.  That bit makes sense.
What does not make sense is that it is
adios2.bindings.adios2_bindings_mpi.ADIOS reporting this error,
i.e. the MPI build is saying it's using the serial engine.

Looks like something's gone wrong in adios2.
Perhaps they're not properly updated for HDF5 v2.
#1145118#15
Date:
2026-08-31 09:33:07 UTC
From:
To:
doing anything in the package itself, which means the problem was
really in some build-dependency.

I'm retesting this one right now, and will close it if it works today.

(Normally I like to use debbisect to reassign the bug where it belongs,
but there are too many affected packages for that).

Thanks.
#1145118#20
Date:
2026-08-31 09:39:12 UTC
From:
To:
this one was not in the set of packages that fixed themselves.

Thanks.
#1145118#25
Date:
2026-09-03 17:34:44 UTC
From:
To:
Hi,
I believe that the issue is somewhere in adios2 and/or hdf5, not here.



In a clean container with only
# apt install openmpi-bin python3-adios2-mpi python3-mpi4py



use the following test

$ cat test.py
from mpi4py import MPI
import adios2.bindings as adios2

comm = MPI.COMM_WORLD

adios = adios2.ADIOS(comm)
io = adios.DeclareIO("TestIO")
io.SetEngine("HDF5")

engine = io.Open("test.h5", adios2.Mode.Write)
engine.Close()




$ mpirun -n 2 python3 test.py
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/test.py", line 10, in <module>
    engine = io.Open("test.h5", adios2.Mode.Write)
ValueError: [Thu Sep 03 17:01:51 2026] [ADIOS2 EXCEPTION] <Core> <IOHDF5> <MakeEngineHDF5> : A serial HDF5 engine cannot be used with a communicator that is MPI-based.

  File "/home/test.py", line 10, in <module>
    engine = io.Open("test.h5", adios2.Mode.Write)
ValueError: [Thu Sep 03 17:01:51 2026] [ADIOS2 EXCEPTION] <Core> <IOHDF5> <MakeEngineHDF5> : A serial HDF5 engine cannot be used with a communicator that is MPI-based.





Cheers,
Francesco
#1145118#30
Date:
2026-09-12 12:48:55 UTC
From:
To:
Hi,

I think that what may have happened here is that the failure that
Santiago identified in his first message on August 22nd did actually go
away with the rebuild, but then a second, unrelated failure started
happening so that the rebuild on August 30th also failed.  Santiago,
please correct me if that's not the case and you actually verified that
the failure reason was the same.

The reason I'm saying this is that adios4dolfinx is failing to build in
the reproducible builds infrastructure [1], but the log looks very
different. There are tons of errors, one example:

     def test_read_write_P_2D(
         read_comm, family, degree, is_complex, mesh_2D, get_dtype,
write_function, read_function
     ):
         mesh = mesh_2D
         f_dtype = get_dtype(mesh.geometry.x.dtype, is_complex)

         el = basix.ufl.element(
             family,
Looking at the list of tests done by the reproducible builds
infrastructure [2], we can see this successes and failures.

For experimental:
2026-08-16 06:17:00 	0.11.0-1exp1 	experimental 	amd64 	FTBFS
2026-08-05 02:18:00 	0.11.0-1exp1 	experimental 	amd64 	reproducible

For unstable:
2026-08-30 02:43:00 	0.10.0.post1-3 	unstable 	amd64 	FTBFS
2026-07-31 03:44:00 	0.10.0.post1-3 	unstable 	arm64 	reproducible

For forky:
2026-08-31 13:39:00 	0.10.0.post1-3 	forky 	amd64 	FTBFS
2026-08-03 17:18:00 	0.10.0.post1-3 	forky 	arm64 	reproducible

So, we don't know exactly when things broke, but some time in August
there must have been a change that caused adios4dolfinx to stop
building.

I suspected that this might be caused by fenics-dolphinx uploads, as the
new 0.11 version was being uploaded to experimental during the window
and to unstable also inside that window, but the failure in forky seems
to disprove that, as the last migration to testing for fenics-dolphinx
happened on 2026-07-29. Unless that failure is due to a different thing?

Indeed, the error seen forky (on August 31st) is the error that titles
this bug [3]. So, perhaps it is the fact that the new fenics-dolphinx
breaks adios4dolfinx, which requires a new upload, while something else
broke something related to HDF5 temporarily, and while that's gone from
unstable now, it's still happening in forky.

Regards,
Marga

[1]:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/adios4dolfinx.html
[2]:
https://tests.reproducible-builds.org/debian/history/adios4dolfinx.html
[3]:
https://tests.reproducible-builds.org/debian/rb-pkg/forky/amd64/adios4dolfinx.html
#1145118#33
Date:
2026-09-12 12:48:55 UTC
From:
To:
Hi,

I think that what may have happened here is that the failure that
Santiago identified in his first message on August 22nd did actually go
away with the rebuild, but then a second, unrelated failure started
happening so that the rebuild on August 30th also failed.  Santiago,
please correct me if that's not the case and you actually verified that
the failure reason was the same.

The reason I'm saying this is that adios4dolfinx is failing to build in
the reproducible builds infrastructure [1], but the log looks very
different. There are tons of errors, one example:

     def test_read_write_P_2D(
         read_comm, family, degree, is_complex, mesh_2D, get_dtype,
write_function, read_function
     ):
         mesh = mesh_2D
         f_dtype = get_dtype(mesh.geometry.x.dtype, is_complex)

         el = basix.ufl.element(
             family,
Looking at the list of tests done by the reproducible builds
infrastructure [2], we can see this successes and failures.

For experimental:
2026-08-16 06:17:00 	0.11.0-1exp1 	experimental 	amd64 	FTBFS
2026-08-05 02:18:00 	0.11.0-1exp1 	experimental 	amd64 	reproducible

For unstable:
2026-08-30 02:43:00 	0.10.0.post1-3 	unstable 	amd64 	FTBFS
2026-07-31 03:44:00 	0.10.0.post1-3 	unstable 	arm64 	reproducible

For forky:
2026-08-31 13:39:00 	0.10.0.post1-3 	forky 	amd64 	FTBFS
2026-08-03 17:18:00 	0.10.0.post1-3 	forky 	arm64 	reproducible

So, we don't know exactly when things broke, but some time in August
there must have been a change that caused adios4dolfinx to stop
building.

I suspected that this might be caused by fenics-dolphinx uploads, as the
new 0.11 version was being uploaded to experimental during the window
and to unstable also inside that window, but the failure in forky seems
to disprove that, as the last migration to testing for fenics-dolphinx
happened on 2026-07-29. Unless that failure is due to a different thing?

Indeed, the error seen forky (on August 31st) is the error that titles
this bug [3]. So, perhaps it is the fact that the new fenics-dolphinx
breaks adios4dolfinx, which requires a new upload, while something else
broke something related to HDF5 temporarily, and while that's gone from
unstable now, it's still happening in forky.

Regards,
Marga

[1]:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/adios4dolfinx.html
[2]:
https://tests.reproducible-builds.org/debian/history/adios4dolfinx.html
[3]:
https://tests.reproducible-builds.org/debian/rb-pkg/forky/amd64/adios4dolfinx.html
#1145118#38
Date:
2026-09-12 13:18:02 UTC
From:
To:
I confirm that the package fails to build for a different reason now.

I've put a recent build log in the same directory as before:

https://people.debian.org/~sanvila/build-logs/202608/

Sorry, I did not verify that the failure reason was the same (this is
why I just said "it still FTBFS").

In cases like this one there are two options. Option A is to close the
bug and open a new one. Option B is to retitle the bug to the new mode
of failure. I prefer option B but will not object if you decide to go
for option A.

Thanks.
#1145118#43
Date:
2026-09-12 13:27:44 UTC
From:
To:
packages that fixed themselves". I just meant that the package did not
magically start building ok again, but in retrospective I see that
saying that could have been a little bit misleading (sorry).

The reason for the original failure (the hdf5 thing) was probably
fixed (in another package).

Thanks.
#1145118#48
Date:
2026-09-12 14:54:53 UTC
From:
To:
Hi,
I just uploaded 0.11.0-2 to unstable, which should fix the incompatibility with
the upgrade fenics-dolfinx 0.11.0.

The error

ValueError: [Sat Aug 22 10:11:38 2026] [ADIOS2 EXCEPTION] <Core> <IOHDF5> <MakeEngineHDF5> : A serial HDF5 engine cannot be used with a communicator that is MPI-based

will remain, so the build will not actually succeed, but at least it is "less"
broken now.

As I mentioned in a previous email (with a minimal reproducible example), the
above error is not due to this package, and there is nothing we can do from
here (or at least, nothing I can see for myself).

Thanks for the patience and the reporting,
Francesco
#1145118#53
Date:
2026-09-12 16:39:40 UTC
From:
To:
Hi,

Right, those were the same errors that were seen in the Python 3.15
rebuild in debusine.

However, I've noticed that Francesco has uploaded a new version of
adios4dolfinx today and it failed to build with the same error as
reported in this bug:

https://buildd.debian.org/status/fetch.php?pkg=adios4dolfinx&arch=all&ver=0.11.0-2&stamp=1789226652&raw=0

     @contextmanager
     def ADIOSFile(
         adios: adios2.ADIOS,
         filename: Union[Path, str],
         engine: str,
         mode: adios2.Mode,
         io_name: str,
         comm: MPI.Intracomm | None = None,
     ):
         io = adios.DeclareIO(io_name)
         io.SetEngine(engine)
         # ADIOS2 sometimes struggles with existing files/folders it
should overwrite
         if mode == adios2.Mode.Write:
             filename = Path(filename)
             if filename.exists() and comm is not None and comm.rank ==
0:
                 if filename.is_dir():
                     shutil.rmtree(filename)
                 else:
                     filename.unlink()
             if comm is not None:
                 comm.Barrier()

So, it seems that this error is still present, only somewhat flaky?