#1102465 petsc: PETSc was configured with one MPICH/Open MPI mpi.h version but now appears to be compiling using an older version

#1102465#5
Date:
2025-04-09 09:15:43 UTC
From:
To:
https://tracker.debian.org/pkg/petsc

Issues preventing migration:
∙ ∙ autopkgtest for dolfin/2019.2.0~legacy20240219.1c52e83-18: amd64: Pass, arm64: Pass, armel: Regression or new test ♻ (reference ♻), armhf: Regression or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻), ppc64el: Pass, riscv64: Test in progress (will not be considered a regression), s390x: Pass
∙ ∙ autopkgtest for fenics-dolfinx/1:0.9.0-6: amd64: Pass, arm64: Pass, armel: Regression or new test ♻ (reference ♻), armhf: Regression or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻), ppc64el: Test in progress (will not be considered a regression), riscv64: Test in progress (will not be considered a regression), s390x: Pass
∙ ∙ autopkgtest for fenicsx-performance-tests/0.9.0-2: amd64: Pass, arm64: Pass, armel: Regression or new test ♻ (reference ♻), armhf: Regression or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻), ppc64el: Pass, riscv64: Pass, s390x: Pass
∙ ∙ autopkgtest for petsc/3.22.5+dfsg1-1: amd64: Pass, arm64: Pass, armel: Regression or new test ♻ (reference ♻), armhf: Regression or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻), ppc64el: Pass, riscv64: Pass, s390x: Pass
∙ ∙ autopkgtest for slepc/3.22.2+dfsg1-1: amd64: Pass, arm64: Pass, armel: Regression or new test ♻ (reference ♻), armhf: Regression or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻), ppc64el: Pass, riscv64: Pass, s390x: Pass
∙ ∙ autopkgtest for sundials/7.1.1+dfsg1-8: amd64: Pass, arm64: Pass, armel: Regression or new test ♻ (reference ♻), armhf: Regression or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻), ppc64el: Pass, riscv64: Pass, s390x: Pass



This is due to:
https://sources.debian.org/src/petsc/3.22.5%2Bdfsg1-1/include/petscsys.h/#L96
https://sources.debian.org/src/petsc/3.22.5%2Bdfsg1-1/include/petscsys.h/#L104


Assuming the dependencies generated by the MPICH and Open MPI
packages are correct, PETSc should drop these #errors.

#1102465#10
Date:
2025-04-09 13:49:03 UTC
From:
To:
Control: block -1 by 1101686
...

That's the point.  The mpich package is not in good order.  Bug#1101686.

#1102465#17
Date:
2025-04-09 13:49:03 UTC
From:
To:
Control: block -1 by 1101686
...

That's the point.  The mpich package is not in good order.  Bug#1101686.

#1102465#22
Date:
2025-04-11 10:23:45 UTC
From:
To:
Thinking about it further, I don't think we have a bug here as such,
and I'm therefore lowering severity.

I don't think any action is needed either. The latest petsc build requires
the latest mpich (on 32-bit arches). But the latest mpich has not yet
migrated to testing.  petsc will pass fine once mpich migrates
(petsc is not blocking mpich).

Two alternative actions could be taken.

1) We could remove upstream's mpich version check, as you suggested.
I'm reluctant to take this step, however. Upstream is careful with
their code, I imagine they put the version check there for a reason.

2) We could add an mpich versioned depends to the petsc packages, so
when a new petsc build is tested in testing, it pulls any new mpich
version with it.  But this would not actually achieve anything. It
would give a clean migration test, but end up "ready to migrate,
waiting for mpich". Which is the state it's currently in, anyway.
So this option would significantly complicate build scripts
(significantly, because only the 32-bit arches use mpich, so complex
logic would need to be added to debian/rules to differentiate the
different mpi-default conditions), and would provide no functional gain.

So I think for this bug it's just patience that is required,
and perhaps assistance with mpich issues (Bug#1102612 is weird).

#1102465#31
Date:
2025-04-12 05:27:48 UTC
From:
To:
Hi Drew,


But in Debian as you explain below, it doesn't really serve a purpose as
we're a distribution that ships things together. But ...


It does. It shows to all people inspecting the situation exactly what's
going on. It would ensure that the migration software triggers the right
combinations. The other way around, it might also prevent mpich from
migrating too early in the future (with an upper bound, I haven't
checked if that's the case here). It would also prevent people from
doing partial upgrades (yes, not officially supported yet but we're
getting better at it since autopkgtest catch so many of those and a lot
of maintainers add versioned Depends). It would prevent running those
tests (several are quite long running) over and over again (we retry
after a day) because a passing test is remembered for a long time.


This is a valid argument too, I don't deny that at all. (Although, you
don't have to make it complex and could add things manually (now). I
agree that's probably a PITA.)


But this I disagree with. Why would we even have invented versioned
Depends in the first place if people agreed this were true?

I'm not saying the petsc package should have this logic, I leave that up
to you, but I do think there's more value in it than you seem to see.

So, coming back to point 1, I think that version testing in Debian
usually is wrong. The best case is when upstream gets it right. The
great thing is we have versioned Depends to express that at install
time. Worse case upstream is wrong, usually too tight. We see that *a
lot* and the version checks make it harder for packages to migrate as
they need packages to be upgraded in lock step. Regularly because the
migration doesn't see *why* the tests fail, it doesn't know there's a
solution by testing together and migrating together if not expressed in
relations. And then even upstream may have overseen a problem that we
only catch while testing in Debian. Again, a versioned dependency can
express what we find during unstable-to-testing integration.

Paul

#1102465#36
Date:
2025-04-15 19:21:16 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
petsc, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1102465@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Drew Parsons <dparsons@debian.org> (supplier of updated petsc package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Tue, 15 Apr 2025 19:07:42 +0200
Source: petsc
Architecture: source
Version: 3.22.5+dfsg1-2
Distribution: unstable
Urgency: medium
Maintainer: Debian Science Maintainers <debian-science-maintainers@lists.alioth.debian.org>
Changed-By: Drew Parsons <dparsons@debian.org>
Closes: 1102465
Changes:
 petsc (3.22.5+dfsg1-2) unstable; urgency=medium
 .
   * dev packages: complement Depends: mpi-default-dev with
     ${mpi-dev:Depends} as lower-versioned dependency on specific mpi
     implementation (openmpi or mpich), since petscsys.h checks that
     the current MPI version is compatible with the version petsc was
     compiled against. Will help avoid testing migration confusion when
     MPI packages are updated. Closes: #1102465.
Checksums-Sha1:
 f9092da9dff58077e14c3998c387cfce3192f031 4523 petsc_3.22.5+dfsg1-2.dsc
 5080ecd5ef20e8d1b19a1ea2c638bbf6cf1a1dd5 114196 petsc_3.22.5+dfsg1-2.debian.tar.xz
Checksums-Sha256:
 a21e4e1ad998ca9344bc506693693d71481ff3454c2c3fd9e4a27bee28586427 4523 petsc_3.22.5+dfsg1-2.dsc
 549137edd7515e1d679576c59afda70b0c7b370ba4bc120932396b5e67d20601 114196 petsc_3.22.5+dfsg1-2.debian.tar.xz
Files:
 b3cd8425fb3e391b6ab3585772eabe3b 4523 devel optional petsc_3.22.5+dfsg1-2.dsc
 2ab7658ba2b6043ee19cbe76ce64f8c8 114196 devel optional petsc_3.22.5+dfsg1-2.debian.tar.xz
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEI8mpPlhYGekSbQo2Vz7x5L1aAfoFAmf+qyoACgkQVz7x5L1a
AfqpwA/+J0FnHG9f8fq1aSBDTU3e6fJaO/FLpTI1ni9RaGZKWpF8ZBMQCUwhW2+b
LMEsjyCXaC9bdEYOrbeKDG5xzaVKHyFJRgu4UAkO7kMqipchnSUeowU92Y0KoXcl
z4qjNQEeSpyKlxMmfIUSbsxyLHQ3GXYPqaFcVRIh6oWZ+J8NsofAZ5LlPN3PqLeG
5/jr1KqPhIj+X9Kmru/p+PPNU5O+yi8woix3SieaFMntvtMVbWVXMFNze9PTJud2
gJOXtAD23IZuylusaRh9IX7w7E96UHkdehG+C1WgorDdVhV7jb1W2jlzyeu3dqHH
F7aPjS/CQIUol8v+VoT8nrGgEJzrflbE73XcGqDAxAjUV/IwhQhLP3IM/kBu05LE
LRlUZMz1Fh5Jl8eu96W/gFFfok/Xr9MDZ8r8lKOTemzE3xrmH0vjyr5WO4lu4/Ld
UdMLnaOEK0+V9GBOB//LuJlquhXRxZULcMuVywE9bYyCtbaAIzPD4uJNFGgDNIM7
H+J/rVx0+ZyexPvYoxrjmimxbL7r0TtIqifO2jXrgiW8/pipE5UMJWoNjJ07kM+F
lR6jUicv1GFgXU0ESFFm2jgs5icoxUBoq4YHa/4QOg/hlYAWQDwUm+dxFMBWRlaj
RhlOB1zqIqaXXVdZMYYPn55piN877tttPSZcuQnWOenieVZZ/NQ=
=J2Zr
-----END PGP SIGNATURE-----

#1102465#53
Date:
2026-03-19 01:13:25 UTC
From:
To:
As pointed out already in this bug, this warning is not a bug in PETSc.
It is working entirely as intended.

No information was given with this bug reopening.

I presume it refers to the 32-bit build failures of reverse
dependencies, which is happening due to the upgrade of mpich from v4 to v5.

petsc (and all other MPI packages) needs to be rebuilt against
the new mpich (on 32-bit arches)
(the mpich upgrade needs a transition bug).

I will close this bug again if no justification for reopening is given.

Drew

#1102465#58
Date:
2026-03-19 10:30:34 UTC
From:
To:
In any case, there is a new upstream release waiting for package.
The build for the new version will update the package for mpich v5.

#1102465#63
Date:
2026-03-19 10:47:40 UTC
From:
To:
but you should have received the personal copy of it:

Date: Wed, 18 Mar 2026 13:43:22 +0200
From: Adrian Bunk <bunk@debian.org>
To: 1102465@bugs.debian.org, Drew Parsons <dparsons@debian.org>
Subject: Re: Bug#1102465 closed by Debian FTP Masters <ftpmaster@ftp-master.debian.org> (reply to Drew Parsons <dparsons@debian.org>) (Bug#1102465: fixed
        in petsc 3.22.5+dfsg1-2)
Message-ID: <abqP2P5pnH-j_39O@localhost>

The bug is still present in 3.24.4+dfsg1-1 (see i386):

https://tracker.debian.org/pkg/mpich

Issues preventing migration:
∙ ∙ Autopkgtest for dolfin/2019.2.0~legacy20240219.1c52e83-27: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Failed (not a
+regression) ♻ (reference ♻), s390x: Pass
∙ ∙ Autopkgtest for fenics-dolfinx/1:0.10.0.post5-7: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Test triggered (failure will be
+ignored), s390x: Pass
∙ ∙ Autopkgtest for fenicsx-performance-tests/0.10.0-2: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Pass, s390x: Pass
∙ ∙ Autopkgtest for liggghts/3.8.0+repack1-14: amd64: Pass, arm64: Failed (not a regression) ♻ (reference ♻), i386: Pass, ppc64el: Failed (not a regression)
+♻ (reference ♻), s390x: Regression ♻ (reference ♻)
∙ ∙ Autopkgtest for mpi4py/4.1.1-1: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Pass, s390x: Pass
∙ ∙ Autopkgtest for mpich/5.0.0-3: amd64: Pass, arm64: Pass, i386: Pass, ppc64el: Pass, s390x: Pass
∙ ∙ Autopkgtest for nwchem/7.3.1-1: amd64: Pass, arm64: Pass, i386: Pass, ppc64el: Test triggered (failure will be ignored), s390x: Pass
∙ ∙ Autopkgtest for petsc/3.24.4+dfsg1-1: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Pass, s390x: Pass
∙ ∙ Autopkgtest for python-parsl/2026.02.23+ds-1: amd64: Pass, arm64: Pass, i386: Pass, ppc64el: Pass, s390x: Reference test triggered, but real test failed
+already ♻
∙ ∙ Autopkgtest for slepc/3.24.2+dfsg1-1: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Pass, s390x: Pass
∙ ∙ Autopkgtest for sundials/7.1.1+dfsg1-10: amd64: Pass, arm64: Pass, i386: Regression ♻ (reference ♻), ppc64el: Pass, s390x: Pass
∙ ∙ Autopkgtest for vtk9/9.5.2+dfsg4-1: amd64: Failed (not a regression) ♻ (reference ♻), arm64: Failed (not a regression) ♻ (reference ♻), i386: Pass,
+ppc64el: Regression ♻ (reference ♻), s390x: Regression ♻ (reference ♻)

Please remove the bogus version check from include/petscsys.h.

Yes.
rename) to ensure that packages built against v4 won't use v5.

One of the following claims is incorrect:
- mpich claims v5 is compatible with v4
- petsc claims v4 and v5 are incompatible

cu
Adrian

#1102465#68
Date:
2026-03-19 11:49:23 UTC
From:
To:
...
...
...

Thanks for the extra detail

I think part of the situation comes from the historical development of
the MPI implementations,
and from PETSc trying to manage bug reports coming from different MPI
libraries and their different versions.

True, mpich is still provided by libmpich12, so it's not a question of
transitioning libmpich4 to libmpich5 (I had forgotten that point).
So there is some ABI compatibility. It is possible that PETSc is being
over-sensitive due to problems that had been arising in the past but are
no longer a great problem.

The question of ABI compatibility will get even more complex, or simpler
depending on your perspective, in the future when the MPI
implementations move to MPI standard 5, which will provide a common
stable ABI that will enable swap-out between OpenMPI and MPI (and other
implementations).  I say "future", but actually that's what mpich v5 is
already, it's supporting the new MPI-5 standard.

The apparent compatibility that we're seeing between mpich v4 and v5
might be part of that move to a more stable interface, which is a new
development. Once the MPI-5 standard and its common ABI is fully
implemented and operating (i.e. once users are routinely using openmpi
v5 or mpich v5), I can expect PETSc upstream will be able reevaluate and
relax their version tests (at the moment their attention is on ensuring
past library versions continue to work, with, as an example of the types
of bugs they have to deal with, nvidia's cuda fortran compiler still not
supporting an 8-year-old fortran standard [1]).

In the short term I think the simplest thing right now for petsc is to
just rebuild for the new version 3.24.5, which will reset the 32-bit
builds to mpich v5.

Beyond that, perhaps both the openmpi and mpich packages will want to be
reconfigured so they can equally and alternatively be installed as the
preferred MPI on any system. The situation would then probably be
similar to what we do with BLAS/LAPACK and the various alternative
optimised BLAS implementations.

Drew

[1] true story.  https://gitlab.com/petsc/petsc/-/work_items/1879

#1102465#73
Date:
2026-03-19 13:06:54 UTC
From:
To:
There is (supposed to be) no transition; MPI standard v5 designates a
standard ABI, but no change in the API.

I will need to dig into PETSc to understand how it sees two versions.

MPICH provides a new libmpi_abi.so that supplies the new ABI, so as to
not break libmpich.so.12

I will need to see if this is specified in v5, or what OpenMPI is
expected to do.
  Yes, work will be needed in both to make them intercompatible.

Alastair

#1102465#78
Date:
2026-03-19 15:23:31 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
petsc, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1102465@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Drew Parsons <dparsons@debian.org> (supplier of updated petsc package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Thu, 19 Mar 2026 11:50:34 +0100
Source: petsc
Architecture: source
Version: 3.24.5+dfsg1-1
Distribution: unstable
Urgency: medium
Maintainer: Debian Science Maintainers <debian-science-maintainers@lists.alioth.debian.org>
Changed-By: Drew Parsons <dparsons@debian.org>
Closes: 1102465
Changes:
 petsc (3.24.5+dfsg1-1) unstable; urgency=medium
 .
   * New upstream release
   * fresh build updates 32-bit arches for mpich v5. Closes: #1102465.
Checksums-Sha1:
 947e51675dde7908a307834e4c921a9bd735af7b 4591 petsc_3.24.5+dfsg1-1.dsc
 fc01c65c1731d9f63375e3f01169ab0ca7fb25f8 39778860 petsc_3.24.5+dfsg1.orig.tar.xz
 671a9c50ef63eec3c6c8cc2aac493903095a34a0 111148 petsc_3.24.5+dfsg1-1.debian.tar.xz
 be64b395b703d001b1ea1a204cd73fceb2ee1073 16938 petsc_3.24.5+dfsg1-1_source.buildinfo
Checksums-Sha256:
 4850b4923c3b49d7c0ee353208355b1f94fdda04e431335478fd290c43d908a0 4591 petsc_3.24.5+dfsg1-1.dsc
 50cf65ec160c106e69191460093f89394cd0616806564cf44d9ba137abafcd9c 39778860 petsc_3.24.5+dfsg1.orig.tar.xz
 1f7a3ec2124b940b2ee0c3cd279e57af27d08a3ed98160af20570c70b8050ebf 111148 petsc_3.24.5+dfsg1-1.debian.tar.xz
 df062284df817036ee828597fb1b9c5e052f596b66d704b9fc9a6b183d7371e7 16938 petsc_3.24.5+dfsg1-1_source.buildinfo
Files:
 480123df40dc38a79aeed86801aff740 4591 devel optional petsc_3.24.5+dfsg1-1.dsc
 cefd7b1831d24d25ffb365a65af5efe5 39778860 devel optional petsc_3.24.5+dfsg1.orig.tar.xz
 3aae24db7d56db25011d758ad57a7d56 111148 devel optional petsc_3.24.5+dfsg1-1.debian.tar.xz
 4b8457b1d04a63082d1427d0fafae899 16938 devel optional petsc_3.24.5+dfsg1-1_source.buildinfo
-----BEGIN PGP SIGNATURE-----

iQJIBAEBCgAyFiEEI8mpPlhYGekSbQo2Vz7x5L1aAfoFAmm8BzEUHGRwYXJzb25z
QGRlYmlhbi5vcmcACgkQVz7x5L1aAfouLA//XVj/aYjyGJbCfQEPUURvtqThWQ//
JmDHRM3O7PcWZPXDFxJ5J8NMDHEr/KyGqr1MquSK2fy4tODDgfV4ZisAQ2S6iJGN
nfWLpuN/p97MbUTx67vfR6h0XLWI7ZFciXDmOwPtFVM8X+d/RfgxH1bznSeLW4IG
1GhDzZdI8o7izQu5tgb1QRMHi0ogBST4TR7eY5tgRsRvzogbEI6sgW3OYkDHb0NA
igw52F4QB4aElcI5SVT5kHN96pfG3M6QdzjZ79ytGdv6gCM4M7pPClgmdB4ruS7j
GKgyFLBFU1few6ceQ6txHsLxi/Iqqt7PByjRUbFuvHo/vwa65xH0IAjRQ0AuVs65
/WGASZPGPEtXKQTjkRg5MI5TH6aNiUz8OzhJPepLTk5oLecfWZUiS4pACyBiulvB
xf/ixLgdNV8xobXSCbeHpXADXuBXmfXqhBN1b05PsO+4yqJkx2+cupcdzZjsboCQ
CpBpG2wssXNCAlfwKaCDAy/NGgHu8nsC//oCULdFs9yXiXYlWHJYkBuYkIUEuKbO
clkhISDQsVAuky+/daUmDaGGXvsSIcMWy3Il8dH4ZjPfgsrSmyLW6Oy58GtgEUo/
k1aVod7X7976oQhoWYlm+L91BIPu/J+AiUbnYeOfX6KHbFgxYBFTu6d9cN6jdaVi
xQBwRc+7Ce2bXp4=
=v5q4
-----END PGP SIGNATURE-----

#1102465#83
Date:
2026-03-19 16:21:50 UTC
From:
To:
I should correct what I wrote here.  The MPI-5 common ABI is a new
development, enabling exchange of different MPI-5 implementations (i.e.
swapping between openmpi and mpich, etc).

But MPICH ABI stability with respect to mpich's own recent versions and
derivatives has been in place for a decade or more,
the MPICH ABI Compatibility Initiative, https://www.mpich.org/abi/
https://github.com/pmodels/mpich/blob/main/doc/wiki/testing/ABI_Compatibility_Initiative.md
cf.
https://www.anl.gov/mcs/article/new-initiative-on-runtime-compatibility-for-mpi-implementations
This is why libmpich12 in principle hasn't needed ABI transitions.

It's a separate question why PETSc nevertheless felt the need to be more
restrictive about the mpich version.
I haven't found much PETSc discussion upstream about the common ABIs,
except there is brief reference in
https://www.mpich.org/static/docs/slides/2024-cass-bof/zhang.pdf
and a source comment at
https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/pinit.c#L799
about the MPICH ABI Compatibility Initiative
before checking for older mpich (before ABI compatibility) at
https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/pinit.c#L829