As discussed on [1]debian-science, the way mpi-default-dev works does
not ensure
that the right MPI implementation is used.  For instance, if libmpich2-
dev is
installed on the system, it will _always_ be used by default because
it has the
same priority as OpenMPI and sorts before OpenMPI in lexicological
order.

Several solutions were [2]discussed. The easiest and most robust is to
conflict
with all the non-default implementations of mpi-dev.  A smoother
solution is to
provide links like "mpicc.default" pointing to the right
implementation, and
install them as alternatives for mpicc et al. with a high priority.
This should
always work on buildds but leaves a possibility to build with the wrong
implementation on a box where the admin has set the alternative
manually.

I provide a patch implementing the first of these two solutions.

Best regards, Thibaut.

[1] http://lists.debian.org/debian-science/2010/03/msg00033.html
[2] http://lists.debian.org/debian-science/2010/03/msg00056.html

Actually, the information I was missing is that some packages depend on
mpi-default-dev. I thought that mpi-default-dev was only used as a
build-dependency. In that case, conflicting is indeed harmful, so the
only solution is the alternatives hack...

Le 24 mars 10 à 16:59, Lucas Nussbaum a écrit :

New patch implementing the mpicc.default alternative.

I have built and installed this package, but not tried to build
packages using it.

I have only implemented .default links for the files which were
present in both openmpi and lam, so no mpif90.default.

Regards, Thibaut.

This is new to me as well. mpi-defaults-dev was supposed to be a build
dependency only. If packages depend on it, this is a bug.

Am Donnerstag, den 25.03.2010, 13:54 +0100 schrieb Thibaut Paumard:

Thanks for the patch!
patch respectively.

But there is still one thing I do not get: What problem is solved by
installing mpicc.default as alternative? If the sysadmin did change the
MPI implementation via update-alternatives, it will stay that way. It
has no benefit over simply providing mpicc.default as symlink to the
implementation prefered on the arch, not handled via alternatives. If
the sysadmin did not change it, increasing the priority of the openmpi
alternative will save the same purpose. So updating the priority and
just providing symlinks should be enough, AIUI. But please correct me if
I'm wrong. I did not test this, yet.

The thing we need to rely on is that the alternative is not changed. I
mentioned quite a while back (in a different discussion) that I consider
the alternatives system wrong for the case of MPI and that pretty much
shows here. Nevertheless, it is the (best?) solution we have at the
moment and we simply have to make the assumption that noone fiddles with
the alternatives systems on the buildds. I guess that is quite safe to
assume. I do not care that much for the developers' systems right now,
but only for the transition; a more developer-friendly solution has to
be (and will) be found after the release.

Thanks to both of you for your efforts! It's very much appreciated!

Best regards,
Manuel

Le 25 mars 10 à 14:30, Manuel Prinz a écrit :

Installing mpicc.default as an alternative is to make sure the default
implementation has a high priority. It is to fix the problem at hand
for the packages which already exist, since new packages should call
mpicc.default directly (IMHO).

Currently, LAM is the default on some architectures and has a lower
priority than MPICH2. OpenMPI is the default on the other arches, and
MPICH2 still wins.

The solution I propose fixes the problem with a single upload, no
transition needed. To really fix the problem without
installing .default as an alternative, you need at least two uploads:

  1- raise the priority in OpenMPI
  2- raise the priority of LAM OR make the transition to MPICH2.

More profoundly, what I suggest here is to control which
implementation is default entirely from the mpi-defaults package.
Playing with the priorities of the various implementations is more
complex. Also, as Lucas mentioned, what happens if you decide that
MPICH2 has to be the default on a given arch where OpenMPI also exists
because the former is more stable on that particular arch? If you rely
on the relative priorities of the two packages, you have to install
the alternatives with a different priority depending on the
architecture... Becomes really nasty.

Best regards, Thibaut.

Hi Thibaut,

thanks for clarifying! It seems to me that we share the same opinion in
(almost) all points but have a different view on the topic.

Am Donnerstag, den 25.03.2010, 16:57 +0100 schrieb Thibaut Paumard:

Right, though my preferred solution would not use mpicc.default at all.
But this is not possible with touching some packages and needs thorough
testing. It's nothing to be done now. I plan to do this before the next
stable release after squeeze. Using update-alternatives is the wrong
approach. It would work if the implementations were ABI compatible. But
they aren't. (Hope the MPI 3 standard will fix this.)

LAM is not a show-stopper here, since we're in the MPI transition to get
rid of it, or at least drop all dependencies on it.

This is exactly what we're going for and what is started already (see
#573187). I thought you were aware of that.

I just did that with the latest upload.

As said above, the later one is already in progress. It just needs a
working upload of mpi-defaults and the preparation of some patches which
I currently work on.

Full ACK. But using update-alternatives is (generally speaking) the
wrong approach. But we have to live with it for now.

Agreed.
all. This is possible and I could elaborate on that but as said, it will
take larger modifications. And it is not really relevant for this
transition, IMO.

So, given that the priority in Open MPI changed and we will transition,
do you think the patch is useful? I do not refrain from applying it to
mpi-default but my understanding is that we do not need to have it as of
now.

To be clear: I'm very much in favor of having a solution that works
correctly with whatever MPI implementation is chosen as the local
default. But u-a can not be used to realize that. Once the sysadmin set
the default manually, we have lost, no matter how high the priority of
the mpi-defaults alternative is.

Best regards,
Manuel

Hi Manuel,

Le 25 mars 10 à 18:07, Manuel Prinz a écrit :

No, with the prioriy of OpenMPI raised above that of MPICH2, and with
the idea of switching from LAM to MPICH2 on the other arches, my patch
does not bring any _immediate_ benefit. It's a proposal for a longer
term solution, but you seem to have other ideas and you have time to
let them mature.
_Debian_ default, not local default. The problem with the sysadmin
setting the alternative is not relevant for buildds, and is addressed
by using the mpicc.default et al. links instead of mpicc. Dealing with
a local default looks more difficult, requiring a rebuild a little bit
like kernel modules, which IMHO would be more complex than providing
the two variant openmpi and mpich2 in the first place.

Regards, Thibaut.

Am Donnerstag, den 25.03.2010, 18:28 +0100 schrieb Thibaut Paumard:
interested in getting over with the transition soon. As your patch does
no harm, I guess I will apply it. I might make the step to the "real"
solution easier.

You're right with the long term solution: I have something different in
mind. If you're interested, we can discuss that outside this bug report.
Every discussion about that will hopefully avoid further screw-up. ;)

Exactly. I currently only care for the buildds. But switching the local
default and having mpi-defaults-dev installed on a developer machine
should be possible; this is the long-term solution mentioned above. This
I also care for, but I do not care for it with respect to the
transition.

Anyway, thanks again for your help and this discussion! It's always nice
to see people sharing the same goals!

Best regards,
Manuel