#644397 Provide runtime detection for CPU SIMD capabilities

Package:
gromacs
Source:
gromacs
Description:
Molecular dynamics simulator, with building and analysis tools
Submitter:
groenedraeck
Date:
2014-07-03 00:00:10 UTC
Severity:
normal
#644397#5
Date:
2011-10-05 13:37:45 UTC
From:
To:
It was found that the pdb2gmx (v4.5.5) from the GROMACS (gromacs v4.5.5-1) fails with an 'Illegal instruction' after the 'Making bonds...' operation (see below).

The bug was first noted in gromacs 4.5.4-2 (bug report not filed, prior versions of gromacs not tested at this workstation), and persists in the recent gromacs 4.5.5-1. In both cases the gromacs packages and all dependencies were installed through aptitude. Although reproducible on a virtual machine (see below), the bug first appeared on a physical computer, equipped with a single 1 GHz AMD Duron processor, 514340k RAM, md0 raid1 controlled by mdadm.

In order to help in locating the source of the problem, all gromacs packages and dependencies were purged from the system. Hereafter, fftw-3.3 was obtained, compiled from source and installed, followed by obtaining, compiling and installing gromacs-4.5.5 from source. The procedure completed according to instructions (c.f. www.gromacs.org - Installation Instructions) and without any errors. However, the pdb2gmx bug was found to be present here also.

On all occasions, the bug was found to be reproducible by following the lysozyme (pdb: 1AKI) GROMACS tutorial (located at http://goo.gl/dxzex). Briefly, the pdb file was obtained, crystal water molecules were removed (two methods have been tried: (i) manual removal of all HOH's in vi; (ii) automatically using CHIMERA), and the resulting 1AKI_waterless.pdb was processed by pdb2gmx using the following command:

$ pdb2gmx -f 1AKI_waterless.pdb -o 1AKI_processed.gro -water spce

This command was executed in a user homedir as well as in /tmp, furthermore the filenames have been varied (w/o underscore, short filename variants), and the '-water spce' directive has also been omitted.

In all cases, the pdb2gmx program starts normally and prompts the user for the force field. Only the following force fields have been tried, but both were found to result in the same illegal instruction:
- 14: OPLS-AA/L all-atom force field (2001 aminoacid dihedrals)
-  1: AMBER03 protein, nucleic AMBER94 (Duan et al., J. Comp. Chem. 24, 1999-2012, 2003)

The program creates an empty topol.top file and returns to the prompt after displaying:

[..]
Linking CYS-76 SG-601 and CYS-94 SG-724...
Start terminus LYS-1: NH3+
End terminus LEU-129: COO-
Checking for duplicate atoms....
Now there are 129 residues with 1960 atoms
Making bonds...
Illegal instruction
$

I will try to attach the full output to the bugreport.

Furthermore, the bug was found to be reproducable on a virtualbox-ose installation of Debian GNU/Linux testing (Wheezy) i368 (clean install of base system, standard system utilities, gromacs packages and dependencies). In contrary, on an amd64 virtualbox installation of the base system, standard system utilities and gromacs plus all dependencies, the command completes normally without any error. Hence, the bug seems to be i368-specific.

In conclusion, this bug report concerns the pdb2gmx command of gromacs 4.5.5-1, which fails with an 'Illegal instruction' on i368/i686 architectures running Debian GNU/Linux testing (Wheezy) on both a physical computer (1 GHz AMD Duron) as well as on a virtualized vbox i386 installation.

#644397#10
Date:
2011-10-06 09:18:21 UTC
From:
To:
Hello,
This could be a problem with some optimisation that goes beyond what the
Duron can do. The folks from debichem may have a different opinion, but
I suggest to compile it yourself to see what happens.  You may want to
substitute the compiler with gcc-4.4, but I think, given how simple it
seems to reproduce your problem on your platform, I would just start
with the default:

    apt-get source --build gromacs

should be doing that. You may need to install some extra build
dependencies. Then use dpkg -i to install the packages. If this solved
the issue, then please report with your compiler version. If the error
still persists, I see two things to do:
 * use another compiler (clang, gcc-snapshot  or gcc-4.4)
 * add a debug package for gromacs and start pdb2gmx from within gdb

I have pasted a respective patch for the debug package below and have
just built gromacs with it. Worked here. @Debichem: It would be nice to
see a future version of gromacs with something like it. For the problem
at hand I do not think the -dbg package to be helping too much since
illegal instuctions just should not happen and may IMHO more be pointing
to a compiler issue.
--- debian/control (revision 3036) +++ debian/control (working copy) @@ -112,3 +112,12 @@ This package contains only the core simulation engine with parallel support using the OpenMPI interface. It is suitable for nodes of a processing cluster, or for multiprocessor machines. + +Package: gromacs-dbg +Architecture: any +Depends: gromacs (= ${binary:Version}) +Description: debug sysmbols for gromacs + This package contains information to identify problems with the executed + code. Those may be platform-specific issues or programming errors. You + may be asked to install the package prior to reproducing the issue you + were reporting. Index: debian/rules =================================================================== --- debian/rules (revision 3036) +++ debian/rules (working copy) @@ -239,7 +239,7 @@ dh_testroot -s dh_installchangelogs -s dh_installdocs -s - dh_strip -A + dh_strip -A --dbg-package=gromacs-dbg dh_link -s dh_compress -s dh_fixperms -s Hope to have somewhat helped Steffen
#644397#15
Date:
2011-10-08 18:26:37 UTC
From:
To:
thank you for your quick and detailed reponse. I have little experience
with non-trivial compiling issues. However, I have tried some of the
things you suggested, those are reported below.

On the workstation for which the bug was reported, the installed gromacs
and gromacs-data were purged. Hereafter, the following packages
(including dependencies) were installed: dpkg-dev, debhelper, dpatch,
libmpich2-dev, libopenmpi-dev, lesstif2-dev, libgsl0-dev, libxml2-dev,
cmake. Thereafter, the system was restarted and the following command
was issued:

$ apt-get source --build gromacs > log.20111007 2>&1

When finished, the log was inspected for apparent errors. The log file
frequently contained a MPI implementation error:
 > -- Using manually set binary suffix: "_mpi.mpich"
 > -- Using manually set library suffix: "_mpi.mpich"
 > CMake Warning at CMakeLists.txt:192 (MESSAGE):
 >
 >
 >             There are known problems with some MPI implementations:
 >                        OpenMPI version < 1.4.1
 >                        MVAPICH2 version <= 1.4.1
 >
 >
 > -- Checking for MPI_IN_PLACE
 > -- Performing Test MPI_IN_PLACE_COMPILE_OK
 > -- Performing Test MPI_IN_PLACE_COMPILE_OK - Success
 > -- Checking for MPI_IN_PLACE - yes

The built packages were installed using:

$ dpkg -i gromacs_4.5.5-1_i386.deb gromacs-data_4.5.5-1_all.deb

The lysozyme (pdb: 1AKI) tutorial (http://goo.gl/dxzex) was followed and
the bug reported above repeated:

$ pdb2gmx -f 1AKI_waterless.pdb -o 1AKI_processed.gro -water spce
[.. truncated log ..]
 > Linking CYS-76 SG-601 and CYS-94 SG-724...
 > Start terminus LYS-1: NH3+
 > End terminus LEU-129: COO-
 > Checking for duplicate atoms....
 > Now there are 129 residues with 1960 atoms
 > Making bonds...
 > Illegal instruction

The self-built and installed gromacs packages were purged, the official
gromacs source package was obtained, extracted and the following two
compilers were used:

== gcc-4.4 ==
$ ./configure CC=gcc-4.4 --prefix=/homedir --with-fft=fftw3
$ make CC=gcc-4.4
$ make install

The pdb2gmx program failed with the same error, at seemingly the same
time. Install files were removed as well as the automake stuff($ make
distclean)

== clang == (all dependencies were installed first)
$ ./configure CC=clang --prefix=/home/not/gromacs/install
--with-fft=fftw3 && make CC=clang && make install

 From the start, this produced errors in the make phase, a lot of
'clang: warning: argument unused during compilation' warnings were
reported, i.e.:

 > /bin/bash ../../../../libtool   --mode=compile clang   -O3
-fomit-frame-pointer -finline-functions -Wall -Wno-unused -msse2 
-std=gnu99 -pthread -I./include -c -o nb_kernel_ia32_sse_test_asm.lo 
nb_kernel_ia32_sse_test_asm.s
 >  clang -O3 -fomit-frame-pointer -finline-functions -Wall -Wno-unused
-msse2 -std=gnu99 -pthread -I./include -c nb_kernel_ia32_sse_test_asm.s 
-o nb_kernel_ia32_sse_test_asm.o
 > clang: warning: argument unused during compilation:
'-fomit-frame-pointer'
 > clang: warning: argument unused during compilation: '-finline-functions'
 > clang: warning: argument unused during compilation: '-Wall'
 > clang: warning: argument unused during compilation: '-Wno-unused'
 > clang: warning: argument unused during compilation: '-msse2'
 > clang: warning: argument unused during compilation: '-std=gnu99'
 > clang: warning: argument unused during compilation: '-pthread'
 > clang: warning: argument unused during compilation: '-I ./include'

This time, the pdb2gmx error was totally broken and returned to prompt
after displaying the GROMACS motd (i.e. names team).

At this point I decided to stop trying to get things to work at the 1
GHz Duron workstation, because I was not sure the CC=clang option was
the right way to do it and after the command completed, results got worse.

So, to verify one of my earlier statements, in virtualbox-ose
(3.2.10_OSE r66523), was started an i386 guest running Debian GNU/Linux
testing (Wheezy), kernel 3.0.0-1-686-pae. Only the base system, standard
system utilities and gromacs packages (including dependencies) were
freshly installed. The error could not be reproduced (following the
lysozyme (pdb: 1AKI) tutorial at http://goo.gl/dxzex).

Indeed, I am starting to think that the problem might be Duron-specific,
as already suggested by Steffen. So, given the nature of the current
problem, the age of the problematic workstation versus the highly
advanced capabilities and applications of GROMACS, I think trying to
squash this bug might not be worth the time and energy of those
involved. At the other hand, however, it is the first occasion that a
Debian package does not work appropriately on the computer I am  trying
to run it (and that's what made me report the bug in the first place).

In conclusion, I am willing to contribute and help solving the problem
documented in the bugreport on the given workstation, but only if the
suggested steps are described in sufficient detail for me to quickly
execute, compile, build, install or implement them. I have the given
computer at my disposal, I am subscribed to this bug and -when time
permits- gladly like to try whatever is suggested.

#644397#20
Date:
2011-10-08 19:17:25 UTC
From:
To:
Thank you for investing so much energy into this. The chip is soon ten years old and indeed, if there are no other voices from the
off coming up with similar issues, then there is little to be gained. And, hey, who knows, maybe the warnings you mention clang to
give will actually help already.

If you need some help with some computation quickly, you may consider running your calculation on an Amazon cloud instance. I am
tempted to claim that this may pay off alone by the energy you save by not running your old system and you should get your results
in half the time or earlier.

Good luck

Steffen

#644397#25
Date:
2011-10-08 20:41:33 UTC
From:
To:
Hi,

Indeed!

I think it would help if we'd pass -DVERBOSE_MAKEFILE or what's that
flag called to see what the compile flags actually are.  The build log
is not very helpful:

https://buildd.debian.org/status/fetch.php?pkg=gromacs&arch=i386&ver=4.5.5-1&stamp=1316662884


Michael

#644397#28
Date:
2011-10-08 20:41:33 UTC
From:
To:
Hi,

Indeed!

I think it would help if we'd pass -DVERBOSE_MAKEFILE or what's that
flag called to see what the compile flags actually are.  The build log
is not very helpful:

https://buildd.debian.org/status/fetch.php?pkg=gromacs&arch=i386&ver=4.5.5-1&stamp=1316662884


Michael

#644397#33
Date:
2011-10-24 03:31:44 UTC
From:
To:
When building that package, it's also possible to both turn on debugging (i.e.
turn off binary stripping at the end) and disable compiler optimization.  The
latter might help avoid the bug, and the former should help identify what
exactly is causing the failure.  A command such as:

 $ DEB_BUILD_OPTIONS=noopt,debug apt-get source --build (etc.)

will enable both options.  I would be curious to learn if that changes the bug
behavior in any way.  Unfortunately I have no access to Duron hardware and
cannot test these myself.

These are harmless, it issues those warnings regardless of the OpenMPI version
on the system (your installation is probably using 1.4.3), and pdb2gmx doesn't
use MPI anyhow.

Best,

#644397#42
Date:
2011-11-16 16:30:01 UTC
From:
To:
Dear Nicholas,

thank you for your suggestion. I had some time and tried the following:


First, it was verified that gromacs was not installed system-wide and
that the package's dependencies (libc6, libfftw3-3, libx11-6, libxml2)
were all satisfied. In other words, installing gromacs through
apt(itude) would lead to installation of only gromacs and gromacs-data
packages. The following command was executed:

$ DEB_BUILD_OPTIONS="noopt debug" apt-get source --build gromacs >
log.20111116 2>&1

The log was checked multiple times during the build and although I do
not know the significance of it, a "defined but not used
[-Wunused-function]"-error showed up and was repeated frequently for
many .h files, e.g.:

#644397#47
Date:
2011-11-21 06:10:02 UTC
From:
To:
Using the binaries with debugging symbols that you built, could you please
collect a backtrace using "gdb" when the program crashes?  It may help to
install the "libc6-dbg" package for additional symbols within libc, which will
probably be *somewhere* in the stack.

http://wiki.debian.org/HowToGetABacktrace#Running_gdb

With luck this will be enough information to localize the problem to a single
function.

- N

#644397#60
Date:
2014-07-02 23:51:39 UTC
From:
To:
package gromacs
retitle 644397 Provide runtime detection for CPU SIMD capabilities
forwarded 644397 http://redmine.gromacs.org/issues/1165
tags 644397 = upstream
thanks

More background to this bug is in #725013 and #732400, but those are both
closed and archived now, and not clearly visible to anyone browsing the
buglist.  So, in this more visible location:

Since version 4.5, GROMACS autodetects the compilation CPU and chooses the SIMD
compilations based on that; as a result, with the default options, it'll
compile binaries that will not run on any CPU lacking those extensions.  This
is what happened here, since the Duron is an old processor with nothing beyond
SSE1.  The long-term fix is runtime detection; the short-term workaround, in
place since 4.6.5-1, is to force nothing above SSE2 extensions for i386 and
amd64 builds.  This still excludes Duron systems, but will work on at least
every processor sold since 2004 and most of them since 2002.

Further discussion is in README.Debian in the gromacs binary package.  Users
compiling their own copies of the package can re-enable the autodetection and
get all the possible optimizations for their CPU by passing the environment
variable DEB_BUILD_OPTIONS=cpuopt at build time.