It was found that the pdb2gmx (v4.5.5) from the GROMACS (gromacs v4.5.5-1) fails with an 'Illegal instruction' after the 'Making bonds...' operation (see below). The bug was first noted in gromacs 4.5.4-2 (bug report not filed, prior versions of gromacs not tested at this workstation), and persists in the recent gromacs 4.5.5-1. In both cases the gromacs packages and all dependencies were installed through aptitude. Although reproducible on a virtual machine (see below), the bug first appeared on a physical computer, equipped with a single 1 GHz AMD Duron processor, 514340k RAM, md0 raid1 controlled by mdadm. In order to help in locating the source of the problem, all gromacs packages and dependencies were purged from the system. Hereafter, fftw-3.3 was obtained, compiled from source and installed, followed by obtaining, compiling and installing gromacs-4.5.5 from source. The procedure completed according to instructions (c.f. www.gromacs.org - Installation Instructions) and without any errors. However, the pdb2gmx bug was found to be present here also. On all occasions, the bug was found to be reproducible by following the lysozyme (pdb: 1AKI) GROMACS tutorial (located at http://goo.gl/dxzex). Briefly, the pdb file was obtained, crystal water molecules were removed (two methods have been tried: (i) manual removal of all HOH's in vi; (ii) automatically using CHIMERA), and the resulting 1AKI_waterless.pdb was processed by pdb2gmx using the following command: $ pdb2gmx -f 1AKI_waterless.pdb -o 1AKI_processed.gro -water spce This command was executed in a user homedir as well as in /tmp, furthermore the filenames have been varied (w/o underscore, short filename variants), and the '-water spce' directive has also been omitted. In all cases, the pdb2gmx program starts normally and prompts the user for the force field. Only the following force fields have been tried, but both were found to result in the same illegal instruction: - 14: OPLS-AA/L all-atom force field (2001 aminoacid dihedrals) - 1: AMBER03 protein, nucleic AMBER94 (Duan et al., J. Comp. Chem. 24, 1999-2012, 2003) The program creates an empty topol.top file and returns to the prompt after displaying: [..] Linking CYS-76 SG-601 and CYS-94 SG-724... Start terminus LYS-1: NH3+ End terminus LEU-129: COO- Checking for duplicate atoms.... Now there are 129 residues with 1960 atoms Making bonds... Illegal instruction $ I will try to attach the full output to the bugreport. Furthermore, the bug was found to be reproducable on a virtualbox-ose installation of Debian GNU/Linux testing (Wheezy) i368 (clean install of base system, standard system utilities, gromacs packages and dependencies). In contrary, on an amd64 virtualbox installation of the base system, standard system utilities and gromacs plus all dependencies, the command completes normally without any error. Hence, the bug seems to be i368-specific. In conclusion, this bug report concerns the pdb2gmx command of gromacs 4.5.5-1, which fails with an 'Illegal instruction' on i368/i686 architectures running Debian GNU/Linux testing (Wheezy) on both a physical computer (1 GHz AMD Duron) as well as on a virtualized vbox i386 installation.
Hello,
This could be a problem with some optimisation that goes beyond what the
Duron can do. The folks from debichem may have a different opinion, but
I suggest to compile it yourself to see what happens. You may want to
substitute the compiler with gcc-4.4, but I think, given how simple it
seems to reproduce your problem on your platform, I would just start
with the default:
apt-get source --build gromacs
should be doing that. You may need to install some extra build
dependencies. Then use dpkg -i to install the packages. If this solved
the issue, then please report with your compiler version. If the error
still persists, I see two things to do:
* use another compiler (clang, gcc-snapshot or gcc-4.4)
* add a debug package for gromacs and start pdb2gmx from within gdb
I have pasted a respective patch for the debug package below and have
just built gromacs with it. Worked here. @Debichem: It would be nice to
see a future version of gromacs with something like it. For the problem
at hand I do not think the -dbg package to be helping too much since
illegal instuctions just should not happen and may IMHO more be pointing
to a compiler issue.
--- debian/control (revision 3036)
+++ debian/control (working copy)
@@ -112,3 +112,12 @@
This package contains only the core simulation engine with parallel
support using the OpenMPI interface. It is suitable for nodes of a
processing cluster, or for multiprocessor machines.
+
+Package: gromacs-dbg
+Architecture: any
+Depends: gromacs (= ${binary:Version})
+Description: debug sysmbols for gromacs
+ This package contains information to identify problems with the executed
+ code. Those may be platform-specific issues or programming errors. You
+ may be asked to install the package prior to reproducing the issue you
+ were reporting.
Index: debian/rules
===================================================================
--- debian/rules (revision 3036)
+++ debian/rules (working copy)
@@ -239,7 +239,7 @@
dh_testroot -s
dh_installchangelogs -s
dh_installdocs -s
- dh_strip -A
+ dh_strip -A --dbg-package=gromacs-dbg
dh_link -s
dh_compress -s
dh_fixperms -s
Hope to have somewhat helped
Steffen
thank you for your quick and detailed reponse. I have little experience with non-trivial compiling issues. However, I have tried some of the things you suggested, those are reported below. On the workstation for which the bug was reported, the installed gromacs and gromacs-data were purged. Hereafter, the following packages (including dependencies) were installed: dpkg-dev, debhelper, dpatch, libmpich2-dev, libopenmpi-dev, lesstif2-dev, libgsl0-dev, libxml2-dev, cmake. Thereafter, the system was restarted and the following command was issued: $ apt-get source --build gromacs > log.20111007 2>&1 When finished, the log was inspected for apparent errors. The log file frequently contained a MPI implementation error: > -- Using manually set binary suffix: "_mpi.mpich" > -- Using manually set library suffix: "_mpi.mpich" > CMake Warning at CMakeLists.txt:192 (MESSAGE): > > > There are known problems with some MPI implementations: > OpenMPI version < 1.4.1 > MVAPICH2 version <= 1.4.1 > > > -- Checking for MPI_IN_PLACE > -- Performing Test MPI_IN_PLACE_COMPILE_OK > -- Performing Test MPI_IN_PLACE_COMPILE_OK - Success > -- Checking for MPI_IN_PLACE - yes The built packages were installed using: $ dpkg -i gromacs_4.5.5-1_i386.deb gromacs-data_4.5.5-1_all.deb The lysozyme (pdb: 1AKI) tutorial (http://goo.gl/dxzex) was followed and the bug reported above repeated: $ pdb2gmx -f 1AKI_waterless.pdb -o 1AKI_processed.gro -water spce [.. truncated log ..] > Linking CYS-76 SG-601 and CYS-94 SG-724... > Start terminus LYS-1: NH3+ > End terminus LEU-129: COO- > Checking for duplicate atoms.... > Now there are 129 residues with 1960 atoms > Making bonds... > Illegal instruction The self-built and installed gromacs packages were purged, the official gromacs source package was obtained, extracted and the following two compilers were used: == gcc-4.4 == $ ./configure CC=gcc-4.4 --prefix=/homedir --with-fft=fftw3 $ make CC=gcc-4.4 $ make install The pdb2gmx program failed with the same error, at seemingly the same time. Install files were removed as well as the automake stuff($ make distclean) == clang == (all dependencies were installed first) $ ./configure CC=clang --prefix=/home/not/gromacs/install --with-fft=fftw3 && make CC=clang && make install From the start, this produced errors in the make phase, a lot of 'clang: warning: argument unused during compilation' warnings were reported, i.e.: > /bin/bash ../../../../libtool --mode=compile clang -O3 -fomit-frame-pointer -finline-functions -Wall -Wno-unused -msse2 -std=gnu99 -pthread -I./include -c -o nb_kernel_ia32_sse_test_asm.lo nb_kernel_ia32_sse_test_asm.s > clang -O3 -fomit-frame-pointer -finline-functions -Wall -Wno-unused -msse2 -std=gnu99 -pthread -I./include -c nb_kernel_ia32_sse_test_asm.s -o nb_kernel_ia32_sse_test_asm.o > clang: warning: argument unused during compilation: '-fomit-frame-pointer' > clang: warning: argument unused during compilation: '-finline-functions' > clang: warning: argument unused during compilation: '-Wall' > clang: warning: argument unused during compilation: '-Wno-unused' > clang: warning: argument unused during compilation: '-msse2' > clang: warning: argument unused during compilation: '-std=gnu99' > clang: warning: argument unused during compilation: '-pthread' > clang: warning: argument unused during compilation: '-I ./include' This time, the pdb2gmx error was totally broken and returned to prompt after displaying the GROMACS motd (i.e. names team). At this point I decided to stop trying to get things to work at the 1 GHz Duron workstation, because I was not sure the CC=clang option was the right way to do it and after the command completed, results got worse. So, to verify one of my earlier statements, in virtualbox-ose (3.2.10_OSE r66523), was started an i386 guest running Debian GNU/Linux testing (Wheezy), kernel 3.0.0-1-686-pae. Only the base system, standard system utilities and gromacs packages (including dependencies) were freshly installed. The error could not be reproduced (following the lysozyme (pdb: 1AKI) tutorial at http://goo.gl/dxzex). Indeed, I am starting to think that the problem might be Duron-specific, as already suggested by Steffen. So, given the nature of the current problem, the age of the problematic workstation versus the highly advanced capabilities and applications of GROMACS, I think trying to squash this bug might not be worth the time and energy of those involved. At the other hand, however, it is the first occasion that a Debian package does not work appropriately on the computer I am trying to run it (and that's what made me report the bug in the first place). In conclusion, I am willing to contribute and help solving the problem documented in the bugreport on the given workstation, but only if the suggested steps are described in sufficient detail for me to quickly execute, compile, build, install or implement them. I have the given computer at my disposal, I am subscribed to this bug and -when time permits- gladly like to try whatever is suggested.
Thank you for investing so much energy into this. The chip is soon ten years old and indeed, if there are no other voices from the off coming up with similar issues, then there is little to be gained. And, hey, who knows, maybe the warnings you mention clang to give will actually help already. If you need some help with some computation quickly, you may consider running your calculation on an Amazon cloud instance. I am tempted to claim that this may pay off alone by the energy you save by not running your old system and you should get your results in half the time or earlier. Good luck Steffen
Hi, Indeed! I think it would help if we'd pass -DVERBOSE_MAKEFILE or what's that flag called to see what the compile flags actually are. The build log is not very helpful: https://buildd.debian.org/status/fetch.php?pkg=gromacs&arch=i386&ver=4.5.5-1&stamp=1316662884 Michael
Hi, Indeed! I think it would help if we'd pass -DVERBOSE_MAKEFILE or what's that flag called to see what the compile flags actually are. The build log is not very helpful: https://buildd.debian.org/status/fetch.php?pkg=gromacs&arch=i386&ver=4.5.5-1&stamp=1316662884 Michael
When building that package, it's also possible to both turn on debugging (i.e. turn off binary stripping at the end) and disable compiler optimization. The latter might help avoid the bug, and the former should help identify what exactly is causing the failure. A command such as: $ DEB_BUILD_OPTIONS=noopt,debug apt-get source --build (etc.) will enable both options. I would be curious to learn if that changes the bug behavior in any way. Unfortunately I have no access to Duron hardware and cannot test these myself. These are harmless, it issues those warnings regardless of the OpenMPI version on the system (your installation is probably using 1.4.3), and pdb2gmx doesn't use MPI anyhow. Best,
Dear Nicholas, thank you for your suggestion. I had some time and tried the following: First, it was verified that gromacs was not installed system-wide and that the package's dependencies (libc6, libfftw3-3, libx11-6, libxml2) were all satisfied. In other words, installing gromacs through apt(itude) would lead to installation of only gromacs and gromacs-data packages. The following command was executed: $ DEB_BUILD_OPTIONS="noopt debug" apt-get source --build gromacs > log.20111116 2>&1 The log was checked multiple times during the build and although I do not know the significance of it, a "defined but not used [-Wunused-function]"-error showed up and was repeated frequently for many .h files, e.g.:
Using the binaries with debugging symbols that you built, could you please collect a backtrace using "gdb" when the program crashes? It may help to install the "libc6-dbg" package for additional symbols within libc, which will probably be *somewhere* in the stack. http://wiki.debian.org/HowToGetABacktrace#Running_gdb With luck this will be enough information to localize the problem to a single function. - N
package gromacs retitle 644397 Provide runtime detection for CPU SIMD capabilities forwarded 644397 http://redmine.gromacs.org/issues/1165 tags 644397 = upstream thanks More background to this bug is in #725013 and #732400, but those are both closed and archived now, and not clearly visible to anyone browsing the buglist. So, in this more visible location: Since version 4.5, GROMACS autodetects the compilation CPU and chooses the SIMD compilations based on that; as a result, with the default options, it'll compile binaries that will not run on any CPU lacking those extensions. This is what happened here, since the Duron is an old processor with nothing beyond SSE1. The long-term fix is runtime detection; the short-term workaround, in place since 4.6.5-1, is to force nothing above SSE2 extensions for i386 and amd64 builds. This still excludes Duron systems, but will work on at least every processor sold since 2004 and most of them since 2002. Further discussion is in README.Debian in the gromacs binary package. Users compiling their own copies of the package can re-enable the autodetection and get all the possible optimizations for their CPU by passing the environment variable DEB_BUILD_OPTIONS=cpuopt at build time.