#603603 parallel safe /usr/bin/ar

Package:
binutils
Source:
binutils
Description:
GNU assembler, linker and binary utilities
Submitter:
Date:
2014-12-19 09:30:05 UTC
Severity:
normal
#603603#5
Date:
2010-11-15 18:44:08 UTC
From:
To:
Invoking "make -j 12" with a Makefile containing a line like

  libfoo.a: libfoo.a(a.o b.o c.o)

can make the multiple invocations of ar to update each of the .o files
into the .a library in parallel.  This proceeds to fail miserably.

The "right" solution, I think, would be for ar to lock the .a file
using flock(2) or lockf() or whatever, and queue up nicely via
blocking and retrying when there is contention.

If there is some serious efficiency issue associated with this (which
I rather doubt) a switch to disable locking would be appropriate.
That would make ar work in a multicore-build-safe way by default.

#603603#10
Date:
2014-12-18 18:29:16 UTC
From:
To:
Closing binutils bug reports reported for ancient versions
of binutils, or for outdated packages built with outdated
versions of binutils.  This doesn't mean that this issue
is fixed, but your help is needed to confirm the issue with
a recent binutils version (currently 2.24.90.20141209-1,
available in unstable).

Please reopen the issue if you think this still needs fixing,
adding all relevant files (including system libraries) in
a tarball attached to the bug report.

#603603#17
Date:
2014-12-19 09:26:18 UTC
From:
To:
Just sent control@ message to reopen, as this issue remains present.

There is a discussion in make's info pages

http://www.gnu.org/software/make/manual/html_node/Archive-Pitfalls.html

which reads

  11.3 Dangers When Using Archives

  It is important to be careful when using parallel execution (the -j
  switch; see Parallel Execution) and archives.  If multiple ar commands
  run at the same time on the same archive file, they will not know
  about each other and can corrupt the file.

  Possibly a future version of make will provide a mechanism to
  circumvent this problem by serializing all recipes that operate on the
  same archive file.  But for the time being, you must either write your
  makefiles to avoid this problem in some other way, or not use -j.

A workaround suggested on stackoverflow is to use

  AR := flock make-ar-tempfile.lock $(AR)

to serialize ar.  Ideally ar itself would have an option to take a lock
on the libXXX.a file it is manipulating, and make would pass that option
by default when running in non-serial mode.
--
Barak A. Pearlmutter <barak@cs.nuim.ie>
 Hamilton Institute & Dept Comp Sci, NUI Maynooth, Co. Kildare, Ireland
http://www.bcl.hamilton.ie/~barak/