Dear Kurt,
thanks for helping me along with this issue on IRC, as dicussed there
I am submitting this bug-report for documentation purposes.

I have just realized that the Debian specific libtool enhancement for
reducing unneeded linkage (by not recursivly linking dependency_libs
when dynamic linking is used) does not work when libtool convenience
libraries (noinst_LTLIBRARIES) are used.

This hit me in 405239. Gnutls is using a convenience library
liblgnu.la which links against libgcrypt. libtool adds libgcrypt.la
_and_ *its* dependency_libs to liblgnu.la's dependency_libs.
Later when gnutls is generated, libgnutls is linked directly against liblgnu.la's
dependency_libs which contains libgcrypt's indirect dependencies.

So afaict the cause of the issue is that libtool acts differently
depending whether it generates a convenience library or a real one.
For a conv-lib it will _recursively_ add dependencies to
dependency_libs.

As discussed in 347650 there is a workaround:

Limitations of this approach and more discussion on the topic can
also be found in 347650.

cu andreas

See
http://www.mail-archive.com/libtool@gnu.org/msg08136.html for a possible
way to solve this.

Anyway, this is going to require some work to really solve.


Kurt

Hello Kurt, Andreas, all,

* Kurt Roeckx wrote on Sat, Apr 14, 2007 at 04:01:46PM CEST:
convenience archives.  There is *no* indirection that you can take
advantage of.  All objects of the convenience archive will be included
as they are in the library that they end up in.  Thus, this library will
*explicitly* depend upon all symbols that are referenced from those
objects.

Trying to remove deplibs from this list is like saying: while I said a
minute ago that I need these libraries, now I am saying that I don't
need them.  It's nonsensical.  Such a patch will not find its way into
upstream GNU Libtool.

If you want less libs linked against, them simply remove them from the
link line when you _create_ the convenience archives.  They won't mind.

Hope that helps.

Cheers,
Ralf

[...]
hmm. There are a couple of projects that use convenience libraries
simply as a way to structure the source in different directories
without implying different linkage semanthics.

The libtool manual lists this as a common usage scenario:
| This technique is often used to overcome GNU automake's lack of
| support for linking object files built from sources in other
| directories, because it supports linking with libraries from other
| directories. This limitation applies to GNU automake up to release
| 1.4; newer releases should support sources in other directories.
[...]

Eh, no. I was saying that I was going to directly reference symbols
from and was needing *libgcrypt*. And now, at final link time libtool
is linking stuff as if I had been saying: "I am needing and directly
referencing symbols from libgcrypt *and* *its* dependencies."

It would probably work for gnutls and if this is indeed not a
hotfix that will break horribly on strange libtool supported archs
I'll try to push it upstream.
cu andreas

* Andreas Metzler wrote on Wed, Apr 18, 2007 at 08:16:56PM CEST:

Which is fine.  But why are they adding library dependencies on the
convenience archives then?  Convenience archives are not shared
libraries, they do not need dependency information; think of them as a a
short name for a bunch of object files.  The only reason to add library
dependencies is to tell libtool: please, when I ever subsume this bunch
of objects (convenience archive) into a real shared library, remember
for me to also add this library dependency to the real shared library;
because the object files that will be part of it need that library.  And
libtool remembers by putting the library dependency into the
libconvenience.la file.

You are implying some sort of indirection here, which happens when you
have shared library interdependencies.  But please note that this simply
does not happen with convenience archives; it can not, by design.

Sure, but at least I will give you the same answer there, unless you can
somehow convince me of your case; maybe I've still not understood it.
As far as I can see:

Bugs 405239 and 347650 are two different issues from a Libtool POV.
The former is a Libtool limitation (it doesn't handle -Wl,--as-needed
correctly), while the latter is merely a user misunderstanding in what
should be happening.  If you don't want libgnutls to depend on
libgcrypt, then why is -lgcrypt added to the link line of liblgnu.la?
liblgnu.la is nothing more than a shorthand for some objects that all
end up in libgcrypt.  If those objects need symbols from libgcrypt,
then it's only the right thing to do of libtool to put libcgrypt.so.N
in DT_NEEDED of liblibgnutls.so.  Conversely, if those objects do not
reference any symbol from libgcrypt, then adding -lgcrypt to the link
line of liblgnu.la in the first place is the bug that needs fixing.

I sense that we are talking past each other all the time.  But also I
don't see how I can explain things any differently.  Maybe let's try
with a simple example project: I want a shared library libfoo, and it
will collect some objects from a convenience archive.  For my
convenience, I'll use the libtool feature of transporting library
dependency information from the convenience archive:

--- snip example ---
mkdir src lib

cat > src/foo.c <<\EOF
extern double mysine (double);

double foo (double x)
{
  return mysine(x);
}
EOF

cat > lib/mysine.c <<\EOF
#include <math.h>

double mysine (double x)
{
  return sin (x);
}
EOF

: ${CC=gcc}
libtool --mode=compile --tag=CC $CC -c lib/mysine.c -o lib/mysine.lo
libtool --mode=compile --tag=CC $CC -c src/foo.c -o src/foo.lo
# Create the convenience archive:
libtool --mode=link --tag=CC $CC -o lib/libmysine.la lib/mysine.lo -lm
# Create a shared library that uses the convenience archive:
libtool --mode=link --tag=CC $CC -o src/libfoo.la -rpath \
    /usr/local/lib src/foo.lo lib/libmysine.la -no-undefined
--- snip end example ---

If I understand you correctly, then you argue that after this sequence,

objdump -p src/.libs/libfoo.so | grep NEEDED.\*libm
|   NEEDED      libm.so.6

should produce empty output.  My counter argument is that libm *is* a
direct dependency of libfoo, not an indirect one:

nm src/.libs/libfoo.so | grep sin
| 0000000000000700 T mysine
|                  U sin@@GLIBC_2.2.5

simply because, as shared entity, no such thing as libmysine exists;
only libfoo, and it depends on libm.  Which part of this is unclear,
and in which way does this differ from the usage case you're talking
about?

Cheers,
Ralf

Hello,
Because it is the natural thing to do? If one compartmentalized part
of the source grows an external dependency, adding the dependency
exactly there is simply the natural thing to do. (Otherwise if you
choose to re-use only this part somewhere else you are going to forget
the dependency for sure.

And that is exactly what goes wrong. It does not put the library
dependency in the la file, it puts the library dependency and *its*
dependencies in the la file.

[ Snip ]

I do want liblgnu.la to be linked with against -lgcrypt. I also want
libgnutls to be linked against -lgcrypt (because it needs it itself and
because it is linked against liblgnu.la.) However I definitily do not
want libgnutls to be linked against libgpg-error. libgnutls does
(currently) not reference libgpg-error directly. (liblgnu does not
either.) It just ends up on the link line because it is an indirect
dependency but should not.

I also get that feeling. Perhaps you did miss the initial paragraph in
my bug report?

| I have just realized that the Debian specific libtool enhancement
                                ^^^^^^^^^^^^^^^
| [...] does not work when libtool convenience libraries
| (noinst_LTLIBRARIES) are used.

I agree with you.

libm itself does not have any dependencies except
libc. However if it did depend on libbar, I would argue that
objdump -p src/.libs/libfoo.so | grep NEEDED.\*libbar
should be empty, since neither libmysine nor the final product
libfoo.so directly use symbols from libbar, the dependency is only
an indirect one, pulled in by libtool.

thanks, cu andreas

* Andreas Metzler wrote on Thu, Apr 19, 2007 at 08:02:36PM CEST:

D'oh.

Thanks for bearing with me, I feel a bit better now.

OK, now that I (think I) understand the issue, I guess it's the right
thing for Debian libtool (with its link_all_deplibs=no) to also not link
against these indirect deplibs stemming from a convenience archive's
deplibs.  This feature should be specific to systems with
link_all_deplibs=no, though.  It can be pulled into upstream once the
indirect deplibs issues are fixed (one of them being that uninstalled
indirect deplibs aren't found).

Cheers,
Ralf

So would the attached patch be acceptable for now?  I'm thinking about
adding that to the Debian patch.


Kurt

Hello Kurt, all,

Thanks for the patch.

* Kurt Roeckx wrote on Sun, Apr 22, 2007 at 01:22:30AM CEST:

Here's how I tested your patch, as a new test against the testsuite of
CVS HEAD (apologies, but I rather prefer writing new tests for HEAD;
the patch contains two tests; the first one belongs to this bug).
If you're inclined, here's how you can use HEAD's new testsuite with
an 1.5.x libtool:  get CVS HEAD, bootstrap and compile it.  Then run

  make check-local TESTSUITEFLAGS="-v -d -x -k indirect \
       LIBTOOL=/path/to/the/1.5.x/libtool"

Both of those two tests work with Debian's current libtool, but break
with your patch.  Note that they also break if -static is used for all
libraries (add LDFLAGS=-static to TESTSUITEFLAGS to try out), rather
than only to the final ones.

Libtool developers: ok to apply the patch, which serves to document
behavior that works and should continue to do so?

Cheers,
Ralf

	* tests/indirect_deps.at: New file, with tests to ensure
	`link_all_deplibs=no' does not break functionality.
	* Makefile.am: Adjust.

Index: Makefile.am
===================================================================
RCS file: /cvsroot/libtool/libtool/Makefile.am,v
retrieving revision 1.216
diff -u -r1.216 Makefile.am
--- Makefile.am	29 Mar 2007 18:09:37 -0000	1.216
+++ Makefile.am	23 Apr 2007 17:16:59 -0000
@@ -439,6 +441,7 @@
 		  tests/export.at \
 		  tests/search-path.at \
 		  tests/destdir.at \
+		  tests/indirect_deps.at \
 		  tests/old-m4-iface.at \
 		  tests/am-subdir.at \
 		  tests/lt_dlexit.at \
--- /dev/null	2007-04-15 17:46:43.220064750 +0200
+++ tests/indirect_deps.at	2007-04-23 19:20:14.000000000 +0200
@@ -0,0 +1,117 @@
+# indirect_deps.at -- support for link_all_deplibs=no   -*- Autotest -*-
+
+#   Copyright (C) 2007 Free Software Foundation, Inc.
+#   Written by Ralf Wildenhues, 2007.
+#
+#   This file is part of GNU Libtool.
+#
+# GNU Libtool is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation; either version 2 of
+# the License, or (at your option) any later version.
+#
+# GNU Libtool is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Libtool; see the file COPYING.  If not, a copy
+# can be downloaded from  http://www.gnu.org/licenses/gpl.html,
+# or obtained by writing to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+####
+
+AT_SETUP([indirect convenience])
+AT_KEYWORDS([libtool])
+
+AT_DATA([a.c], [[
+#include <math.h>
+int a () { return 0; }
+double ad (double x) { return sin (x); }
+]])
+AT_DATA([b.c], [[
+extern int a ();
+extern double ad (double);
+int b () { return a () + (int) ad (0.0); }
+]])
+AT_DATA([c.c], [[
+extern int b ();
+int c () { return b (); }
+]])
+AT_DATA([m.c], [[
+extern int c ();
+int main () { return c (); }
+]])
+
+LDFLAGS="$LDFLAGS -no-undefined"
+for file in a.c b.c c.c; do
+  $LIBTOOL --mode=compile --tag=CC $CC $CPPFLAGS $CFLAGS -c $file
+done
+$CC $CPPFLAGS $CFLAGS -c m.c
+
+# liba is an indirect dependency of libd and of libconv.
+$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o liba.la a.lo -rpath /nowhere -lm
+$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o libb.la b.lo liba.la -rpath /nowhere
+$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o libconv.la c.lo libb.la
+$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o libd.la libconv.la -rpath /nowhere
+
+for st in '' -static; do
+  AT_CHECK([$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS $st -o m1 m.$OBJEXT libd.la],
+	   [], [ignore], [ignore])
+  AT_CHECK([$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS $st -o m2 m.$OBJEXT libconv.la],
+	   [], [ignore], [ignore])
+  LT_AT_EXEC_CHECK([./m1])
+  LT_AT_EXEC_CHECK([./m2])
+done
+
+AT_CLEANUP
+
+
+AT_SETUP([indirect uninstalled])
+AT_KEYWORDS([libtool])
+
+AT_DATA([a.c], [[
+int a () { return 0; }
+]])
+AT_DATA([b.c], [[
+extern int a ();
+int b () { return a (); }
+]])
+AT_DATA([c.c], [[
+extern int b ();
+int c () { return b (); }
+]])
+
+AT_DATA([m1.c], [[
+extern int b ();
+int main () { return b (); }
+]])
+AT_DATA([m2.c], [[
+extern int c ();
+int main () { return c (); }
+]])
+
+mkdir a b c
+LDFLAGS="$LDFLAGS -no-undefined"
+for file in a.c b.c c.c; do
+  $LIBTOOL --mode=compile --tag=CC $CC $CPPFLAGS $CFLAGS -c $file
+done
+for file in m1.c m2.c; do
+  $CC $CPPFLAGS $CFLAGS -c $file
+done
+
+for st in '' -static; do
+  $LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o a/liba.la a.lo -rpath /nowherea
+  $LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o b/libb.la b.lo a/liba.la -rpath /nowhereb
+  $LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o c/libcee.la c.lo b/libb.la -rpath /nowherec
+
+  AT_CHECK([$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS $st -o m1 m1.$OBJEXT b/libb.la],
+	   [], [ignore], [ignore])
+  AT_CHECK([$LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS $st -o m2 m2.$OBJEXT c/libcee.la],
+	   [], [ignore], [ignore])
+  LT_AT_EXEC_CHECK([./m1])
+  LT_AT_EXEC_CHECK([./m2])
+done
+
+AT_CLEANUP

This is a great trick, thanks!

Right, so the problem is with static libraries.  And the solution I had
in mind was that it needs to look at the .la files recursivly.  It
doesn't seem to be doing this.


Kurt