#749120 Missing X11 include to cause type conflict

#749120#5
Date:
2014-05-24 10:14:02 UTC
From:
To:
During a rebuild of all Debian packages in a clean sid chroot (using cowbuilder
and pbuilder) the build failed with the following error. Please note that we
use our research compiler tool-chain (using tools from the cbmc package), which
permits extended reporting on type inconsistencies at link time.

[...]
libtool: link:  gcc -shared  -fPIC -DPIC  .libs/FSCloseFt.o .libs/FSClServ.o .libs/FSConnServ.o .libs/FSErrDis.o .libs/FSErrHndlr.o .libs/FSFlush.o .libs/FSFontInfo.o .libs/FSFtNames.o .libs/FSGetCats.o .libs/FSlibInt.o .libs/FSListCats.o .libs/FSListExt.o .libs/FSMisc.o .libs/FSNextEv.o .libs/FSOpenFont.o .libs/FSOpenServ.o .libs/FSQGlyphs.o .libs/FSQuExt.o .libs/FSQXExt.o .libs/FSQXInfo.o .libs/FSServName.o .libs/FSSetCats.o .libs/FSSync.o .libs/FSSynchro.o .libs/fs_transport.o    -O2 -Wl,-z -Wl,relro   -Wl,-soname -Wl,libFS.so.6 -Wl,-version-script -Wl,.libs/libFS.ver -o .libs/libFS.so.6.0.0

error: conflicting function declarations "_FSTransBytesReadable"
old definition in module FSConnServ file /usr/include/X11/Xtrans/Xtrans.h line 338
signed int (struct _XtransConnInfo *, signed int *)
new definition in module fs_transport file /usr/include/X11/Xtrans/Xtrans.c line 897
signed int (struct _XtransConnInfo *ciptr, signed long int *pend)
Makefile:467: recipe for target 'libFS.la' failed
make[3]: *** [libFS.la] Error 64
make[3]: Leaving directory '/srv/jenkins-slave/workspace/sid-goto-cc-libfs/libfs-1.0.5/build/src'
Makefile:509: recipe for target 'all-recursive' failed

While Xtrans.h and Xtrans.c can reasonably be expected to be consistent, indeed
the second parameter's type depends on whether LONG64 is defined or not. This is
set in /usr/include/X11/Xmd.h, which is transitively included by FSConnServ.c
(via #include "FSlibint.h"). fs_transport.c, however, solely has:

#include "config.h" /* Need to know which transport types to support */
#include <X11/Xtrans/transport.c>

This causes a mismatch of types, with undefined behaviour as dereferencing the
"pend" pointer will cause invalid memory accesses. Consequently fs_transport.c
should include FSlibint.h before transport.c.

Best,
Michael

#749120#10
Date:
2015-07-26 18:16:59 UTC
From:
To:
Xtrans defines BytesReadable_t as int or long depending on whether
LONG64 is defined.  So on a 64bit build it would be int when building
FSConnServ.c, but long for fs_transport.c.  To fix this inconsistency,
make fs_transport.c include <X11/Xmd.h>.

Debian bug#749120

Reported-by: Michael Tautschnig <mt@debian.org>
Signed-off-by: Julien Cristau <jcristau@debian.org>
---
 src/fs_transport.c | 1 +
 1 file changed, 1 insertion(+)

I'm not sure if this should be fixed in xtrans instead...

diff --git a/src/fs_transport.c b/src/fs_transport.c
index a924f91..347bd43 100644
--- a/src/fs_transport.c
+++ b/src/fs_transport.c
@@ -28,4 +28,5 @@
 #endif

 #include "config.h" /* Need to know which transport types to support */
+#include <X11/Xmd.h>
 #include <X11/Xtrans/transport.c>

#749120#15
Date:
2015-07-26 18:57:47 UTC
From:
To:
Yeah - I think we should add the #include <X11/Xmd.h> to Xtrans.h, not
force every consumer to add it before including Xtrans.h.

#749120#20
Date:
2015-07-29 18:41:58 UTC
From:
To:
Xtrans.h defines BytesReadable_t as int or long depending on whether
LONG64 is defined.  We need to make sure Xmd.h is included so our idea
of BytesReadable_t is consistent across compilation units.

Debian bug#749120

Reported-by: Michael Tautschnig <mt@debian.org>
Signed-off-by: Julien Cristau <jcristau@debian.org>
---
 Xtrans.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Xtrans.h b/Xtrans.h
index 026fbd7..22e0440 100644
--- a/Xtrans.h
+++ b/Xtrans.h
@@ -52,6 +52,7 @@ from The Open Group.

 #include <X11/Xfuncproto.h>
 #include <X11/Xos.h>
+#include <X11/Xmd.h>

 #ifndef WIN32
 #include <sys/socket.h>

#749120#33
Date:
2015-09-24 21:19:31 UTC
From:
To: