As far as I've understood, removal of a symbol requires a major SONAME
bump. Between version 4.3.0 and 4.4.0 the symbol '_TIFFsetString' was
removed while the SONAME was bumped from 5.7.0 -> 5.8.0. The missing
symbol is easily seen in debian/libtiff5.symbols, and the upstream
commit [1].

This is causing some breakage further down the line with these messages,

The error message belongs to the package linuxcnc while the file
'libtifftcl4.1.0.so' belongs to libtk-img.


Regards,
Håvard


[1]
https://gitlab.com/libtiff/libtiff/-/commit/9aeb9f836a444411675a4684ca47683cf556722e

Nasty issue.  I created an issue upstream about it.

How did you discover the problem?  Can it be automatically detected in a
autopkgtest script?

It was actually a bug [1] in linuxcnc that made me look. Then it was just
to search for the library that previously had that symbol.
Should probably have reported back with my findings.
Not sure about autopkgtest, but in this specific case, I believe lintian could
have caught it, if the test existed.
The maintainer removed three symbols from 'debian/libtiff5.symbols' without
a major SONAME bump. Might be an edge case, since the symbols file need to exists
beforehand and also requires that the symbols file is updated.

I've included a diff for libtiff5.symbols between version 4.3.0-8 and 4.4.0-2, three
symbols removed, and four added.

diff -Nru tiff-4.3.0/debian/libtiff5.symbols tiff-4.4.0/debian/libtiff5.symbols
--- tiff-4.3.0/debian/libtiff5.symbols  2022-04-25 22:24:06.000000000 +0200
+++ tiff-4.4.0/debian/libtiff5.symbols  2022-05-30 18:04:05.000000000 +0200
@@ -46,6 +46,8 @@
  TIFFFieldName@LIBTIFF_4.0 4.0.3
  TIFFFieldPassCount@LIBTIFF_4.0 4.0.3
  TIFFFieldReadCount@LIBTIFF_4.0 4.0.3
+ TIFFFieldSetGetCountSize@LIBTIFF_4.0 4.4.0~rc1
+ TIFFFieldSetGetSize@LIBTIFF_4.0 4.4.0~rc1
  TIFFFieldTag@LIBTIFF_4.0 4.0.3
  TIFFFieldWithName@LIBTIFF_4.0 4.0.3
  TIFFFieldWithTag@LIBTIFF_4.0 4.0.3
@@ -88,6 +90,7 @@
  TIFFInitDumpMode@LIBTIFF_4.0 4.0.3
  TIFFInitJBIG@LIBTIFF_4.0 4.0.3
  TIFFInitJPEG@LIBTIFF_4.0 4.0.3
+ (arch=!s390x !hppa !powerpc !ppc64 !sparc64)TIFFInitLERC@LIBTIFF_4.0 4.4.0
  TIFFInitLZMA@LIBTIFF_4.0 4.0.3
  TIFFInitLZW@LIBTIFF_4.0 4.0.3
  TIFFInitNeXT@LIBTIFF_4.0 4.0.3
@@ -100,6 +103,7 @@
  TIFFInitZIP@LIBTIFF_4.0 4.0.3
  TIFFInitZSTD@LIBTIFF_4.0 4.0.9+git181026
  TIFFIsBigEndian@LIBTIFF_4.0 4.0.3
+ TIFFIsBigTIFF@LIBTIFF_4.0 4.4.0~rc1
  TIFFIsByteSwapped@LIBTIFF_4.0 4.0.3
  TIFFIsCODECConfigured@LIBTIFF_4.0 4.0.3
  TIFFIsMSB2LSB@LIBTIFF_4.0 4.0.3
@@ -212,7 +216,6 @@
  _TIFFCheckRealloc@LIBTIFF_4.0 4.0.3
  _TIFFClampDoubleToFloat@LIBTIFF_4.0 4.0.10+git190814
  _TIFFCreateAnonField@LIBTIFF_4.0 4.0.3
- _TIFFDataSize@LIBTIFF_4.0 4.0.3
  _TIFFDefaultStripSize@LIBTIFF_4.0 4.0.3
  _TIFFDefaultTileSize@LIBTIFF_4.0 4.0.3
  _TIFFFax3fillruns@LIBTIFF_4.0 4.0.3
@@ -241,7 +244,6 @@
  _TIFFRewriteField@LIBTIFF_4.0 4.0.3
  _TIFFSeekOK@LIBTIFF_4.0 4.0.9
  _TIFFSetDefaultCompressionState@LIBTIFF_4.0 4.0.3
- _TIFFSetGetFieldSize@LIBTIFF_4.0 4.1.0+git201212
  _TIFFSetupFields@LIBTIFF_4.0 4.0.3
  _TIFFSwab16BitData@LIBTIFF_4.0 4.0.3
  _TIFFSwab24BitData@LIBTIFF_4.0 4.0.3
@@ -266,7 +268,6 @@
  _TIFFsetFloatArray@LIBTIFF_4.0 4.0.3
  _TIFFsetLongArray@LIBTIFF_4.0 4.0.3
  _TIFFsetShortArray@LIBTIFF_4.0 4.0.3
- _TIFFsetString@LIBTIFF_4.0 4.0.3
  _TIFFwarningHandler@LIBTIFF_4.0 4.0.3
  _TIFFwarningHandlerExt@LIBTIFF_4.0 4.0.3
  uv_decode@LIBTIFF_4.0 4.0.3



Regards,
Håvard

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012789

[Håvard F. Aasen]

Aha.  I have linked the two bugs together now, to allow BTS to track the
two alongside each other.

As this bug make the linuxcnc package fail to start, ie 'make unreleated
software on the system break', I believe the correct severity for this
issue is critical.

Note, in the upstream bug there seem to be reluctance to both reinsert
the symbol and bump the SONAME, claiming programs using this symbol
should be changed instead.  It seem like a good idea in any case to fix
libtk-img, but that would be independent of the need for a SONAME bump
due to the ABI change.

Until upstream decide to reinsert the symbol or bump the SONAME, I
suggest to carry this patch.  It should get the broken packages in
Debian working again.

diff -ur tiff-4.4.0/libtiff/tif_dir.c tiff-4.4.0-new/libtiff/tif_dir.c
--- tiff-4.4.0/libtiff/tif_dir.c        2022-05-20 18:15:09.000000000 +0200
+++ tiff-4.4.0-new/libtiff/tif_dir.c    2022-06-24 10:45:10.465540590 +0200
@@ -56,6 +56,8 @@
 }
 void _TIFFsetByteArray(void** vpp, const void* vp, uint32_t n)
     { setByteArray(vpp, vp, n, 1); }
+void _TIFFsetString(char** cpp, char* cp)
+    { setByteArray((void**) cpp, (void*) cp, strlen(cp)+1, 1); }
 static void _TIFFsetNString(char** cpp, const char* cp, uint32_t n)
     { setByteArray((void**) cpp, cp, n, 1); }
 void _TIFFsetShortArray(uint16_t** wpp, const uint16_t* wp, uint32_t n)
diff -ur tiff-4.4.0/libtiff/tiffiop.h tiff-4.4.0-new/libtiff/tiffiop.h
--- tiff-4.4.0/libtiff/tiffiop.h        2022-05-20 18:15:09.000000000 +0200
+++ tiff-4.4.0-new/libtiff/tiffiop.h    2022-06-24 10:45:40.709737278 +0200
@@ -340,6 +340,7 @@
 extern uint32_t _TIFFDefaultStripSize(TIFF* tif, uint32_t s);
 extern void _TIFFDefaultTileSize(TIFF* tif, uint32_t* tw, uint32_t* th);

+extern void _TIFFsetString(char**, const char*);
 extern void _TIFFsetByteArray(void**, const void*, uint32_t);
 extern void _TIFFsetShortArray(uint16_t**, const uint16_t*, uint32_t);
 extern void _TIFFsetLongArray(uint32_t**, const uint32_t*, uint32_t);

In addition the debian/libtiff5.symbols file will need an update.

Control: tags 1012825 patch
Control: severity 1012825 normal
Control: unblock 1012789 by 1012825
 As noted by libtiff upstream as well, this function was internal of
tiff and its declaration was in a header file that was not installed
publicly. They could change that without prior notice.
If linuxcnc-uspace still wants to use it, then copy that function[1]
and its details to their source tree. They already done copying with
the _TIFFsetString() function declaration.
Then I can add a break for its older versions for tiff.

Regards,
Laszlo/GCS
[1] https://gitlab.com/libtiff/libtiff/-/blob/master/libtiff/tif_dir.c#L43

[László Böszörményi]

While it might sound sensible on the surface, the realities is that the
libraries binary interface (aka ABI) changed, removing a public symbol
from the library.  Such API change require a no major SONAME number to
avoid breaking programs using the library.

It is not linuxcnc-uspace that is using it.  It is the tcl/tk Img
library.  To test for yourself, try running 'wish' and give it the
'package require Img' command to load the Img library.  linuxcnc do the
equivalent loading, but using the python Tk library.

Removing the symbol without bumping the SONAME made it impossible for
programs using the symbol to keep the old working library version.  This
was the ratinale for my severity setting critical.  Given that the
symbol removal without bumping SONAME broken libtk-img and linuxcnc,
what is your argument for lowering the severity to normal?

Control: tags 1012825 -patch
Control: clone 1012789 -1
Control: reassign -1 libtk-img
Control: retitle -1 libtk-img: add _TIFFsetString to its internal tiff library
Control: tags -1 +patch
 You are right that removing public symbols from a library interface
is an ABI break and requires a SONAME change. Per coding standards
function names starting with underscore are part of the private API
and a) not to be used outside of the library, b) if used nevertheless,
it's accepted that the other code can break anytime.
 Yup, I was tricked by the bug reports. . In this case, the Python Tk
library must follow the internal change of tiff.
 First of all, critical is used for several issues like making the
system unbootable or causing huge data loss. That's not the case. Then
as noted above, projects using others internal structures and/or
functions must follow that when the latter changes.
What you proposed is to diverge from tiff upstream and adding back the
mentioned function, then forcing a SONAME change, doing a transition
with over two hundred code rebuilds on fourteen architectures. This
makes no sense.
As noted above, the Python Tk library copies an internal tiff function
and probably not just one but a whole set of those (just check its
compat/libtiff/libtiff source directory). It must accept to follow
tiff development and act on such changes. Especially that the
mentioned removed function is a one liner, being a wrapper for another
function. If libtk-img needs that function, right. Copy it to their
code like it copied others already.
See the attached patch, basically it's a one liner. Sergei just needs
to add it to the libtk-img package source.

Regards,
Laszlo/GCS

Control: merge 1013877 1013878
 Then it's just one bug, sorry for the duplication. As soon as Sergei
uploads this fix, linuxcnc will work again as well.
Just a note that tiff will no longer export its private functions,
breaking libtk-img entirely. Of course, tiff upstream reported it for
libtk-img. See its bug tracker [1].

Hope this helps,
Laszlo/GCS
[1] https://sourceforge.net/p/tkimg/bugs/109/

[László Böszörményi]

Which coding standards?  I believe the Debian policy require SONAME
changes when the ABI break.  Did I misunderstand?

I base my understanding on
<URL: https://www.debian.org/Bugs/Developer#severities >, which state
the following for critical:

  * critical - makes unrelated software on the system (or the whole
    system) break, or causes serious data loss, or introduces a security
    hole on systems where you install the package.

My observation is that the libtiff change 'made unreleated software on
the system break', making this a bug with severity 'critical'.

To me it is the only approach that make sense when the ABI is broken.

Note, I do not try to defend the libtk-img developers.  To me their
behaviour is beside the point, which is that the tiff ABI broke (a
public symbol was removed) and the SONAME as not bumped.