Package: debugedit
Version: 4.16.1.2+dfsg1-3
Severity: important
User: helmutg@debian.org
Usertags: rebootstrap
User: debian-cross@lists.debian.org
Usertags: ftcbfs
Control: affects -1 + src:binutils
X-Debbugs-Cc: pkg-gnutls-maint@lists.alioth.debian.org
This is a strange bug. It all started as a cross build failure of
binutils. Upon closer inspection, I figured that debugedit is
segfaulting in the following line of the binutils cross build:
| : # Strip shared libraries and binaries
| set -e; nfiles=0; for i in debian/libbinutils/usr/lib/m68k-linux-gnu/libbfd-*so debian/libbinutils/usr/lib/m68k-linux-gnu/libopcodes-*so debian/libbinutils/usr/lib/m68k-linux-gnu/libctf*.so.0.0.0 $(file debian/libbinutils/usr/bin/* |awk -F: '$0 !~ /script/ {print $1}'); do test ! -h $i || continue; test -f $i || continue; files="$files $i"; nfiles=$(expr $nfiles + 1); done; for i in $files; do id=$(debugedit --build-id --build-id-seed='libbinutils-2.37-7' $i); done; mkdir -p debian/libbinutils-dbg/usr/lib/debug/.dwz/m68k-linux-gnu; dwz=usr/lib/debug/.dwz/m68k-linux-gnu/libbinutils.debug; if [ $nfiles -gt 1 ]; then dwz -m debian/libbinutils-dbg/$dwz -M /$dwz $files; m68k-linux-gnu-objcopy --compress-debug-sections debian/libbinutils-dbg/$dwz; else dwz $files; fi; for i in $files; do b_id=$(LC_ALL=C m68k-linux-gnu-readelf -n $i | sed -n 's/ *Build ID: *\([0-9a-f][0-9a-f]*\)/\1/p'); if [ -z "$b_id" ]; then id=$(echo $i | sed -r 's,debian/[^/]+, debian/libbinutils-dbg/usr/lib/debug,'); echo strip $i; mkdir -p $(dirname $id); m68k-linux-gnu-objcopy --only-keep-debug $i $id; chmod 644 $id; m68k-linux-gnu-strip -R .comment -R .note $i; m68k-linux-gnu-objcopy --add-gnu-debuglink $id $i; else echo "ID: ${b_id} -> $(echo $i | sed 's,debian/libbinutils,,')"; d=usr/lib/debug/.build-id/${b_id:0:2}; f=${b_id:2}.debug; mkdir -p debian/libbinutils-dbg/$d; m68k-linux-gnu-objcopy --only-keep-debug --compress-debug-sections $i debian/libbinutils-dbg/$d/$f; chmod 644 debian/libbinutils-dbg/$d/$f; m68k-linux-gnu-strip -R .comment -R .note $i; fi; done
If that sounds a bit complex, you're not alone. I managed to trim it
down to this:
| debugedit --build-id --build-id-seed=x libbfd-2.37-system.so
I'm attaching a suitable libbfd-2.37-system.so that makes it segfault to
this mail. If you prefer building your own, run:
| sbuild -d unstable --host=s390x --anything-failed-commands=%SBUILD_SHELL binutils
Now rpm wasn't uploaded in a while, so it seems unlikely that it
actually caused the failure. Fortunately, there is debbisect and
debbisect tells us:
| bisection finished successfully
| last good timestamp: 2021-09-04 03:49:25+00:00
| first bad timestamp: 2021-09-04 08:39:14+00:00
| only one package differs: libgcrypt20:amd64 1.8.7-6 -> 1.9.4-2
Accordingly, I'm putting libgcrypt20 maintainers into Cc.
Now for the actual segfault. valgrind has this to say:
| Invalid read of size 1
| at 0x48434A0: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
| by 0x4B9497A: ??? (in /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4)
| by 0x4B8C0FE: ??? (in /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4)
| by 0x4890D11: rpmDigestUpdate (in /usr/lib/x86_64-linux-gnu/librpmio.so.9.1.2)
| by 0x10B463: ??? (in /usr/lib/rpm/debugedit)
| by 0x49C5E49: (below main) (libc-start.c:314)
| Address 0x0 is not stack'd, malloc'd or (recently) free'd
And gdb says:
| #0 __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:311
| No locals.
| #1 0x00007ffff7ba797b in ?? () from /usr/lib/x86_64-linux-gnu/libgcrypt.so.20
| No symbol table info available.
| #2 0x00007ffff7b9f0ff in ?? () from /usr/lib/x86_64-linux-gnu/libgcrypt.so.20
| No symbol table info available.
| #3 0x00007ffff7f8dd12 in rpmDigestUpdate (ctx=ctx@entry=0x5555555e1350, data=<optimized out>, len=<optimized out>)
| at digest_libgcrypt.c:94
| No locals.
| #4 0x0000555555557464 in handle_build_id (build_id_size=20, build_id_offset=16, build_id=0x55555559e288,
| dso=0x55555559f710) at tools/debugedit.c:3206
| d = <optimized out>
| u = {ehdr = {e_ident = "\000\000\000\323\000\000\000\b\000\000\000\000\000\000\000\003", e_type = 0,
| e_machine = 0, e_version = 1822624256, e_entry = 0, e_phoff = 14123569906410586112, e_shoff = 0, e_flags = 0,
| e_ehsize = 0, e_phentsize = 1024, e_phnum = 0, e_shentsize = 0, e_shnum = 0, e_shstrndx = 0}, phdr = {
| p_type = 3539992576, p_flags = 134217728, p_offset = 216172782113783808, p_vaddr = 7828111572416331776,
| p_paddr = 0, p_filesz = 14123569906410586112, p_memsz = 0, p_align = 288230376151711744}, shdr = {
| sh_name = 3539992576, sh_type = 134217728, sh_flags = 216172782113783808, sh_addr = 7828111572416331776,
| sh_offset = 0, sh_size = 14123569906410586112, sh_link = 0, sh_info = 0, sh_addralign = 288230376151711744,
| sh_entsize = 0}}
| x = {d_buf = 0x7fffffffdfc0, d_type = ELF_T_SHDR, d_version = 1, d_size = 64, d_off = 0, d_align = 0}
| ctx = 0x5555555e1350
| algorithm = <optimized out>
| i = 23
| digest = 0x0
| len = 140737488347072
| ctx = <optimized out>
| algorithm = <optimized out>
| i = <optimized out>
| digest = <optimized out>
| len = <optimized out>
| print = <optimized out>
| bad = <optimized out>
| u = {ehdr = {e_ident = {<optimized out> <repeats 16 times>}, e_type = <optimized out>,
| e_machine = <optimized out>, e_version = <optimized out>, e_entry = <optimized out>,
| e_phoff = <optimized out>, e_shoff = <optimized out>, e_flags = <optimized out>, e_ehsize = <optimized out>,
| e_phentsize = <optimized out>, e_phnum = <optimized out>, e_shentsize = <optimized out>,
| e_shnum = <optimized out>, e_shstrndx = <optimized out>}, phdr = {p_type = <optimized out>,
| p_flags = <optimized out>, p_offset = <optimized out>, p_vaddr = <optimized out>, p_paddr = <optimized out>,
| p_filesz = <optimized out>, p_memsz = <optimized out>, p_align = <optimized out>}, shdr = {
| sh_name = <optimized out>, sh_type = <optimized out>, sh_flags = <optimized out>, sh_addr = <optimized out>,
| sh_offset = <optimized out>, sh_size = <optimized out>, sh_link = <optimized out>, sh_info = <optimized out>,
| sh_addralign = <optimized out>, sh_entsize = <optimized out>}}
| x = {d_buf = <optimized out>, d_type = <optimized out>, d_version = <optimized out>, d_size = <optimized out>,
| d_off = <optimized out>, d_align = <optimized out>}
| d = <optimized out>
| id = <optimized out>
| hex = <optimized out>
| #5 main (argc=<optimized out>, argv=<optimized out>) at tools/debugedit.c:3512
| dso = <optimized out>
| fd = <optimized out>
| i = <optimized out>
| file = <optimized out>
| optCon = <optimized out>
| nextopt = <optimized out>
| args = <optimized out>
| stat_buf = {st_dev = 29, st_ino = 25946610, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000,
| __pad0 = 0, st_rdev = 0, st_size = 690352, st_blksize = 4096, st_blocks = 1352, st_atim = {tv_sec = 1632767282,
| tv_nsec = 537189525}, st_mtim = {tv_sec = 1632764390, tv_nsec = 742576993}, st_ctim = {tv_sec = 1632767282,
| tv_nsec = 537189525}, __glibc_reserved = {0, 0, 0}}
| build_id = <optimized out>
| build_id_offset = <optimized out>
| build_id_size = <optimized out>
| need_update = <optimized out>
| macro_sec = <optimized out>
| types_sec = <optimized out>
Maybe understanding the relevant ELF sections a bit better helps. Here
is the readelf output:
| Section Headers:
| [Nr] Name Type Addr Off Size ES Flg Lk Inf Al
| [ 0] NULL 00000000 000000 000000 00 0 0 0
| [ 1] .note.gnu.bu[...] NOTE 00000114 000114 000024 00 A 0 0 4
| [ 2] .hash HASH 00000138 000138 0013f8 04 A 4 0 4
| [ 3] .gnu.hash GNU_HASH 00001530 001530 001478 04 A 4 0 4
| [ 4] .dynsym DYNSYM 000029a8 0029a8 002f30 10 A 5 2 4
| [ 5] .dynstr STRTAB 000058d8 0058d8 003f65 00 A 0 0 1
| [ 6] .gnu.version VERSYM 0000983e 00983e 0005e6 02 A 4 0 2
| [ 7] .gnu.version_r VERNEED 00009e24 009e24 0000e0 00 A 5 3 4
| [ 8] .rela.dyn RELA 00009f04 009f04 0117d8 0c A 4 0 4
| [ 9] .rela.plt RELA 0001b6dc 01b6dc 001470 0c AI 4 21 4
| [10] .init PROGBITS 0001cb4c 01cb4c 000026 00 AX 0 0 2
| [11] .plt PROGBITS 0001cb74 01cb74 002224 14 AX 0 0 4
| [12] .text PROGBITS 0001ed98 01ed98 0699ac 00 AX 0 0 4
| [13] .fini PROGBITS 00088744 088744 000016 00 AX 0 0 2
| [14] .rodata PROGBITS 0008875a 08875a 0167b3 00 A 0 0 2
| [15] .eh_frame_hdr PROGBITS 0009ef10 09ef10 000024 00 A 0 0 4
| [16] .eh_frame PROGBITS 0009ef34 09ef34 000080 00 A 0 0 4
| [17] .init_array INIT_ARRAY 000a2222 0a0222 000004 04 WA 0 0 2
| [18] .fini_array FINI_ARRAY 000a2226 0a0226 000004 04 WA 0 0 2
| [19] .data.rel.ro PROGBITS 000a222a 0a022a 005cd6 00 WA 0 0 2
| [20] .dynamic DYNAMIC 000a7f00 0a5f00 000100 08 WA 5 0 4
| [21] .got PROGBITS 000a8000 0a6000 001ca4 04 WA 0 0 4
| [22] .data PROGBITS 000a9ca4 0a7ca4 0006c8 00 WA 0 0 4
| [23] .bss NOBITS 000aa36c 0a836c 0001c4 00 WA 0 0 4
| [24] .gnu_debugaltlink PROGBITS 00000000 0a836c 000049 00 0 0 1
| [25] .shstrtab STRTAB 00000000 0a83b5 0000ea 00 0 0 1
In the above, i is the section index and it happens to be 23, which
refers to the .bss section of type NOBITS. In theory, the condition
https://sources.debian.org/src/rpm/4.16.1.2+dfsg1-3/tools/debugedit.c/#L3201
should cause the relevant rpmDigestUpdate call to be skipped, but that's
not what is happening here. Indeed, u.shdr.sh_type happens to be
0x8000000, which happens to be the big endian of SHT_NOBITS.
In rebootstrap, I've seen more binutils builds fail for other
architectures. Those were ppc64, s390x, sparc and sparc64. It's all big
endian!
So here comes my hypothesis:
debugedit's handle_build_id function iterates over all elf section
headers and attempts to skip all SHT_NOBITS section headers. For big
endian ELF objects, the check fails and the thus the .bss section is
hashed, which causes a NULL pointer dereference since libgcrypt20 got
stricter.
I have no clue why this ever worked.
Helmut