The following program

#include <mcheck.h>
#include <stdio.h>

int main (void)
{
  int r;

  r = mcheck (NULL);
  printf ("%d\n", r);
  return 0;
}

outputs -1, which is incorrect. It should have been 0.

The glibc manual says

  It is too late to begin allocation checking once you have allocated
  anything with ‘malloc’.  So ‘mcheck’ does nothing in that case.
  The function returns ‘-1’ if you call it too late, and ‘0’
  otherwise (when it is successful).

Since there hasn't been any malloc yet, 0 should have been output.

Similarly, the example given in the mcheck(3) man page fails
unexpectedly.

Hello,
I did a series [1] of tests and found this seems to worked last in Buster.

With Bullseye it fails because there is a malloc before main is reached,
somewhere inside the dynamic loader [2].
Maybe this would work with a static build.

In Bookworm and Trixie [3] the function mcheck always returns -1,
because "IS_IN (libc)" seems to be true at compile time.

Reading the upstream commit comment [4] it looks like mcheck should
just work when using libc_malloc_debug.so.

And indeed following also prints 0 in Trixie:
   LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libc_malloc_debug.so.0 ./mcheck-test

Kind regards,
Bernhard



[1]
glibc-2.28: prints 0    (10 Buster/oldoldstable)
glibc-2.31: prints -1   (11 Bullseye/oldstable)
glibc-2.36: prints -1   (12 Bookworm/stable)
glibc-2.40: prints -1   (13 Trixie/testing)




[2] Bullseye: malloc before main is reached.
(rr) bt
#0  ptmalloc_init () at arena.c:400
#1  0x00007f63d8948101 in ptmalloc_init () at arena.c:291
#2  malloc_hook_ini (sz=32, caller=<optimized out>) at hooks.c:31
#3  0x00007f63d8948d5a in __libc_calloc (n=<optimized out>, elem_size=<optimized out>) at malloc.c:3387
#4  0x00007f63d88bdb18 in _dlerror_run (operate=operate@entry=0x7f63d88bd3a0 <dlsym_doit>, args=args@entry=0x7ffd28cf5d10) at dlerror.c:148
#5  0x00007f63d88bd41c in __dlsym (handle=<optimized out>, name=<optimized out>) at dlsym.c:70
#6  0x00007f63d8a9e1c6 in init_process () at ./src/preload/syscallbuf.c:778
#7  0x00007f63d8ac0fe2 in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7ffd28cf5fc8, env=env@entry=0x7ffd28cf5fd8) at dl-init.c:72
#8  0x00007f63d8ac10e9 in call_init (env=0x7ffd28cf5fd8, argv=0x7ffd28cf5fc8, argc=1, l=<optimized out>) at dl-init.c:30
#9  _dl_init (main_map=0x7f63d8add180, argc=1, argv=0x7ffd28cf5fc8, env=0x7ffd28cf5fd8) at dl-init.c:119
#10 0x00007f63d8ab20ca in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#11 0x0000000000000001 in ?? ()



[3] Bookworm, Trixie:
(rr) list
32
33      int
34      mcheck (void (*func) (enum mcheck_status))
35      {
36      #if IS_IN (libc)
37        return -1;
38      #else
39        return __mcheck_initialize (func, false);
40      #endif
41      }
(rr) disassemble mcheck
Dump of assembler code for function mcheck:
=> 0x00007f290d5bd290 <+0>:     mov    $0xffffffff,%eax
    0x00007f290d5bd295 <+5>:     ret
End of assembler dump.



[4] https://sourceware.org/git/?p=glibc.git;a=commit;f=malloc/mcheck.c;h=c142eb253f3814f46527e9b37484041dd85702cf

On 2024-09-09 11:42:43 +0200, Bernhard Übelacker wrote:
[...]
[...]

Thanks. I've just reported the bug upstream.

In short:

Returning -1 is incorrect as it can introduce an error, while
not preloading libc_malloc_debug is not an error (this just
means that one does not wish to check memory). Returning 0
would be OK, since when one does not check memory, mcheck()
is never called too late (as its call should not have any
consequence).

Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie
Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran,
Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun.

Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie
für weitere Details.

Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie
Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran,
Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun.

Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie
für weitere Details.