OCaml programs which create Cairo surfaces via Cairo_bigarray leak memory.
Following test program reproduces the bug. The program allocates many 100Mb arrays, but
explicit garbage collector invocation should keep the total memory usage low.


$ cat bug.ml
#use "topfind"
#require "bigarray,cairo"

open Bigarray

let test () =
   let u = Array2.create int8_unsigned c_layout 10000 10000 in
   ignore (Cairo_bigarray.of_bigarr_8 u)

let () =
   for k = 1 to 100 do Gc.compact (); test () done

$ ocaml bug.ml
Out of memory during evaluation.


It seems that the OCaml runtime does not free memory associated with the bigarray "u".
Curiously, following program works as expected. Note the last line of function "test", which
prevents deallocation of the bigarray in preceding GC run.


$ cat nobug.ml
#use "topfind"
#require "bigarray,cairo"

open Bigarray

let test () =
   let u = Array2.create int8_unsigned c_layout 10000 10000 in
   ignore (Cairo_bigarray.of_bigarr_8 u);
   Gc.compact ();
   ignore u

let () =
   for k = 1 to 100 do Gc.compact (); test () done

$ ocaml nobug.ml
(no output)

tags 524908 + moreinfo
thanks

I cannot reproduce this with version 20090223-5 (current testing), nor
with 1:1.2.0-1 (current unstable). Is this bug still relevant?


Cheers,

Hi,

   Thank you for investigating. I've just tried to run the old test code and I can still
reproduce the bug. I use Debian Stable, package versions

  libcairo-ocaml-dev        20090223-3+b2
  ocaml-findlib             1.2.5+debian-1+b1
  ocaml-nox                 3.11.2-2

I do not have debian testing installed, so I did not test version 20090223-5.

   I've run the test programs bug.ml and nobug.ml (mentioned in the original bug report)
under strace.

$ strace ocaml bug.ml 2> bug.strace
$ strace ocaml nobug.ml 2> nobug.strace

Please look at the mmap/munmap calls at the very end of the strace logs. Clearly, bug.ml
allocates until exhaustion of the address space. On the other hand, nobug.ml correctly
releases the memory. Note that I have 32-bit system.

Could you please run similar tests? If you test on 64-bit system and cannot reproduce the
bug, you can try bigger array size / iteration count.

Best wishes,

   Oto Havle.

Stéphane Glondu wrote:

tags 524908 - moreinfo
tags 524908 + confirmed
found 524908 1:1.2.0-2
thanks

Le 02/08/2011 21:52, Oto Havle a écrit :

I didn't take this into account, and I can indeed reproduce the bug in
an up-to-date i386 sid chroot, with version 1:1.2.0-2.


Cheers,

tags 524908 + help upstream
thanks

Le 03/08/2011 08:22, Stéphane Glondu a écrit :

By looking a bit deeper, I can observe that bug.ml fails on both
bytecode and native, but nobug.ml fails only on native (and succeeds on
bytecode as originally stated in the bug report).

Looking even further at the failing bytecode case, after instrumenting
the code (cairo-ocaml and ocaml GC code), I can see that cairo_surface_t
finalizer is always called when expected.
ml_cairo_image_surface_create_for_data does register a global root, but
caml_{register/remove}_global_root calls are always paired. I tried to
remove the global root registration, and the problem disappears.

By the way, not all the recommendations from the section "Living in
harmony with the garbage collector" of the manual are respected: there
are many missing CAMLparam*, CAMLlocal* and CAMLreturn* invocations. I
tried to add some, but was unsuccessful in solving the problem.

Stéphane Glondu wrote:
 > I can see that cairo_surface_t finalizer is always called when expected.
 > ml_cairo_image_surface_create_for_data does register a global root, but
 > caml_{register/remove}_global_root calls are always paired.

   The bug scenario in fact involves two custom blocks - cairo_surface_t and bigarray.
The cairo_surface_t finalizer (ml_final_cairo_surface_t) is called correctly, but the
bigarray finalizer (caml_ba_finalize) is not called at all. The bigarray value somehow
disappears from the OCaml heap. The corresponding bigarray data on the C heap is not reclaimed.

   I've found a thread from caml-list archives which suggests that calling
caml_{register/remove}_global_root from custom block finalizers might not be safe:

http://www.mail-archive.com/caml-list@yquem.inria.fr/msg03746.html

   I've tried to relate the bug to the OCaml GC. My current idea is as follows:

The custom block

- is always allocated in the major heap
- is reclaimed and finalizer is called in sweep_slice() in major_gc.c
- is also reclaimed in caml_compact_heap() in compact.c - and finalizer is not called!

compact.c:
     310           /* No pointers to the header and no infix header:
     311              the object was free. */
     312           *p = Make_header (Wosize_ehd (q), Tag_ehd (q), Caml_blue);
     313           p += Whsize_ehd (q);

   The custom block is manipulated in this part of the GC code (confirmed by inserting a tag
check here, and running bug.ml through the modified ocamlrun). Moreover, the bug does not
appear if Gc.compact is replaced by Gc.full_major.

   Maybe someone who really understands OCaml GC code can confirm it?

No, this is not the case, quoting Xavier Leroy from that same thread: