Dear Maintainer,

When I try to add the noise suppress filter on my mic, obs stops with a
segfault.


This is the output on the console:

info: User added filter 'Noise Suppression' (noise_suppress_filter_v2) to
source 'Capture Audio (Entrée PulseAudio)'
Erreur de segmentation (core dumped)


This is what gdp says about the core file:

Core was generated by `obs'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fd338c7eb71 in _celt_autocorr () from /lib/x86_64-linux-
gnu/libcodec2.so.0.9
[Current thread is 1 (Thread 0x7fd2f25c7700 (LWP 3578386))]
(gdb) bt
#0  0x00007fd338c7eb71 in _celt_autocorr () at /lib/x86_64-linux-
gnu/libcodec2.so.0.9
#1  0x00007fd338c7ed2a in pitch_downsample () at /lib/x86_64-linux-
gnu/libcodec2.so.0.9
#2  0x00007fd2f3880006 in rnnoise_process_frame () at /usr//lib/x86_64-linux-
gnu/obs-plugins/obs-filters.so
#3  0x00007fd2f3892f4c in  () at /usr//lib/x86_64-linux-gnu/obs-plugins/obs-
filters.so
#4  0x00007fd33c155b8b in obs_source_output_audio () at /lib/x86_64-linux-
gnu/libobs.so.0
#5  0x00007fd32800be15 in  () at /usr//lib/x86_64-linux-gnu/obs-plugins/linux-
pulseaudio.so
#6  0x00007fd336823b2d in  () at /lib/x86_64-linux-gnu/libpulse.so.0
#7  0x00007fd33426dbf5 in  () at /usr/lib/x86_64-linux-
gnu/pulseaudio/libpulsecommon-13.0.so
#8  0x00007fd3342705ab in  () at /usr/lib/x86_64-linux-
gnu/pulseaudio/libpulsecommon-13.0.so
#9  0x00007fd334270966 in  () at /usr/lib/x86_64-linux-
gnu/pulseaudio/libpulsecommon-13.0.so
#10 0x00007fd3342711ba in  () at /usr/lib/x86_64-linux-
gnu/pulseaudio/libpulsecommon-13.0.so
#11 0x00007fd3368391a3 in pa_mainloop_dispatch () at /lib/x86_64-linux-
gnu/libpulse.so.0
#12 0x00007fd3368394ce in pa_mainloop_iterate () at /lib/x86_64-linux-
gnu/libpulse.so.0
#13 0x00007fd336839570 in pa_mainloop_run () at /lib/x86_64-linux-
gnu/libpulse.so.0
#14 0x00007fd336847439 in  () at /lib/x86_64-linux-gnu/libpulse.so.0
#15 0x00007fd334280268 in  () at /usr/lib/x86_64-linux-
gnu/pulseaudio/libpulsecommon-13.0.so
#16 0x00007fd33b1c0ea7 in start_thread (arg=<optimized out>) at
pthread_create.c:477
#17 0x00007fd33b0f0d4f in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)


Possibly the issue comes from libcodec2.so.0.9, or due to some incompatibility
with obs filters.

Best regards
Christophe

Dear Maintainer,
I tried to have a look at this crash and could it reproduce it
inside a minimal testing amd64 VM.

A backtrace with full debug symbols in [1].

The crash itself happens because of a stack exhaustion, because in
_celt_autocorr a 2015 MB array should be constructed.

But this seems to be just a consequence of function compute_frame_features
calling function pitch_downsample, which is unfortunately first found by
the dynamic linker in libcodec2.so.0.9 instead of obs-filters.so.
And the function signatures are quite different.

Therefore a cheap workaround currently might be to start obs-studio with
obs-filters preloaded. That way at least the denoise filter can be added.

     LD_PRELOAD=/usr/lib/x86_64-linux-gnu/obs-plugins/obs-filters.so obs

But might break when libcodec2.so.0.9 wants to call its function pitch_downsample.
Another more reliable option might be to rename
that and the other common functions.
The last part of the attached file shows all common function
between obs-filters.so and libcodec2.so.0.9.

Kind regards,
Bernhard


[1]
     (gdb) bt
     #0  0x00007f1bee6e4b71 in _celt_autocorr (x=x@entry=0x7f1aacfee8e8, ac=ac@entry=0x7f1aacfee7c0, window=window@entry=0x0, overlap=overlap@entry=0, lag=lag@entry=4, n=n@entry=-1392576048) at ./lpcnet/src/celt_lpc.c:216
     #1  0x00007f1bee6e4d2a in pitch_downsample (x_lp=x_lp@entry=0x7f1aacfee8e8, len=len@entry=-1392576048) at ./lpcnet/src/pitch.c:160
     #2  0x00007f1b30293006 in compute_frame_features (in=0x7f1aacfeeac0, features=0x7f1aacfeea10, Exp=0x7f1aacfee9b0, Ep=0x7f1aacfee950, Ex=0x7f1aacfee8f0, P=0x7f1aacff2560, X=0x7f1aacff1650, st=0x562d5f38cff0) at ./plugins/obs-filters/rnnoise/src/denoise.c:326
     #3  rnnoise_process_frame (st=0x562d5f38cff0, out=<optimized out>, in=<optimized out>) at ./plugins/obs-filters/rnnoise/src/denoise.c:471
     #4  0x00007f1b302a5f4c in process_rnnoise (ng=0x562d5f373f40) at ./plugins/obs-filters/noise-suppress-filter.c:343
     #5  process (ng=<optimized out>) at ./plugins/obs-filters/noise-suppress-filter.c:392
     #6  noise_suppress_filter_audio (data=0x562d5f373f40, audio=<optimized out>) at ./plugins/obs-filters/noise-suppress-filter.c:468
     #7  0x00007f1bf1beab8b in filter_async_audio (in=<optimized out>, source=0x562d5f26bf00) at ./libobs/obs-source.c:3068
     #8  obs_source_output_audio (source=0x562d5f26bf00, audio=audio@entry=0x7f1aacff81f0) at ./libobs/obs-source.c:3246
     #9  0x00007f1bcc002774 in _alsa_listen (attr=0x7f1be000dc20) at ./plugins/linux-alsa/alsa-input.c:574
     #10 0x00007f1bf0c26ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
     #11 0x00007f1bf0b56d4f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95