#1087146 libiscsi7:amd64: coredump when starting qemu/libiscsi with iser

Package:
libiscsi7
Source:
libiscsi7
Description:
iSCSI client shared library
Submitter:
IriKa Q
Date:
2024-11-10 21:12:01 UTC
Severity:
normal
Tags:
#1087146#5
Date:
2024-11-09 03:22:34 UTC
From:
To:
diff --git a/lib/iser.c b/lib/iser.c
index 3740564..4e062d8 100644
--- a/lib/iser.c
+++ b/lib/iser.c
@@ -770,7 +770,11 @@ iser_prepare_read_cmd(struct iser_conn *iser_conn,struct iser_pdu *iser_pdu)
        if (data_size > 0) {

                if (task->iovector_in.iov == NULL) {
-                       iser_pdu->iscsi_pdu.indata.data = iscsi_malloc(iscsi, data_size);
+                       if (data_size <= iscsi->smalloc_size) {
+                               iser_pdu->iscsi_pdu.indata.data = iscsi_smalloc(iscsi, data_size);
+                       } else {
+                               iser_pdu->iscsi_pdu.indata.data = iscsi_malloc(iscsi, data_size);
+                       }
                        if (iser_pdu->iscsi_pdu.indata.data == NULL) {
                                iscsi_set_error(iscsi, "Failed to aloocate data buffer");
                                return -1;

#1087146#10
Date:
2024-11-10 08:22:47 UTC
From:
To:
return -ECONTEXT;

Thanks,

/mjt

#1087146#17
Date:
2024-11-10 20:00:49 UTC
From:
To:
I don't understand what you mean?
The patch I sent only fixes a mismatch between allocation and release of
indata.data for iser pdu.
Applying the patch to master branch or 1.20.0 of libscsi has been tested
for a few days and no more coredumps occur.
But from my local test, just applying this patch to 1.19.0 version of
libiscsi cannot completely fix all coredump issues. This may require
updating libiscsi to 1.20.0 version.

Michael Tokarev <mjt@tls.msk.ru> 于2024年11月10日周日 16:22写道:

#1087146#22
Date:
2024-11-10 20:29:57 UTC
From:
To:
10.11.2024 23:00, IriKa Qiu wrote:

I mean that sending patch for an issue without mentioning a word
what the issue actually IS - this sounds a bit strange, don't you find?

I don't have iscsi targets to test anymore, so don't have an (easy) way
to test if a change fixes something or not, but again, without knowing
what the problem is about, it's hardly possible to test it.

I'd suggest to poke the upstream bugtracker too.

Thanks,

/mjt

#1087146#27
Date:
2024-11-10 20:51:46 UTC
From:
To:
Sorry, I am a newbie in the community.
Next, I will try to describe the issue I encountered and the investigation
process.

I tried to build a virtual hypervisor platform based on iser in debian 12,
and the software platform is based on libvirt/qemu.
In my test, the target is located in a debian 12 virtual machine, providing
services through an mxc5 vf virtual NIC, and the service software is based
on LIO.
The initiator is located on the host and connected to the target through
mxc5 pf NIC. The initiator enable iser transport of libvirt/qemu/libiscsi.
In this environment, the initiator (qemu) will have random crashes.
Then I tried to upgrade libiscsi to master (by compiling), and the
probability of crashes was reduced. Then I tried to compile with
-fsanitize=address, and saw the following information from the crashed
process:
functions and may produce false positives in some cases!
0x60200032249a at pc 0x7faf98a47681 bp 0x7fad65173060 sp 0x7fad65172810
../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799
/work/libiscsi/lib/iscsi-command.c:206
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/iscsi.c:1099
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3210
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/raw-format.c:423
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3208
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1655
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1676
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../util/coroutine-ucontext.c:177
[0x602000322490,0x60200032249a)
../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
/work/libiscsi/lib/iser.c:773
/work/libiscsi/lib/iser.c:1001
/work/libiscsi/lib/iscsi-command.c:283
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/iscsi.c:1099
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3210
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/raw-format.c:423
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/io.c:3208
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1655
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../block/block-backend.c:1676
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../util/coroutine-ucontext.c:177
../../../../src/libsanitizer/asan/asan_interceptors.cpp:207
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../util/qemu-thread-posix.c:545
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../iothread.c:215
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../event-loop-base.c:86
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../qom/object_interfaces.c:28
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../qom/object_interfaces.c:125
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../qom/object_interfaces.c:157
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/vl.c:1718
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/vl.c:1896
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/vl.c:3547
/home/builder/debian/qemu-7.2+dfsg/bin/debug/native/../../../softmmu/main.c:47
../sysdeps/nptl/libc_start_call_main.h:58
../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799
in __interceptor_memset

Therefore, it is determined that the allocation of iscsi_pdu->indata.data
is iscsi_malloc, and based on an uncertain size, it may be released through
iscsi_sfree. From the code analysis, the size of the memory released by
iscsi_sfree should be equal to iscsi->smalloc_size, not an uncertain value.
Therefore, a judgment was added to my patch to use iscsi_smalloc to
allocate memory with data_size <= iscsi->smalloc_size. The actual size of
the memory allocated by iscsi_smalloc is equal to iscsi->smalloc_size.

I have submitted a pull request to the above, refer to
https://github.com/sahlberg/libiscsi/pull/431, and the issue refers to
https://github.com/sahlberg/libiscsi/issues/430.

Michael Tokarev <mjt@tls.msk.ru> 于2024年11月11日周一 04:29写道: