- Package:
- libmimalloc3
- Source:
- libmimalloc3
- Description:
- compact general purpose allocator with excellent performance
- Submitter:
- Date:
- 2025-09-03 08:55:01 UTC
- Severity:
- normal
~# apt-get install --purge context
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libmimalloc3 luametatex
The following NEW packages will be installed:
libmimalloc3 luametatex
The following held packages will be changed:
context
The following packages will be upgraded:
context
1 upgraded, 2 newly installed, 0 to remove and 1 not upgraded.
Need to get 99.2 MB of archives.
After this operation, 6,011 kB disk space will be freed.
Do you want to continue? [Y/n]
Get:1 https://deb.debian.org/debian unstable/main amd64 libmimalloc3
amd64 3.0.3+ds-1 [58.1 kB]
Get:2 https://deb.debian.org/debian trixie/main amd64 context all
2024.04.01.20240428+dfsg-2 [97.8 MB]
Get:3 https://deb.debian.org/debian trixie/main amd64 luametatex amd64
2.11.02+really2.11.01+ds-1+b1 [1,344 kB]
Fetched 99.2 MB in 10s (9,876 kB/s)
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
Reading changelogs... Done
apt-listchanges: Do you want to continue? [Y/n]
Selecting previously unselected package libmimalloc3:amd64.
(Reading database ... 470453 files and directories currently installed.)
Preparing to unpack .../libmimalloc3_3.0.3+ds-1_amd64.deb ...
Unpacking libmimalloc3:amd64 (3.0.3+ds-1) ...
Preparing to unpack .../context_2024.04.01.20240428+dfsg-2_all.deb ...
Unpacking context (2024.04.01.20240428+dfsg-2) over
(2021.03.05.20230120+dfsg-1+deb12u1) ...
Selecting previously unselected package luametatex.
Preparing to unpack .../luametatex_2.11.02+really2.11.01+ds-1+b1_amd64.deb ...
Unpacking luametatex (2.11.02+really2.11.01+ds-1+b1) ...
Setting up libmimalloc3:amd64 (3.0.3+ds-1) ...
Setting up luametatex (2.11.02+really2.11.01+ds-1+b1) ...
Setting up context (2024.04.01.20240428+dfsg-2) ...
Processing triggers for auctex (13.2-1) ...
Processing triggers for tex-common (6.19) ...
Running mktexlsr. This may take some time... done.
Running mtxrun --generate. This may take some time...
mtxrun --generate failed. Output has been stored in
/tmp/mtxrun.p2dVjw0e
Please include this file if you report a bug.
dpkg: error processing package tex-common (--configure):
installed tex-common package post-installation script subprocess
returned error exit status 1
Processing triggers for fontconfig (2.15.0-2.3) ...
Processing triggers for libc-bin (2.41-8) ...
Processing triggers for man-db (2.13.1-1) ...
Errors were encountered while processing:
tex-common
E: Sub-process /usr/bin/dpkg returned an error code (1)
The contents of /tmp/mtxrun.p2dVjw0e are:
~# cat /tmp/mtxrun.p2dVjw0e
Illegal instruction
I've taken a look at the mimalloc-3.0.3 source. The setting
MI_OPT_ARCH=OFF in debian/rules appears to
invoke a falsehood if my interpretation of
mimalloc-3.0.3+ds/CMakeLists.txt is correct:
# Compiler and architecture specific flags
if(CMAKE_C_COMPILER_ID MATCHES "AppleClang|Clang|GNU|Intel" AND NOT
CMAKE_SYSTEM_NAME MATCHES "Haiku")
# ^true
if(MI_OPT_ARCH)
# ^false - go to continue...
if(APPLE AND CMAKE_C_COMPILER_ID MATCHES "AppleClang|Clang" AND
CMAKE_OSX_ARCHITECTURES) # to support multi-arch binaries (#999)
if("arm64" IN_LIST CMAKE_OSX_ARCHITECTURES)
list(APPEND MI_OPT_ARCH_FLAGS "-Xarch_arm64;-march=armv8.1-a")
endif()
if("x86_64" IN_LIST CMAKE_OSX_ARCHITECTURES)
list(APPEND MI_OPT_ARCH_FLAGS
"-Xarch_x86_64;-march=haswell;-Xarch_x86_64;-mavx2")
endif()
# continue...
elseif(MI_ARCH STREQUAL "x64")
# ^true. Architecture is x86-64
set(MI_OPT_ARCH_FLAGS "-march=haswell;-mavx2") # fast bit scan
(since 2013)
# ^These flags force Haswell architecture or above and allow the
generation of AVX2 code
elseif(MI_ARCH STREQUAL "arm64")
set(MI_OPT_ARCH_FLAGS "-march=armv8.1-a") # fast atomics
(since 2016)
endif()
endif()
endif()
arch baselines violations are serious. C.
[...] I don't think this diagnosis can be the right one, because https://buildd.debian.org/status/fetch.php?pkg=mimalloc&arch=amd64&ver=3.0.3%2Bds-1&stamp=1743673297&raw=0 is verbose enough to show compiler command lines and they don't include -march. Something else must be going on ... Thanks,
Control: severity -1 important
I'm downgrading this for the moment as I can't currently find evidence
that it's a baseline violation. I've tried this in various ancient qemu
CPU models ("-cpu Conroe", "-cpu qemu64", "-cpu core2duo"), and it seems
fine there. I'm prepared to believe that I've missed something, but
figuring it out seems like a bit of a fishing expedition.
Adam, could you please give some details about your machine, and attach
your /proc/cpuinfo file? That might help narrow things down.
Thanks,
On Wed, 13 Aug 2025 at 23:06, Colin Watson <cjwatson@debian.org> wrote:
...
Hi Colin! It's a baseline violation. Your analysis of the build files
was helpful but ultimately I just had to check the dmesg log for the
segfault and look up the offset in the shared library:
traps: mtxrun[62011] trap invalid opcode ip:7fe6e4f64988
sp:7ffe42301c80 error:0 in libmimalloc.so.3.0[c988,7fe6e4f5e000+15000]
c988: f3 48 0f b8 c2 popcnt rax,rdx
Here are the additional occurrences of popcnt in the binary:
$ objdump -d -m i386:x86-64:intel
./x86_64-linux-gnu/libmimalloc.so.3.0|grep popcnt
c988: f3 48 0f b8 c2 popcnt rax,rdx
cbec: f3 49 0f b8 c1 popcnt rax,r9
cd56: f3 4c 0f b8 d6 popcnt r10,rsi
d4e3: f3 48 0f b8 c2 popcnt rax,rdx
d585: f3 4d 0f b8 f5 popcnt r14,r13
d5ca: f3 48 0f b8 c2 popcnt rax,rdx
def4: f3 4c 0f b8 df popcnt r11,rdi
e273: f3 48 0f b8 c7 popcnt rax,rdi
ecd2: f3 49 0f b8 c2 popcnt rax,r10
ee21: f3 49 0f b8 e8 popcnt rbp,r8
f095: f3 48 0f b8 c3 popcnt rax,rbx
f1f9: f3 4d 0f b8 d8 popcnt r11,r8
This library is going to segfault on hardware pre-Sandy Bridge.
The primary source uses a generic implementation of popcount or
built-in intrinsics, which should be OK if baseline code is in fact
being generated:
https://github.com/microsoft/mimalloc/blob/09a27098aa6e9286518bd9c74e6ffa7199c3f04e/include/mimalloc/internal.h#L1069
Regards,
Adam
OK. I think this is coming from supposedly CPUID-guarded code:
$ git grep -i popcnt
include/mimalloc/bits.h:extern bool _mi_cpu_has_popcnt;
include/mimalloc/bits.h: if mi_unlikely(!_mi_cpu_has_popcnt) { return _mi_popcount_generic(x); }
include/mimalloc/bits.h: __asm ("popcnt\t%1,%0" : "=r"(r) : "r"(x) : "cc");
include/mimalloc/bits.h: if mi_unlikely(!_mi_cpu_has_popcnt) { return _mi_popcount_generic(x); }
include/mimalloc/bits.h: return (size_t)mi_msc_builtinz(__popcnt)(x);
include/mimalloc/bits.h: return (size_t)mi_msc_builtinz(__popcnt)(x);
src/init.c:mi_decl_cache_align bool _mi_cpu_has_popcnt = false;
src/init.c: _mi_cpu_has_popcnt = ((cpu_info[2] & (1 << 23)) != 0); // bit 23 of ECX : see <https://en.wikipedia.org/wiki/CPUID#EAX=1:_Processor_Info_and_Feature_Bits>
src/init.c: _mi_cpu_has_popcnt = true;
Here's the relevant code (for GCC/amd64):
#if !defined(__BMI1__)
if mi_unlikely(!_mi_cpu_has_popcnt) { return _mi_popcount_generic(x); }
#endif
size_t r;
__asm ("popcnt\t%1,%0" : "=r"(r) : "r"(x) : "cc");
return r;
And:
static void mi_detect_cpu_features(void) {
// FSRM for fast short rep movsb/stosb support (AMD Zen3+ (~2020) or Intel Ice Lake+ (~2017))
// EMRS for fast enhanced rep movsb/stosb support
uint32_t cpu_info[4];
if (mi_cpuid(cpu_info, 7)) {
_mi_cpu_has_fsrm = ((cpu_info[3] & (1 << 4)) != 0); // bit 4 of EDX : see <https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features>
_mi_cpu_has_erms = ((cpu_info[1] & (1 << 9)) != 0); // bit 9 of EBX : see <https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features>
}
if (mi_cpuid(cpu_info, 1)) {
_mi_cpu_has_popcnt = ((cpu_info[2] & (1 << 23)) != 0); // bit 23 of ECX : see <https://en.wikipedia.org/wiki/CPUID#EAX=1:_Processor_Info_and_Feature_Bits>
}
}
In principle this sort of thing should be OK. But maybe __BMI1__ is
defined and so the generic fallback isn't present, or maybe the CPUID
check is incorrect for your CPU? I'll check the former when I have a
little more time, but if you could check the latter that would be
helpful.
(Note that I don't know this library well. I'm just trying to figure
this out since it's been blocking some other things I work on.)
This observation may help. The relevant code (for GCC/amd64) should
include a runtime check before executing the popcnt instruction if
__BMI1__ is not defined.
If we look at the surrounding assembly before popcnt is executed there
should be a simple compare for the boolean value _mi_cpu_has_popcnt
and a jump skipping the popcnt instruction if the result is false:
c976: f0 49 0f b1 0c 24 lock cmpxchg QWORD PTR [r12],rcx
c97c: 75 f2 jne c970
<mi_arena_reload@@Base+0x7a0>
c97e: 48 8d 0d bb ce 01 00 lea rcx,[rip+0x1cebb]
# 29840 <mi_stats_get_json@@Base+0x10a70>
c985: 48 21 c2 and rdx,rax
c988: f3 48 0f b8 c2 popcnt rax,rdx
Here's another snippet from the binary (no check is apparent):
ded9: b8 01 00 00 00 mov eax,0x1
dede: 48 d3 e0 shl rax,cl
dee1: 48 83 e8 01 sub rax,0x1
dee5: 89 d1 mov ecx,edx
dee7: 48 d3 e0 shl rax,cl
deea: 49 89 c7 mov r15,rax
deed: 48 8d 05 4c b9 01 00 lea rax,[rip+0x1b94c]
# 29840 <mi_stats_get_json@@Base+0x10a70>
def4: f3 4c 0f b8 df popcnt r11,rdi
Locally there is no indication these simple checks are being performed.
This indicates __BMI1__ is incorrectly defined.
(Or perhaps there is no runtime detection going on and what we are
seeing is output from intrinsics compiled for a platform that supports
the popcnt instruction.)
Regards,
Adam
I have two 64-bit laptops, both running Debian unstable. On the newish one, which has an Intel(R) Core(TM) i7-5600U CPU, the mtxrun works fine (no "illegal instruction"). On the ancient one, which has an Intel(R) Core(TM)2 Duo CPU T9300, it fails right away. According to x86-64-level <https://github.com/HenrikBengtsson/x86-64-level>, it's a level-1 CPU. Here is its cpuinfo, in case it's useful: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz stepping : 6 microcode : 0x60f cpu MHz : 1197.124 cache size : 6144 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm pti tpr_shadow flexpriority vpid dtherm ida vnmi vmx flags : vnmi flexpriority tsc_offset vtpr vapic bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown bogomips : 4988.01 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz stepping : 6 microcode : 0x60f cpu MHz : 1197.124 cache size : 6144 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm pti tpr_shadow flexpriority vpid dtherm ida vnmi vmx flags : vnmi flexpriority tsc_offset vtpr vapic bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown bogomips : 4988.01 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:
On the ancient laptop, I applied the attached patch and rebuilt libmimalloc3. The patch completely avoids the popcnt instruction and forces the use of _mi_popcount_generic(). Now mtxrun works (so the long-missing formats have finally been generated). Of course, this patch is not a viable solution in general. But it indicates a problem in the test for whether the CPU has popcnt. So, it might point someone who understands how this test should work toward a proper patch.