#1070168 zstd: Poor performance of Debian build

Package:
zstd
Source:
zstd
Description:
fast lossless compression algorithm -- CLI tool
Submitter:
Laurent Bonnaud
Date:
2025-11-28 20:07:01 UTC
Severity:
normal
#1070168#5
Date:
2024-05-01 09:15:47 UTC
From:
To:
Dear Maintainer,

here is a quick benchmark of the zstd build in Debian:

$ zstd -V
*** Zstandard CLI (64-bit) v1.5.5, by Yann Collet ***

$ zstd -b
  3#Synthetic 50%     :  10000000 ->   3230847 (x3.095),  168.9 MB/s, 1310.7 MB/s

and here is the same benchmark in the flatpak build:

$ flatpak run org.freedesktop.Platform//23.08

[📦 org.freedesktop.Platform ~]$ zstd -V
*** Zstandard CLI (64-bit) v1.5.5, by Yann Collet ***

[📦 org.freedesktop.Platform ~]$ zstd -b
  3#Synthetic 50%     :  10000000 ->   3230847 (x3.095),  201.1 MB/s, 1502.0 MB/s

The performance difference is significant IMHO.

#1070168#10
Date:
2025-04-09 20:03:42 UTC
From:
To:
Hi,

here is an update with more recent versions:

Debian:

$ zstd -V
*** Zstandard CLI (64-bit) v1.5.7, by Yann Collet ***

$ zstd -b
  3#Lorem ipsum       :  10000000 ->   2981954 (x3.354),  161.3 MB/s,  598.8 MB/s

Fedora:

$ docker run fedora zstd -V
*** Zstandard CLI (64-bit) v1.5.7, by Yann Collet ***

$ docker run fedora zstd -b
  3#Lorem ipsum       :  10000000 ->   2981954 (x3.354),  180.8 MB/s,  948.4 MB/s

Thanks,

#1070168#15
Date:
2025-11-28 20:04:36 UTC
From:
To:
I am no sure saying it is poor is best description. It is not poor even
in your report, but maybe there is some difference.

On my machine I do not really see a difference:

Fedora:

docker run --rm -it fedora zstd -v
*** Zstandard CLI (64-bit) v1.5.7, by Yann Collet ***
user@debian:~/tmp-live-build$ docker run --rm -it fedora zstd -b
 3#Lorem ipsum       :  10000000 ->   2983345 (x3.352),  213.4 MB/s,  901.7 MB/s


Debian testing:

*** Zstandard CLI (64-bit) v1.5.7, by Yann Collet ***

 3#Lorem ipsum       :  10000000 ->   2983345 (x3.352),  252.3 MB/s,  879.3 MB/s



This is on AMD Ryzen Threadripper 2950X (16-core cpu), 4 channel of DDR3
memory, and with schedutil cpu governor.

I did repeated tests (about 200 each), and compared distribution of
results. And yes, Debian is a bit slower, but not by a lot. median for
compression is about 2% slower on Debian, and median for decompression
is about 3% slower on Debian than in the Fedora container. Could be many
things, different compiler version, different security hardening
options, memory allocation scheme, compiler options, and zstd build
options itself.


I think you should provide more details, like CPU type, model,
frequency, memory type and speed, machine type, cpu frequency governor
used and cpu idle driver used. And also do repeated tests, on otherwise
idle system. I also do see you are using RT kernel - could you try
with standard kernel too?


To get my numbers I run something like this:


while true; do zstd -b; done

or

while true; do schedtool -F -p 99 -e zstd -b; done

if possible, and set performance cpu frequency governor first, then let
it run for some time to gether enough samples.

Then in gnuplot I did plot distributions:

gnuplot> set xrange [0:]
gnuplot> plot "<awk '{print $8;}' debian.txt | sort -n" u 1:($0 / 156.0) w l, "<awk '{print $8;}' fedora.txt | sort -n" u 1:($0 / 132.0) w l


Where 156 and 132 are number of lines in each file to normalize data.
Change $8 to $10 to see decompression speeds.