Hi,

The lzma executable from the lzma package (vs. xz-utils) returns zero
exit status when decompressing some files not compressed by lzma, even
when such decompression results in no output. The specific case I'm
interested in is:

  $ lzma -dc "/boot/vmlinuz-$(uname -r)"; echo $?
  0

  $ lzma -t "/boot/vmlinuz-$(uname -r)" && echo "yes"
  yes

It looks like it's about zeros:

  $ head -c18 "/boot/vmlinuz-$(uname -r)" | xxd
  00000000: 4d5a 0000 0000 0000 0000 0000 0000 0000  MZ..............
  00000010: 0000                                     ..

  # fails with -c17
  $ head -c18 "/boot/vmlinuz-$(uname -r)" | lzma -d; echo $?
  0

  $ head -c18 "/dev/zero" | lzma -d; echo $?
  0

I'm not sure if this is actually a valid lzma file, but xz and zstd
(with their lzma support) seem to reject it as I expected lzma should:

  $ xz --format=lzma -dc "/boot/vmlinuz-$(uname -r)"; echo $?
  xz: /boot/vmlinuz-7.1.8+deb14.1-amd64: File format not recognized
  1

  # --format=* doesn't seem to take effect here, but still
  $ zstd --format=lzma -dc "/boot/vmlinuz-$(uname -r)"; echo $?
  zstd: /boot/vmlinuz-7.1.8+deb14.1-amd64: unsupported format
  1

I was using the exit status with many decompressors to get the actual
kernel file from a potentially unknown-compressed file so I can do
bootloader stuff in depthcharge-tools, I'll have to actually check the
decompressed size for robustness...

Hello!

I'm replying because I maintained LZMA Utils until 2008-2009, and I
hope I can provide some information. However, I won't attempt to fix
the reported issue.

I hadn't realized that Debian still ships lzmp.cpp from the long
obsolete LZMA Utils. Somehow I thought only LZMA SDK's own command line
tool was packaged nowadays.

The behavior you see was known 20 years ago, but it was never changed.
The section "LZMA UTILS COMPATIBILITY" in the xz man page documents
this. Specifically, xz fails (and lzmp.cpp succeeds) with your test
input for two reasons:

  - 0x4D as the first byte indicates pb=1, lp=3, and lc=5. xz doesn't
    support that combination. See the subsection "Unsupported .lzma
    files".[1]

  - Even if xz supported the .lzma header, then xz would fail due to
    the reason in the subsection "Trailing garbage".[2]

Debian's version of lzmp.cpp was forked from LZMA Utils to make
lzmp.cpp work with a newer LZMA SDK. I see now that the Debian version
lacks some of the fixes that were made in the upstream project in
2007-2008, for example [3]. If someone wants to maintain lzmp.cpp, I
suggest going through the commits in lzma.git[4], there aren't that
many.

[1] https://tukaani.org/xz/man/xz.1.html#LZMA_UTILS_COMPATIBILITY:_Unsupported_.lzma_files
[2] https://tukaani.org/xz/man/xz.1.html#LZMA_UTILS_COMPATIBILITY:_Trailing_garbage
[3] https://git.tukaani.org/?p=lzma.git;a=commitdiff;h=a6d49bfb97a8b8db3dd86a4814f41826fe4bddad
[4] git clone https://git.tukaani.org/lzma.git

Hello!

I'm replying because I maintained LZMA Utils until 2008-2009, and I
hope I can provide some information. However, I won't attempt to fix
the reported issue.

I hadn't realized that Debian still ships lzmp.cpp from the long
obsolete LZMA Utils. Somehow I thought only LZMA SDK's own command line
tool was packaged nowadays.

The behavior you see was known 20 years ago, but it was never changed.
The section "LZMA UTILS COMPATIBILITY" in the xz man page documents
this. Specifically, xz fails (and lzmp.cpp succeeds) with your test
input for two reasons:

  - 0x4D as the first byte indicates pb=1, lp=3, and lc=5. xz doesn't
    support that combination. See the subsection "Unsupported .lzma
    files".[1]

  - Even if xz supported the .lzma header, then xz would fail due to
    the reason in the subsection "Trailing garbage".[2]

Debian's version of lzmp.cpp was forked from LZMA Utils to make
lzmp.cpp work with a newer LZMA SDK. I see now that the Debian version
lacks some of the fixes that were made in the upstream project in
2007-2008, for example [3]. If someone wants to maintain lzmp.cpp, I
suggest going through the commits in lzma.git[4], there aren't that
many.

[1] https://tukaani.org/xz/man/xz.1.html#LZMA_UTILS_COMPATIBILITY:_Unsupported_.lzma_files
[2] https://tukaani.org/xz/man/xz.1.html#LZMA_UTILS_COMPATIBILITY:_Trailing_garbage
[3] https://git.tukaani.org/?p=lzma.git;a=commitdiff;h=a6d49bfb97a8b8db3dd86a4814f41826fe4bddad
[4] git clone https://git.tukaani.org/lzma.git