#990097 duperemove: Searching duplicate extends aborts with ERROR: hash-tree.h:80

Package:
duperemove
Source:
duperemove
Description:
extent-based deduplicator for file systems
Submitter:
"Benedikt Wildenhain (BO)"
Date:
2024-10-12 12:00:01 UTC
Severity:
normal
#990097#5
Date:
2021-06-20 13:00:35 UTC
From:
To:
Dear Maintainer,

   * What led up to the situation?

I ran the following command:
duperemove -dr --hashfile=/root/$VOLUME.hash /mnt/$VOLUME
with /mnt/$VOLUME being a btrfs device. This has worked before. (I plan
do rerun the job at a later point of time to check reproducability,
however this process takes several hours so it won't be possible today)

   * What was the outcome of this action?

The action was aborted with the following messages:

[28851/28851] (100.00%) csum: /mnt/$VOLUME/$somefile
Total files:  28851
Total extent hashes: 961397
Loading only duplicated hashes from hashfile.
Hashing completed. Using 2 threads to calculate duplicate extents. This may take some time.
ERROR: hash-tree.h:80
ERROR: hash-tree.h:80
[stack trace follows]
duperemove(print_stack_trace+0x2e) [0x55584fc9c93e]
duperemove(+0xcd2c) [0x55584fc9ed2c]
duperemove(+0xd0ac) [0x55584fc9f0ac]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x7b9a4) [0x7f3ac07079a4]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x7b0bd) [0x7f3ac07070bd]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7) [0x7f3ac02f7ea7]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f3ac0481def]
Aborted

   * What outcome did you expect instead?
duperemove should have find duplicate files and deduplicate them.

#990097#10
Date:
2024-10-12 11:23:03 UTC
From:
To:
This looks like upstream bug https://github.com/markfasheh/duperemove/issues/265

- workaround: remove hash file
- not reproducible
- may be fixed with a current version? (Debian contains a very old one.)