Hi there,

When inspecting tar balls created tar's --posix option, file paths
(path + file name) exceeding about 100 characters in length are not
shown properly, subdirectories are skipped alltogether.

How to repeat (sorry, overlong lines):

echo 'asdf' >012345678-10--------20--------30--------40--------50--------60--------70--------80--------.file1
touch 012345678-10--------20--------30--------40--------50--------60--------70--------80--------.file2
mkdir posix-tar-test/012345678-10--------20--------30--------40--------50--------60--------70--------80--------
touch posix-tar-test/012345678-10--------20--------30--------40--------50--------60--------70--------80--------/subfile
tar --posix -cvf posix-tar-test.tar posix-tar-test

Then start mc and inspect posix-tar-test.tar.

Expected behaviour:

In posix-tar-test/, two files, first length 5, second length 0, and a
sub-directory. In that sub-directory, a single file length 0.
Compare to the output to 'tar -tvf posix-tar-test.tar'.

Observed behaviour:

Three items named
"012345678-10--------20--------30--------40--------50--------60--------70--------80---"
all type file and length five (i.e. taken from .file1).


Examining the tar ball ...

00000800  31 32 31 20 70 61 74 68  3d 70 6f 73 69 78 2d 74  |121 path=posix-t|
00000810  61 72 2d 74 65 73 74 2f  30 31 32 33 34 35 36 37  |ar-test/01234567|
00000820  38 2d 31 30 2d 2d 2d 2d  2d 2d 2d 2d 32 30 2d 2d  |8-10--------20--|
00000830  2d 2d 2d 2d 2d 2d 33 30  2d 2d 2d 2d 2d 2d 2d 2d  |------30--------|
00000840  34 30 2d 2d 2d 2d 2d 2d  2d 2d 35 30 2d 2d 2d 2d  |40--------50----|
00000850  2d 2d 2d 2d 36 30 2d 2d  2d 2d 2d 2d 2d 2d 37 30  |----60--------70|
00000860  2d 2d 2d 2d 2d 2d 2d 2d  38 30 2d 2d 2d 2d 2d 2d  |--------80------|
00000870  2d 2d 2e 66 69 6c 65 31  0a 33 30 20 6d 74 69 6d  |--.file1.30 mtim|
00000880  65 3d 31 33 30 37 30 31  30 35 32 34 2e 30 31 32  |e=1307010524.012|
00000890  31 37 32 36 35 31 0a 33  30 20 61 74 69 6d 65 3d  |172651.30 atime=|
000008a0  31 33 30 37 30 31 30 35  32 34 2e 30 31 32 31 37  |1307010524.01217|
000008b0  32 36 35 31 0a 33 30 20  63 74 69 6d 65 3d 31 33  |2651.30 ctime=13|
000008c0  30 37 30 31 30 36 30 34  2e 38 37 32 32 39 31 32  |07010604.8722912|
000008d0  34 33 0a 00 00 00 00 00  00 00 00 00 00 00 00 00  |43..............|
000008e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000a00  70 6f 73 69 78 2d 74 61  72 2d 74 65 73 74 2f 30  |posix-tar-test/0|
00000a10  31 32 33 34 35 36 37 38  2d 31 30 2d 2d 2d 2d 2d  |12345678-10-----|
00000a20  2d 2d 2d 32 30 2d 2d 2d  2d 2d 2d 2d 2d 33 30 2d  |---20--------30-|
00000a30  2d 2d 2d 2d 2d 2d 2d 34  30 2d 2d 2d 2d 2d 2d 2d  |-------40-------|
00000a40  2d 35 30 2d 2d 2d 2d 2d  2d 2d 2d 36 30 2d 2d 2d  |-50--------60---|
00000a50  2d 2d 2d 2d 2d 37 30 2d  2d 2d 2d 2d 2d 2d 2d 38  |-----70--------8|
00000a60  30 2d 2d 2d 30 30 30 30  36 34 34 00 30 30 30 31  |0---0000644.0001|
00000a70  37 35 30 00 30 30 30 31  37 35 30 00 30 30 30 30  |750.0001750.0000|
00000a80  30 30 30 30 30 30 35 00  31 31 35 37 31 36 36 32  |0000005.11571662|
00000a90  37 33 34 00 30 32 34 33  32 34 00 20 30 00 00 00  |734.024324. 0...|
00000aa0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

... suggests mc's tar viewer picks file names from the second (legacy
format) blob and caches that information, while tar --list does the
right thing and reads the pax header (first blob).

Regards,

    Christoph

I am not sure if this is the same problem, but I have an archive 72MB in
size (~2GB before compression), compressed into tar.zst with about 1.1M
files (0 - 100 bytes each) and about 120 characters in path of each file.

Listing content is fast:

$ time zstdcat ../gnuplot-fuzzing-instrumented-snapshot-2018-11-27T01\:12.tar.zst | tar tf - | wc -l
1139010

real	0m4.619s
user	0m3.935s
sys	0m4.387s
$

With metadataa:

$ time zstdcat ../gnuplot-fuzzing-instrumented-snapshot-2018-11-27T01\:12.tar.zst | tar tvf -  | wc -l
1139010

real	0m10.418s
user	0m6.176s
sys	0m8.681s
$

Via index file:

$ time zstdcat ../gnuplot-fuzzing-instrumented-snapshot-2018-11-27T01\:12.tar.zst | tar -t -v --index-file=/tmp/zlk -f -  | wc -l
0

real	0m9.234s
user	0m5.837s
sys	0m5.323s
$ wc -l /tmp/zlk
1139010 /tmp/zlk
$


Yes, mc is stuck for 20 minutes when trying to enter the same archive.
mc is using 100% CPU (one thread). tar or zstd processes nowhere
to be found tho.

strace -p `pidof mc`

is showing enormous amount of lseeks and small reads:

lseek(9, 512, SEEK_CUR)                 = 1075074048
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075075072
read(9, "././@LongLink\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075077120
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075078144
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075079168
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075080192
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075081216
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075082240
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075083264
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075084288
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075085312
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075086336
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075087360
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075088384
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075089408
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075090432
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075091456
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075092480
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075093504
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075094528
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075095552
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075096576
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075097600
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075098624
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075099648
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075100672
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075101696
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075102720
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075103744
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075104768
read(9, "././@LongLink\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075106816
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075107840
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075108864
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075109888
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075110912
read(9, "././@LongLink\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075112960
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075113984
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075115008
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075116032
read(9, "././@LongLink\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075118080
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075119104
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075120128
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075121152
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075122176
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075123200
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 1024, SEEK_CUR)                = 1075124736
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075125760
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075126784
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075127808
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075128832
read(9, "././@LongLink\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
lseek(9, 512, SEEK_CUR)                 = 1075130880
read(9, "././@LongLink\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512
read(9, "gnuplot-fuzzing-instrumented/out"..., 512) = 512

In /tmp/mc-user/ I found this file

$ ls -lh /tmp/mc-user/sfsB89YSZ
-rw------- 1 user user 1.9G Nov 27 00:12 /tmp/mc-user/sfsB89YSZ
$ file /tmp/mc-user/sfsB89YSZ
/tmp/mc-user/sfsB89YSZ: POSIX tar archive (GNU)
$

So I am guessing mc did unpacked the file (instead of just streaming the
data on the fly) and now is trying to read metadata on its own without
the help of tar or using worse method than tar (that finishes in 10
seconds):

$ time tar tf /tmp/mc-user/sfsB89YSZ | wc -l
1139010

real	0m3.664s
user	0m2.777s
sys	0m2.644s
$
$ time tar tvf /tmp/mc-user/sfsB89YSZ | wc -l
1139010

real	0m10.144s
user	0m4.809s
sys	0m6.990s
$



PS. All files (/tmp and source file are in RAM on unionfs/tmpfs. I am
running out of the livecd with entire file system in memory).