Hi,

While updating plexus-archiver I noticed that uscan filters more files
than expected.

The watch file is:

  version=3
  opts=uversionmangle=s{-alpha-}{~alpha} \
https://github.com/codehaus-plexus/plexus-archiver/tags .*/plexus-archiver-(.*).tar.gz

The file downloaded is:

https://github.com/codehaus-plexus/plexus-archiver/archive/plexus-archiver-3.5.tar.gz

And debian/copyright contains:

  Files-Excluded: jira

In this case the file plexus-archiver-plexus-archiver-3.5/src/test/resources/zeroFileMode/foobar.zip
and a few others are removed from the tarball. It doesn't happen
if the Files-Excluded field is removed.

Control: reassign -1 tar 1.29b-1.1
Control: affects -1 devscripts
Control: retitle -1 tar corrupts filenames when using --delete

I tested this out manually and it seems to be an issue with tar.

[jamessan@freya] $ wget https://github.com/codehaus-plexus/plexus-archiver/archive/plexus-archiver-3.5.tar.gz
--2017-07-20 21:38:49--  https://github.com/codehaus-plexus/plexus-archiver/archive/plexus-archiver-3.5.tar.gz
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/codehaus-plexus/plexus-archiver/tar.gz/plexus-archiver-3.5 [following]
--2017-07-20 21:38:49--  https://codeload.github.com/codehaus-plexus/plexus-archiver/tar.gz/plexus-archiver-3.5
Resolving codeload.github.com (codeload.github.com)... 192.30.253.121, 192.30.253.120
Connecting to codeload.github.com (codeload.github.com)|192.30.253.121|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘plexus-archiver-3.5.tar.gz’

plexus-archiver-3.5.tar.gz                            [ <=>                                                                                                         ] 545.30K  3.43MB/s    in 0.2s

2017-07-20 21:38:50 (3.43 MB/s) - ‘plexus-archiver-3.5.tar.gz’ saved [558383]

[jamessan@freya] $ gunzip plexus-archiver-3.5.tar.gz
[jamessan@freya] $ tar --delete  $(tar tf plexus-archiver-3.5.tar | grep jira | sort -r) < plexus-archiver-3.5.tar > plexus.tar
[jamessan@freya] $ diff -u <(tar tvf plexus-archiver-3.5.tar) <(tar tvf plexus.tar)
--- /proc/self/fd/11    2017-07-20 21:47:49.999199728 -0400
+++ /proc/self/fd/12    2017-07-20 21:47:49.999199728 -0400
@@ -5,286 +5,6 @@
 -rw-rw-r-- root/root     11358 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/LICENSE
 -rw-rw-r-- root/root      1153 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/README.md
 -rw-rw-r-- root/root     12709 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/ReleaseNotes.md
-drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/jira/
--rw-rw-r-- root/root     17770 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/jira/PLXCOMP-1.json
--rw-rw-r-- root/root      8733 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/jira/PLXCOMP-10.json
…
--rw-rw-r-- root/root      7741 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/jira/PLXCOMP-97.json
--rw-rw-r-- root/root      4116 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/jira/PLXCOMP-98.json
--rw-rw-r-- root/root     26543 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/jira/PLXCOMP-99.json
 -rw-rw-r-- root/root      4808 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/pom.xml
 drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/
 drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/main/
@@ -546,17 +266,17 @@
 drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/
 -rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
 -rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
-drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/world-writable/
--rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/world-writable/foo.txt
-drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zeroFileMode/
--rwxrwxr-x root/root       453 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zeroFileMode/foobar.zip
--rw-rw-r-- root/root      1877 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zeroFileMode/mixed-file-mode.zip
-drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/
--rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/file-with-even-time.txt
--rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/file-with-odd-time.txt
-drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/foo/
--rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/foo/file-with-even-time.txt
--rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/foo/file-with-odd-time.txt
--rw-rw-r-- root/root       368 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/foo/macOsZipFile.jar
--rw-rw-r-- root/root    164170 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/foo/plexus-archiver-2.4.4-SNAPSHOT.jar
--rw-rw-r-- root/root       368 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/zip-timestamp/macOsZipFile.zip
+drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rwxrwxr-x root/root       453 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root      1877 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+drwxrwxr-x root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root         0 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root       368 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root    164170 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
+-rw-rw-r-- root/root       368 2017-06-29 12:27 plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt


All the jira files (and directory) get deleted, as expected.  However,
the filenames for every member after
plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt
had its name replaced with
plexus-archiver-plexus-archiver-3.5/src/test/resources/utf8/D12345679012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890æøå.txt

The filesizes and other metadata remain correct … just the filenames are
wrong.

Cheers,

control: severity -1 important

Hi,

Raising the severity here since it causes data loss that tickles through
the whole build chain (#901303).

Bdale do you have an idea what would trigger this or if this is known
upstream already?

A while back, a Debian developer observed that using --delete can in
at least some cases cause filenames after the deleted ones to be
corrupted.  There's a test case in the bug log at

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=869087

I was able to confirm that this still happens with tar 1.30.  Any idea
what's happening and why?

Bdale

Hi Bdale,

I believe it was fixed by

http://git.savannah.gnu.org/cgit/tar.git/commit/?id=8e2898ab11

Regards,
Sergey

Hello,

This bug looks to be the same as #944118

I opened a MR with the upstream fix mentioned in #33

https://salsa.debian.org/debian/tar/-/merge_requests/2