#1082886 cp --sparse defeated by ficlone

Package:
coreutils
Source:
coreutils
Description:
GNU core utilities
Submitter:
Matthew Wilcox
Date:
2024-09-29 11:15:02 UTC
Severity:
normal
#1082886#5
Date:
2024-09-27 17:18:20 UTC
From:
To:
strace cp --sparse=always dd dd-sparse
[extraneous stuff skipped]
openat(AT_FDCWD, "dd", O_RDONLY)        = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=134248, ...}) = 0
openat(AT_FDCWD, "dd-sparse", O_WRONLY|O_CREAT|O_EXCL, 0755) = 4
ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = 0
close(4)                                = 0
close(3)                                = 0

It makes no attempt to look for sparse regions in the file, just uses
FICLONE (which succeeds because it's on XFS).

I worked around this by copying to /tmp, which is on a different
filesystem (tmpfs):

openat(AT_FDCWD, "dd", O_RDONLY)        = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=134248, ...}) = 0
openat(AT_FDCWD, "/tmp/dd-sparse", O_WRONLY|O_CREAT|O_EXCL, 0755) = 4
ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = -1 EXDEV (Invalid cross-device link)
fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0@>\0\0\0\0\0\0"..., 131072) = 131072
write(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0@>\0\0\0\0\0\0"..., 81920) = 81920
lseek(4, 45056, SEEK_CUR)               = 126976
fallocate(4, FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE, 81920, 45056) = 0
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "\320\34\0\0\0\0\0\0\320\34\0\0\0\0\0\0\320\34\0\0\0\0\0\0\320\34\0\0\0\0\0\0"..., 131072) = 3176
write(4, "\320\34\0\0\0\0\0\0\320\34\0\0\0\0\0\0\320\34\0\0\0\0\0\0\320\34\0\0\0\0\0\0"..., 3176) = 3176
read(3, "", 131072)                     = 0
close(4)                                = 0
close(3)                                = 0

Without looking at the source code, it seems likely that cp blindly
tries FICLONE without checking to see whether the sparse flag is set.
I suggest that setting --sparse=always should disable the FICLONE
optimisation.

#1082886#10
Date:
2024-09-27 18:41:39 UTC
From:
To:
I tend to disagree; the reflink is going to be more efficient in the
default case than making an explicit copy that has holes in it--so why
should cp assume you don't want a reflink? If you want to make an
existing file sparse, just use fallocate -d. Or, you could turn off
reflink if you really want to make a full copy, though this seems
strange.

#1082886#15
Date:
2024-09-27 20:31:37 UTC
From:
To:
Maybe for cp --sparse=auto that makes sense, but when I've explicitly
specified that I want to make a sparse copy, cp is ignoring my
instructions.  Yes, there are other commands that can make a file sparse,
but if cp's current behaviour is correct, then this at least deserves
a mention in the manpage because it's confusing as hell.

#1082886#20
Date:
2024-09-27 22:23:05 UTC
From:
To:
It would be just as confusing for cp to ignore reflink, right?
#1082886#25
Date:
2024-09-28 00:20:17 UTC
From:
To:
I didn't ask cp to use reflink.  It chose to do that all by itself.
I asked for a sparse file to be made, and it didn't do that.

#1082886#30
Date:
2024-09-28 01:25:11 UTC
From:
To:
You did, by not selecting reflink=always or reflink=never.
#1082886#35
Date:
2024-09-28 03:45:10 UTC
From:
To:
I didn't even know cp had grown a --reflink option.  So no, I didn't
ask cp to use reflink.  But I did ask it to use sparse.  That should
disable reflink.

#1082886#40
Date:
2024-09-29 11:10:30 UTC
From:
To:
I see your point, however `cp --reflink=auto --sparse=always`
was documented as the way to make a copy taking the least
amount of space supported by the file system.
That would be a more common use case than
making a separate copy as sparse as possible.