#804338 coreutils: provide a clean way for optional global cp --reflink=auto

Package:
coreutils
Source:
coreutils
Description:
GNU core utilities
Submitter:
Christoph Anton Mitterer
Date:
2015-11-08 00:03:04 UTC
Severity:
wishlist
#804338#5
Date:
2015-11-07 14:34:17 UTC
From:
To:
Hi.

It would be nice if the package could provide a clean out-of-the-box
way to get global --reflink=auto behaviour.

Starting with 8.24, mv --reflink=auto will be the default, but
for cp it stays at "never", and probably always will, see Pádraig's
comment here:
https://unix.stackexchange.com/questions/80351/why-is-cp-reflink-auto-not-the-default-behaviour/152639#152639


I have no opinion on whether these arguemnts are good or not,
some may argue they justify not using reflink per default, some
may say that people choosing filesystems as btrfs typically expect/want
their drawbacks/advantages.


Anyway, it should be easily possible to switch the global default.

Shell aliases are not a solution, they're typically not expanded in
non-interactive shells (and while at least bash allows to do so, I'd
say this is rather dangerous as it potentially breaks countless
scripts) and as such any system scripts, cron jobs (that make e.g.
backups) or any other program that directly invokes cp wouldn't benefit
from it.

Creating a wrapper shell script and dpkg-diverting the whole story
doesn't seem proper either, as the wrapper would require additional
shell execution... and why should people who want the more "native"
btrfs behaviour have to suffer from such penalty?

Maybe the coreutils package could provide a 2nd cp binary with
changed default and people could have things more easily switched,
either debconf (probably difficult for coreutils ;) ) or some
proper end-user friendly documentation on how to dpkg-divert.


Cheers,
Chris.

PS: Oh and having 8.24 in Debian would be nice, either
PS2: I've CCed Pádraig so he may comment if there are any better ways
to provide solution for this upstream, or even if upstream's view
may change on the cp/reflink=auto question.

#804338#10
Date:
2015-11-07 16:15:21 UTC
From:
To:
The reason --reflink=auto is not default for cp is because it
changes the end state of the copy. In this case the main issue is
that later changes to the file may give ENOSPC or result in fragmentation.
Now I do agree that you'll get the same issue if you later fill holes
in the file, or change data on deduplicating file systems/devices, ...
So perhaps we should just consider reflink as an implementation option,
and have a higher level --prealloc or whatever for those who care about that.
I.E. instead have cp to do the lightest weight copy possible,
and have --prealloc etc. change that for the edge cases that care
(with the caveat that existing systems may already be depending
 on the current non CoW default behavior).

There is work currently in the kernel for providing vfs_copy_file_range()
which will copy at the lowest level possible. That will do in prefered order;
server copy offload, SCSI COPY, reflink on BTRFS (or XFS etc. when available),
or splice at the VFS level. If cp calls into that by default then
you get your desired behavior by default. I'm looking at that logic
at present, and hope to have it in the next coreutils release,
which would be v9.0 since there are other invasive changes being considered.

cheers,
Pádraig.

#804338#15
Date:
2015-11-07 19:22:52 UTC
From:
To:
Hey Pádraig.
Sure..
That's the argument which may justify to make the default reflink=auto
- these issues may happen anyway, and they're core part of CoW
filesystems.
So everyone who uses them, must expect this,... or should avoid them.
Whatever one does, there will surely be at least one person that
complains ;-)
But your idea doesn't sound bad.
Looking forward to it.

So do you see it coming that this is actually "solved" (meaning refcopy
works per default out of the box for e.g. btrfs users) upstream ... or
is it worth to look into making it configurable at the distro level, as
I proposed above?


Thanks,
Chris.

#804338#20
Date:
2015-11-07 19:43:40 UTC
From:
To:
I don't see this as something a distro would want to diverge on,
nor something a distro would change in a minor release.
I.E. we should solve this upstream.

cheers,
Pádraig.

#804338#25
Date:
2015-11-07 19:51:53 UTC
From:
To:
Sure,.. but the idea here was, the distro could provide two binaries...
one (the default one) which is identical to upstream... and another one
(which people could manually dpkg-divert to) which does =auto.


Anyway,.. if this is going to be sorted out upstream... then the Debian
maintainers may either just close this bug, or leave it open till it's
"fixed" upstream.


Thanks,
Chris.

#804338#30
Date:
2015-11-07 23:52:01 UTC
From:
To:
Agree.

Mike Stone