Typical problem: suppose we want to parse 'ls -l' to get file sizes.
The 'ls -l' file size field is right justified, separated by spaces,
and its rightmost columnar position depends on the width of the user
and group names.  Example:

    # file size (0 in this case) for 'user.log' ends at
    # column 25.
    % ls -l /var/log/clamav/clamav.log /var/log/user.log
    -rw-r----- 1 clamav adm 0 Jul 16 07:44 /var/log/clamav/clamav.log
    -rw-r----- 1 root   adm 0 Jul 21 08:09 /var/log/user.log

    # same file, but now it ends at column 23.
    % ls -l /var/log/user.log
    -rw-r----- 1 root adm 0 Jul 21 08:09 /var/log/user.log

No option in 'cut' can handle that:

    1) changing the field separator won't work, because the
       number of spaces in 'ls -l' can vary with file size.
    2) fixed columnar positions won't work, because the columnar
       position in 'ls -l' varies with the width of the user and
       group names.

It'd be easier if 'cut' had an option to handle white space like the
shell's positional parameters.  Here's how to cut the file size field
using any Bourne compatible shell:

	# print the 5th field.
	cd foo; ls -l | while read x ; do set - $x ; echo $5 ; done

So, it'd be nice there was a '-p' (for positional parameters) option
for 'cut'.  Suggested usage:

	cd foo; ls -l | cut -p -f 5

If '-p' is too unmemorable, perhaps '-w' for "whitespace".

NB:  'ls -l' is just a familiar example -- of course there are
other ways to get file sizes that work with the current version of
'cut'; e.g. using 'ls' with other options, or utils like 'wc'.  The
point is that there's always input data where there are no such
options.

Hope this helps...

Your first problem is trying to use ls as programmatic input. It's
really a utility intended for human readability. Try using stat, either
with -t or --printf, or find --printf.

If you really, really want to parse ls output, use something like awk or
perl; cut is just not the right tool for that job.

Mike Stone

Thanks, but I was aware of other methods of parsing 'ls', as noted in
my report:

	NB:  'ls -l' is just a familiar example -- of course there are
	other ways to get file sizes that work with the current version of
	'cut'; e.g. using 'ls' with other options, or utils like 'wc'.  The
	point is that there's always input data where there are no such
	options.

While we can devise many roundabout ways to parse such output, it
would be better to do the coding once and encapsulate that function in
some utility, either a new one or an existing one well suited to it...

	% whatis cut
	cut (1)              - remove sections from each line of files

Note the plural noun "sections" is general and doesn't specify a fixed
number of delimiters or bytes.  'man cut' uses the noun "field", which
is also general.  Both nouns without qualifying adjectives are too
general, if fixed-length fields or sections were a necessary design
feature.

Adding an option to use shell-like white space fields fits in with the
"Software Tools" philosophy.  Indeed, a 'tab' character, (the default
'cut' delimiter), is itself a kind of variable width field, at least
from a human readable view, in the sense that it substitutes for a
varying number of spaces, usually from 1 to 8.  Conceptually, parsing
white space fields would be more like taking away an unnecessary
obstruction, (that of fixed lengths), rather than adding a new feature.

Which is why I suggested perl and awk or even sed. They've been around a
really long time, and they are suited for parsing complicated formats.

Mike Stone

On Sun, 23 Jul 2006 10:20:02 -0400 Michael Stone <mstone@debian.org> wrote:
general purpose languages, but 'cut' is a single purpose utility.
Solving a problem in a general purpose language does little to improve a
single purpose utility.

It's puzzling that you say: "complicated formats".  Are you
suggesting that the Bourne shell's traditional positional parameters
constitute a "complicated" format, or are relatively more complicated
than the fixed-length fields that 'cut' already supports?  If so, that
is a debatable point -- it might be argued that the shell is a much
_simpler_ format, because there's _less_ for humans to remember, since
there's no need for users to think about the widths of white space
delimiters.  A math analogy:  Topology is "simpler" than Euclidean
Geometry because Topology disregards angles and length.

Granted, in some cases a simpler format may be harder to program, and
perhaps this is such a case -- but it's not especially hard, since every
Bourne compatible shell already does it.  For maximum programmer
laziness 'cut' could even call the shell to do its parsing, then
translate that to a fixed length data format, then do
whatever it needs to.

Speculation:  apparently we're using terms like 'complicated' and
'simple' as applied to data formats in different ways -- I tend to
use these terms from an abstract view, whereas you seem to be (correct
me if I'm wrong) using them from the view of how difficult they'd be to
program, or reprogram and retrofit.

tags 379187 +wontfix
thanks

And when you add enough functionality to a single-purpose utility it
becomes a general purpose language. It's unnecessary cruft on top of
cut, and I'm not going to put it in the debian version. If you want to
try to get it added upstream, fine. Personally I don't see that adding
another set of switches to cut is in any way going to result in
something simpler than
awk '{print $5}'
for the problem you presented. (Or, as I originally suggested, stat -c
'%s'.) You already mentioned a way to do have the shell parse it. So
there's a million ways to solve this problem--why does the world need
another one?

Mike Stone