The bash builtin command "printf" is not UTF-8 aware (i.e. does not
handle UTF-8 multibyte characters). Example:

  $ /bin/bash -c 'type printf'
  printf is a shell builtin

  $ /bin/bash -c 'printf "!%5s!\n" a ä'
  !    a!
  !   ä!

"a" is "U+0061 LATIN SMALL LETTER A" (UTF-8: 0x61)
"ä" is "U+00E4 LATIN SMALL LETTER A WITH DIAERESIS" (UTF-8: 0xc3a4)

The problem is that the field width "%5s" is calculated wrong because
the letter "ä" takes two bytes in UTF-8 but is only one character. It
displays correctly because the terminal is UTF-8 aware.

(By the way, /usr/bin/printf [in coreutils] has the same bug.)

Teemu Likonen <tlikonen@iki.fi> writes:

According to SUSv3, the printf utility interprets the format
string as specified under "File Format Notation", apart from
various exceptions which do not apply here.  And there, it says
that the field width and precision control the number of bytes,
rather than characters.  This is also how the printf function
works.

I suppose bash without --posix could be changed to behave
otherwise.  However, I doubt the usefulness of merely counting
Unicode characters.  If you want to align columns of a table, you
should also consider control characters, spacing and nonspacing
combining characters, and fullwidth ideographic characters.
If a simple character-counting feature were added now, perhaps
with new syntax such as "%5Uc", extending it later to handle
those cases too might cause compatibility problems.

Kalle Olavi Niemitalo kirjoitti:

This sounds good idea. Yes, maybe the best would be to save the current
byte-counting method and add separate character-counting method.

This "%5s"-thing affects also mawk and gawk. I wish someone would update
the basic Unix/Linux tools to Unicode world. :) They are useful with
human languages too, not only with computer code.

severity 459413 wishlist
tags 459413 + upstream
quit

Teemu Likonen wrote:

I agree.  Maybe someone should file it with http://austingroupbugs.net/?