#650381 fmt: incorrect formatting of UTF-8 text

Package:
coreutils
Source:
coreutils
Description:
GNU core utilities
Submitter:
Heikki Piirainen
Date:
2023-06-16 16:36:03 UTC
Severity:
normal
#650381#5
Date:
2011-11-29 12:24:27 UTC
From:
To:
The fmt command appears to count bytes rather than characters when
determining line lengths. This causes too short lines when the text
contains multibyte characters.

Example output (fmt invoked without options):

foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo
foo foo foo foo foo foo foo þøø þøø þøø þøø þøø þøø
þøø þøø þøø þøø þøø þøø þøø þøø þøø þøø
þøø þøø þøø þøø þøø þøø þøø þøø þøø foo foo
foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo
foo foo foo foo foo þøø þøø þøø þøø þøø þøø þøø
þøø þøø þøø þøø þøø þøø þøø þøø þøø þøø
þøø þøø þøø þøø þøø þøø þøø þøø

#650381#10
Date:
2023-06-16 16:33:28 UTC
From:
To:
Still holds on bookworm:
  $ echo "Since I first posted this, procps free(1) started using MemAvailable to evaluate free/used, so sure. I don't feel strongly either way." | fmt
  Since I first posted this, procps free(1) started using MemAvailable to
  evaluate free/used, so sure. I don't feel strongly either way.
  $ echo "Śińćę I firśt póśtęd thiś, próćpś fręę(1) śtąrtęd uśińg MęmĄvąiląblę tó ęvąluątę fręę/uśęd, śó śurę. I dóń't fęęl śtróngly ęithęr wąy." | fmt
  Śińćę I firśt póśtęd thiś, próćpś fręę(1) śtąrtęd uśińg
  MęmĄvąiląblę tó ęvąluątę fręę/uśęd, śó śurę. I dóń't
  fęęl śtróngly ęithęr wąy.

  $ echo 开 花 馒 头 好 吃 方 法 很 简 单 , 学 会 这 个 做 法 , 蓬 松 柔 软 个 个 开 花 【 海 娟 美 食 】 | fmt
  开 花 馒 头 好 吃 方 法 很 简 单 , 学 会 这 个 做 法
  , 蓬 松 柔 软 个 个 开 花 【 海 娟 美 食 】
  $ echo аа аа аа аа аа аа аа аа аа аа аа аа аа aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa | fmt
  аа аа аа аа аа аа аа аа аа аа аа аа аа aa aa
  aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa

I doubt this is, strictly UTF-8, related at all, and would most likely
happen with any multibyte encoding. Retitling accordingly.

Best,
наб