#906091 man: 8 bit characters from the '.tm' request are displayed as \[u00XY]

Package:
groff
Source:
groff
Description:
GNU troff text-formatting system
Submitter:
Bjarni Ingi Gislason
Date:
2023-07-21 13:45:04 UTC
Severity:
normal
Tags:
#906091#5
Date:
2018-08-14 04:05:36 UTC
From:
To:
  Test example:

.pl 30v
.TH testman 1 2018-MM-DD
.SH NAME
testman \- test some macros
.LP
Sævör grét áðan því úlpan var ónýt.
.tm Sævör grét áðan því úlpan var ónýt.

  Output from "man -d -l <file> with

MAN_KEEP_STDERR=yes
MANPATH=/home/bg/.local/share/man:/usr/local/share/man:/usr/share/man
MANOPT=--encoding=latin1 --warnings=w --no-justification --no-hyphenation
MANWIDTH=80

  is

ruid=1000, euid=1000
rgid=1000, egid=1000
++priv_drop_count = 1
From the config file /home/bg/.manpath:

Defined `troff' as `test-groff -b -mandoc -ww'.
Defined `nroff' as `test-nroff -mandoc -ww'.
From the config file /etc/manpath.config:

Mandatory mandir `/usr/man'.
Mandatory mandir `/usr/share/man'.
Mandatory mandir `/usr/local/share/man'.
Path `/bin' mapped to mandir `/usr/share/man'.
Path `/usr/bin' mapped to mandir `/usr/share/man'.
Path `/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/share/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/share/man'.
Path `/usr/X11R6/bin' mapped to mandir `/usr/X11R6/man'.
Path `/usr/bin/X11' mapped to mandir `/usr/X11R6/man'.
Path `/usr/games' mapped to mandir `/usr/share/man'.
Path `/opt/bin' mapped to mandir `/opt/man'.
Path `/opt/sbin' mapped to mandir `/opt/man'.
Global mandir `/usr/man', catdir `/var/cache/man/fsstnd'.
Global mandir `/usr/share/man', catdir `/var/cache/man'.
Global mandir `/usr/local/man', catdir `/var/cache/man/oldlocal'.
Global mandir `/usr/local/share/man', catdir `/var/cache/man/local'.
Global mandir `/usr/X11R6/man', catdir `/var/cache/man/X11R6'.
Global mandir `/opt/man', catdir `/var/cache/man/opt'.
Added section `1'.
Added section `n'.
Added section `l'.
Added section `8'.
Added section `3'.
Added section `2'.
Added section `3posix'.
Added section `3pm'.
Added section `3perl'.
Added section `3am'.
Added section `5'.
Added section `4'.
Added section `9'.
Added section `6'.
Added section `7'.
`troff'	`test-groff -b -mandoc -ww'	`-4'
`nroff'	`test-nroff -mandoc -ww'	`-4'
`/usr/man'	`'	`1'
`/usr/share/man'	`'	`1'
`/usr/local/share/man'	`'	`1'
`/bin'	`/usr/share/man'	`0'
`/usr/bin'	`/usr/share/man'	`0'
`/sbin'	`/usr/share/man'	`0'
`/usr/sbin'	`/usr/share/man'	`0'
`/usr/local/bin'	`/usr/local/man'	`0'
`/usr/local/bin'	`/usr/local/share/man'	`0'
`/usr/local/sbin'	`/usr/local/man'	`0'
`/usr/local/sbin'	`/usr/local/share/man'	`0'
`/usr/X11R6/bin'	`/usr/X11R6/man'	`0'
`/usr/bin/X11'	`/usr/X11R6/man'	`0'
`/usr/games'	`/usr/share/man'	`0'
`/opt/bin'	`/opt/man'	`0'
`/opt/sbin'	`/opt/man'	`0'
`/usr/man'	`/var/cache/man/fsstnd'	`-1'
`/usr/share/man'	`/var/cache/man'	`-1'
`/usr/local/man'	`/var/cache/man/oldlocal'	`-1'
`/usr/local/share/man'	`/var/cache/man/local'	`-1'
`/usr/X11R6/man'	`/var/cache/man/X11R6'	`-1'
`/opt/man'	`/var/cache/man/opt'	`-1'
`1'	`'	`-5'
`n'	`'	`-5'
`l'	`'	`-5'
`8'	`'	`-5'
`3'	`'	`-5'
`2'	`'	`-5'
`3posix'	`'	`-5'
`3pm'	`'	`-5'
`3perl'	`'	`-5'
`3am'	`'	`-5'
`5'	`'	`-5'
`4'	`'	`-5'
`9'	`'	`-5'
`6'	`'	`-5'
`7'	`'	`-5'
real user = 1000; effective user = 1000

using pager as pager
add_nls_manpaths(): processing /home/bg/.local/share/man:/usr/local/share/man:/usr/share/man
checking for locale is_IS.iso88591
manpath search path (with duplicates) = /home/bg/.local/share/man:/usr/local/share/man:/usr/share/man
adding /home/bg/.local/share/man to manpathlist
adding /usr/local/share/man to manpathlist
adding /usr/share/man to manpathlist
final search path = /home/bg/.local/share/man:/usr/local/share/man:/usr/share/man
++priv_drop_count = 2
pre-processors `t' from default
page_encoding = iso88591
source_encoding = iso88591
locale_charset = ISO-8859-1
roff_encoding = ISO-8859-1
output_encoding = ISO-8859-1
Terminal width 80
Terminal width 80 within cat page range [80, 80]
formatted_encoding = ISO-8859-1
format: 1, save_cat: 0, found: 1
++priv_drop_count = 3
loading seccomp filter (permissive: 0)
loading seccomp filter (permissive: 0)
loading seccomp filter (permissive: 1)
loading seccomp filter (permissive: 1)
loading seccomp filter (permissive: 0)
loading seccomp filter (permissive: 0)
opening -
opened -
seccomp already enabled
seccomp already enabled
trying encoding UTF-8 -> UTF-8//IGNORE
trying encoding iso88591 -> UTF-8//IGNORE
S\[u00E6]v\[u00F6]r gr\[u00E9]t \[u00E1]\[u00F0]an \[u00FE]v\[u00ED] \[u00FA]lpan var \[u00F3]n\[u00FD]t.
testman(1)                  General Commands Manual                 testman(1)

NAME
       testman - test some macros

       Sævör grét áðan því úlpan var ónýt.

                                  2018-MM-DD                        testman(1)
--priv_drop_count = 2
--priv_drop_count = 1

#906091#10
Date:
2018-08-14 08:04:53 UTC
From:
To:
Control: reassign -1 groff
[...]

This is because man runs the input file through preconv(1), and then .tm
reads its argument in copy mode.

Short of skipping preconv entirely, which isn't a viable option, I don't
think man can do much about this.  Either preconv needs to understand
enough about the input to avoid encoding the argument to .tm (although I
wonder what this would do if the input text were in some more exotic
encoding), or .tm needs to evaluate \[...] escapes, or we need to accept
that this debugging command will produce slightly strange output.
Reassigning to groff.

#906091#23
Date:
2022-09-27 20:39:25 UTC
From:
To:
package groff groff-base
severity 906091 wishlist
reassign 906091 groff-base
retitle groff-base: troff `tm` request has no means of writing non-ASCII characters
tag 906091 + upstream
thanks

This would be a significant new feature.

See https://savannah.gnu.org/bugs/?63074#comment9 for lengthy analysis
of this and related feature gaps.

Regards,
Branden