#903373 "not a valid UTF-8 string" errors creating compose table in en_IN locale

Package:
libxkbcommon-x11-0
Source:
libxkbcommon
Description:
library to create keymaps with the XKB X11 protocol
Submitter:
Ritesh Raj Sarraf
Date:
2021-12-04 06:12:03 UTC
Severity:
normal
#903373#5
Date:
2018-07-09 05:45:49 UTC
From:
To:
Thank you for packaging the newer version of kitty.

With this version, the `icat` and `diff` commands fail with the
following error:


rrs@priyasi:~$ kitty diff rrs-home/Data/Pictures/debian-transparent.png
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:87:34: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:88:29: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:89:29: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:90:29: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:91:29: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:92:27: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:93:27: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:94:27: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:95:27: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:96:29: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:97:29: string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:97:29: too many errors
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:97:29: failed to parse file
[190 11:11:39.666312] [glfw error 65544]: Failed to create XKB compose table


Looking at upstream bug report:
https://github.com/kovidgoyal/kitty/issues/495
It looks like you need to bump a dependency on xkbcommon to >= 0.9.
FYI, xkbcommon (0.9) isn't yet in Debian.
#903373#10
Date:
2018-07-09 11:15:57 UTC
From:
To:
What do “locale” and “env | grep -e ^LC -e ^LANG” report?

The 0.9 being referred to in that issue is kitty 0.9.

Cheers,
#903373#15
Date:
2018-07-09 16:57:55 UTC
From:
To:
This is what I have:

rrs@priyasi:~/rrs-home/Data/Pictures$ locale
LANG=en_IN
LANGUAGE=en_IN:en
LC_CTYPE="en_IN"
LC_NUMERIC="en_IN"
LC_TIME="en_IN"
LC_COLLATE="en_IN"
LC_MONETARY="en_IN"
LC_MESSAGES="en_IN"
LC_PAPER="en_IN"
LC_NAME="en_IN"
LC_ADDRESS="en_IN"
LC_TELEPHONE="en_IN"
LC_MEASUREMENT="en_IN"
LC_IDENTIFICATION="en_IN"
LC_ALL=
22:26 ♒♒♒   ☺ 😄
rrs@priyasi:~/rrs-home/Data/Pictures$ env | grep -e ^LC -e ^LANG
LANG=en_IN
LANGUAGE=en_IN:en
22:26 ♒♒♒   ☺ 😄
#903373#20
Date:
2018-07-10 02:18:21 UTC
From:
To:
Control: reassign -1 libxkbcommon-x11-0 0.8.0-2
Control: retitle -1 "not a valid UTF-8 string" errors creating compose table in en_IN locale

This also happens simply when starting kitty, but those go into your
session error log.

That error is directly from xkbcommon's code[0].  If you instead use
LANG=en_IN.UTF-8, then there are no issues.  It appears that xkbcommon
is falling back incorrectly when there isn't an explicit codeset.

[0]: https://sources.debian.org/src/libxkbcommon/0.8.0-2/src/compose/parser.c/?hl=207#L207

That being said, just "en_IN" is what "dpkg-reconfigure locales"
generates when selecting that locale.  Both reportbug and "locale -c
charmap" are able to figure out that the codeset is UTF-8.

kitty is using essentially the exact code[1] that xkbcommon's
documentation suggests to use when calling
xkb_compose_table_new_from_locale().

I'm reassigning to xkbcommon to see if there's something they can do to
better handle this.

Cheers,
#903373#33
Date:
2019-10-18 10:12:07 UTC
From:
To:
Hello,
I faced exactly same issue while using `i3lock` from command line although
it was working fine (I assume). Setting `LC_ALL` to `en_IN.UTF-8`was the
solver. But as I think it messes up with language and font rendering, I
hope it doesn't screw up other things.
#903373#38
Date:
2021-12-04 06:09:22 UTC
From:
To:
I'm getting these exact errors on weston.

```
$ weston
...
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:39:34:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:40:29:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:41:29:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:42:29:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:43:29:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:44:27:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:45:27:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:46:27:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:47:27:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:48:29:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:49:29:
string literal is not a valid UTF-8 string
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:49:29: too many errors
xkbcommon: ERROR: /usr/share/X11/locale/iso8859-1/Compose:49:29:
failed to parse file
could not create XKB compose table for locale 'en_IN'.  Disabiling compose
```

Thanks!