#1074599 command-not-found: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 15: invalid continuation byte

Package:
src:command-not-found
Source:
src:command-not-found
Submitter:
Otto Kekäläinen
Date:
2025-06-05 15:13:05 UTC
Severity:
normal
#1074599#5
Date:
2024-07-01 20:11:04 UTC
From:
To:
Running /usr/lib/cnf-update-db failed on:

Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 32, in <module>
    col.create(db)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 96, in create
    self._fill_commands(con)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 141, in _fill_commands
    self._parse_single_contents_file(con, f, sub.stdout)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 242, in _parse_single_contents_file
    if not (l.startswith('usr/sbin') or l.startswith('usr/bin') or
        ^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position
15: invalid continuation byte

The command is currently unusable.

I added a print statement and seems the parser is failing on file from aspell:


b'usr/lib/aspell/xh.dat\t\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/xh.multi\t\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/xh.rws\t\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/xhosa.alias\t\t\t\t    universe/text/aspell-xh\n'
b'usr/lib/aspell/zu.dat\t\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/zu.multi\t\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/zu.rws\t\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/zulu.alias\t\t\t\t    universe/text/aspell-zu\n'
b'usr/lib/aspell/\xedslenska.alias\t\t\t\t    universe/text/aspell-is\n'
Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 32, in <module>
    col.create(db)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 96, in create
    self._fill_commands(con)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 141, in _fill_commands
    self._parse_single_contents_file(con, f, sub.stdout)
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 242, in _parse_single_contents_file
    if not (l.startswith('usr/sbin') or l.startswith('usr/bin') or
        ^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position
15: invalid continuation byte


This seems to be trigered by the i in "Islenska":

/usr/lib/aspell/íslenska.alias aspell-is

See https://packages.debian.org/search?mode=path&suite=bookworm§ion=all&arch=any&searchon=contents&keywords=slenska.alias

As a temporary workaround I used this try-statement:

    def _parse_single_contents_file(self, con, f, fp):
        # read header
        suite=None      # FIXME

        for l in fp:
            try:
               l = l.decode("utf-8")
            except UnicodeDecodeError:
                continue

            if not (l.startswith('usr/sbin') or l.startswith('usr/bin') or
                    l.startswith('bin') or l.startswith('sbin')):
                continue
            try:
                command, pkgnames = l.split(None, 1)
            except ValueError:
                continue
#1074599#10
Date:
2024-08-17 06:29:10 UTC
From:
To:
Hi!

I saw this again today in a container but when I tried to reproduce it
didn't happen again..
#1074599#15
Date:
2025-06-05 14:57:46 UTC
From:
To:
I saw this again in a container, and it is easy to reproduce now.

Just install the package 'aspell-is' that includes filename
'usr/lib/aspell/\xedslenska.alias' and running /usr/lib/cnf-update-db
will fail:

(extra output is my custom print() in creator.py line 241)
l: b'usr/lib/aspell/xh.dat\t\t\t\t\t    universe/text/aspell-xh\n'
l: b'usr/lib/aspell/xh.multi\t\t\t\t\t    universe/text/aspell-xh\n'
l: b'usr/lib/aspell/xh.rws\t\t\t\t\t    universe/text/aspell-xh\n'
l: b'usr/lib/aspell/xhosa.alias\t\t\t\t    universe/text/aspell-xh\n'
l: b'usr/lib/aspell/zu.dat\t\t\t\t\t    universe/text/aspell-zu\n'
l: b'usr/lib/aspell/zu.multi\t\t\t\t\t    universe/text/aspell-zu\n'
l: b'usr/lib/aspell/zu.rws\t\t\t\t\t    universe/text/aspell-zu\n'
l: b'usr/lib/aspell/zulu.alias\t\t\t\t    universe/text/aspell-zu\n'
l: b'usr/lib/aspell/\xedslenska.alias\t\t\t\t    universe/text/aspell-is\n'
Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 32, in <module>
    col.create(db)
    ~~~~~~~~~~^^^^
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 96, in create
    self._fill_commands(con)
    ~~~~~~~~~~~~~~~~~~~^^^^^
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 141, in _fill_commands
    self._parse_single_contents_file(con, f, sub.stdout)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/command-not-found/CommandNotFound/db/creator.py",
line 242, in _parse_single_contents_file
    l = l.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position
15: invalid continuation byte