#933325 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 749: invalid start byte

#933325#5
Date:
2019-07-29 08:53:17 UTC
From:
To:
This is what I get from cron:

/etc/cron.daily/apt-compat:
Traceback (most recent call last):
  File "/usr/bin/unattended-upgrade", line 1538, in <module>
    main(options)
  File "/usr/bin/unattended-upgrade", line 1476, in main
    log_content = get_dpkg_log_content(logfile_dpkg, install_start_time)
  File "/usr/bin/unattended-upgrade", line 1116, in get_dpkg_log_content
    for line in fp.readlines():
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 749: invalid start byte

No idea what's up with that error message.
*t

#933325#10
Date:
2019-08-06 18:51:12 UTC
From:
To:
Hi Tomas,

Tomas Pospisek <tpo_deb@sourcepole.ch> ezt írta (időpont: 2019. júl. 29.,
H, 10:54):
invalid start byte

Could you please attach the part of
/var/log/unattended-upgrades/unattended-upgrades-dpkg.log
that triggers the error?

In theory this is what unattended-upgrade does and this should crash on the
bad log:

#!/usr/bin/python3
# reproducer for Debian #933325
import io
with io.open("test-data", encoding='utf-8', errors='replace') as f:
    for line in f.readlines():
        pass

Cheers,
Balint

#933325#17
Date:
2019-08-07 10:46:06 UTC
From:
To:
Hello Bálint,

thanks a lot for having a look at this!

# ls -l /var/log/unattended-upgrades/unattended-upgrades-dpkg.log*
-rw-r----- 1 root adm    0 Aug  2 06:01 /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
-rw-r----- 1 root adm  663 Jul 28 06:25 /var/log/unattended-upgrades/unattended-upgrades-dpkg.log.1.gz
-rw-r----- 1 root adm  931 Jun 29 06:33 /var/log/unattended-upgrades/unattended-upgrades-dpkg.log.2.gz
[...]

the one from August is empty, so I guess it must have been the one from
July. I'm attaching the log (in unzipped form) to this mail.

I did run the "reproducer" you provided on:
* the attached log
* the gzipp'ed version of it
* an empty file

but "reproducer" ran through on each of them without an error.
*t

#933325#22
Date:
2026-07-31 13:47:38 UTC
From:
To:
It seems nothing new was learned since 2019, and thus keeping the
bug open does not provide anything actionable. If new info/insight
becomes available, please reopen the bug.

Chris