#916796 IEEEfull.bib: cannot be read in encoding 'utf8' by biber

Package:
texlive-publishers
Source:
texlive-extra
Submitter:
"Damir R. Islamov"
Date:
2021-10-31 06:03:03 UTC
Severity:
normal
#916796#5
Date:
2018-12-18 17:20:57 UTC
From:
To:
Dear Maintainer,

The file /usr/share/texlive/texmf-dist/bibtex/bib/IEEEtran/IEEEfull.bib contains a symbol
`á' in ISO-8859-1 8bit charset:
line 23, see `Nicolás Barabino' name.

This character breakes biblatex+biber.

Recoding from ISO-8859-1 to UTF-8 charset fixes the issue.

Below is scenario and input file.

$ pdflatex test
...
$ biber test
INFO - This is Biber 2.12
INFO - Logfile is 'test.blg'
INFO - Reading 'test.bcf'
INFO - Found 1 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file 'IEEEfull.bib' for section 0
ERROR - Data file '/usr/share/texlive/texmf-dist/bibtex/bib/IEEEtran/IEEEfull.bib' cannot be read in encoding 'utf8': Can't decode ill-formed UTF-8 octet sequence <E1> at /usr/share/perl5/File/Slurper.pm line 59.

INFO - ERRORS: 1

minimal input file:
##################################
\documentclass{article}
\usepackage[backend=biber]{biblatex}
\addglobalbib[datatype=bibtex]{IEEEfull.bib}
\begin{document}
\cite{bib1}
\printbibliography
\end{document}
##################################

######################################
 List of ls-R files
#916796#10
Date:
2018-12-18 22:32:38 UTC
From:
To:
Am 18.12.2018 um 18:20 teilte Damir R. Islamov mit:

Hi Damir,
Unfortunately the most active member from the Debian TeX group stopped
working for Debian. So if you expect to get a solution for that problem
in a foreseeable time frame please be so kind to contact upstream author
yourself.

Thanks,
   Hilmar
#916796#15
Date:
2018-12-19 11:12:10 UTC
From:
To:
Am 18.12.2018 um 23:32 teilte Hilmar Preuße mit:

Hi Damir,
...finally it could be that the author of the bib file is not aware of
biber and therefore the file is only usable in bibtex. Are you able to
use that file using bibtex?

H.
#916796#20
Date:
2018-12-25 02:00:17 UTC
From:
To:
Hi, Hilmar

IEEEfull.bib wirks fine with bibtex.
I will try to contact TeX group.

---
Sincerely yours
Damir Islamov

email: damir@trefle.ru


Hilmar Preuße писал 2018-12-19 17:12:
#916796#25
Date:
2018-12-25 02:22:47 UTC
From:
To:
Dear Dr. Dhell,

The file /usr/share/texlive/texmf-dist/bibtex/bib/IEEEtran/IEEEfull.bib
contains a symbol
`á' in ISO-8859-1 8bit charset:
line 23, see `Nicolás Barabino' name.

This character breaks biblatex+biber.

Recoding from ISO-8859-1 to UTF-8 fixes the issue.

Below is scenario and input file.

$ pdflatex test
...
$ biber test
INFO - This is Biber 2.12
INFO - Logfile is 'test.blg'
INFO - Reading 'test.bcf'
INFO - Found 1 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file 'IEEEfull.bib' for section 0
ERROR - Data file
'/usr/share/texlive/texmf-dist/bibtex/bib/IEEEtran/IEEEfull.bib' cannot
be read in encoding 'utf8': Can't decode ill-formed UTF-8 octet sequence
<E1> at /usr/share/perl5/File/Slurper.pm line 59.

INFO - ERRORS: 1

minimal input file:
##################################
\documentclass{article}
\usepackage[backend=biber]{biblatex}
\addglobalbib[datatype=bibtex]{IEEEfull.bib}
\begin{document}
\cite{bib1}
\printbibliography
\end{document}
##################################



Also, the biber shows warnings on all plain text strings in
/usr/share/texlive/texmf-dist/bibtex/bib/IEEEtran/IEEE{abrv,example,full}.bib
files after ISO-8859-1 -> UTF-8 recoding.

Example:
$ pdflatex test.tex
...
$ biber test
INFO - This is Biber 2.12
INFO - Logfile is 'test.blg'
INFO - Reading 'test.bcf'
INFO - Found 1 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file 'IEEEfull.bib' for section 0
INFO - LaTeX decoding ...
INFO - Found BibTeX data source
'/usr/share/texlive/texmf-dist/bibtex/bib/IEEEtran/IEEEfull.bib'
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
97, warning: 2630 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
107, warning: 46 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
116, warning: 41 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
123, warning: 19 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
131, warning: 67 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
145, warning: 14 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
155, warning: 36 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
158, warning: 20 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
173, warning: 3 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
179, warning: 33 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
187, warning: 43 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
209, warning: 18 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
218, warning: 38 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
226, warning: 77 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
233, warning: 18 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
236, warning: 19 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
240, warning: 19 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
245, warning: 49 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
253, warning: 20 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
259, warning: 23 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
268, warning: 47 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
280, warning: 51 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
297, warning: 25 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
312, warning: 32 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
326, warning: 29 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
331, warning: 22 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
338, warning: 10 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
340, warning: 21 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
346, warning: 20 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
353, warning: 40 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
367, warning: 28 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
375, warning: 61 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
388, warning: 11 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
394, warning: 59 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
402, warning: 7 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
407, warning: 4 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
417, warning: 34 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
430, warning: 22 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
438, warning: 17 characters of junk seen at toplevel
WARN - BibTeX subsystem: /tmp/sTx6tRw3yS/IEEEfull.bib_5064.utf8, line
466, warning: 29 characters of junk seen at toplevel
WARN - I didn't find a database entry for 'bib1' (section 0)
INFO - Overriding locale 'en-US' defaults 'normalization = NFD' with
'normalization = prenormalized'
INFO - Overriding locale 'en-US' defaults 'variable = shifted' with
'variable = non-ignorable'
INFO - Sorting list 'nty/global//global/global' of type 'entry' with
template 'nty' and locale 'en-US'
INFO - No sort tailoring available for locale 'en-US'
INFO - Writing 'test.bbl' with encoding 'UTF-8'
INFO - Output to test.bbl
INFO - WARNINGS: 41


It would be nice to escape them as latex comments by '%'.
#916796#30
Date:
2018-12-25 11:16:34 UTC
From:
To:
On 25.12.18 03:00, Damir Islamov wrote:

Hi Damir,
So, I'll mark that bug for me as "submitter cares about the issue".
Please be so kind to call back, once you get an answer.

Many thanks!
  Hilmar
#916796#35
Date:
2019-01-15 08:59:12 UTC
From:
To:
I got response from Michael Shell, maintainer of IEEEtran.bst package.

--- 
Sincerely yours
Damir Islamov

======================================================
Subject: Re: IEEEfull.bib: cannot be read in encoding 'utf8' by biber
Date: 2019-01-04 16:53
 From: Michael Shell <ieeetran@michaelshell.org <mailto:ieeetran@michaelshell.org> >
To: Damir Islamov <damir@trefle.ru <mailto:damir@trefle.ru> >



   Damir,

OK I removed the accent from Nicolas' name. So, this change will
be in the next release of the IEEEtran.bst package. I hope
to have that out this month. It will be a big release with lots
of long awaited updates.

...

   Cheers and thanks,

   Mike
#916796#40
Date:
2019-01-15 21:24:57 UTC
From:
To:
On 15.01.19 09:59, Damir Islamov wrote:

Hi Damir,
Please make sure the file uploaded to CTAN, else we won't get the fix.

Many thanks!

Hilmar
#916796#45
Date:
2021-10-20 20:39:13 UTC
From:
To:
Am 15.01.2019 um 22:24 teilte Hilmar Preuße mit:

Hi Damir,
Did your hear anything from Michael Shell for this?

This issue is still present and the bib file on CTAN is still the broken
version,

Thanks,
   Hilmar
#916796#50
Date:
2021-10-31 05:58:50 UTC
From:
To:
Dear Hilmar,

I've the following response from Mike:
----
  Damir,

I sure do apologize for my lengthly delay. I've been through a lot, but
there will be the update. Thanks for not yelling at me - I'd deserve it
too. Sigh.


  Mike
----

--------------------
Sincerely yours
*Damir Islamov*

В письме от четверг, 21 октября 2021 г. 03:39:13 +07 пользователь Hilmar Preuße
написал: