#763598 docbook-xml: xmllint fails to identify local copy of docbook entities file

#763598#5
Date:
2014-10-01 07:58:01 UTC
From:
To:
Consider the test document attached, it's starting with this:

<?xml version='1.0' encoding='utf-8' ?>
<!DOCTYPE section [
<!ENTITY % BOOK_ENTITIES SYSTEM "Users_Guide.ent">
%BOOK_ENTITIES;
<!ENTITY % sgml.features "IGNORE">
<!ENTITY % xml.features "INCLUDE">
<!ENTITY % DOCBOOK_ENTS PUBLIC "-//OASIS//ENTITIES DocBook Character Entities V4.5//EN" "http://www.oasi
s-open.org/docbook/xml/4.5/dbcentx.mod">
%DOCBOOK_ENTS;
]>

Now I want to parse it (with publican which uses libxml internally) but I always ends
up loading http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod from the network instead
of finding the local copy. I can reproduce the problem with xmllint:

    $ XML_DEBUG_CATALOG=1 xmllint --debugent --nonet --noent --noout test.xml
    [...]
    Resolve: pubID -//OASIS//ENTITIES DocBook Character Entities V4.5//EN sysID http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod
    0 Parsing catalog file:///etc/xml/catalog
    file:///etc/xml/catalog added to file hash
    file:///etc/xml/docbook-xml.xml not found in file hash
    0 Parsing catalog file:///etc/xml/docbook-xml.xml
    file:///etc/xml/docbook-xml.xml added to file hash
    Trying system delegate file:///etc/xml/docbook-xml.xml
    Resolve URI http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod
    I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod
    [...]

This is not normal. It looks like only the system idendifier (the URL) is used
while the public identifier (for which there's a match in /etc/xml/docbook-xml.xml)
is not used:

    $ grep -- "-//OASIS//ENTITIES DocBook Character Entities V4.5//EN" /etc/xml/docbook-xml.xml
    <delegatePublic publicIdStartString="-//OASIS//ENTITIES DocBook Character Entities V4.5//EN" catalog="file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml"/>

To confirm this impression I modified /etc/xml/docbook-xml.xml to replace this line:

    <delegateSystem systemIdStartString="http://docbook.org/xml/4.5/docbookx.dtd" catalog="file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml"/>

With this one:

    <delegateSystem systemIdStartString="http://docbook.org/xml/4.5/" catalog="file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml"/>

This allowed to go one step further in the catalog lookup:

    Resolve: pubID -//OASIS//ENTITIES DocBook Character Entities V4.5//EN sysID http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod
    0 Parsing catalog file:///etc/xml/catalog
    file:///etc/xml/catalog added to file hash
    file:///etc/xml/docbook-xml.xml not found in file hash
    0 Parsing catalog file:///etc/xml/docbook-xml.xml
    file:///etc/xml/docbook-xml.xml added to file hash
    Trying system delegate file:///etc/xml/docbook-xml.xml
    file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml not found in file hash
    0 Parsing catalog file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml
    file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml added to file hash
    Trying system delegate file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml
    Resolve URI http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod
    I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod

And to finally get it to work, I had to add this line in
/usr/share/xml/docbook/schema/dtd/4.5/catalog.xml:

    <system systemId="http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod"
	    uri="dbcentx.mod"/>

Now I have this:

    Resolve: pubID -//OASIS//ENTITIES DocBook Character Entities V4.5//EN sysID http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod
    0 Parsing catalog file:///etc/xml/catalog
    file:///etc/xml/catalog added to file hash
    file:///etc/xml/docbook-xml.xml not found in file hash
    0 Parsing catalog file:///etc/xml/docbook-xml.xml
    file:///etc/xml/docbook-xml.xml added to file hash
    Trying system delegate file:///etc/xml/docbook-xml.xml
    file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml not found in file hash
    0 Parsing catalog file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml
    file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml added to file hash
    Trying system delegate file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml
    Found system match http://www.oasis-open.org/docbook/xml/4.5/dbcentx.mod, using file:///usr/share/xml/docbook/schema/dtd/4.5/dbcentx.mod
    new input from file: file:///usr/share/xml/docbook/schema/dtd/4.5/dbcentx.mod

There's something fishy either in the catalog files, or in the logic of libxml2, I'm not
sure which one. Looking at
https://www.oasis-open.org/committees/download.php/14809/xml-catalogs.html#s.ext.resx
it looks like that the catalog file is at fault since libxml2 does the
right thing by trying to use the system identifier in the first place.

FWIW, I investigated this with the upstream author of Publican in this bugzilla ticket:
https://bugzilla.redhat.com/show_bug.cgi?id=1143060#c18

It's really blocking me to release the new version of Publican in Debian so it would
be nice to find a fix quickly if possible, because the freeze is approaching.

#763598#10
Date:
2014-10-07 12:59:39 UTC
From:
To:
Am Mittwoch, den 01.10.2014, 09:58 +0200 schrieb Raphael Hertzog:
[..]

Indeed the above will fail. According to the XML catalog standard you
cite in
https://www.oasis-open.org/committees/download.php/14809/xml-catalogs.html#s.ext.ent (see the end of the section and section 4.1.1) it is specified, that if system identifiers are provided together with public identifiers  the default is to only lookup the system identifier. This cbehaviour can be modified by the prefer attribute of the catalog entry. The DocBook XML catalogs use this attribute to prefer the public identifier whereas the Debian catalogs for DocBook XML do not do this. AFAIK we even don't have a switch for dh_installxmlcatalogs to set this attribute.

So what happens, happens accordingly to the standard and you are to
blame to use a system identifier together with the public one :)

Seriously, atm I have no clue, how to easily fix this globally. The XML
catalog system of Debian has some serious flaws and needs a complete
rewrite. In the future I'd like to use the <nextCatalog> element instead
of registering system and public identifiers on the system catalog
itself or maybe use the <group> element around them. This could solve
the problem. We cannot set the "prefer" attribute globally, because this
would conflict with catalog files, where the prefer mode conflicts with
the chosen global mode. I'd suggest to
use /usr/share/xml/docbook/schema/dtd/4.5/catalog.xml as catalog for
your DocBook files atm as a quick workaround. I'll check for possible
solutions.

Regards, Daniel

#763598#13
Date:
2014-10-07 12:59:39 UTC
From:
To:
Am Mittwoch, den 01.10.2014, 09:58 +0200 schrieb Raphael Hertzog:
[..]

Indeed the above will fail. According to the XML catalog standard you
cite in
https://www.oasis-open.org/committees/download.php/14809/xml-catalogs.html#s.ext.ent (see the end of the section and section 4.1.1) it is specified, that if system identifiers are provided together with public identifiers  the default is to only lookup the system identifier. This cbehaviour can be modified by the prefer attribute of the catalog entry. The DocBook XML catalogs use this attribute to prefer the public identifier whereas the Debian catalogs for DocBook XML do not do this. AFAIK we even don't have a switch for dh_installxmlcatalogs to set this attribute.

So what happens, happens accordingly to the standard and you are to
blame to use a system identifier together with the public one :)

Seriously, atm I have no clue, how to easily fix this globally. The XML
catalog system of Debian has some serious flaws and needs a complete
rewrite. In the future I'd like to use the <nextCatalog> element instead
of registering system and public identifiers on the system catalog
itself or maybe use the <group> element around them. This could solve
the problem. We cannot set the "prefer" attribute globally, because this
would conflict with catalog files, where the prefer mode conflicts with
the chosen global mode. I'd suggest to
use /usr/share/xml/docbook/schema/dtd/4.5/catalog.xml as catalog for
your DocBook files atm as a quick workaround. I'll check for possible
solutions.

Regards, Daniel

#763598#18
Date:
2014-10-07 22:41:58 UTC
From:
To:
Fedora uses a catalog with <rewriteSystem> and <rewriteURI>:
https://bugzilla.redhat.com/attachment.cgi?id=943232
(see https://bugzilla.redhat.com/show_bug.cgi?id=1143060#c26 for the
context)

Couldn't we use the same here? It seems to work very well for their needs.

I want to push the latest publican version to jessie and I would like to
see a fix for this issue, and one that doesn't involve a full rewrite
seems more reasonable :)

Cheers,

#763598#23
Date:
2015-05-21 19:48:03 UTC
From:
To:
Hi Daniel,

We're now in the stretch cycle and more important changes can be tried.
But I would like to see this issue fixed so that I can drop the ugly
work-around that I introduced in publican.

Thank you!