#1106066 python-w3lib: FTBFS: E ValueError: 'example.com' does not appear to be an IPv4 or IPv6 address

#1106066#5
Date:
2025-05-19 09:50:44 UTC
From:
To:
Dear maintainer:

During a rebuild of all packages in bookworm, your package failed to build:

[ Note: The bug is closed with the version in trixie/sid, but
  packages in bookworm must still build in bookworm ].
--------------------------------------------------------------------------------
[...]
 debian/rules clean
dh clean --buildsystem=pybuild
   dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:240: python3.11 setup.py clean
running clean
removing '/<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.11' does not exist -- can't clean it
   dh_autoreconf_clean -O--buildsystem=pybuild
   dh_clean -O--buildsystem=pybuild
 debian/rules binary
dh binary --buildsystem=pybuild
   dh_update_autotools_config -O--buildsystem=pybuild
   dh_autoreconf -O--buildsystem=pybuild
   dh_auto_configure -O--buildsystem=pybuild
I: pybuild base:240: python3.11 setup.py config
running config
   dh_auto_build -O--buildsystem=pybuild
I: pybuild base:240: /usr/bin/python3 setup.py build
running build
running build_py
creating /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/_infra.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/http.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/_types.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/encoding.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/_url.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/__init__.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/util.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/url.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
copying w3lib/html.py -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
running egg_info
writing w3lib.egg-info/PKG-INFO
writing dependency_links to w3lib.egg-info/dependency_links.txt
writing top-level names to w3lib.egg-info/top_level.txt
writing manifest file 'w3lib.egg-info/SOURCES.txt'
reading manifest file 'w3lib.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.txt' under directory 'tests'
adding license file 'LICENSE'
writing manifest file 'w3lib.egg-info/SOURCES.txt'
copying w3lib/py.typed -> /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build/w3lib
   dh_auto_test -O--buildsystem=pybuild
I: pybuild base:240: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build; python3.11 -m pytest tests
============================= test session starts ==============================
platform linux -- Python 3.11.2, pytest-7.2.1, pluggy-1.0.0+repack
rootdir: /<<PKGBUILDDIR>>, configfile: pytest.ini
collected 314 items

tests/test_encoding.py ...................                               [  6%]
tests/test_html.py ..................................................... [ 22%]
..                                                                       [ 23%]
tests/test_http.py .......                                               [ 25%]
tests/test_url.py x...xxx....x..x.................................xxxxxx [ 42%]
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.....Fx...xx......xxxFx....xx.x.. [ 65%]
xx....x......xxxxxx..xxxx..x....x....................................... [ 88%]
..............................                                           [ 98%]
tests/test_util.py .....                                                 [100%]

=================================== FAILURES ===================================
_ test_safe_url_string_url[https://"%;<=>@[]^`{|}\x7f:"%;<=>@[]^`{|}\x7f:@example.com-https://%22%25%3B%3C%3D%3E%40%5B%5D%5E%60%7B%7C%7D%7F:%22%25%3B%3C%3D%3E%40%5B%5D%5E%60%7B%7C%7D%7F%3A@example.com] _

url = 'https://"%;<=>@[]^`{|}\x7f:"%;<=>@[]^`{|}\x7f:@example.com'
output = 'https://%22%25%3B%3C%3D%3E%40%5B%5D%5E%60%7B%7C%7D%7F:%22%25%3B%3C%3D%3E%40%5B%5D%5E%60%7B%7C%7D%7F%3A@example.com'

    @pytest.mark.parametrize(
        "url,output",
        tuple(
            case
            if case[0] not in KNOWN_SAFE_URL_STRING_URL_ISSUES
            else pytest.param(*case, marks=pytest.mark.xfail(strict=True))
            for case in SAFE_URL_URL_CASES
        ),
    )
    def test_safe_url_string_url(
        url: StrOrBytes, output: Union[str, Type[Exception]]
    ) -> None:

tests/test_url.py:435:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_url.py:343: in _test_safe_url_string
    return _test_safe_url_func(
tests/test_url.py:332: in _test_safe_url_func
    actual = func(url, **kwargs)
w3lib/url.py:142: in safe_url_string
    parts = urlsplit(_strip(decoded))
/usr/lib/python3.11/urllib/parse.py:516: in urlsplit
    _check_bracketed_netloc(netloc)
/usr/lib/python3.11/urllib/parse.py:454: in _check_bracketed_netloc
    _check_bracketed_host(hostname)
/usr/lib/python3.11/urllib/parse.py:463: in _check_bracketed_host
    ip = ipaddress.ip_address(hostname) # Throws Value Error if not IPv6 or IPv4
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = 'example.com'

    def ip_address(address):
        """Take an IP string/int and return an object of the correct type.

        Args:
            address: A string or integer, the IP address.  Either IPv4 or
              IPv6 addresses may be supplied; integers less than 2**32 will
              be considered to be IPv4 by default.

        Returns:
            An IPv4Address or IPv6Address object.

        Raises:
            ValueError: if the *address* passed isn't either a v4 or a v6
              address

        """
        try:
            return IPv4Address(address)
        except (AddressValueError, NetmaskValueError):
            pass

        try:
            return IPv6Address(address)
        except (AddressValueError, NetmaskValueError):
            pass
/usr/lib/python3.11/ipaddress.py:54: ValueError
_______ test_safe_url_string_url[http://[2a01:5cc0:1:2:3:4]-ValueError] ________
[XPASS(strict)]
=========================== short test summary info ============================
FAILED tests/test_url.py::test_safe_url_string_url[https:/"%;<=>@[]^`{|}\x7f:"%;<=>@[]^`{|}\x7f:@example.com-https:/%22%25%3B%3C%3D%3E%40%5B%5D%5E%60%7B%7C%7D%7F:%22%25%3B%3C%3D%3E%40%5B%5D%5E%60%7B%7C%7D%7F%3A@example.com]
FAILED tests/test_url.py::test_safe_url_string_url[http:/[2a01:5cc0:1:2:3:4]-ValueError]
================== 2 failed, 236 passed, 76 xfailed in 1.54s ===================
E: pybuild pybuild:388: test: plugin distutils failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_w3lib/build; python3.11 -m pytest tests
dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.11 returned exit code 13
make: *** [debian/rules:6: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
--------------------------------------------------------------------------------

The above is just how the build ends and not necessarily the most relevant part.
If required, the full build log is available here:

https://people.debian.org/~sanvila/build-logs/bookworm/

About the archive rebuild: The build was made on virtual machines from AWS,
using sbuild and a reduced chroot with only build-essential packages.

If you could not reproduce the bug please contact me privately, as I
am willing to provide ssh access to a virtual machine where the bug is
fully reproducible.

If this is really a bug in one of the build-depends, please use
reassign and add an affects on src:python-w3lib, so that this is still
visible in the BTS web page for this package.

Note: The following packages fail in a similar weird way:

asyncpg
python-taskflow
python-w3lib
yarl

Maybe there is a common reason, or maybe the packages need to adapt anyway.

Thanks.

#1106066#16
Date:
2025-05-19 10:13:56 UTC
From:
To:
[...]

Hi! This is #1042345 aka https://github.com/scrapy/w3lib/issues/212
The upstream fix is https://github.com/scrapy/w3lib/pull/213/files (note
that it checks the interpreter version for one of the failing tests, which
won't work in Debian, because the CVE-2024-11168 fix is included upstream
in 3.11.4 but in Debian it was backported into 3.11.2-6+deb12u5).

All of that should be fallout from https://github.com/python/cpython/issues/103848