#1111574 python3-tldextract: tldextract with empty suffix_list_urls fails

#1111574#5
Date:
2025-08-19 15:27:20 UTC
From:
To:
Dear Maintainer,

When initiating and using TLDExtract with an empty suffix_list_urls,
subsequent calls fail. Please see the example:
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tldextract/suffix_list.py", line 104, in _get_suffix_lists
    text = find_first_response(
        cache, urls, cache_fetch_timeout=cache_fetch_timeout, session=session
    )
  File "/usr/lib/python3/dist-packages/tldextract/suffix_list.py", line 56, in find_first_response
    raise SuffixListNotFound(
    ...<2 lines>...
    )
tldextract.suffix_list.SuffixListNotFound: No remote Public Suffix List found. Consider using a mirror, or avoid this fetch by constructing your TLDExtract with `suffix_list_urls=()`.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    extract("debian.org")
    ~~~~~~~^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/tldextract/tldextract.py", line 228, in __call__
    return self.extract_str(url, include_psl_private_domains, session=session)
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/tldextract/tldextract.py", line 257, in extract_str
    return self._extract_netloc(
           ~~~~~~~~~~~~~~~~~~~~^
        lenient_netloc(url), include_psl_private_domains, session=session
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3/dist-packages/tldextract/tldextract.py", line 308, in _extract_netloc
    suffix_index, is_private = self._get_tld_extractor(
                               ~~~~~~~~~~~~~~~~~~~~~~~^
        session=session
        ^^^^^^^^^^^^^^^
    ).suffix_index(labels, include_psl_private_domains=include_psl_private_domains)
    ^
  File "/usr/lib/python3/dist-packages/tldextract/tldextract.py", line 356, in _get_tld_extractor
    public_tlds, private_tlds = get_suffix_lists(
                                ~~~~~~~~~~~~~~~~^
        cache=self._cache,
        ^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        session=session,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3/dist-packages/tldextract/suffix_list.py", line 81, in get_suffix_lists
    return cache.run_and_cache(
           ~~~~~~~~~~~~~~~~~~~^
        func=_get_suffix_lists,
        ^^^^^^^^^^^^^^^^^^^^^^^
    ...<8 lines>...
        hashed_argnames=["urls", "fallback_to_snapshot"],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3/dist-packages/tldextract/cache.py", line 211, in run_and_cache
    result = func(**kwargs)
  File "/usr/lib/python3/dist-packages/tldextract/suffix_list.py", line 109, in _get_suffix_lists
    maybe_pkg_data = pkgutil.get_data("tldextract", ".tld_set_snapshot")
  File "/usr/lib/python3.13/pkgutil.py", line 453, in get_data
    return loader.get_data(resource_name)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 1217, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/python3/dist-packages/tldextract/.tld_set_snapshot'

An empty suffix_list_urls is a perfectly valid option as can be seen in https://github.com/john-kurkowski/tldextract?tab=readme-ov-file#note-about-caching

Best regards,
Michel Le Bihan

#1111574#10
Date:
2025-08-19 15:49:58 UTC
From:
To:
I think that the best solution would be to load the system public suffix
list file (/usr/share/publicsuffix/public_suffix_list.dat) there instead
of having it in the package. |PUBLIC_SUFFIX_LIST_URLS (as changed in the
patch debian/patches/pubsuffix.diff) could then be made an empty list.
That would make sure that the system doesn't load the file from a remote
location and in turn would also fix bug #1076826.|


Best regards,
Michel Le Bihan

#1111574#15
Date:
2025-08-20 09:59:14 UTC
From:
To:
Hello,

I prepared a patch to fix this issue. It should replace
patches/pubsuffix.diff. A dependency on |publicsuffix also needs to be
added.|

Best regards,
Michel Le Bihan

#1111574#18
Date:
2025-08-20 11:37:30 UTC
From:
To:
Hello,

Bug #1111574 in tldextract reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/python-team/packages/python-tldextract/-/commit/5f8ba283abc6156a6f4dfe57e2006c1913f7af25
------------------------------------------------------------------------
patch: Use list of suffixes from publicsuffix package

Closes: #1076826, #1111574
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1111574

#1111574#25
Date:
2025-08-20 14:53:28 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
tldextract, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1111574@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Alexandre Detiste <tchet@debian.org> (supplier of updated tldextract package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Wed, 20 Aug 2025 16:02:43 +0200
Source: tldextract
Architecture: source
Version: 5.1.2-2
Distribution: unstable
Urgency: medium
Maintainer: Debian Python Team <team+python@tracker.debian.org>
Changed-By: Alexandre Detiste <tchet@debian.org>
Closes: 1076826 1111574
Changes:
 tldextract (5.1.2-2) unstable; urgency=medium
 .
   * Team Upload
   * Set Team as Maintainer per new DPT Policy
   * Bump Standards-Version to 4.7.2
   * Drop "Rules-Requires-Root: no", it is the default now
   * Reformat d/control with debputy
 .
   [ Michel Le Bihan ]
   * patch: Use list of suffixes from publicsuffix package
            (Closes: #1076826, #1111574)
   * Declare runtime dependency on publicsuffix
   * Make autopkgtest-pkg-pybuild work
Checksums-Sha1:
 829a47af5f3fa3448bdc42a3ccd6c0d6ccaf1eb3 2353 tldextract_5.1.2-2.dsc
 29d34ea8eb4291d4b8f01402e8b3682fb6d7d8ff 4248 tldextract_5.1.2-2.debian.tar.xz
 790d0cff5ac1c66e95f0eef9e4b5a80bf61147fa 7820 tldextract_5.1.2-2_source.buildinfo
Checksums-Sha256:
 d995fdec8f95536968c97a8d2339ca13f672d9054c3e19236203f8dcf0e74cd0 2353 tldextract_5.1.2-2.dsc
 bd755c9103eb9d0cfb3b3c6b14ccf86ff2315940ef34196160cb861c114d1bc7 4248 tldextract_5.1.2-2.debian.tar.xz
 623efce95b616e9bf67ab7f10ef561820c5f52290fb1e1eae2a990c430af162f 7820 tldextract_5.1.2-2_source.buildinfo
Files:
 229ab28e87d191a88629b62713b2578a 2353 python optional tldextract_5.1.2-2.dsc
 38e54af5d5e6a85a95ae3a752652a799 4248 python optional tldextract_5.1.2-2.debian.tar.xz
 43350fa2ca844c615ee97e4fd49b6fd7 7820 python optional tldextract_5.1.2-2_source.buildinfo
-----BEGIN PGP SIGNATURE-----

iQJFBAEBCgAvFiEEj23hBDd/OxHnQXSHMfMURUShdBoFAmil3ekRHHRjaGV0QGRl
Ymlhbi5vcmcACgkQMfMURUShdBoWHBAAgGvcnDCyvKpoeboX4H9TUKb8TrD+PKVk
yzqKZLwDWTMHl3dfB5hqXpeDprxxlUibU2dI6PpAuKZPWC4IDbEYD1giS/QZsSnk
UbWhUEF6iURYBgupUjgFID616PG1EtCHjAxR2UutZWJfThcBn+Izw1PQoec1S5JW
SSig4zgQn4njPcIZ7wSUR72TvJAjWq5KT2dTnIff+42Aqf1a1N44i+8BpuQH6e/5
FwAlSHoh4aeOo4I+gRxFR4qGXQyqQ9myelnrV1sf4GYrL+1ZB+jOep4V+vGzrB4H
MD7jS+bxF8i8wKj23V68yPfgg7l7KgTt2iixndvSrOeV1uUn/HWR2156aotclXu2
Q3dkhY5644h5sxdqQ+MKjPydhLnvJMMnP+RmREigbERxOJepyarIuFrCMoug4ZZC
iKsck4IvAYRdETTTOnugintWtid/Yhu6pU4EdlKyFIw3DYBz+dIt+1nBpSOeBKhG
UwCA07M4oosCNgvBk/SnngrYTthyrTPMtJCPGXa27mKeYFEOqdJyl6GFtqAkPiey
rt4dAeFDjdZJCK7Z4GYDVQXNTZqJVu1JhtEPmOpEP2gQkBkAeIQNrB/g5jwDOM5Z
mPxHfoVST8MFfHzbzYnvlUm3MFdlLL44Ng82ywvhIJjrV1jkIxLsGQEp2l84sdsd
wkONa6S/MC8=
=dD+w
-----END PGP SIGNATURE-----