#1140270 python-scrapy: FTBFS: FAILED tests/test_robotstxt_interface.py::TestPythonRobotParser::test_order_based_precedence

Package:
src:python-scrapy
Source:
src:python-scrapy
Submitter:
Santiago Vila
Date:
2026-06-21 14:53:02 UTC
Severity:
normal
Tags:
#1140270#5
Date:
2026-06-17 16:15:38 UTC
From:
To:
Dear maintainer:

During a rebuild of all packages in unstable, this package failed to build.

Below you will find the last part of the build log (probably the most
relevant part, but not necessarily). If required, the full build log
is available here:

https://people.debian.org/~sanvila/build-logs/202606/

About the archive rebuild: The build was made on virtual machines from AWS,
using sbuild and a reduced chroot with only build-essential packages.

If you cannot reproduce the bug please contact me privately, as I
am willing to provide ssh access to a virtual machine where the bug is
fully reproducible.

If this is really a bug in one of the build-depends, please use
reassign and add an affects on src:python-scrapy, so that this is still
visible in the BTS web page for this package.

Thanks.
--------------------------------------------------------------------------------
[...]
 debian/rules clean
dh clean --buildsystem=pybuild
   dh_auto_clean -O--buildsystem=pybuild
   dh_autoreconf_clean -O--buildsystem=pybuild
   dh_clean -O--buildsystem=pybuild
 debian/rules binary
dh binary --buildsystem=pybuild
   dh_update_autotools_config -O--buildsystem=pybuild
   dh_autoreconf -O--buildsystem=pybuild
   dh_auto_configure -O--buildsystem=pybuild
   dh_auto_build -O--buildsystem=pybuild
I: pybuild plugin_pyproject:142: Building wheel for python3.14 with "build" module
I: pybuild base:385: python3.14 -m build --skip-dependency-check --no-isolation --wheel --outdir /<<PKGBUILDDIR>>/.pybuild/cpython3_3.14_scrapy
* Building wheel...
Successfully built scrapy-2.16.0-py3-none-any.whl

[... snipped ...]


docs/faq.rst ......                                                      [  0%]
docs/intro/overview.rst .                                                [  0%]
docs/intro/tutorial.rst .......................................          [  1%]
docs/news.rst ......                                                     [  1%]
docs/topics/addons.rst .....                                             [  1%]
docs/topics/api.rst .                                                    [  1%]
docs/topics/asyncio.rst ......                                           [  1%]
docs/topics/autothrottle.rst .                                           [  1%]
docs/topics/broad-crawls.rst .......                                     [  1%]
docs/topics/commands.rst ..........                                      [  1%]
docs/topics/components.rst ..                                            [  1%]
docs/topics/contracts.rst .....                                          [  1%]
docs/topics/coroutines.rst ......                                        [  2%]
docs/topics/debug.rst .........                                          [  2%]
docs/topics/developer-tools.rst .......                                  [  2%]
docs/topics/download-handlers.rst ...                                    [  2%]
docs/topics/downloader-middleware.rst .......                            [  2%]
docs/topics/dynamic-content.rst ......................                   [  3%]
docs/topics/exceptions.rst .                                             [  3%]
docs/topics/exporters.rst .....                                          [  3%]
docs/topics/extensions.rst ...                                           [  3%]
docs/topics/feed-exports.rst ........                                    [  3%]
docs/topics/item-pipeline.rst .......                                    [  3%]
docs/topics/items.rst ..............................                     [  4%]
docs/topics/jobs.rst .                                                   [  4%]
docs/topics/leaks.rst ...................                                [  4%]
docs/topics/link-extractors.rst ..                                       [  4%]
docs/topics/loaders.rst ............................                     [  5%]
docs/topics/logging.rst ..............                                   [  5%]
docs/topics/media-pipeline.rst ........................                  [  6%]
docs/topics/practices.rst ............                                   [  6%]
docs/topics/request-response.rst .......................                 [  7%]
docs/topics/selectors.rst .............................................. [  8%]
..............................................................           [  9%]
docs/topics/settings.rst ..............................                  [ 10%]
docs/topics/shell.rst .................                                  [ 10%]
docs/topics/signals.rst ...                                              [ 10%]
docs/topics/spider-middleware.rst ......                                 [ 10%]
docs/topics/spiders.rst .......................                          [ 11%]
docs/topics/stats.rst .........                                          [ 11%]
docs/topics/telnetconsole.rst ......                                     [ 11%]
tests/test_addons.py .........                                           [ 11%]
tests/test_closespider.py .......                                        [ 12%]
tests/test_cmdline/__init__.py .....                                     [ 12%]
tests/test_cmdline_crawl_with_pipeline/__init__.py ..                    [ 12%]
tests/test_command_check.py .............                                [ 12%]
tests/test_command_crawl.py ......                                       [ 12%]
tests/test_command_fetch.py .....                                        [ 12%]
tests/test_command_genspider.py ..............................           [ 13%]
tests/test_command_parse.py ......................                       [ 13%]
tests/test_command_runspider.py .........................sssssssssssssss [ 14%]
ssssssssss                                                               [ 15%]
tests/test_command_shell.py ....................                         [ 15%]
tests/test_command_startproject.py ........                              [ 15%]
tests/test_command_version.py ..                                         [ 15%]
tests/test_commands.py ....................................              [ 16%]
tests/test_contracts.py ..............                                   [ 16%]
tests/test_core_downloader.py ..s..........                              [ 17%]
tests/test_core_scraper.py .                                             [ 17%]
tests/test_crawl.py .....................................s....x.x....... [ 18%]
..........                                                               [ 18%]
tests/test_crawler.py ............................................s....  [ 19%]
tests/test_crawler_subprocess.py .....s..........................s...... [ 20%]
.....................................                                    [ 21%]
tests/test_dependencies.py s                                             [ 21%]
tests/test_downloader_handler_httpx.py ................................. [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
.......s..s.......s.....................ssssssss                         [ 26%]
tests/test_downloader_handler_twisted_ftp.py ..............              [ 26%]
tests/test_downloader_handler_twisted_http11.py ........................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 30%]
................s..........s...................s.ssssssss                [ 31%]
tests/test_downloader_handler_twisted_http2.py ......................... [ 32%]
......................................................ssssss....s....ss. [ 34%]
x...........ssssssssssssssss                                             [ 34%]
tests/test_downloader_handlers.py ..............s.......                 [ 35%]
tests/test_downloadermiddleware.py .............                         [ 35%]
tests/test_downloadermiddleware_cookies.py ............xx.x............. [ 36%]
...........                                                              [ 36%]
tests/test_downloadermiddleware_defaultheaders.py ..                     [ 36%]
tests/test_downloadermiddleware_downloadtimeout.py ....                  [ 36%]
tests/test_downloadermiddleware_httpauth.py .......                      [ 36%]
tests/test_downloadermiddleware_httpcache.py ........................... [ 37%]
..............................                                           [ 37%]
tests/test_downloadermiddleware_httpcompression.py .....s..s............ [ 38%]
......s...s...s...s...s...s...s...                                       [ 39%]
tests/test_downloadermiddleware_httpproxy.py ........................... [ 39%]
                                                                         [ 39%]
tests/test_downloadermiddleware_offsite.py ............................. [ 40%]
.................                                                        [ 40%]
tests/test_downloadermiddleware_redirect.py ............................ [ 41%]
.................................................                        [ 42%]
tests/test_downloadermiddleware_redirect_metarefresh.py ................ [ 42%]
........................................................................ [ 44%]
.......                                                                  [ 44%]
tests/test_downloadermiddleware_retry.py ............................... [ 45%]
.                                                                        [ 45%]
tests/test_downloadermiddleware_robotstxt.py .............sssssssssssss  [ 45%]
tests/test_downloadermiddleware_stats.py ...                             [ 46%]
tests/test_downloadermiddleware_useragent.py ...                         [ 46%]
tests/test_downloaderslotssettings.py .....                              [ 46%]
tests/test_dupefilters.py ..........                                     [ 46%]
tests/test_engine.py .............................                       [ 47%]
tests/test_engine_loop.py ......                                         [ 47%]
tests/test_engine_stop_download_bytes.py .                               [ 47%]
tests/test_engine_stop_download_headers.py .                             [ 47%]
tests/test_exporters.py ................................................ [ 48%]
........................................................................ [ 49%]
................................                                         [ 50%]
tests/test_extension_periodic_log.py ...                                 [ 50%]
tests/test_extension_statsmailer.py ....                                 [ 50%]
tests/test_extension_telnet.py ...                                       [ 50%]
tests/test_extension_throttle.py ....................................... [ 51%]
....                                                                     [ 51%]
tests/test_feedexport.py ........................................        [ 52%]
tests/test_feedexport_batch.py .........                                 [ 52%]
tests/test_feedexport_postprocess.py ...............                     [ 53%]
tests/test_feedexport_storages.py ................................sss... [ 54%]
..                                                                       [ 54%]
tests/test_feedexport_uri_params.py ..........                           [ 54%]
tests/test_http2_client_protocol.py ..........................           [ 54%]
tests/test_http_cookies.py ............                                  [ 55%]
tests/test_http_headers.py ..................                            [ 55%]
tests/test_http_request.py ............................................. [ 56%]
.........                                                                [ 56%]
tests/test_http_request_form.py ........................................ [ 57%]
........................................................................ [ 59%]
.......                                                                  [ 59%]
tests/test_http_request_json.py ........................................ [ 60%]
........................                                                 [ 60%]
tests/test_http_response.py ............................                 [ 61%]
tests/test_http_response_text.py ....................................... [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
.............................................................            [ 67%]
tests/test_item.py ...................                                   [ 67%]
tests/test_link.py ...                                                   [ 67%]
tests/test_linkextractors.py ....................................        [ 68%]
tests/test_loader.py ................................................... [ 69%]
....................                                                     [ 69%]
tests/test_logformatter.py ........................                      [ 70%]
tests/test_logstats.py ...                                               [ 70%]
tests/test_mail.py .......                                               [ 70%]
tests/test_middleware.py .....                                           [ 70%]
tests/test_pipeline_crawl.py ..........                                  [ 71%]
tests/test_pipeline_files.py .................................ss.....    [ 71%]
tests/test_pipeline_images.py ............................               [ 72%]
tests/test_pipeline_media.py ........................................    [ 73%]
tests/test_pipelines.py ...s................                             [ 73%]
tests/test_poet.py .                                                     [ 73%]
tests/test_pqueues.py .s...s.........                                    [ 74%]
tests/test_request_attribute_binding.py .......                          [ 74%]
tests/test_request_cb_kwargs.py .                                        [ 74%]
tests/test_request_dict.py ...............                               [ 74%]
tests/test_request_left.py ....                                          [ 74%]
tests/test_responsetypes.py .......                                      [ 75%]
tests/test_robotstxt_interface.py ........sFssssssss......s              [ 75%]
tests/test_scheduler.py .................                                [ 76%]
tests/test_scheduler_base.py ..........                                  [ 76%]
tests/test_scrapy__getattr__.py .                                        [ 76%]
tests/test_selector.py ..........s                                       [ 76%]
tests/test_settings/__init__.py ..................xx.................... [ 77%]
.......................................................................  [ 79%]
tests/test_signals.py ..                                                 [ 79%]
tests/test_spider.py .................................                   [ 79%]
tests/test_spider_crawl.py ..................................            [ 80%]
tests/test_spider_sitemap.py ........................................... [ 81%]
.                                                                        [ 81%]
tests/test_spider_start.py .....                                         [ 81%]
tests/test_spiderloader/__init__.py .................                    [ 82%]
tests/test_spidermiddleware.py .....................................     [ 82%]
tests/test_spidermiddleware_base.py ....                                 [ 82%]
tests/test_spidermiddleware_depth.py .                                   [ 82%]
tests/test_spidermiddleware_httperror.py ............                    [ 83%]
tests/test_spidermiddleware_output_chain.py ..........                   [ 83%]
tests/test_spidermiddleware_process_start.py .....                       [ 83%]
tests/test_spidermiddleware_referer.py ................................. [ 84%]
........                                                                 [ 84%]
tests/test_spidermiddleware_start.py .                                   [ 84%]
tests/test_spidermiddleware_urllength.py ..                              [ 84%]
tests/test_spiderstate.py ...                                            [ 84%]
tests/test_squeues.py ......x...............x...............x........... [ 85%]
....x...............x...............x...................x............... [ 87%]
....x...................x...................x...................x....... [ 89%]
.......x...........                                                      [ 89%]
tests/test_squeues_request.py .s.s.s.s.s.s.s.s.s.s.s.s                   [ 89%]
tests/test_stats.py .......                                              [ 90%]
tests/test_toplevel.py ......                                            [ 90%]
tests/test_utils_asyncgen.py ..                                          [ 90%]
tests/test_utils_asyncio.py ........                                     [ 90%]
tests/test_utils_conf.py .............                                   [ 90%]
tests/test_utils_console.py .ss                                          [ 90%]
tests/test_utils_curl.py ..............                                  [ 91%]
tests/test_utils_datatypes.py .......................................... [ 92%]
                                                                         [ 92%]
tests/test_utils_defer.py .......x................s                      [ 92%]
tests/test_utils_deprecate.py .................                          [ 93%]
tests/test_utils_display.py ........                                     [ 93%]
tests/test_utils_gz.py ......                                            [ 93%]
tests/test_utils_httpobj.py .                                            [ 93%]
tests/test_utils_iterators.py ....................................       [ 94%]
tests/test_utils_log.py ........................                         [ 94%]
tests/test_utils_misc/__init__.py .........                              [ 94%]
tests/test_utils_misc/test_return_with_argument_inside_generator.py .... [ 95%]
..                                                                       [ 95%]
tests/test_utils_project.py .....                                        [ 95%]
tests/test_utils_python.py .....................                         [ 95%]
tests/test_utils_reactor.py ..                                           [ 95%]
tests/test_utils_request.py .........................                    [ 96%]
tests/test_utils_response.py ..................                          [ 96%]
tests/test_utils_serialize.py ......                                     [ 96%]
tests/test_utils_signal.py ..........                                    [ 97%]
tests/test_utils_sitemap.py ...........                                  [ 97%]
tests/test_utils_spider.py ..                                            [ 97%]
tests/test_utils_template.py .                                           [ 97%]
tests/test_utils_trackref.py .....                                       [ 97%]
tests/test_utils_url.py ................................................ [ 98%]
........s....................................................            [ 99%]
tests/test_zz_resources.py ....                                          [100%]

=================================== FAILURES ===================================


tests/test_robotstxt_interface.py:160:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <tests.test_robotstxt_interface.TestPythonRobotParser object at 0x7f1198e6a490>

    def test_order_based_precedence(self):
        robotstxt_robotstxt_body = b"User-agent: * \nDisallow: / \nAllow: /page"
        rp = self.parser_cls.from_crawler(
            crawler=None, robotstxt_body=robotstxt_robotstxt_body
        )
E        +    where allowed = <scrapy.robotstxt.PythonRobotParser object at 0x7f117c4d7230>.allowed

tests/test_robotstxt_interface.py:71: AssertionError
=============================== warnings summary ===============================
scrapy/extensions/statsmailer.py:14
  /<<PKGBUILDDIR>>/.pybuild/cpython3_3.13_scrapy/build/scrapy/extensions/statsmailer.py:14: ScrapyDeprecationWarning: The scrapy.mail module is deprecated and will be removed in a future release. Please use a dedicated Python mail library instead.
    from scrapy.mail import MailSender

tests/test_extension_statsmailer.py:16
  /<<PKGBUILDDIR>>/.pybuild/cpython3_3.13_scrapy/build/tests/test_extension_statsmailer.py:16: ScrapyDeprecationWarning: The scrapy.extensions.statsmailer module is deprecated and will be removed in a future release.
    from scrapy.extensions import statsmailer  # noqa: E402

../../../../../../usr/lib/python3/dist-packages/pyftpdlib/handlers/ftp/control.py:5
  /usr/lib/python3/dist-packages/pyftpdlib/handlers/ftp/control.py:5: DeprecationWarning: asynchat was removed in Python 3.12. The recommended replacement is asyncio. Please be aware that you are currently NOT using standard 'asynchat', but instead a separately installed 'standard-asynchat'.
    import asynchat

tests/test_contracts.py::TestContractsManager::test_returns_async
  /<<PKGBUILDDIR>>/.pybuild/cpython3_3.13_scrapy/build/scrapy/contracts/__init__.py:189: RuntimeWarning: coroutine 'DemoSpider.returns_request_async' was never awaited
    results.addError(case, sys.exc_info())
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

#1140270#14
Date:
2026-06-21 14:50:13 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
python-scrapy, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1140270@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Andrey Rakhmatullin <wrar@debian.org> (supplier of updated python-scrapy package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Sat, 20 Jun 2026 18:52:39 +0500
Source: python-scrapy
Architecture: source
Version: 2.16.0-2
Distribution: unstable
Urgency: medium
Maintainer: Debian Python Team <team+python@tracker.debian.org>
Changed-By: Andrey Rakhmatullin <wrar@debian.org>
Closes: 1140270
Changes:
 python-scrapy (2.16.0-2) unstable; urgency=medium
 .
   * Fix FTBFS with Python 3.13.14+ (Closes: #1140270).
   * Switch to the debhelper compat level 14.
   * Add X-Style: black and reformat.
Checksums-Sha1:
 92b54ea32ee4c80b8d67d2d77b9768795b592c77 3724 python-scrapy_2.16.0-2.dsc
 ca0aa8148f88def3fd3f4f96937732c7fe5d3598 11572 python-scrapy_2.16.0-2.debian.tar.xz
 a5ea7d06b13e26ca63452bc31f856b388203380d 13413 python-scrapy_2.16.0-2_amd64.buildinfo
Checksums-Sha256:
 6aac8cd33f3f266cf575c075a88227df8c6fbb76c9fc5f4016fde4b06b1b5457 3724 python-scrapy_2.16.0-2.dsc
 8844a10ebad91918b81255ffadbeee0ae82d28470545300d40f373955e6c268f 11572 python-scrapy_2.16.0-2.debian.tar.xz
 f15535dfb1581b7fd9f401d52c702fc8a28bff61578f2b7b378c22129068c3fa 13413 python-scrapy_2.16.0-2_amd64.buildinfo
Files:
 f46530aa2718d57ba57cb80824624562 3724 python optional python-scrapy_2.16.0-2.dsc
 c867c6f0d359a3b222c91ad747fbe0a2 11572 python optional python-scrapy_2.16.0-2.debian.tar.xz
 d1104c2d825e7d3fbc277e0c61ec9522 13413 python optional python-scrapy_2.16.0-2_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEtf6ieDcfC1EgtGkao+OWn23e7IYFAmo2n/QACgkQo+OWn23e
7Ia8WQ/+KIHT6+pfJT55GM/fBMORNzNNNgOmnnh2woxTOKxFjlU1qhQvbKiCFJR1
JVl7gccOCZKPKidoTectNfVbYtf4mNOZpoQGCW6Pnq7DWJCEkRljG5fHv/9f+AnX
5VTqq46DsLi6zvnnmqVNTDbl913EVWajcBZs28K3OF2FPIccJz9xc1xnF37UyKuX
gP/00kpKRMgkNbqKHAgw001CAAXFKOaq0m7d+yw0n1Hjbwvoz2JxzXWUTrj0PeH+
fgDgH5QTavyHfe+OPEEsT1a2oEQvZYXDSkqemAktKmTDYOXMEg5a/KSkUEt+IAFx
dzteVMbre0mHBoShL+ut2TSEcaLpcHwuBM/365eqV1FlQqt0olyNq9mh0xZYwjxw
6QvZlZeQyUh0hbA5Xgu9KLKcrTfX4XOEUPqy5X9kPE7tsfQJbSUlTzcmaY+uzVCL
udxO70ZCz/adoxnUWjqvJuyYuTRJZi/xLbk3yXI5NeJ7gp5xyJUqqZkwKNp1NSHt
TMAgh0Mj34JPcPCBHe3lXwXqg4GnswtI3ps7vs0vG/IjwoKZCL60ADB3dDhCU+TD
B+xUG8WGHY4uTqlc93WRyJ5JMidZIM8a/L1uVOk6t1U1R+FQwbReSUdSGCM5ZQHY
+tp2NrcayzoF55EG7wr1okiO0EsIJlwQH1WZDTQrjyDV8bBljcI=
=LXOP
-----END PGP SIGNATURE-----