#1104495 r-cran-httr2: flaky tests on riscv64

#1104495#5
Date:
2025-05-01 08:42:03 UTC
From:
To:
Hi,

your package's tests fail very often (but not always) on riscv64:

https://ci.debian.net/packages/r/r-cran-httr2/unstable/riscv64/

Example log:

https://ci.debian.net/data/autopkgtest/testing/riscv64/r/r-cran-httr2/60319307/log.gz
but also
https://ci.debian.net/data/autopkgtest/unstable/riscv64/r/r-cran-httr2/59659816/log.gz
which looks completely different.

Please ensure the tests are stable enough for automated testing, on
all archs.

Thanks,
Chris

#1104495#10
Date:
2025-05-01 08:55:49 UTC
From:
To:
Hi Chris,

we are struggling hard with bigger issues, and risc64 is not supported
upstream.  Also the whole architecture as a popcon score of 29.  Please
allow us to ignore the issue in this release.

Thanks,

Charles


Le Thu, May 01, 2025 at 10:42:03AM +0200, Chris Hofstaedtler a écrit :

#1104495#17
Date:
2025-05-01 13:56:52 UTC
From:
To:
Hi,

* Charles Plessy <plessy@debian.org> [250501 10:56]:

I think you are mostly hurting your own team, as packages will be
stuck migrating etc.

Chris

#1104495#22
Date:
2025-05-02 01:15:30 UTC
From:
To:
Le Thu, May 01, 2025 at 03:56:52PM +0200, Chris Hofstaedtler a écrit :

Hi Chris,

thanks for your help in general; if my actions make your work harder, I will
stop changing severities.

This said, please correct me if anything is wrong in the following:

 - setting the severity from serious to normal is not going to block
   testing migration.

 - r-cran-httr2 is in Trixie at the same version as in Sid, does
   not need migration at the moment, and does not block any migration.

 - Nobody knows how to make r-cran-httr2 tests more robust on risc64
   (except of course by disabling them).

 - keeping severity to serious can cause r-cran-httr2 and its reverse-depencies
   to be removed from Trixie.

Have a nice day,

Charles

#1104495#27
Date:
2025-05-02 09:41:34 UTC
From:
To:
Hi Charles,

* Charles Plessy <plessy@debian.org> [250502 03:15]:
[..]

Yes, but other packages also trigger the r-cran-httr2 test. For
example, r-cran-base. Which was blocked from migrating for a while
because of r-cran-httr2 - at least until someone noticed.

It looks like the tests have an internal timeout, until when they
expect something to start. I imagine this timeout can be increased.


I've briefly spoken to Paul here at MiniDebConf Hamburg. It might be
reasonable to rejectlist r-cran-httr2 on riscv64, but please
consider increasing the r-cran-httr2 tests internal timeout first.

Chris

#1104495#32
Date:
2025-05-03 23:05:11 UTC
From:
To:
Le Fri, May 02, 2025 at 11:41:34AM +0200, Chris Hofstaedtler a écrit :

Thanks Chris for checking this,

Indeed it seems that there is a timeout of 30s that seems enough to
cause the tests to fail around 5% of the times.  I also note that
some of the failures were synchronised on some architecture (like
failing on the same day in amd64 and arm64), so it may be network
issues that are not solvable by changing the timeout.

Indeed, the failure rate on risc64 is way higher than on other release
architectures.  The tests run also 10 times slower, which is a lot.  I
am not familiar with risc64, but if the usage pattern of this
architecture is narrower than amd64 and arm64 and does not include
scientific computing, it may make sense to just remove all
team-maintained r-cran-* packages there too, as it is not supported
upstream.  Risc64 users who just want to perform HTTP requests with a
script language have better alternatives such as Perl or Python.

Have a nice day,

Charles

#1104495#37
Date:
2025-05-04 08:18:38 UTC
From:
To:
* Charles Plessy <charles@plessy.org> [250504 01:05]:

30s might be too slow on all archs, on "loaded" runners.

The expectation is that riscv64 will get significantly faster
hardware (in general and in Debian), but probably during forky.

I can't make the call for you or the riscv porters.

Chris

#1104495#42
Date:
2025-05-04 19:01:25 UTC
From:
To:
I agree this is probably caused by the timeout being too short for slow
hardware (such as the current riscv64 test runners).  My (untested)
attempt to increase the timeout is below.

It's *not* an urgent problem, as retrying failures usually works.

 > Indeed it seems that there is a timeout of 30s

Where did you find 30s?  The places I've seen timeout errors are:
- Timeout of 1s at
https://sources.debian.org/src/r-cran-httr2/1.1.1-1/tests/testthat/test-req-perform-parallel.R/?hl=28#L17
- Timeout starting the test server (in various tests), which seems to
use the default of 5s:
https://sources.debian.org/src/r-cran-httr2/1.1.1-1/R/test.R/#L54
https://sources.debian.org/src/r-cran-webfakes/1.3.2-1/R/app-process.R/?hl=91#L89

 > some of the failures were synchronised on some architecture (like
 > failing on the same day in amd64 and arm64)

What does this refer to?  I don't see any failures on those, except when
there was (plausibly) a real bug (e.g. r-cran-rlang/1.1.5-2).

 > so it may be network
 > issues that are not solvable by changing the timeout.

Probably not: the failing tests are ones that use a local test server,
not the actual internet.
--- a/R/test.R
+++ b/R/test.R
@@ -53,7 +53,8 @@ example_app <- function() {

    webfakes::new_app_process(
      app,
-    opts = webfakes::server_opts(num_threads = 20, enable_keep_alive =
TRUE)
+    opts = webfakes::server_opts(num_threads = 20, enable_keep_alive =
TRUE),
+    process_timeout = 10000 # ms, default is 5000
    )
  }
--- a/tests/testthat/test-req-perform-parallel.R +++ b/tests/testthat/test-req-perform-parallel.R @@ -18,14 +18,14 @@ test_that("requests happen in parallel", { # test works best if webfakes has ample threads and keepalive reqs <- list2( request_test("/delay/:secs", secs = 0), - request_test("/delay/:secs", secs = 0.25), - request_test("/delay/:secs", secs = 0.25), - request_test("/delay/:secs", secs = 0.25), - request_test("/delay/:secs", secs = 0.25), - request_test("/delay/:secs", secs = 0.25), + request_test("/delay/:secs", secs = 0.5), + request_test("/delay/:secs", secs = 0.5), + request_test("/delay/:secs", secs = 0.5), + request_test("/delay/:secs", secs = 0.5), + request_test("/delay/:secs", secs = 0.5), ) time <- system.time(req_perform_parallel(reqs)) - expect_lt(time[[3]], 1) + expect_lt(time[[3]], 2) }) test_that("can perform >128 file uploads in parallel", {
#1104495#49
Date:
2025-05-05 00:20:04 UTC
From:
To:
Hi Rebecca,

I was wondering about what to do yesterday, and decided that I was not
convinced that deviation from upstream is needed and if I were wrong,
somebody would step in.  So please feel free too go ahead.

One explanation for my behaviour is that I have been running hundreads
of routine updates and that has let me seen that every patch is a toxic
asset.  They eat time unpredictably (except for those who live near time
stamps; those you can be sure that any routine update you are going to
have to edit them by hand).  If we let them them accumulate they will
paralyse us little by little.  Dirk's external repositories are mostly
patchless and probably have more satisfied users than our packages.

Here is a point to point answer to your questions in case it is useful.

Le Sun, May 04, 2025 at 08:01:25PM +0100, Rebecca N. Palmer a écrit :

I have seen a value of 30 at the following line.  That is obviously
not enough for a strong conclusion.  I am sorry if I was wrong and
wasted people time.

https://github.com/r-lib/httr2/blob/main/tests/testthat/helper-promise.R#L2

For instance 1.0.0-1 has failed migration-reference/0 on 2024-01-04 06:36:10 UTC
and 2023-12-07 03:45:08 UTC on amd64 and arm64, but passed on 2023-12-09 22:39:47 UTC
and 2024-01-05 01:35:14 UTC on armel.  There is at least one more
example like this in earlier years.

Have a good day!

Charles

#1104495#54
Date:
2025-05-05 12:59:20 UTC
From:
To:
Thank you.

I intend to leave this until after release, since Normal bugs aren't
supposed to be fixed during freeze.

That appears to be used only in test-req-promise.R; since I haven't seen
that one failing, it probably does not need to be increased.

Those are in unstable not testing (which makes real bugs more likely),
and long enough ago that the logs have been deleted.

#1104495#57
Date:
2025-09-18 14:31:10 UTC
From:
To:
Hello,

Bug #1104495 in r-cran-httr2 reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/r-pkg-team/r-cran-httr2/-/commit/9dcb1a7355fa590615794ec08c10fbe0cd369ebe

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1104495

#1104495#62
Date:
2025-09-21 13:24:14 UTC
From:
To:
Hello,

Bug #1104495 in r-cran-httr2 reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/r-pkg-team/r-cran-httr2/-/commit/9dcb1a7355fa590615794ec08c10fbe0cd369ebe

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1104495

#1104495#67
Date:
2025-09-21 17:58:19 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
r-cran-httr2, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1104495@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Rebecca N. Palmer <rebecca_palmer@zoho.com> (supplier of updated r-cran-httr2 package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Sun, 21 Sep 2025 13:37:15 +0100
Source: r-cran-httr2
Architecture: source
Version: 1.1.1-2
Distribution: unstable
Urgency: medium
Maintainer: Debian R Packages Maintainers <r-pkg-team@alioth-lists.debian.net>
Changed-By: Rebecca N. Palmer <rebecca_palmer@zoho.com>
Closes: 1104495
Changes:
 r-cran-httr2 (1.1.1-2) unstable; urgency=medium
 .
   * Team upload.
   * Tests: increase timeouts to pass on slow runners (closes: #1104495)
Checksums-Sha1:
 82be8d7d6cb37d07c81ffb9c067f031ff7718e59 2501 r-cran-httr2_1.1.1-2.dsc
 112d19ea277a0479985fd457cbcc3465a78af9ab 3948 r-cran-httr2_1.1.1-2.debian.tar.xz
 39c4a68620579ca3d99614d630295766805ed656 5545 r-cran-httr2_1.1.1-2_source.buildinfo
Checksums-Sha256:
 de927aed9e8b8f4b211017bc596fcc94c053f0651ac34b52db332134fd47924c 2501 r-cran-httr2_1.1.1-2.dsc
 1a34d5a73c31c8143ea33c35525c2c62a73e245e2f07e101f17290289fb46879 3948 r-cran-httr2_1.1.1-2.debian.tar.xz
 1136ad4ba0dba0a19ec1ff17a2c37e050beae498d53613f69127144fd5d0eec7 5545 r-cran-httr2_1.1.1-2_source.buildinfo
Files:
 d6921f280dedce2d4ec71dc69168d1d4 2501 gnu-r optional r-cran-httr2_1.1.1-2.dsc
 dc5f229b3d8d298e65e9a4d16349cfb8 3948 gnu-r optional r-cran-httr2_1.1.1-2.debian.tar.xz
 f5fb85d6d755a8428d5d15eeccfaa5ad 5545 gnu-r optional r-cran-httr2_1.1.1-2_source.buildinfo
-----BEGIN PGP SIGNATURE-----

iQJMBAEBCgA2FiEEZ8sxEAXE7b4yF1MI3uUNDVZ+omYFAmjQNMYYHHJlYmVjY2Ff
cGFsbWVyQHpvaG8uY29tAAoJEN7lDQ1WfqJm3c8QAJ2+VR9u6F1mweRh69njf8kD
EPQmxhRCkspWofBQfkdem811BL8fz1AP4LT87P6aHHAz5gbisLkDkSZswl1/jyA2
0JVTkqQaw+hazQe4/KzK4zHWMtCk0VlDJdJ8jH/+wzOj7T34bmgPNzgVzjZmKs4y
eotNhTfl3BJ4Z/CZoHvJ3CZbU5oeVVJlJ/NvEReke9YPwWnaT6pmu/i3GhQZ5N6E
IYQZw10TuJaMDTgJJbBekm8z2zQO7TXNpXf/rjExjDca0+s+fIVTEHiLgEJOCVfD
cXWPZ2l+Gw72SojAaYGtwZWaHObubBsblrWmuuzN6PdmTTey3DYMlLD/Gi99FrHp
Daw/lblpGGLYJpejWmjvCxYuJOSHrJKUHs4U1+GBPjvqHMxH0UfDb+MkO2Qxr2Ke
+rOTWXG7/jccT7OOT8+q55rJq+OENL0F2ivf3gC6jh9exqs+4SQ3aOPu0VbyELIz
hlH35Wbf/fJ01LW4XHMROhk9QIJ4OjPlJ/UpEPt0w6JadotV72q+hxQctOjJQke4
46fmTISE7wDSLdMnIsChsG3e6dHcioe4toK5h5hh3eeP4zWUg9ZkU1+BDeNkeo1j
9xCpk62acHG2G3W8Q+k3XxCG6w631QbecjBXNUTPwwYytfRQY7+LZOuVpsrEAXe0
+MR7tGZ+qqhoSwLLcWpb
=6s94
-----END PGP SIGNATURE-----