- Package:
- r-base-dev
- Source:
- r-base
- Submitter:
- Ximin Luo
- Date:
- 2026-10-19 18:29:03 UTC
- Severity:
- wishlist
- Tags:
Dear Maintainer, I've been asked to file this bug report so that others working on the Reproducible Builds project can better track the ongoing issue of build-path-independent reproducibility in various tools and compilers. I don't mean to imply to rush this process nor pressure you, this report is only for informational purposes. As you know, I've already contacted upstream with a proposed patch: - https://stat.ethz.ch/pipermail/r-devel/2017-April/074138.html - https://stat.ethz.ch/pipermail/r-devel/2017-May/074181.html (patch amended) These may be browsed as git commits here: - https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds The built packages may be downloaded here: - https://reproducible.alioth.debian.org/debian/ - https://tests.reproducible-builds.org/debian/index_repositories.html (overview) The reproducibility status of dependent packages is being tracked on these pages: - https://tests.reproducible-builds.org/debian/issues/unstable/randomness_in_r_rdb_rds_databases_issue.html We're currently waiting to hear back from upstream to see if they think the patch will negatively impact the behaviour of any R packages. At minimum, we likely have to amend it to guard the behaviour behind some sort of run-time option, as was done for a similar patch for timestamps in #774031. Ximin
On 8 May 2017 at 19:06, Ximin Luo wrote: | Package: r-base-dev | Version: 3.4.0-1.0~reproducible2 | Severity: wishlist | Tags: upstream patch | User: reproducible-builds@lists.alioth.debian.org | Usertags: build-path | | Dear Maintainer, | | I've been asked to file this bug report so that others working on the | Reproducible Builds project can better track the ongoing issue of | build-path-independent reproducibility in various tools and compilers. I don't | mean to imply to rush this process nor pressure you, this report is only for | informational purposes. | | As you know, I've already contacted upstream with a proposed patch: | - https://stat.ethz.ch/pipermail/r-devel/2017-April/074138.html | - https://stat.ethz.ch/pipermail/r-devel/2017-May/074181.html (patch amended) | | These may be browsed as git commits here: | - https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds | | The built packages may be downloaded here: | - https://reproducible.alioth.debian.org/debian/ | - https://tests.reproducible-builds.org/debian/index_repositories.html (overview) | | The reproducibility status of dependent packages is being tracked on these pages: | - https://tests.reproducible-builds.org/debian/issues/unstable/randomness_in_r_rdb_rds_databases_issue.html | | We're currently waiting to hear back from upstream to see if they think the | patch will negatively impact the behaviour of any R packages. At minimum, we | likely have to amend it to guard the behaviour behind some sort of run-time | option, as was done for a similar patch for timestamps in #774031. At work now with limited time, but I think I already told you twice that there were two of the three parts of the patch you proposed to upstream that I would not take, were I upstream (which I am not). A decent longer-term strategy may well to stress-test the patch by including it in our builds for a while. We can work on that. Dirk | | Ximin | | -- System Information: | Debian Release: 9.0 | APT prefers testing | APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (300, 'unstable'), (200, 'experimental'), (1, 'experimental-debug') | Architecture: amd64 | (x86_64) | Foreign Architectures: i386 | | Kernel: Linux 4.9.0-2-amd64 (SMP w/4 CPU cores) | Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8) | Shell: /bin/sh linked to /bin/dash | Init: systemd (via /run/systemd/system) | | Versions of packages r-base-dev depends on: | ii build-essential 12.3 | ii cdbs 0.4.150 | ii g++ 4:6.3.0-4 | ii gcc 4:6.3.0-4 | ii gfortran 4:6.3.0-4 | ii libatlas-base-dev 3.10.3-1+b1 | ii libblas-dev 3.7.0-1 | ii libbz2-dev 1.0.6-8.1 | ii libicu-dev 57.1-6 | ii libjpeg-dev 1:1.5.1-2 | ii libjpeg62-turbo-dev [libjpeg-dev] 1:1.5.1-2 | ii liblapack-dev 3.7.0-1 | ii liblzma-dev 5.2.2-1.2+b1 | ii libncurses5-dev 6.0+20161126-1 | ii libpcre3-dev 2:8.39-3 | ii libpng-dev 1.6.28-1 | ii libreadline-dev 7.0-2 | ii pkg-config 0.29-4+b1 | ii r-base-core 3.4.0-1.0~reproducible2 | ii xauth 1:1.0.9-1+b2 | ii zlib1g-dev 1:1.2.8.dfsg-5 | | r-base-dev recommends no packages. | | Versions of packages r-base-dev suggests: | ii debhelper 10.2.5 | ii texinfo 6.3.0.dfsg.1-1+b2 | ii texlive-base 2016.20170123-5 | ii texlive-extra-utils 2016.20170123-5 | ii texlive-fonts-extra 2016.20170123-5 | ii texlive-fonts-recommended 2016.20170123-5 | ii texlive-generic-recommended 2016.20170123-5 | ii texlive-latex-base 2016.20170123-5 | ii texlive-latex-extra 2016.20170123-5 | ii texlive-latex-recommended 2016.20170123-5 | | -- no debconf information
On 8 May 2017 at 19:06, Ximin Luo wrote: | Package: r-base-dev | Version: 3.4.0-1.0~reproducible2 | Severity: wishlist | Tags: upstream patch | User: reproducible-builds@lists.alioth.debian.org | Usertags: build-path | | Dear Maintainer, | | I've been asked to file this bug report so that others working on the | Reproducible Builds project can better track the ongoing issue of | build-path-independent reproducibility in various tools and compilers. I don't | mean to imply to rush this process nor pressure you, this report is only for | informational purposes. | | As you know, I've already contacted upstream with a proposed patch: | - https://stat.ethz.ch/pipermail/r-devel/2017-April/074138.html | - https://stat.ethz.ch/pipermail/r-devel/2017-May/074181.html (patch amended) | | These may be browsed as git commits here: | - https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds | | The built packages may be downloaded here: | - https://reproducible.alioth.debian.org/debian/ | - https://tests.reproducible-builds.org/debian/index_repositories.html (overview) | | The reproducibility status of dependent packages is being tracked on these pages: | - https://tests.reproducible-builds.org/debian/issues/unstable/randomness_in_r_rdb_rds_databases_issue.html | | We're currently waiting to hear back from upstream to see if they think the | patch will negatively impact the behaviour of any R packages. At minimum, we | likely have to amend it to guard the behaviour behind some sort of run-time | option, as was done for a similar patch for timestamps in #774031. At work now with limited time, but I think I already told you twice that there were two of the three parts of the patch you proposed to upstream that I would not take, were I upstream (which I am not). A decent longer-term strategy may well to stress-test the patch by including it in our builds for a while. We can work on that. Dirk | | Ximin | | -- System Information: | Debian Release: 9.0 | APT prefers testing | APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (300, 'unstable'), (200, 'experimental'), (1, 'experimental-debug') | Architecture: amd64 | (x86_64) | Foreign Architectures: i386 | | Kernel: Linux 4.9.0-2-amd64 (SMP w/4 CPU cores) | Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8) | Shell: /bin/sh linked to /bin/dash | Init: systemd (via /run/systemd/system) | | Versions of packages r-base-dev depends on: | ii build-essential 12.3 | ii cdbs 0.4.150 | ii g++ 4:6.3.0-4 | ii gcc 4:6.3.0-4 | ii gfortran 4:6.3.0-4 | ii libatlas-base-dev 3.10.3-1+b1 | ii libblas-dev 3.7.0-1 | ii libbz2-dev 1.0.6-8.1 | ii libicu-dev 57.1-6 | ii libjpeg-dev 1:1.5.1-2 | ii libjpeg62-turbo-dev [libjpeg-dev] 1:1.5.1-2 | ii liblapack-dev 3.7.0-1 | ii liblzma-dev 5.2.2-1.2+b1 | ii libncurses5-dev 6.0+20161126-1 | ii libpcre3-dev 2:8.39-3 | ii libpng-dev 1.6.28-1 | ii libreadline-dev 7.0-2 | ii pkg-config 0.29-4+b1 | ii r-base-core 3.4.0-1.0~reproducible2 | ii xauth 1:1.0.9-1+b2 | ii zlib1g-dev 1:1.2.8.dfsg-5 | | r-base-dev recommends no packages. | | Versions of packages r-base-dev suggests: | ii debhelper 10.2.5 | ii texinfo 6.3.0.dfsg.1-1+b2 | ii texlive-base 2016.20170123-5 | ii texlive-extra-utils 2016.20170123-5 | ii texlive-fonts-extra 2016.20170123-5 | ii texlive-fonts-recommended 2016.20170123-5 | ii texlive-generic-recommended 2016.20170123-5 | ii texlive-latex-base 2016.20170123-5 | ii texlive-latex-extra 2016.20170123-5 | ii texlive-latex-recommended 2016.20170123-5 | | -- no debconf information
control: found -1 3.4.0-1 control: notfound -1 3.4.0-1.0~reproducible2 Hi Dirk, I've asked Ximin to file this bug so that we have something to track and to refer in discussions… you wrote: Dirk, could you please point out (here in this bug report) which of the two parts you consider unsuitable? Ximin, looking at https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds I believe it would be best to merge those two (top most) commits into one? actually we've been using Ximin's patches on tests.reproducible-builds.org since the 2nd of May on amd64 and since the 3rd on arm64, armhf and i386. According to https://tests.reproducible-builds.org/debian/index_performance.html (top row labeled "oldest build result) in the first table on the right) this means we've almost done a full rebuild with the patch on amd64+arm64 (probably 85% done now) and pretty close to that on i386… According to this, this patch seems to work well…
actually, we've done a full rebuild of all r-* packages on amd64, not just 85%: Ximin scheduled them all manually on amd64 before filing this bug… :)
Holger Levsen: We talked about this over private email, this refers to the patch hunks in: - src/library/tools/R/admin.R - src/library/tools/R/parseRd.R The suggestion was to guard them behind a command-line option using getOption, so presumably Debian could set this whilst upstream / other distros do not. However, since I'm not familiar with the full R codebase I was waiting to see if upstream had any further comments, because even with the getOption() change there might be other consequences that I didn't foresee. In this case it would be beneficial for Debian's behaviour to exactly match upstream, so we get the same bugs (if any). The patch works well for getting stuff built, but I haven't tested all of the *behaviour* of these packages. (Some probably have unit tests, but these don't cover everything at any rate.) That is why I wanted to wait for upstreams' comments first, before adding the getOption guards - which are relatively minor IMO, compared to what the full patch does. I also haven't tested other potential tools that might process R rdb files, who might for some reason expect absolute build paths. X
Sorry, was out traveling / running a relay. Now back. On 15 May 2017 at 17:12, Ximin Luo wrote: | Holger Levsen: | > control: found -1 3.4.0-1 | > control: notfound -1 3.4.0-1.0~reproducible2 | > | > Hi Dirk, | > | > I've asked Ximin to file this bug so that we have something to track and to | > refer in discussions… | > | > you wrote: | > | >> At work now with limited time, but I think I already told you twice that | >> there were two of the three parts of the patch you proposed to upstream that | >> I would not take, were I upstream (which I am not). | > | > Dirk, could you please point out (here in this bug report) which of the two parts | > you consider unsuitable? | > | | We talked about this over private email, this refers to the patch hunks in: | | - src/library/tools/R/admin.R | - src/library/tools/R/parseRd.R | | The suggestion was to guard them behind a command-line option using getOption, so presumably Debian could set this whilst upstream / other distros do not. Corret. As posted, the patch changes existing behaviour _unconditionally_ so I don't not think I can say with a straight face to upstream that they should take this. And also: you can turn them on for your builds, I won't at first, but advertise so that more users can test it. "Eventually" I may turn them on as well. The R "universe" is _at least_ the (currently around) 10600+ packages (yes, really, 10 thousand) on CRAN. The fact that we successfully rebuilt our 400+ is important aspect but nowhere near comprehensive enough. | However, since I'm not familiar with the full R codebase I was waiting to see if upstream had any further comments, because even with the getOption() change there might be other consequences that I didn't foresee. In this case it would be beneficial for Debian's behaviour to exactly match upstream, so we get the same bugs (if any). | | > Ximin, looking at https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds | > I believe it would be best to merge those two (top most) commits into one? | > | >> A decent longer-term strategy may well to stress-test the patch by including | >> it in our builds for a while. We can work on that. | > | > actually we've been using Ximin's patches on tests.reproducible-builds.org | > since the 2nd of May on amd64 and since the 3rd on arm64, armhf and i386. | > | > According to https://tests.reproducible-builds.org/debian/index_performance.html | > (top row labeled "oldest build result) in the first table on the right) this | > means we've almost done a full rebuild with the patch on amd64+arm64 (probably | > 85% done now) and pretty close to that on i386… | > | > According to this, this patch seems to work well… | > | | The patch works well for getting stuff built, but I haven't tested all of | the *behaviour* of these packages. (Some probably have unit tests, but these | don't cover everything at any rate.) Exactly. | That is why I wanted to wait for upstreams' comments first, before adding the getOption guards - which are relatively minor IMO, compared to what the full patch does. Correct if I am wrong but you have not heard back from upstream, have you? [ That is not uncommon for posts on r-devel. Sometimes one needs to be persistent, and/or ally with an R Core maintainer. Which is pretty much what I suggested early one. ] | I also haven't tested other potential tools that might process R rdb files, who might for some reason expect absolute build paths. Correct as well. We are moving in the right direction, but we should not rush this. Upstream *is* sympathetic, they have taken an earlier 'repro build' patch. Dirk | X | | -- | GPG: ed25519/56034877E1F87C35 | GPG: rsa4096/1318EFAC5FBBDBCE | https://github.com/infinity0/pubkeys.git
It took a jolly few years, and some assistance from upstream, friends, and even LLMs but we got to. r-base 4.6.1-9, the current build of the the current upstream, builds reproducibly. I will over the next few weeks unsure this gets into upstream in a way that is not-Debian dependent so by R 4.7.0 next April it will (at long last) build reproducibly everywhere. Dirk