Hi, Once build-arch and build-indep are supported by dpkg-buildpackage, hopefully in the next week, and/or are required by Policy, please could you apply the attached patch to move build-arch and build-indep from recommended to required? I kept the debian-rules-missing-recommended-target check and description in case it's of potential use in the future, but otherwise these could also be removed. Thanks, Roger
Hi, dpkg/experimental now supports build-arch/build-indep with the "make -qn" fallback[1]. The tech-ctte's multi-arch ruling[2] suggests we may see this change in sid in 14 days time (unless that change is reverted etc.). That being said, I am not sure this is sufficient to bump these targets to "required". I am not aware of anything on the Policy front or the tech-ctte (build-arch) front to ratify the recommended -> required change. I am hesistant because bumping them has a side-effect of making them "fatal auto-rejects". Despite the steady drop in missing targets[3] there are still 4000-4500 packages that "overnight" would be auto-reject candidates. ~Niels [1] http://packages.qa.debian.org/d/dpkg/news/20120206T000222Z.html [2] http://lists.debian.org/debian-ctte/2012/02/msg00018.html [3] http://people.debian.org/~nthykier/rg-build-arch-target/affected-packages-graphs.png Numbers are available on lintian.debian.org in /srv/lintian.debian.org/history/tags/debian-rules-missing-recommended-target.dat Note, both sources are slightly inflated due to lintian currently processing "outdated" source packages. Some cut | uniq | wc -l magic on the lintian.log should produce more accurate results, but it is unlikely to drop the numbers below 4000 at the moment.
Things have changed a bit since we talked about this last year. In sid and Wheezy by now. Ratified in Policy 3.9.4, but as mentioned in [1] it is "Not for Wheezy". This number is now about 3700, which is still a bit much. In the interest of not getting a lot of mail from people aggrevated by their package being auto-rejected, I still feel the tags should remain split for now (until that number drops a bit more and Wheezy has been released). I am open to bumping the severity of the recommended-target tag (possibly including a rename) to make the tag more visible and hopefully increasing the adoption rate of this tag (well, the post-freeze adoption rate). [1] https://lists.debian.org/debian-devel-announce/2012/09/msg00006.html
Ccing the dpkg maintainers, since the lintian checks will be coupled to changes in the tools, and it's really down to them when this happens. When we discussed introduction of the targets, my understanding what that we agreed to use auto-detection of the build-arch and build-indep targets on a strictly temporary basis to allow a migration to having them be optional but recommended for wheezy, and that for jessie we would be removing the autodetection logic (or at least having it not be the default) and making them a hard requirement. I still see this as being the goal we want to achieve, and given that the autodetection is a horrible hack, I think removal for jessie is still needed, and our tools can then start to rely on these targets being present. From the POV of lintian, I think that we would want to encourage adoption as quickly as possible post-wheezy in order that dpkg-buildpackage can remove the autodetection by default. Would it be possible to make this a "required" check, perhaps with an exception for being auto-rejected to begin with? If we can agree on a timeframe for migration, perhaps with a hard deadline when the change will be made, this would spur more complete adoption before packages start being rejected and maintainers are forced to use the targets. Of course, we don't yet know when wheezy will release, but it would be good to have some sort of tentative plan in place for doing this. We wouldn't need to turn off autodetection until later into the jessie development cycle, but probably would be best to have done well before the freeze to allow the remaining packages to be fixed, so that would give maintainers a good year(?) to fix their stuff on top of the 1.5+ years they will have had since autodetection was introduced. That's my thoughts anyway, any other suggestions welcome! Kind regards, Roger
Hi! Well, I'm really not comfortable deciding unilaterally on a flag day when thousands of packages will start FTBFS, for something that will affect so many people. I think this should be discussed and agreed with the project at large. In any case my opinion on this is that yes, getting rid of the autodetection hack before jessie is out, would be ideal, but if that cannot happen, then oh well, this has taken a looong time, having to wait a bit longer should not be the end of the world. I think the less painful way to achieve that would be by a staged increase of the enforcing level of those targets, where changing dpkg to require them should be the last stage when really few packages still do not provide it, because otherwise mass rebuilds, binNMUs and similar become very painful. The first stage could be to wait a bit after testing thaws to see the progress; after a bit, change/rename the tag to an error w/o autoreject. Wait and see how it progresses, and after a bit more (several months) change it to autoreject, but not for binNMUs if that's possible? to avoid disrupting the release process. And then only a small tail should remain which could be handled by a MBF etc. After or during this last stage dpkg could be switched. Thanks, Guillem
That can be done. It is my understanding that Lintian is only run on source uploads (or human uploads). So any upload from buildds should be ignored, but I suggest we get that confirmed with the ftp-masters if we go with that. We can (also?) ask the ftp-masters to do a "non-fatal" autoreject for that tag at that point (that would allow people NMUing packages to ignore it by overriding it). Not sure if that has any merrit though. ~Niels
Actually after some sleep I realized that this should not be an issue as even if lintian is run on binNMUs then the source checks should not be performed as no source will be included. Right, that crossed my mind too, but given the amount of work involved (most of the time) in fixing this, if a human is involved and has to edit a file to make it uploadable, the human can as well fix the root cause. :) Thanks, Guillem
Absolutely, this definitely needs wider discussion. I really just intended this to be preliminary to that to see if getting this done for jessie was a mutually desirable goal. Which seems to be the case. But how we go about achieving that goal is definitely in need of wider discussion. I think this all makes a good deal of sense. It's certainly logistically impractical to "force" the issue by changing dpkg until the vast majority of packages are converted, so we certainly need to encourage adoption by other means and do this as the final step. As for when build-arch and build-indep were introduced, I'll be happy to do a set of whole-archive rebuilds to obtain concrete numbers once wheezy is released, and onward from that as needed, if I can get access to the hardware to do this. Regards, Roger
Hi, Restarting this as Wheezy has been out for a while. At March 1st, we had 3630 packages missing at least one recommended target according to Lintian. Yesterday, the number was 3122. Both numbers include source packages in sid and in experimental, so it may be slightly inflated[1]. The change translates to about 85 packages a month are being "fixed"[2], but our graph suggests that somewhere between May and July the rate increased[3]. Indeed, for the last month, a total of 96 (~3 a day) were "fixed". If the current rate is sustained, we are looking at ~3 years for this problem to fix itself. Even if we assume 10% of these to only affect experimental (see [1]) and all fixes affect sid, we are still look at ~2.5 years. So, the question is now - do we want to scale up the enforcement level, and, if so, to what? As mentioned earlier, I am willing to increase the severity of the tag (provided it does not become an auto-reject overnight). I would be interested in seeing the results from this kind of build-testing. ~Niels [1] That is, if a package is available in sid and experimental, it can count for up to two packages missing a recommended target. From memory, during the freeze about 10% of all packages in sid were also available in experimental. [2] I write "fixed" because removing the package from experimental can cause this number to drop as well despite the sid version still being affected. [3] http://lintian.debian.org/tags/debian-rules-missing-recommended-target.html
Hi! Well, if we take into account the dynamics of normal transitions the remaining long tail usually takes a very long time to get done w/o active incentives. I've been passively tracking the Source-Version substvar migration since around 2007, out of curiosity on how this kind of migrations go w/o any active external intervention (just as an observer), which I'll try to post on -devel at some point, but in any case the first year around 900~ packages got fixed, next year 190~, then 110~, 30~, 15~, and a year up to now 3. I doubt the remaning 43 packages will get fixed soon (as in 1-2 years) if no MBF or possibly NMUs are performed. I think if we'd want to get this done relatively soon, then it needs “active herding”. Increasing the lintian tag to a non auto-reject error, mails to debian-devel (or d-d-a) and possibly blog posts encouraging people to switch packages, someone to possibly handle it as a release goal to give it visibility, etc. After a while, and depending on the amount still remaning, probably switching to more aggressive methods, like I described above would help with the remaining straddlers. Thanks, Guillem
Indeed. My estimate was based on the rate being sustained, which gets increasingly more and more unlikely as there are fewer packages left. For the record, we are now 2½ after the last mail (from me) and "only" down to 1694 packages remaining. In the past year, it has dropped by about ~500 without any nagging. No one ever confirmed that "E" tags are more likely to be fixed than "W" tags. That said, nagging people on d-d(-a) or via blog posts might help. Alternatively, if the FTP masters agree we could make it an auto-reject tag. Indeed. Thanks, ~Niels
Enviado desde mi iPhone
Hi, With many people now using dh over the older explicit targets [1], is this patch still needed? Kind regards, Felix Lechner [1] According to trends.debian.net, only about 6% of packages use debhelper currently.
Hi! We discussed this over IRC some weeks ago. I checked the current numbers derived from the debian-rules-missing-recommended-target tag and how these get mapped to the different problem types, and here's the number of sources of what I came up with (which I think is correct but didn't record a reproducer :/): 0 building arch-indep + arch-dep binaries (the non-trivial case) 340 building only arch-dep binaries 249 building only arch-indep binaries So we are actually in a way better position than we were some years ago. As I mentioned there, I don't think those numbers are at a point where dpkg-source can stop using its fallback code to avoid a FTBFS. But I guess lintian could make it a non-fatal error already, because these have been policy violation for a while. Felix then came up with the great idea of involving Jelmer Vernooij with Debian Janitor, who was happy to take this on during the coming weeks, and implemented already initial support for the automatic fix. Thanks, Guillem
...
That request is #657390 (in cc).
When this was discussed some years ago, Niels Thykier pointed out that
debian-rules-missing-required-target is on the ftp team's list of Lintian
tags that cause automatic rejection[1], so making that change in Lintian
would make it impossible to do a sourceful upload of the affected packages
(for example to fix some unrelated RC issue) without also adding the
required targets.
This does not necessarily mean the Lintian change is a bad idea, it's just
something we should be aware of - expanding the scope of autorejections
should be intentional rather than accidental.
smcv
[1] https://ftp-master.debian.org/static/lintian.tags
Ack, in that case let's wait until this mbf is done and some time is given for the packages to get fixed. After that, I think this should become an error and autorejection. Cheers, Emilio
Hello, Bug #657390 in lintian reported by you has been fixed in the Git repository and is awaiting an upload. You can see the commit message below and you can check the diff of the fix at: https://salsa.debian.org/lintian/lintian/-/commit/23f836f91c03b78df76743fc002a105403a5bc14 At the time of writing, 264 sources were still affected, down from 421 in November. That worked out more or less to the monthly decline of 85 sources originally predicted by N. Thykier in 2013. Lintian's tag description was amended with the simplest possible fix, which has two-lines. They are a reasonable addition to sources that do not receive a lot of attention. For more information, please check the thread on debian-devel in the tag references. ------------------------------------------------------------------------ (this message was generated automatically) -- Greetings https://bugs.debian.org/657390
We believe that the bug you reported is fixed in the latest version of
lintian, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 657390@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Axel Beckert <abe@debian.org> (supplier of updated lintian package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Mon, 20 Jun 2022 13:23:02 +0200
Source: lintian
Architecture: source
Version: 2.115.0
Distribution: unstable
Urgency: medium
Maintainer: Debian Lintian Maintainers <lintian-maint@debian.org>
Changed-By: Axel Beckert <abe@debian.org>
Closes: 657390 932634 941656 963099 989381 995286 996740 999768 999810 1000234 1000977 1001655 1002828 1003131 1003272 1003353 1003456 1003668 1003817 1003913 1003941 1004231 1004239 1004240 1004660 1005046 1005184 1005762 1006390 1006859 1007140 1007257 1012090
Changes:
lintian (2.115.0) unstable; urgency=medium
.
The Lintian Resurrection Release.
.
* Summary of tag changes:
+ Added:
- alien-tag
- chown-with-dot
- conflicting-test-fields
- declare-python-versions-for-test
- drop-python-version-declaration
- invalid-override-restriction
- missing-prerequisite-for-pyproject-backend
- old-devhelp-standard
- stray-devhelp-documentation
- test-leaves-python-version-untested
- uses-poetry-cli
+ Removed:
- crossing-screens
- debhelper-compatibility-level-not-a-number
- debian-tests-control-and-control-autodep8
- exclusive-runtime-tests-field
- package-contains-devhelp-file-without-symlink
.
[ Axel Beckert ]
* Adopting Lintian. (Changes #1012289 from ITA to pure RFH.)
+ Remove Chris Lamb from Uploaders (see #1012289) and re-add myself.
* Workarounds until
https://github.com/Perl-Critic/Perl-Critic/issues/925 is fixed:
+ Replace all occurrences of "Copyright ©" with "Copyright (C)" again.
+ Remove unnecessary usage of UTF-8 from bin/lintian.
+ Replace UTF-8 characters in mostly Copyright comments.
+ Replace UTF-8 characters in code with \N{…}.
* Remove literal unicode character U+0334 COMBINING TILDE OVERLAY which
likely had been added accidentally. (Triggered by the symptoms of
https://github.com/Perl-Critic/Perl-Critic/issues/925, but permanent.)
* Update copyright years in debian/copyright.
* Run perltidy over lib, bin/lintian, private/refresh-perl-provides,
private/runtests and several files in t/scripts/.
* data/…/perl-provides updated by running "debian/rules
refresh-perl-provides".
* Add Felix Lechner to debian/copyright based on copyright statements
elsewhere. Thanks for all your contributions!
* Update t/recipes/README: "debian/rules runtests" → "private/runtests"
* Follow module renaming: Perl::Critic::Freenode → Perl::…::Community.
* t/s…/h…/tag-coverage.t: Replace "$ENV{'LINTIAN_BASE'}" with
"$ENV{'LINTIAN_BASE'} // '.'" to be able to run it with "prove -l".
* init.d-general check: Avoid relying on line numbers in #DEBHELPER#
replacement code.
* very-long-line-length-in-source-file: Ignore files listed in new data
file binary-file-extensions. (Closes: #1005046)
* Fix false positives for adopted-extended-field with X- prefixed
fields. (Closes: #999768)
+ Empty hints files seem to require a Test-Against field in desc.
* Update own source lintian-overrides for "pointed hints".
+ Make them work with old and new lintian versions by using wildcards.
* Rename README.developers to have a proper file suffix (.pod).
* Switch syntax marker of README.developers.pod from "perl" to "pod".
* Documentation update: Replace directory "frontend/" with "bin/".
* Fix a bunch of "Use of uninitialized value $_ in concatenation"
warnings when running tests with "prove -l" directly.
* README.developers.pod: Explain the difference between check and test.
* lintian(1): Drop mentioning of never existing --no-overrides option.
* Replace unfitting Text::Glob with more flexible Regexp::Wildcards
(Closes: #1003353)
+ Add unit test for Lintian::Util::match_glob. The current testsuite
does not seem to be able to cover such a case.
* Declare compliance with Debian Policy 4.6.1. (No changes needed.)
* Refresh data using private/refresh-data. Skip unreleased policy though
for now.
* Fix "Use of uninitialized value $step in concatenation" in
Lintian::Version which showed up as unrecognized tag (!) when running
the test suite on the git repo already tagged for a release.
* debian/gbp.conf: Declare so far used tag format so that gbp uses it.
* Add lintian override for very-long-line-length-in-source-file in
Lintian::Check::Cruft as well as test-leaves-python-version-untested.
* Use versioned Breaks instead of Conflicts against lzd, see #1001655.
Thanks Lintian for reporting ;-) and Paul Gevers for the sanity check!
.
[ Felix Lechner ]
* Refresh manual references.
* Use Text::Glob to match hint contexts with override patterns. Replaces
a trusted homegrown routine. (Closes: #1003272)
* Refresh list of available Debhelper commands.
* Refresh list of installable fonts.
* Generate section references for Lintian manual from repo; point to
website.
* Accept globbing patterns in profiles when enabling and disabling
checks or tags.
* Refresh data sources in parallel.
* Add the New Maintainer's Guide to the list of quotable authorities.
* Eliminate unpredictable output in the check siles/privacy-breach.
* Honor the environment variable NO_COLOR as specified in
https://no-color.org/.
* More attempts to eliminate unpredictable output in the check
files/privacy-breach.
* Drop the tag debian-tests-control-and-control-autodep8.
* Set authority references apart from other data sources.
* Provide rudimentary Emacs integration. (See: #968758)
* Associate Emacs modules with the 'editors' archive section.
* Recognize /usr/bin/raku as a known interpreter for scripts. (Closes:
#1002828)
* Do not depend on any particular Lzip implementation. (Closes:
#1001655)
* Exempt installables designated as documentation from warning about new
Python2 packages. (Closes: #995286)
* Update citations in two tags. (Closes: #1003131)
* Drop version requirement from
skip-systemd-native-flag-missing-pre-depends. (See: #1003271)
* Import new CSS style sheet from the website.
* Recognize dh-sequence-sphinxdoc as a valid prerequisite for
dh_sphinxdoc. (Closes: #999810)
* Tolerate multiarch acceptors in prerequisites for Debhelper commands
and addons. (Closes: #1000234)
* Issue yet more pointed hints.
* Recognize pybuild-plugin-pyproject as a valid prerequisite for the
python3 Debhelper plugin. (Closes: #1003668)
* Exempt bullseye backports from changelog-file-missing-explicit-entry.
(Closes: #941656)
* Mask long source lines in autotools-generated files. (Closes: #996740)
* Turn embedded-library into a classification tag. (Closes: #932634)
* Require the targets build-arch and build-indep in debian/rules.
(Closes: #657390)
* Do not insist on a particular name for unversioned links to a shared
library. (Closes: #963099)
* Exempt the names of Debian folks associated with a package from
spelling checks. (Closes: #989381)
* Require py3version invocation consistent with presence of
X-Python3-Version in d/control. (See: #1001677)
* Exempt CGI scripts from executable-in-usr-lib. (Closes: #1003941)
* CGI scripts can be ELF executables. (See: #1003941)
* Exempt Python's .dist-info and .egg-info folders everywhere from
documentation-outside-usr-share. (Closes: #1003913)
* Flag an outdated Debian copyright just once; use the most recent
year. (Closes: #1003817)
* Implement '--no-show-overrides'; honor it for overrides and masks
alike. (See: #1004240)
* Allow the command-line option '--no-info' to reverse 'info=yes' in the
configuration file. (Closes: #1004240)
* Elide manual references to ancient Lintian versions; use modern
examples. (Closes: #1004231)
* Deprecate --no-tag-display-limit for '--tag-display-limit 0'; update
documentation. (Closes: #1004239)
* Also provide a default output width for
lintian-annotate-hints. (Closes: #1004660)
* Mask examples in tests from
package-does-not-install-examples. (Closes: #1005184)
* Recognize Java 18 in unstable, and Java 19 as otherwise
available. (Closes: #1005762)
* Leave default Java bytecode version at 56. (See: #1005762)
* Adjust documentation reference to manual page for dh_make. (Closes:
#1006390)
* Warn about devhelp index files that use version 1. (Closes: #1006859)
* Store ELF information from readelf in an MLDBM database. (Closes:
#1003456)
* Issue pedantic hint for dot in 'chown user.group' instead of a
colon. (Closes: #1007140)
* Upgrade missing-systemd-timer-for-cron-script to warning; no longer
experimental. (Closes: #1007257)
.
[ Ryan Finnie ]
* Provide a constant citation for
systemd-service-file-uses-nobody-or-nogroup. (Closes: !385)
.
[ Louis-Philippe Véronneau ]
* Check that tests pulling in all Python versions also query which ones
are available. (Closes: !361)
* Add new Python tags for pyproject.toml build backends according to
PEP-517. (Closes: !384)
* Rename 'python3-flit' to 'flit', as there is no 'python3-flit'
package. (Closes: !386)
.
[ Daniel Kahn Gillmor ]
* Correct lintian-annotate-hints manpage.
.
[ Simon McVittie ]
* Silence a very widespread false positive for detached debug symbols.
(Closes: #1000977, !387)
.
[ Simon Quigley ]
* Add "kinetic" as a known Ubuntu distribution. (Closes: !392)
.
[ xiao sheng wen(肖盛文) ]
* Add riscv64 support (Closes: #1012090, !394)
.
[ Damyan Ivanov ]
* Update releases.json data for Debian policy releases (4.6.1 added;
closes: !393)
.
[ Paul Wise ]
* Add more obsolete domains for former source code hosting services.
Checksums-Sha1:
863a51cffc5b8359ef133d6e6f150d9cac148eb3 2503 lintian_2.115.0.dsc
e20194c63f481a7361969fab6ea7739f3f6e03d5 2170172 lintian_2.115.0.tar.xz
3453b48f59dd2f58e2e78523e9018c58a42faebe 7274 lintian_2.115.0_source.buildinfo
Checksums-Sha256:
431a025b52e185cac6cbf2fedca8b6c60e2f7dfa55d89d86fcd543a34e8232b9 2503 lintian_2.115.0.dsc
f353d372d036daa6ad0341b418728cfe73c11688708b6c3a33d50acb445d2b53 2170172 lintian_2.115.0.tar.xz
e1c1bd41b7f5a8014231f9db422c1711e97cf50146eab83a1d113370735f02d9 7274 lintian_2.115.0_source.buildinfo
Files:
4484e31abe39a15def23b119f92ef3ab 2503 devel optional lintian_2.115.0.dsc
9d7922b69d30a63693825120e049ea00 2170172 devel optional lintian_2.115.0.tar.xz
12f2494d410df4c2bf03f82075ce5f60 7274 devel optional lintian_2.115.0_source.buildinfo
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEERoyJeTtCmBnp12Ema+Zjx1o1yXUFAmKwZiIACgkQa+Zjx1o1
yXUIng/5ARDjo8W4NggeM79xaeIgqtRbCao5XuBX6VyAE6hupaK8vHH/konnGmwg
jfRvPQK6YCXfqHggIKNQnToJgKUxdI+U3e10PYrbc5qt2UWFMiRfEHGqG3cfu+uA
JhqldKXfRkvmY4+9WPChWZ4C8qjdPZNcQwwdwm1Rg6gKMreYDGNIWJ94Hh7BkNsC
lNowSbDW6uE0XLBxVVeHhQGrn0Aki8fkx82W+Hod2A26sCXv/Gx7OSb8KVERa3+7
D05KDZd+ZGXBIdk/zRn/AeCxwcLXXMCS7qNDsfbJ8oUeYyy7y7QrQbGeFrBdOTyk
JbqqG0rTGbvfg3UcviOm/rvkr5QHdA5OMLUKRoljApMJEzbASGzYqNiUSmZNWH8x
v9EZ/qTHrcPFL/bIC9POvjD3FhSs3EkgQxXCuVPWZRruurgEme6DmnxHEj9TAe6L
GcUr+6DDHMw+j9SlO04VDDpUFVO2ftPfMcSckNCNM3eMxfig4XjPmyay3m6BjgaF
Rq6ckTUCUt5hsm7T48I0PAjXMRcAbemfI3PmVOUgwIf3BvzcdvJew0h90ULOY94N
cCWdsZ5SwC1hoz6EsBVmD16XxxYvZM7/wcf40PJkJ+YwtJu2Wm3dAv5WPK8UTrYr
qK1s62DfByho45eV9rXKeLnjvRN37rQYOWl7WwYy6YfpzQYDFWU=
=3dK3
-----END PGP SIGNATURE-----