There seems to be an ongoing transition in Lintian, in which hints that
report a filename or line in an ad-hoc way are being converted to "pointed
hints" that use a new format with the filename in square brackets.
One of the design decisions that keeps Lintian's value high as a
QA tool is that all of its hints (formerly known as tags) can be
overridden if the maintainer has assessed the hint and determined that
it is a false-positive or otherwise inapplicable to this particular
package. Unfortunately, changing a hint to a pointed hint invalidates
most overrides for that hint (unless they use a very broad wildcard match,
which seems like a bad idea since it could hide unrelated instances of the
hint that genuinely indicate a problem). I can see why it is considered
valuable to have a consistent format for all hints, but mass-invalidating
existing overrides seems like a high price to pay for that.
This is particularly frustrating when the overrides that would be required
to silence a hint on lintian.debian.org and the overrides that would
be required to silence a hint with the released version of Lintian are
mutually incompatible. https://lintian.debian.org/tags/mismatched-override
currently reports 15K mismatched overrides among 3K source packages,
which seems like a lot.
It would be useful if Lintian could treat previously-valid overrides as
still being a valid way to override the new form of a tag, particularly
in the common case where "foo usr/share/bar" has been replaced by
"foo [usr/share/bar]".
smcv
Control: unblock 1006348 by -1
I agree with your reasoning for unmerging, and I don't understand why
they were merged either.
#1007002 was marked as blocking #1006348 "lintian: Tag
improbable-bug-number-in-closes condition considers 7-digit bug numbers
improbable", but I think that was a side-effect of the merge and I don't
consider #1007002 to be RC or a blocker for #1006348, so I'm removing
that metadata.
That makes sense to me. I should have reported #1007002 earlier (or,
ideally, a Lintian co-maintainer should have pointed out the problems with
a major tag format transition before it got that far), which would have
made pausing or reverting the transition lower-cost.
Getting a new Lintian release into testing and backports would at least
mitigate #1007002 by making sure that maintainers can write one tag format
that is equally valid for stable-backports Lintian, testing Lintian,
unstable Lintian and lintian.debian.org.
Thank you for picking up this package! It's an important QA tool and
I'm glad someone is looking after it again.
smcv
Hi Simon, Simon McVittie wrote: Thanks! Definitely. All three unmerged bug reports had that block. Actually it was still on my TODO list to find out which of the bug reports was related to #1006348 and why. You reduced that part of my TODO list to 2/3 by no more having to check this bug report. Thanks! :-) There are parts of that tag format transition already committed in Git which aren't yet in Debian Unstable. :-/ There were nearly 200 commits in git since 2.114.0 before I started working again on Lintian. I won't triage and revert them either unless they are outright buggy in a technical sense. Sorry. As mentioned it would need much more man power to "fix" this issue, even with what is so far only in git. And with Felix and Chris left the Lintian development, there's much less man power so far. At least so far I'm the only one who did commits since then, but there were at least three DDs asking for group membership on Salsa (Thanks!), so I hope they will start working on Lintian as well. I'm also updating the development how-tos where I notice that they're outdated. No offence meant. That's more a thing. Actually I already thought if we should setup a similar rule for Lintian tag syntaxes as we did for using epoch in package versions, maybe just not as stringent. I'm though not sure where such a rule should be placed. The Debian Policy seems the wrong place. So that rule should be probably documented inside Lintian itself or so. Ack. Should have noticed the resignation of Felix and Chris (or looked for such a thing after noticing that there were no lintian uploads for a while) earlier, too. Probably would have been less (or at least more distributed) work to get it back in shape as the remainder of Debian Unstable didn't stand still while Lintian did. Especially I found one case where the amount of lines replacing the #DEBHELPER# token caused test suite failures after a debhelper update seem to have added some lines. Took a few hours to understand what went wrong and why. (Granted, because many things I knew about Lintian's test suite had changed since I last used it, I first had to relearn how to use the current setup and this took probably some of that time as well.) Regards, Axel
Hi Andreas, Andreas Tille wrote: Correct, except that it happened for quite a while (7 months at least) and was (and maybe still is — see below) a continuous transition. It is present since at least 2.114.0 from November 2021. According to the git history, the implementation started shortly before the 2.114.0 upload, but the bug report which requested this is actually from 2014: https://bugs.debian.org/743226 And yes, 2.115.0 (but not 2.115.1 or 2.115.2 from today) had more such changes as there were over 200 commits from Felix included which he did after 2.114.0, but which weren't uploaded since then. Many of them were (IMHO irresponsibly) marked "Gbp-Dch: Ignore" and hence didn't show up in debian/changelog when I generated it with "gbp dch". (I though ignored that in some cases and added them manually to the debian/changelog entry, partially even retroactively.) The advantage is to clearly mark what is a file with potentially a line number in the output of lintian so that further processors like the lintian website can do deep links to the proper code position on e.g. sources.debian.org or salsa.debian.org. Felix called it "pointed hints". From my point of view, that's quite an advantage. See also https://bugs.debian.org/1007002 for the proper bug report about the issue with invalidating overrides by this transition. I've added it to the Cc of this thread and dropped lintian-maint@debian.org instead as mails to that bug report get forwarded to the Lintian Team anyway. Felix decided that it's worth to implement this requested feature and I at least partially agree as I do clearly see the advantage and also like the possibilities which this now offers. I clearly won't do that — for multiple reasons: * Far too much work for the gain (IMHO) * Losing a requested feature. * Invalidating 7 months of already updated overrides. * Better ways to invest your (and my) time. (See my proposition below.) Background: This seems not to have been one big commit but many small commits whenever a tag was touched by Felix. Reverting it to the old format would be a huge effort for which I don't have the time as there are way more pressing issues in lintian. And I'm very sure it can't be done by just doing a bunch of "git revert". So far I found 15 commits by Felix mentioning "pointed hint" reaching from November 2021 to March 2022. With about 200 other, partially quite invasive commits inbetween. And in addition to that, it's IMHO _far_ too late, because many maintainers already have updated their overrides in the past 7 months. And I'm currently not sure if I would even accept a Merge Request which tries to do that. But I doubt that anyone would a) make that effort and b) make all those maintainers angry who already updated their overrides in the past 7 months or more. Roberto C. Sánchez wrote: That was Felix' plan, yes. I just have currently no idea how many tags have been and how many haven't been converted yet. If there aren't too many tags left, I might finish this transition. If there are many tags left, I'd only do that if we have a way to cope with it with much less effort than it so far caused. And from my point of view, the only way to get out of this mess without causing too much work for anyone (maintainers of packages with affected overrides as well as the current Lintian maintainers): Someone should write a converter from the old to the new format. That doesn't sound too difficult. Main work would be gathering for each tag involved how it looked beforehand and how it looked now. This probably can be gathered from changes in git to Lintian's test suite. And maybe it should even try to output overrides which are compatible with the old and new format, at least in cases where the order of parameters didn't change. See lintian's own overrides for some examples: https://salsa.debian.org/lintian/lintian/-/blob/master/debian/source/lintian-overrides (Although this also accounts for a bug which only ever was present in Lintian's git repository and never got uploaded, but still got an RC bug report because people started using lintian off the git repo due to it no having been maintained for months: https://bugs.debian.org/1003353) Regards, Axel
Hello, Axel Beckert, le mer. 29 juin 2022 15:49:11 +0200, a ecrit: Yes, I fully agree. The format migration poses problem (I had to patch quite a few lintian files), but it solves a lot other current-and-future problems: we can now just ignore whole sets of warnings for a given file (typically one that is generated by some tool such as doxygen) Samuel
Hi Samuel and Axel,
Am Wed, Jun 29, 2022 at 04:10:49PM +0200 schrieb Samuel Thibault:
OK, thanks for making the advantages visible to me then. I'll
start editing lintian-overrides then.
Kind regards
Andreas.
Hi Axel, Just a note that since the last version of lintian to migrate to testing was 2.111 (which was also the last one to be backported to stable), some of us might not have updated since 2.111 and might be hit by changes that happened since then. (I'm not arguing whether it should be kept or reverted, but I'm just mentioning this as other disruptive changes might be discovered in the coming days) Lucas
Hi Lucas, Lucas Nussbaum wrote: I indeed did not think about that case because I do nearly all Debian development work on Unstable (and occassionally on Stable when a new package starts as a quick and dirty hack to scratch my own itches at work :-). And until my reply to Andreas I also wasn't aware of when Felix actually started with this. Just knew that it wasn't a one-shot thing, but quite distributed over at least the past half year (because that's the timeframe of git commits I most often looked into in the past few weeks). Andreas: Sorry if I was a bit too opposing because of assuming that every DD uses Unstable by default. (Actually I think the Dev Ref says you should, though. ;-) Or if DDs pinned Lintian to 2.111.0 from Testing… Andreas Tille wrote: Maybe wait a bit with that. Given Lucas' comment, I feel a bit more urged to provide such a migration script. I will look into this for the next upload. No promises as of now, though. Regards, Axel
Hello, Bug #1007002 in lintian reported by you has been fixed in the Git repository and is awaiting an upload. You can see the commit message below and you can check the diff of the fix at: https://salsa.debian.org/lintian/lintian/-/commit/2a5ca0ab3adef058fdf360684d7dbbfbc3c63e50 ------------------------------------------------------------------------ [WIP] Add script to migrate overrides to pointed hints format Closes: #1007002 Thanks: Simon McVittie, Andreas Tille, Lucas Nussbaum ------------------------------------------------------------------------ (this message was generated automatically) -- Greetings https://bugs.debian.org/1007002
Hi again, Axel Beckert wrote: A first prototype is now available in git in the branch "migrate-overrides": https://salsa.debian.org/lintian/lintian/-/blob/migrate-overrides/bin/lintian-migrate-overrides-to-pointed-hints So far the script only knows about the tags spelling-error-in-binary and package-contains-documentation-outside-usr-share-doc, but it is explicitly written to be expandable. Of course it will also get support for more tags in the (near) future. Additionally it currently only prints the transformed result to STDOUT. The plan is to also support inline editing, either with an -i option or maybe even by default if it detects that the package is maintained in git. Please give me feedback if this approach (especially after inline editing is supported) is feasible — preferably from those who are annoyed. :-) It's not yet in the master branch as neither Perl::Critic nor myself are happy about the usage of (the expression form of) "eval" in there. (Maybe one of the other JAPH has an idea on this. :-) Patches and other improvements suggestions as well as pattern sets for further tags are of course welcome as well. (Note: I will probably force-push that feature branch over and over again until I'm satisfied. If someone else wants to work on the same branch, too, we can also work without forced pushes and squash-merge the result at the end. Please contact me, if you're interested.) Regards, Axel
Hi Axel,
Am Wed, Jun 29, 2022 at 06:01:33PM +0200 schrieb Axel Beckert:
No need to be sorry, really not. I simply missed the point of that
change.
I'm using testing but pinned lintian on unstable - so at least in my
case it was not what Lucas pointed out. It was just that I beared the
change for some time ... and than my amount of patience spilled over.
;-P
But its correct that some users might use lintian from testing anyway.
Kind regards
Andreas.
It would probably be helpful to update the documentation at https://lintian.debian.org/manual/index.html#format-of-override-files[1] to describe the new pointed hints format.-------- [1] https://lintian.debian.org/manual/index.html#format-of-override-files
Hi Soren, Soren Stoutner wrote: Indeed. We though have the issue that lintian.debian.org no more updates. It's future is unclear, unfortunately. I do not have access there. :-/ And it's also not yet my focus currently. After the freeeze maybe. But I'll at least try to update doc/lintian.rst so that at least the content is ready once we can update the website again. This is also related to https://bugs.debian.org/1004234 (docs: give advice how to debug overrides). Regards, Axel
Thank you.
Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran, Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun. Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie für weitere Details.
Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran, Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun. Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie für weitere Details.
Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran, Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun. Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie für weitere Details.