Hi,

Your package is the only known consumer of Lintian's internal Perl
modules. We would like to stop shipping our modules in the Perl system
path. Here is another way to get the information you require.

Your package loads a default Lintian profile [1] and uses the
Lintian::Data mechanism to get the policy release dates from
data/standard-version/release-dates [2]. It may be tempting to use a
simpler parser, but please consider that the information actually
originates somewhere else.

[1] https://salsa.debian.org/perl-team/modules/packages/libconfig-model-dpkg-perl/-/blob/master/lib/Config/Model/Dpkg/Control/Source/StandardVersion.pm#L19-23
[2] https://salsa.debian.org/lintian/lintian/-/blob/master/data/standards-version/release-dates

Attached is a script that extracts the release dates from the policy
changelog. It uses the network. The data will always be current.

More information may be available in Bug#903220, which caused you to
use Lintian in the first place.

For some time, Lintian has had issues with outdated modules loading
from the system path. Starting with the next release, Lintian will in
addition ship its Perl modules in a private location.

Please let us know when we can stop shipping our modules in the Perl
system path. Thank you!

Kind regards
Felix Lechner

ok.

Actually, only the latest policy number is used by cme (currently 4.5.0)

I don't really understand your point there, but it may not matter much (see
below)

If the source of information is Salsa, I'd rather use gitlab API to extract
the latest policy version from the tags with something like:

mojo get -r https://salsa.debian.org/api/v4/projects/234/repository/tags \
  /0/name | perl -n -E '/(\d+\.\d+\.\d+)/; say $1;'

Since the data returned by gitlab is JSON, parsing is not much of an issue.

Does that sound reasonable ?

All the best

Hi Dominique,

First of all, thanks for your swift reply!

I have no preference how the package gets the information. I suggested
a solution only in order to appear helpful. Thanks for addressing it
so promptly.

Unless you advise otherwise, Lintian will stop shipping the modules in
the system path in the next release.

As a side note, it seems the changelog date could differ from the tag
date, but that is a question for the policy editors.

Kind regards
Felix Lechner

The problem I see here is that dynamically fetching the policy
version can't work during package builds / for tests; having the
version in a file was the big advantage of using the lintian data.

But yeah, at runtime it should be ok.

Cheers,
gregor

Hi gregor,

I figured. How about depending on debian-policy and using the
information in its changelog?

Kind regards
Felix Lechner

Actually, I could use rmadison to get the latest version of debian-policy
without installing this package (like rmadison is used to get the version of
packages in unstable and older releases)

Please wait until the next release of libconfig-model-dpkg-perl.

All the best

Hi Dominique,

No rush, please. We will keep shipping the modules until you close
this bug. Thank you!

Kind regards
Felix Lechner

But if we skip the test phase that would be ok.

What I'm wondering is:
- depending on debian-policy would be another dependency, and the debian-policy
  only installs files to /usr/share/doc which is not guaranteed to
  exist; and there's also no easily parsable file there;
- will lintian continue to ship
  /usr/share/lintian/data/standards-version/release-dates ? Then we
  could still parse it ourselves without the Lintian perl modules;
- even if lintian ships the perl modules in a private path, it would
  be possible to use it; if this makes sense depends on the reason
  why lintian moves the files, which I haven't fully understood yet
  (my impression is more that this has to do with lintian internales
  like tests that with the question if the modules should be uses by
  others).

But yeah, maybe just using rmadison or whatever other online method
to get the lates S-V is easier than to rely on lintian.


Cheers,
gregor

gregor herrmann <gregoa@debian.org> writes:

If it would be helpful for debian-policy to ship this information directly
in the package, that should be relatively easy to do.  Feel free to open a
bug against debian-policy.  I'm not sure if Sean would see any problems
that I'm not seeing, but I personally have no objection to Policy starting
to install some machine-readable information about Policy in files in
/usr/share/debian-policy in a documented format.

If we did that, I would probably ship this information in YAML rather than
in the somewhat ad hoc format of Lintian's data file (which was my fault
originally).

I think having the data provided by debian-policy (in the package,
and maybe-for other purposes-also on the website) would be an
excellent idea, thanks for the offer.

I'm waiting for Dominique's thoughts before cloning/etc. this bug.


Cheers,
gregor

Agreed. It's better than hitting madison regularly to get policy version.

Please go on.

All the best

Control: clone -1 -2
Control: reassign -2 debian-policy
Control: retitle -2 debian-policy: please provide file with policy release versions and dates
Control: severity -2 wishlist
Control: block -1 with -2
Control: affects -2 + libconfig-model-dpkg-perl

Great.

Thanks for the feedback, doing the BTS control dance now.


Cheers,
gregor

Hi,

Please have a look at this commit. It explains how to mitigate
expected upcoming breakage in your package.

https://salsa.debian.org/lintian/lintian/-/commit/d2f692f564ac725a0eb58a7d62abe57f66cdc753

Please load the policy data with:

    my $data = $profile->load_data(...);

instead of

    my $data = Lintian::Data->new(...);

That is a temporary fix; the interface is scheduled to change again in
the near future. Thank you!

Kind regards
Felix Lechner

Probably because the change in lintian isn't uploaded yet.
This means we can't make this change (as in: uploading a change)
right now.

If you want us to make this change please provide a tested and
working MR, including required changes in d/control (aka minimum
required lintian version, which actually is in the archive).

Ehm, sorry, this sounds quite unattractive. My personal motivation to
play catchup with lintian changes is, hm, limited. Could we maybe
wait for a fix for #968154 in debian-policy?

Cheers,
gregor

Hi,

I'm not sure what the policy team's intentions are, but Lintian's Data
module no longer provides the policy information. [1] We now keep the
data in a JSON file that is more reliably exchanged and updated. [2]
As a courtesy to this package (and to the policy team) Lintian now
ships an executable "/usr/share/lintian/private/latest-policy-version"
that provides the latest policy information. [3] Please use that
executable to obtain the information you require.

As a side note, Lintian no longer ships its private modules in the
public Perl path. [4] Please do not use our modules directly anymore.
We would be happy to give you access to additional data via
executables, if needed.

Marking this bug "important" because Debian CI is failing (but
separately, since I copied the policy bug). Thanks!

Kind regards
Felix Lechner

[1] https://salsa.debian.org/lintian/lintian/-/commit/d2f692f564ac725a0eb58a7d62abe57f66cdc753
[2] https://salsa.debian.org/lintian/lintian/-/commit/21fd3c7d1ae24bf3bb00ffbab6a0dd99acd3503c
[3] https://salsa.debian.org/lintian/lintian/-/commit/d43799ae30f55e8a211281295d4508b11d022147
[4] https://bugs.debian.org/968011

Implemented in git, but not uploaded because

- I'd appreciate review from dod
- and I discovered a wrinkle:

The old /usr/share/lintian/data/standards-version/release-dates had

| 4.5.1  1605571543
| 4.5.0  1579549029
| 4.4.1  1569780709
| …

i.e. only the 3-digit "main" policy releases which are relevant for
Standards-Version.

The new /usr/share/lintian/data/debian-policy/releases.json has
sections for all releases, including minor ones, in 4-digit notation:

|       {
|          "author" : "Sean Whitton <spwhitton@spwhitton.name>",
|          "changes" : [
|             "",
|             "debian-policy (4.6.0.1) unstable; urgency=medium",
|             "",
|             "  * Fix header of upgrading checklist entry for last release (Closes: #992414).",
|             "    Thanks to Scott Talbert and Drew Parsons for reporting the problem."
|          ],
|          "closes" : [
|             992414.0
|          ],
|          "epoch" : 1629318110,
|          "timestamp" : "2021-08-18T20:21:50Z",
|          "version" : "4.6.0.1"
|       },

and /usr/share/lintian/private/latest-policy-version also outputs
4.6.0.1.


This leads to

| Config::Model::Value::show_warnings Warning in 'source Standards-Version': Current standards version is '4.6.0.1'. Please read https://www.debian.org/doc/debian-policy/upgrading-checklist.html for the changes that may be needed on your package to upgrade it from standard version '4.6.0' to '4.6.0.1'.
|
| Offending value: '4.6.0' (line 1263)
|
| Changes applied to dpkg-control configuration:
| - source Standards-Version: '4.6.0' -> '4.6.0.1' # applied fix for :Current standards version is '4.6.0.1'. Please read https://www.debian.org/doc/debian-policy/upgrading-checklist.html for the changes that may be needed on your package to upgrade it from standard version '4.6.0' to '4.6.0.1'.

which is not we want …


Now of course we can strip off the fourth level ourselves but I'm wondering
- if /usr/share/lintian/data/debian-policy/releases.json needs to be
  more precise than /usr/share/lintian/data/standards-version/release-dates
  (I guess having more information there is a feature)
  or
- if /usr/share/lintian/private/latest-policy-version should do the
  stripping (i.e. output "4.6.0" currently) as this is the actual
  relevant Standards-Version.


Alright, for now I've added the stripping as that's probably faster
than waiting for a new lintian release …


Cheers,
gregor

Hi,

Thanks for looking at it so promptly, and sorry I did not also submit
an MR like you had asked.

If Lintian becomes a data provider (which the policy team seemed to
prefer) it's probably better—as a general matter—for consuming
packages to apply the changes they require.

You are welcome to borrow the code from here:

https://salsa.debian.org/lintian/lintian/-/blob/master/lib/Lintian/Check/Fields/StandardsVersion.pm#L76-81

Kind regards
Felix Lechner

Hi

Re-reading the thread of this bug, there's still the solution based on debian-policy. Even though there's still no file that provide the policy version, this one can be extracted from debian-policy changelog with something like:

$ zcat /usr/share/doc/debian-policy/changelog.gz | head -1 | perl -n -E '/\((\d+\.\d+\.\d+)/; say $1;'
4.6.0

On the other hand, people might not like yet another dependency on cme. And it may be hard to explain a dependency on a pure doc package.

And this will break in debian-slim container images because docs are not installed... oh well... :-/

Back to gregoa's proposal...
$std_ver =~ s|^(\d+\.\d+\.\d+)\.\d+$|$1|;

I don't know if policy people are committed to used 4 digits version.

It may be better to extract the first 3 digits and call it a day.
(i.e. $std_ver =~ s|^(\d+\.\d+\.\d+)|$1|; )

In any case, the program should croak if $std_ver is undef because the regexp does not match

Which you fixed...

All the best

:)

Good catch!

Ack.

Right.


Thanks for checking; changes pushed.


Feel free to upload if you are happy and have the time, or I can do
it tomorrow.


Cheers,
gregor

Hi,

chiming in as the new Lintian maintainer. :-)

Felix Lechner wrote:

I haven't read all the discussion in this bug report, but I stumbled
upon it due to these lines in debian/lintian.install:

  # the next line will be removed when libconfig-model-dpkg-perl stops using Lintian data (Bug#968000)
  private/latest-policy-version           usr/share/lintian/private

private/latest-policy-version also seems to be the current way, how
libconfig-model-dpkg-perl interacts with this data in Lintian.

Actually the fact that this script is used in
libconfig-model-dpkg-perl's autopkgtest caused a regression in (or by
and for) lintian 2.115.0 because Felix moved one method used in this
script from Lintian::Profile to Lintian::Data and forgot to update
this script (which Lintian itself doesn't seem to use).

So I'll soon upload lintian 2.115.1 to fix this issue to finally get
the current Lintian version into testing.

On the other it seems as if no lintian-internal check caught this
issue, so I'm kinda happy that this has happend. But it also shows
what happens if internal modules or so are used. ;-)

For a potential long term solution:

On #debian-qa, pabs (Cc'ed) was suggesting to the Lintian and DUCK
maintainers (Cc'ed as well) to use a common data package for data
currently synced occassionally between both source packages. I
suggested to build that data out of src:lintian and the duck
maintainer was happy with that.

Hence I suggest that libconfig-model-dpkg-perl, cme, etc. are then
using this planned data packages in the future as well.

I currently the following:

* Create a new binary package lintian-data, which is built from
  src:lintian. It will more or less contain /usr/share/lintian/data/.

* Additionally I will create a file named
  /usr/share/lintian/data/debian-policy/latest.txt (or similar) at
  lintian build time based on the output of
  private/latest-policy-version. (Can even do that already before
  splitting off that data package.)

That way you have the wanted data in an easily consumable form without
having to call any script or module from Lintian and without having to
parse a huge bunch of JSON.

And we can easily ship that file in a pure data package, not requiring
you to pull in all dependencies of Lintian, either.

Would that work out for you?

If so: What are your requirements for such a transition? Do we have to
just care about Unstable and Testing or are Stable-Backports a topic,
too?

		Regards, Axel

Thanks, both for caring about lintian and about
libconfig-model-dpkg-perl :)

Yup.

Great, thanks.

Sounds like a plan.

Sounds good.

With the only caveat, that this will still lag (at least by some
hours :)) behind debian-policy releases.
But as the cloned policy bug #968154 didn't see any activity lately …

Since nice.

Yup.
I'm cc'ing dod as the Config::Model::* tsar to make sure I'm not
missing anything.

I believe we never backported libconfig-model-dpkg-perl. Dominique?

So I guess we can just depend on a new enough lintian or lintian-data
with /usr/share/lintian/data/debian-policy/latest.txt (or whatever)
and use it, and from the lintian side you can then add a versioned
Breaks on old libconfig-model-dpkg-perl versions still using
private/latest-policy-version, when you remove it.


Cheers,
gregor

All good. Thanks for the follow-up

Not that I know of.

Sounds good to me.

All the best