#782486 UDD: adding yum based distro package information

#782486#5
Date:
2015-04-13 01:42:11 UTC
From:
To:
Hi,

In order to aid in cross-distribution collaboration, it would be useful
to have more information about packages from other distributions. A
number of yum based distributions exist that either are a derivative of
Fedora or use Fedora's EPEL (Extra Packages for Enterprise Linux) so I
would suggest this as the starting point.

I wrote some scripts that use the public Koji API although these are
incredibly slow. I was looking at using a Koki database dump, although
this isn't an advertised public interface and may be restricted or taken
away without much notice. Following some discussion in #debian-qa, it
looks like the best approach will be to use the yum repos themselves as
a data source.

It will take time to go through all the packages and extract the
information, but once the information is extracted it will be only
updated packages that will need to be reprocessed.

Fedora uses Red Hat's bugzilla instance for bug tracking. There is a CSV
interface to this that allows the export of bugs information.

This data will also be useful to #685605 in order to provide semantic
linking to other distribution's packages in the "Linked Data Cloud".

Previous efforts at matching packages across distributions have been
documented on the wiki at:

https://wiki.debian.org/Mapping%20package%20names%20across%20distributions

None of these appear to be presently maintained, although I may be
mistaken.

This is a task on my queue, but it's quite near the "future" end of my
queue. If you feel you'd like to have a stab at this, please go ahead.

I'll take ownership of this bug when I start if no one else has stepped
up before then.

Thanks,
Iain.

#782486#10
Date:
2015-04-19 12:02:23 UTC
From:
To:
Hi Iain,

My own feeling about this is that it is better to try and encourage each
distribution to render their data using some standard formats, such as
iCalendar and then there are multiple options available for using the data:

a) use productivity tools such as Mozilla Lightning or GNOME Evolution
to aggregate and render the data into a to-do list for a developer

b) build reporting tools similar to UDD that are independent of any
specific distribution, to scrape data from the distributions and from
other sources like the Github API, upstream bugzilla instances, etc

The benefit of this strategy is that it is more modular and people who
are not involved with Debian may be more likely to contribute to a
generic, high level solution.  Putting too much in UDD may make it
harder to maintain and keep in sync with the other distributions.

Some other comments on the topic:

- you can already use the iCalendar format to see a combined bug list
from Debian bug tracker, Github issue list and Fedora bugzilla:
http://danielpocock.com/debian-maintainer-dashboard-now-provides-icalendar-feeds

- I've had applications from two GSoC students willing to work on some
related concepts, having an additional mentor for this project would be
really helpful.  Details are here:

https://wiki.debian.org/SummerOfCode2015/StudentApplications#Developer_Horizon_:_Dashboard

- maybe upstreams can be encouraged to keep some metadata file in their
tarballs and repositories that helps identify the relevant package names
in each distribution?

- maybe package maintainers could be offered some new field in
debian/control that allows them to identify the corresponding Fedora
packages?

- extracting compiled library packages, it may be possible to identify
SONAMEs and use that data to cross-reference package names

- there are a couple of instances where I do think it is compelling for
Debian UDD to pull data from non-Debian sources, e.g. knowing when there
is a new upstream release, knowing about a security advisory and for
some packages it is useful to know if the next release of Debian is
carrying at least the same version of something that is in Fedora's next
release.

Regards,

Daniel

#782486#15
Date:
2015-04-19 15:35:48 UTC
From:
To:
Hi Daniel,

Thanks for the comments, replies inline.

Brilliant! Because this is the same feeling I have. I would like to be able
to use RDF and linking of RDF datasets between distributions. This allows
for complex queries on the datasets.

Exporting iCalendar from such an RDF dataset should be trivial.

Yep, again, this also becomes trivial with RDF being published or data being
scraped and converted into RDF.

Putting information in UDD is not a final solution, but it helps us get
towards the final solution. The final solution would be to have no
derivatives information in UDD at all, and simply reference the URIs of RDF
datasets elsewhere. Dumps of these datasets can be aggregated and queries
can be performed across them.

Was not aware of this, I'll take a look.

This looks rather exciting and I am interested! Events was something I had
in my mind for adding to rdf.debian.net.

This is entirely possible. CPAN already do have RDF in their packages, and I
don't believe it will be too hard to encourage them to add semantic links at
all.

I would probably be against this. This isn't going to scale. It's a tricky
problem though.

Yes, this is a brilliant idea!

I'd like the information about derivatives and other distributions to be
minimal, and then use RDF to link to the information provided by other
distributions. I would definitely like to avoid pulling in more data than is
necessary.

Thanks,
Iain.

#782486#22
Date:
2021-02-20 15:48:22 UTC
From:
To:
Hallo,

Ich habe dir eine Mail geschickt, aber keine Antwort von dir, warum?

Eddie