Dear Maintainer,

apt_pkg.Dependency objects returned by apt_pkg.Version.depends_list
select a apt_pkg.Dependency.target_pkg that has the same architecture
as the apt_pkg.Version that generated it (Architecture: all packages
aside).  Usually this is the right thing do to.

The one time is isn't is when target_pkg is "multi-arch: Foreign".
In that case you always want the native package, if it is available.

Eg consider a system that wants both libgcc1:amd64 and libgcc1:i386
installed.  libgcc1 PreDepends on multiarch-support.
multiarch-support:amd64 conflicts with multiarch-support:i386, so
you can't have both installed.  But currently if you just use the
suggested target_pkg, you will attempt to install both.  apt-get and
aptitude resolve this by only installing the native version, and it
would be nice if apk_pkg did the same.

Example:

  import apt_pkg
  apt_pkg.init()
  cache = apt_pkg.Cache()
  dep_cache = apt_pkg.DepCache(cache)
  libgcc1_i386 = cache["libgcc1:i386"]
  version = dep_cache.get_candidate_ver(libgcc1_i386)
  depends = version.depends_list["PreDepends"][0][0]
  target_pkg = depends.target_pkg
  print target_pkg.name, target_pkg.architecture
  print depends.all_targets()[0].multi_arch == apt_pkg.Version.MULTI_ARCH_FOREIGN
  print cache["multiarch-support"].architecture, cache["dpkg"].architecture

Prints:
  multiarch-support i386
  True
  amd64 amd64

A better result would be:
  multiarch-support amd64
  True
  amd64 amd64

- -- System Information: Debian Release: 8.0
  APT prefers testing-updates
  APT policy: (500, 'testing-updates'), (500, 'testing') Architecture: amd64 (x86_64) Foreign
Architectures: i386

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores) Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8
(charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system)

Versions of packages python-apt depends on: ii libapt-inst1.5 1.0.9.3 ii libapt-pkg4.12 1.0.9.3 ii
libc6 2.19-13 ii libgcc1 1:4.9.1-19 ii libstdc++6 4.9.1-19 ii python 2.7.8-2 ii python-apt-common
0.9.3.11

Versions of packages python-apt recommends: ii iso-codes 3.57-1 ii lsb-release 4.1+Debian13+nmu1 ii
xz-utils 5.1.1alpha+20120614-2+b2

Versions of packages python-apt suggests: pn python-apt-dbg <none> ii python-apt-doc 0.9.3.11 ii
python-gtk2 2.24.0-4 pn python-vte <none>

- -- no debconf information
iQIVAwUBVIadxqzUn4heVFJuAQpMRA/+P30MjoHRHz6BvVQW74LSUk+GZdO+mW62
yE5+Zp8jahDIaM6bg0CXnItfRI53jFRteQIxTVUBi3WUdQi5lMRxyRvkH523na3U
apln9Juxdr5O8gu2WsPA2Aasv3sR1sD8+19BAQwYWtuqxisglV3ZG8v/pcV7wdVk
lGX2vYgw3nhao1Ad9jmV8oIxLvkhK7OhxBe9F8oaf8PhsUfWuJulxWwM3COxe6SE
igo3mzjfDHY8dvtdXndac0ZVXE6CKd2TPQtRslqPOEjiGvp/9E7O3Q8Rul+92NQi
H4G6XJ2wgyKSB6yLnClvHKS+LOSNoPTvGsy6cAQ/TQk2qCDcKE8/HgUa555TdTyz
sxmYywPt+2MiB+OzpqqSmIaPch4vclpce0lwD83jS0lWqe2PH+tHFnv8jVc/aWkg
iiT+w6qLHWtArqH+hX/hn/R+ObkSvy+dsvAvpCfGdJwukIo12LIRuPQ//HUw2drH
4ya+JTLd+LqlzrVVUMlzpCaXnXXew+54P8TuhUU7ZMjGpaI25XVYxVry9hY52OR5
tkazM0bF8jOsnPaS/a9rKY5IS4NsaD8ahpTqNVqNmvHh0BCOdEzMGUOGTEu2MjjN
hKAaxdaGKWVhbZY23aMZWQ9AtycNMk96t78Lk1czRmrEb2z48+66Ge5d82Z2z5ah
SxPbjHLmGk4=
=hwNm
-----END PGP SIGNATURE-----

[Something for David to look at :)]

Reassigning to APT, python-apt just calls the TargetPkg() method there.

Hi,
marked as foreign, same or whatever, but a package is just a collection
of versions.

Beside from compatibility concerns (old clients might be confused) it is
also wrong from an architectual point of view: A dependency usually is
declared on a package in the same architecture space, so that is the
safest bet. Other versions of this Debian package from other
architectures might or might not provide the same, but we can't know
that beforehand and we certaintly can't safely assume that for all
versions from this architecture (aka: what would be the result for
packages which change from M-A: none to M-A:foreign or vice versa).

You might have noticed that I said "provides" in the paragraph above and
that is in fact what is happening here: An M-A:foreign package is
internally mapped as a (versioned) provides, which on one hand is
a giant hack, but on the other hand I am increadibly proud of it as this
helps tremendously in making M-A:foreign "just work" even for old
clients which have never heard of M-A.

The takeaway for you is that at the point where you evaluate which
package you are going to install/upgrade to resolve a dependency
you can tune your evaluation function to consider the architecture.
That is how apt is doing it for example:
It puts the candidate version (if any) of the target package as well as
every candidate version (if any) of the packages providing the target
package (if any) in a list and then sorts the list considering various
properties (see CompareProviders here: [0]) to pick the best provider,
which for M-A:foreign packages happens to be the one from the native
architecture, but there are other things to consider as well (which got
added over time. This started very innocently with just priorities…).

Now that it does far more than trivial stuff, I could imagine exporting
this in some way, which this bugreport is actually asking for, but that
needs time for someone to think about a sane API…
any takers (after jessie) ?

(My fear is that this would otherwise end up looking like the horrible
car accidents called DepIterator::*Target* methods we currently have…)


Best regards

David Kalnischkies

[0] https://anonscm.debian.org/cgit/apt/apt.git/tree/apt-pkg/depcache.cc#n956

Safe bet maybe (it wasn't in this case), but surely wrong is too a
strong word.  By definition a package declaring itself to be M-A foreign
is saying the arch doesn't matter.  It may be a safer bet because of
bugs mean the package isn't really the same, but in that case the bug is
in the mistaken M-A foreign declaration, not installer thinking M-A
foreign doesn't mean what its documentation says.
(always?) had versioned dependencies to cope with that.  So if a
particular version meets the version preconditions then by definition it
meets the API requirements (for some definition of API that includes
"command line").

"We certainly can't safely assume for all versions from the
architecture" seems wrong to me.  Both sides of their dependency have to
declare their willingness to accept M-A: foreign.  (The depends declares
package:any, and the package declares M-A: foreign.)  If the version
conditions also are fulfilled then surely it is reasonable to assume the
maintainers knew what there were doing.

As for it going wrong if things change later: if later, either side of
the dependency M-A: foreign no longer applies, then they simply remove
the declaration.  It is never a problem for packages pre-multiarch,
because they didn't declare themselves as willing to accept M-A: foreign
in the first place.

Yeah, I saw package:any:i386.  I recall cleaning my glasses and looking
again, and then asking on Debian-mentors what it was I just saw.  No one
knew.
apt_get.Depends.target_pkg target had only one possible package it could
point at.  Now it potentially has several because there are separate
package objects per architecture.  You are essentially saying "we should
always choose the same architecture as the depends, as that is what we
did before".

Well that breaks an underlying assumption I made that was true before:
some version of target_pkg would be installed.  In the example I quoted
its not.  Effectively that means everything in target_pkg except
target_pkg.name is useless in the general case.  If pointing it to the
right architecture is too hard depreciate it and replace it with
target_pkg_name (a string) which express the same thing in an
architecture independent way.

Again, the sane API to me is expressing is exposing it in
apt_pkg.Dependency.target_pkg.  If you aren't going to do that then
surely apt_pkg.Dependency.smart_target_pkg() would be it - but currently
it also returns the wrong architecture in the case I example I gave.

from one datastructure (dependency) to another (package) where the later
is supposed to contain all possible solutions (and some) to the earlier.
The link is created while parsing files, so at a time we have no
complete information at what we are pointing to, we just know that at the
place we are pointing to all the information needed will be available
later. This link hence points to a place where all (if any) possible
solutions can be found, not to the 'best' possible solution, if such
a thing even exists (at that point in time or at all).

Beside, while the real package foo might be M-A: foreign, packages
providing foo might not, so if I have a bar:foreign depends on foo,
foo:foreign and foo:native are potential choices and foo:native probably
the best one, but foo-provider:native providing foo might not be
a solution at all, but it is what we get with TargetPkg().ProvidesList()
which used to work in the past and for all M-A: none/same packages.
(I am not even trying to imagine what that would mean for allowed)
from the depender side. Especially of note is in this regard that an
'M-A: foreign' package can NOT satisfy a 'package:any' dependency.
answered (well, I am a bit as I am subscribed to debian-mentors@, so
I wonder what made me miss it) as this is "super top secret" information.
I would guess only 10 people in the world know about it…

Claiming I said that in binary would be a bit much - I counted very
generously in decimal as I explained that once in a room with 3 people
present as a tongue-in-check remark on how they could do M-A in their
tool (CUDF).
The more interesting claim might be: More associates to various national
agencies know about this than associates to Debian (one of the 3 people
was the [at that time] DPL, so that isn't totally unrealistic).

And now you know, too! But remember, with great knowledge comes great
responsibility… ;)

Yes and no, as I explained further above, TargetPkg() [which is the C++
version of whatever python has; I don't speak parseltongue^Wpython] is
a one-to-one mapping as such, but that only works because the package
structure itself contains pointers to other structures (versions of the
real package in the architecture this structure has and providers of the
virtual package with this name in this architecture, which itself link
back to packages structures…).

I know that heritage is not the most sexy argument, but it is something
we have to considered in apt all the time as many tools are written on
top of us, so implementing M-A was heavily constraint by the fact that
we couldn't just go and reinvent everything. aptitude for example
struggled with the introduction of M-A for a while as it had less than
optimal first solutions to present, but it didn't explode in a big pile
of useless code and that counts as changing package management is a lot
harder than changing some boring stuff like init systems…
(yes, I actually believe that, even through I fear reality would
 disappoint me greatly if that would ever happen)

But that is hardly the only argument, which is why this was an aside
in the initial mail and not the entire essence.

If you extend your assumption to include provides, it is magically
true in single- as it is in multi-arch (which was the point of the
exercise to begin with).
a package name is looking up the package structure for it, so instead of
storing a pointer to the name of the package, we point to the package
itself saving us a lookup at runtime.

And please remember: M-A:foreign ALLOWS to install foo:native instead of
foo:foreign, but that doesn't mean it has to. There are perfectly valid
reasons not to do this, so "the right architecture" isn't always native,
it is just most of the time in most usage cases.

I have no idea what the API from the python side looks like, but the C++
one is one of the more ugly examples. The return of a single package is
probably not ideal anyway, I think a list of versions would be better,
but that would need constrains (only candidate versions? all versions?)
and caller-defined sorting…


Best regards

David Kalnischkies