Hi,

during a test with piuparts I noticed your package fails to upgrade from
'buster'.
It installed fine in 'buster', then the upgrade to 'bullseye' fails.

In order to run 'gnt-cluster upgrade' both ganeti-3.0 and ganeti-2.16
need to be installed, but the package from buster needs to be removed
due to the removal of unversioned python and the Python 2 modules
(python-*) before the local admin could run 'gnt-cluster upgrade'

I'm not sure how a clean upgrade is intended to be performed in this
case. This may also be an interesting question for DSA.

From the attached log (scroll to the bottom...):

  The following packages were automatically installed and are no longer required:
    ganeti-haskell-2.16 ganeti-htools-2.16 libprocps7 libpython2-stdlib
    libpython2.7-minimal libpython2.7-stdlib libreadline5 libreadline7
    python-six python2 python2-minimal python2.7 python2.7-minimal
  Use 'sudo apt autoremove' to remove them.
  The following packages will be REMOVED:
    ganeti-2.16 libpython-stdlib python python-asn1crypto python-bcrypt
    python-bitarray python-cffi-backend python-cryptography python-enum34
    python-fdsend python-ipaddr python-ipaddress python-minimal python-nacl
    python-openssl python-paramiko python-psutil python-pyasn1 python-pycurl
    python-pyinotify python-pyparsing python-simplejson
  The following NEW packages will be installed:
    ganeti-3.0 ganeti-haskell-3.0 ganeti-htools-3.0 gcc-10-base libapt-pkg6.0
    libbpf0 libbrotli1 libcbor0 libcrypt1 libffi7 libfido2-1 libgcc-s1
    libgdbm-compat4 libgdbm6 libhogweed6 libmd0 libmpdec3 libnettle8 libnsl2
    libpcre2-8-0 libperl5.32 libprocps8 libpython3-stdlib libpython3.9-minimal
    libpython3.9-stdlib libreadline8 libtirpc-common libtirpc3 libxxhash0
    logsave mailcap media-types perl perl-modules-5.32 python3 python3-bcrypt
    python3-bitarray python3-cffi-backend python3-cryptography python3-minimal
    python3-nacl python3-openssl python3-paramiko python3-psutil python3-pycurl
    python3-pyinotify python3-pyparsing python3-simplejson python3-six python3.9
    python3.9-minimal runit-helper tzdata
  The following packages will be upgraded:
    [...]
  135 upgraded, 53 newly installed, 22 to remove and 0 not upgraded.
[...]
  Selecting previously unselected package ganeti-3.0.
  Preparing to unpack .../44-ganeti-3.0_3.0.1-2_all.deb ...
  Adding 'diversion of /usr/share/ganeti/2.16/ganeti/utils/version.py to /usr/share/ganeti/2.16/ganeti/utils/version.py.orig by ganeti-3.0'
  Unpacking ganeti-3.0 (3.0.1-2) ...
  Preparing to unpack .../45-libcurl4_7.74.0-1.2_amd64.deb ...
  Unpacking libcurl4:amd64 (7.74.0-1.2) over (7.64.0-4+deb10u2) ...
  Preparing to unpack .../46-libpcre3_2%3a8.39-13_amd64.deb ...
  Unpacking libpcre3:amd64 (2:8.39-13) over (2:8.39-12) ...
  Setting up libpcre3:amd64 (2:8.39-13) ...
  Selecting previously unselected package ganeti-haskell-3.0.
  Preparing to unpack .../ganeti-haskell-3.0_3.0.1-2_amd64.deb ...
  Unpacking ganeti-haskell-3.0 (3.0.1-2) ...
  Selecting previously unselected package ganeti-htools-3.0.
  Preparing to unpack .../ganeti-htools-3.0_3.0.1-2_amd64.deb ...
  Unpacking ganeti-htools-3.0 (3.0.1-2) ...
  Preparing to unpack .../ganeti_3.0.1-2_all.deb ...
  Unpacking ganeti (3.0.1-2) over (2.16.0-5) ...
  Removing ganeti-2.16 (2.16.0-5) ...
  dpkg: error processing package ganeti-2.16 (--remove):
   installed ganeti-2.16 package pre-removal script subprocess returned error exit status 30
  dpkg: python-simplejson: dependency problems, but removing anyway as you requested:
   ganeti-2.16 depends on python-simplejson.

  Removing python-simplejson (3.16.0-1) ...
  dpkg: python-fdsend: dependency problems, but removing anyway as you requested:
   ganeti-2.16 depends on python-fdsend.

  Removing python-fdsend (0.2.1-2) ...
  dpkg: python-paramiko: dependency problems, but removing anyway as you requested:
   ganeti-2.16 depends on python-paramiko.

  Removing python-paramiko (2.4.2-0.1) ...
  dpkg: python-openssl: dependency problems, but removing anyway as you requested:
   ganeti-2.16 depends on python-openssl.

  Removing python-openssl (19.0.0-1) ...
  Removing python-cryptography (2.6.1-3+deb10u2) ...
  Removing python-asn1crypto (0.24.0-1) ...
  Errors were encountered while processing:
   ganeti-2.16


cheers,

Andreas

Hi Andreas

Thanks for the report.

Andreas Beckmann <anbe@debian.org> writes:

Yes, the removal of unversioned python makes things more complicated. I
was hoping that ganeti-2.16's dependencies would be enough to keep the
Python 2 modules around until gnt-cluster upgrade could run.

That said, I think the best way forward is to upgrade buster clusters
to 3.0 using the packages from buster-backports, before attempting to
dist-upgrade to bullseye. This should probably be documented in the
release notes.

Since there's really not much else we can do about it now, I'm
downgrading to important and tagging it as wontfix. Please let me know
what you think.

Regards,
Apollon

Do we know what specifically causes old python to be removed?

Cheers,
Julien

Hi Julien,

Julien Cristau <jcristau@debian.org> writes:

  Broken python2.7:amd64 Breaks on libpython-stdlib:amd64 < 2.7.16-1 @ii mK Ib > (<
 2.7.18)
    Considering libpython-stdlib:amd64 11 as a solution to python2.7:amd64 23
    Added libpython-stdlib:amd64 to the remove list
  Broken python2.7:amd64 Breaks on python:amd64 < 2.7.16-1 @ii mK Ib > (< 2.7.18)
    Considering python:amd64 15 as a solution to python2.7:amd64 23
    Added python:amd64 to the remove list
  Broken python2.7:amd64 Breaks on python-minimal:amd64 < 2.7.16-1 @ii mK Ib > (< 2.7.18)
    Considering python-minimal:amd64 11 as a solution to python2.7:amd64 23
    Added python-minimal:amd64 to the remove list
    Fixing python2.7:amd64 via remove of libpython-stdlib:amd64
    Fixing python2.7:amd64 via remove of python:amd64
    Fixing python2.7:amd64 via remove of python-minimal:amd64

python2.7 in Bullseye explicitly Breaks: python (<< 2.7.18),
python-minimal (<< 2.7.18). This causes the removal of all packages that
Depend on python (including ganeti-2.16, python-simplejson etc).

Cheers,
Apollon

Apollon Oikonomopoulos <apoikos@debian.org> writes:

This was added in

python2.7 (2.7.18-7) unstable; urgency=medium

  [ Andreas Beckmann ]
  * python2.7, libpython2.7-minimal: Add Breaks: python (<< 2.7.18) to ensure
    removal of the unversioned python packages on upgrades from buster. The
    previously added (transitive) Breaks are not sufficient in a few upgrade
    scenarios. Closes: #987661.

Can one upgrade ganeti, keeping the old python2.7 and ganeti-2.16, then
do the cluster upgrade, then finish the dist-upgrade?

Cheers,
Julien

Package: python
Source: python-defaults
Version: 2.7.16-1
Depends: python2.7 (>= 2.7.16-1~), libpython-stdlib (= 2.7.16-1), python2 (= 2.7.16-1)
Pre-Depends: python-minimal (= 2.7.16-1)

bullseye:

Package: python2
Source: python-defaults
Version: 2.7.18-2
Depends: python2.7 (>= 2.7.18~), libpython2-stdlib (= 2.7.18-2)
Pre-Depends: python2-minimal (= 2.7.18-2)

==> python/buster is not co-installable with python2/bullseye
==> python/buster must be removed to upgrade python2

Package: python2.7
Version: 2.7.18-7
Depends: python2.7-minimal (= 2.7.18-7), libpython2.7-stdlib (= 2.7.18-7), mime-support
Breaks: libpython-stdlib (<< 2.7.18), python (<< 2.7.18), python-minimal (<< 2.7.18), python-virtualenv (<< 1.7.1.2-2~), vim-athena (<< 2:7.3.547-4), vim-gnome (<< 2:7.3.547-4), vim-gtk (<< 2:7.3.547-4), vim-nox (<< 2:7.3.547-4)

The explicit versioned Breaks in python2.7 against unversioned
python packages are there to help apt making the right decision:
remove unversioned python and upgrade python2 to bullseye
(otherwise it would try to not upgrade python2/buster in order
to keep python/buster installed in quite a bunch of upgrade
scenarios)

@apollon:
Would it be possible to build ganeti-2.16 in bullseye (with
reduced dependencies?) s.t. it depends on python2 instead of
python? Then it could be reintroduced into bullseye like buster
had ganeti-2.15 in addition to ganeti (2.16).

Andreas

I'd argue that keeping python/buster installed would be a better
decision...

Cheers,
Julien

Andreas Beckmann <anbe@debian.org> writes:

It would be possible, but that would also have to be done for at least
python-simplejson, python-fdsend and possibly others. To be honest, I'd
much prefer promoting upgrades via buster-backports for Ganeti, it's
much more straightforward.

Regards,
Apollon

Hi!

I wrote a mail to ganeti at packages.debian.org which I don't know if you
guys got, on it I commented over this problem and some others I had found
while upgrading.

The thing is that I also got this problem and the solution I found was to do
a:

apt install python-is-python2

And then magically everything fitted, I could install both ganeti versions
and do the upgrade, after upgrading everything I removed all python 2 stuff
and that's it.

Hope it helps.

Regards.