Hi,

I've been talking to Thomas (borgbackup upstream) about what is going to
happen with borg and it turns out this is not entirely trivial.

Once ugrading to borgbackup 2.x, you cannot continue to use borgbackup
1.x repositories. Rather you have to upgrade them which has to be done
with "borg transfer --upgrader=From12To20 ...". In particular, this will
copy all of the backup data.

So one option is to just upload borgbackup 2.x under the existing
package name. Doing so will immediately break all borg setups during the
upgrade. That's bad. Maybe the next thought is automatically converting
all repositories during upgrade, but this is totally out of question,
because the Debian package doesn't know the repositories and if it did,
it wouldn't be clear that they have sufficient space for temporarily
storing another copy. Quite obviously, this is not as simple as it
seems initially.

So I've been talking to Thomas about this, and we figured that it would
be good to move the borg repository upgrade out of the Debian stable
upgrade process. Quite obviously, you cannot upgrade your repositories
before bookworm as buster doesn't include borgbackup 2.x. So if you want
to do it later, bookworm must include borgbackup 1.x and borgbackup 2.x.
Ideally, doing the dist-upgrade would not switch from borgbackup 1.x to
borgbackup 2.x automatically.

So unless we want to loose sanity, we want two borgbackup source
packages in bookworm. One for 1.x and one for 2.x. Then next question is
what happens to /usr/bin/borg. I think we have basically three options:

a) It belongs to borgbackup 1.x and borgbackup 2.x uses a different name
   (e.g. borg2). Prior art: python3. This makes dist-upgrades easy, but
   everyone will have to change their scripts.
b) It belongs to borgbacku 2.x and borgbackup 1.x uses a different name
   (e.g. borg1). Prior art: openssh-client-ssh1. This makes
   dist-upgrades annoying and everyone will have to change their scripts
   twice (once during dist-upgrade).
c) It is flexible. It could be managed using update-alternatives. I
   suggest that bookworm gives a preference to borgbackup 1.x to make
   dist-upgrades painless.

At this time, I prefer the flexible route.

Beyond the cli, there also is the borg Python module. This presently is
located in /usr/lib/python3/dist-packages and thus importable by other
packages. I think this is wrong. Upstream clearly says[1] that the borg
module should not be used directly. As such, I think it would be better
to package it as a private module and move it to
/usr/lib/<binarypackage>/<modulename>.  Once doing so, the
coinstallability of two borg modules becomes trivial.

If we want to follow this plan, borgbackup 2.x should be uploaded rather
sooner than later to clear new.

Do you agree with all of this?

Helmut

[1] https://borgbackup.readthedocs.io/en/stable/internals/frontends.html?highlight=stable%20api#all-about-json-how-to-develop-frontends

Thanks to Helmut, that plan sounds really good to me!

A few little details to add (but they do not influence the suggested
upgrade procedure):

One actually has to create NEW repos for borg2 and then has the options:
- to just have the future backup archives in these new repos, or
- to "borg2 transfer" existing archives from the borg1 repos to the
borg2 repos and afterwards add future backups

The transfer might take quite a while, depending on the archive count
and the amount of data. It does deduplicate, so the transfer of the
first archive usually will take a long time, but subsequent similar
archives will transfer faster. The src and dst repos will be locked, so
they can not be used for anything else while the transfer is running.

If one wants to transfer old archives, one needs to create **related**
borg2 repos ("related" means some of the secret key material from the
old borg key is copied to the new key, specifically the chunker secret
and the ID-hash key). This makes sure that past backups will deduplicate
with future backups.

borg transfer in a client/server setup (ssh:// repos) will need a borg2
on client **and** on server.

Thanks to Helmut, that plan sounds really good to me!

A few little details to add (but they do not influence the suggested
upgrade procedure):

One actually has to create NEW repos for borg2 and then has the options:
- to just have the future backup archives in these new repos, or
- to "borg2 transfer" existing archives from the borg1 repos to the
borg2 repos and afterwards add future backups

The transfer might take quite a while, depending on the archive count
and the amount of data. It does deduplicate, so the transfer of the
first archive usually will take a long time, but subsequent similar
archives will transfer faster. The src and dst repos will be locked, so
they can not be used for anything else while the transfer is running.

If one wants to transfer old archives, one needs to create **related**
borg2 repos ("related" means some of the secret key material from the
old borg key is copied to the new key, specifically the chunker secret
and the ID-hash key). This makes sure that past backups will deduplicate
with future backups.

borg transfer in a client/server setup (ssh:// repos) will need a borg2
on client **and** on server.

Hello Helmut and Thomas,
since I don't use the package directly by myself,
I'll let you choose the preferred transition plan, and feel free to commit/upload just on git at
your convenience, I prefer somebody that uses the package on a daily basis to take care of eventual
issues that might arise :)

I maintain the package just because it contains part of my nickname :D

cheers,

Gianfranco