Dear Maintainer,

In our setup, we share an archives cache directory [Dir::Cache::archives]
between multiple (virtual) machines. This turns out to be an effective
way to save bandwidth and disk space. However, each machine may lock the
archives directory and make apt unavailable for the other machines. We had
hoped that setting a lock timeout [DPkg::Lock::Timeout] would allow the
other machines to wait on the one holding onto the lock. Unfortunately,
that configuration parameter only applies to the 'admin directory'
containing the status file. As a result, apt operations fail immediately
when another machine holds a lock on the archives directory.

We can think of a few ways to make the experience better.
1. Respect DPkg::Lock::Timeout also for the other apt locks
    (e.g. in apt-pkg/acquire.cc:pkgAcquire::GetLock).
2. Add a new lock configuration for the cache/archives
    (e.g. APT::Cache::Lock::Timeout).
3. Implement per-download locking with a timeout as requested in #478961.

If either of these approaches is acceptable, we could propose an implementation
if desired.

Regards,
- Jouke

I do not consider sharing archives between machines to be a valid use
case. It breaks the assumptions of both the apt(8) model of cleaning up
after yourself as well as the opposite historical model of autoclean
(which removes packages no longer available from sources.list).

My suggestion for caching is to use a central squid-deb-proxy.


There are three locks at play in apt:

1. dpkg frontend lock - which apt always holds if it might invoke dpkg later on
2. dpkg lock - similar, but released to run dpkg
3. archives lock - held if we need to download anything

I implemented waiting for the outer most one, as we can somewhat easily
reason about this.

Some stuff might not lock the dpkg locks and that is problematic. Like
apt update should not change lists/ under me, and I should not be able
to acquire the archives/ lock without holding the frontend lock, as
otherwise I break the other process that just acquired the frontend
lock and now wants to install stuff.

I believe that all these different locks were essentially a mistake,
we really should have had just the one frontend lock.

The problem is, if the 2 dpkg locks succeed and we then get to the
archives lock, we can't go back to releasing the dpkg lock and waiting
for both of them, as we ought to.

I see how this could be annoying in our use case. We would need to set
a lot of configuration parameters to reliably prevent cleaning the
archives and I agree that this clearly runs against the assumptions of
apt and autoclean. Thank you for these pointers!

This looks like a reasonable solution. It is slightly more involved
than just mounting the same archives volume in all the containers, but
still reasonably simple. The biggest downside is that packages are
still copied into the local archives on the containers, thus wasting
disk space. The default docker apt configuration does clean the copies
after use, but I feel our initial naive solution of mounting the same
archives directory more cleanly circumvented the whole issue.

Sounds reasonable. I'd say this seals the fate of #478961.

Maybe I'm missing something, but as long as the archives lock is
obtained last, wouldn't it always be possible to wait for it? For the
current situation with 3 locks, this still leaves the possibility of
respecting the lock timeout also for the archives lock. If I read the
code correctly, the timeout logic cannot be added to
pkgAcquire::GetLock since it would apply to not only the archives
lock, but I don't think writing an archives-specific retry-wrapper
would be hard. Would you consider such an addition?

Thanks for your fast response!
- Jouke

Hi,

(somewhat obsoleted by Julians reply, but just so I haven't only wasted
 my time, lets waste yours as well by sending it anyway 😉)

I am not sure we should encourage this setup. It feels a bit brittle,
especially as not every form of directory sharing even supports
sharing the locks (like NFS I think), so that silently works until it
doesn't.

Sadly I don't really have a good alternative suggestion as e.g. the idea
I am on/off playing with of constructing a partial mirror out of
/var/cache/apt/archives (and /var/lib/apt/lists) so that you could us
a mirror+file as a source which would have priority for the
reconstructed partial file:/ mirror and otherwise fall back to the
online https:// source is nice and all, but certainly not as trivial to
set up and maintain as sharing /var/cache/apt/archives and pinky-
promising to each machine involved to not mess up too badly as you are
basically granting each of them full root access on each other.

(I take it that not copying around is your goal, hence me talking about
 file:/ as that uses the files at the place they are rather than copying
 them into /var/cache/apt/archives. Otherwise more "traditional" caching
 solutions are likely an easier/better fit as Julian mentioned)

That isn't really gonna work as its not just the download, you also want
to keep the file around until its no longer "needed", so we would need
to hold onto potentially thousands of locks in a typical full-upgrade.

If we just lock individual files while they are downloaded e.g. an 'apt
clean' operation could remove all the ones we finished downloading
already. Just like 'apt' defaults to removing deb files after
installation nowadays, so that the other machine who wanted to reuse
that file ends up failing to find it even through it was here a second
ago…


Best regards

David Kalnischkies