#461758 apt-cacher: Automatically import already downloaded packages.

Package:
apt-cacher
Source:
apt-cacher
Submitter:
Raúl Sánchez Siles
Date:
2021-01-27 08:45:03 UTC
Severity:
wishlist
#461758#5
Date:
2008-01-20 15:21:51 UTC
From:
To:
On regular use of apt, either apt-get or aptitude, you usually have a
lot of packages cached in /var/cache/apt/archives

It would be nice that apt-cacher would retrieve them and store in its
own cache so you avoid downloading those packages again.

Thanks.

#461758#10
Date:
2008-01-20 16:33:05 UTC
From:
To:
Hello:

  Well, read the documentation and knew about apt-cacher-import. Quite useful.
Still I'd like to keep this bug open. I think importing should be done
automatically on install time.

  Thanks.

#461758#15
Date:
2008-07-29 22:33:17 UTC
From:
To:
Currently I run apt-cacher and after the
/etc/cron.daily
runs (apt) I have update-notifier tell me a bunch of
new packages are available.  I start update-manager
and
the downloads atart (via synaptic I think) and the
debs are downloaded to apt-cacher and headers put on,
but are they also going into /var/cache/apt/archives?
I have a bunch of recent debs in the archives location
so I'm unsure which process(es) put them there.  When
I run 'aptitude autoclean' and it says 0B freed.

Also, it would be nice to see a report from
'apt-cacher-cleanup' to show exactly what got cleaned up.

#461758#20
Date:
2021-01-27 08:41:22 UTC
From:
To:
Here's a setup I've built to optimize my network use.

I have debdelta running every couple of hours to download deltas and
create proper debs in /var/cache/apt/archives/

```
[Unit]
Description=Daily debdelta activity
Documentation=man:debdelta(1)
ConditionACPower=true
After=network.target network-online.target systemd-networkd.service
NetworkManager.service connman.service

[Service]
Type=oneshot
ExecStartPre=/usr/bin/apt update -qq
ExecStart=/usr/bin/debdelta-upgrade -v
ExecStartPost=/usr/bin/debdelta-upgrade -v
```


Then, I have apt-cacher's cron job, slightly modified, to periodically
import .debs from /var/cache/apt/archives/

```
Run the report generator.
# Hourly.
#
*/5 * * * *     root    test -x
/usr/share/apt-cacher/apt-cacher-report.pl &&
/usr/share/apt-cacher/apt-cacher-report.pl

# Run the importer.
# Hourly because a debdelta service runs separately at a frequent rate
@hourly root    test -x /usr/share/apt-cacher/apt-cacher-import.pl &&
/usr/share/apt-cacher/apt-cacher-import.pl /var/cache/apt/archives/ -r

# Run the cache cleaner.
# Weekly (Sunday) by default, but reduce the frequency if this is too
# heavy duty for you.
# Add -v to send root an email showing what the script has done.
#
37 4 * * sun    root    test -x
/usr/share/apt-cacher/apt-cacher-cleanup.pl &&
/usr/share/apt-cacher/apt-cacher-cleanup.pl
```


From the looks of it, it all seems to be working fine. Manually running
the import script tells me that it has already imported the .debs into
its cache.

```
rrs@chutzpah:/var/cache/apt-cacher$ sudo
/usr/share/apt-cacher/apt-cacher-import.pl /var/cache/apt/archives/ -r
[sudo] password for rrs:
Importing from /var/cache/apt/archives/
Hard linking or copying package files from /var/cache/apt/archives/ to
/var/cache/apt-cacher
/var/cache/apt-cacher/headers/gdisk_1.0.6-1_amd64.deb already exists:
skipping
/var/cache/apt-cacher/headers/intel-media-va-driver_21.1.0+dfsg1-1_amd64.deb
already exists: skipping

...snipped...

/var/cache/apt-cacher/headers/sudo_1.9.5p1-1.1_amd64.deb already exists:
skipping
/var/cache/apt-cacher/headers/usb.ids_2021.01.26-1_all.deb already
exists: skipping
Done.
0 files imported
14:04 ♒♒♒   ☺
```


But when I run `apt upgrade` on a different machine, which talks to this
proxy, the proxy doesn't seem to be offering the .debs from the cache.
Instead, it downloads those .debs from the internet.


I did some investigation for an example deb and I see:

```
rrs@chutzpah:/var/cache/apt-cacher$ find . | grep libavcodec58
./packages/deb.debian.org_debian/libavcodec58_4.3.1-8_amd64.deb
./headers/libavcodec58_4.3.1-6_amd64.deb
./headers/deb.debian.org_debian/libavcodec58_4.3.1-8_amd64.deb
./headers/libavcodec58_4.3.1-7_amd64.deb
14:06 ♒♒♒   ☺
```

Can you please explain what is the difference of the same deb in
./packages/ vs ./headers/ folder ? When I import the .deb, it is
imported to ./headers/ folder. But when I do an `apt upgrade` on a
different machine, the proxy doesn't offer the cache. Instead, it
downloads a new copy of the deb and puts it into the ./packages/ folder.

Subsequent requests for the same file do get offered from the cache. But
my ultimate goal is to squeeze every bit of the network I have. And for
that I would like to make use of the `debdelta` setup I outlined above.