#992692 general: Use https for {deb,security}.debian.org by default

Package:
general
Source:
general
Submitter:
Hideki Yamane
Date:
2025-07-27 15:25:03 UTC
Severity:
wishlist
Tags:
#992692#5
Date:
2021-08-22 12:56:57 UTC
From:
To:
Dear developers,

 As we discussed on -devel(*), it seems that we can enable https for
 {deb,security}.debian.org by default. With this bug report, I'll
 collect related things and fix it.

 - Update mirror list (how?)
 - Update security mirror setting in d-i (how?)
 - https://www.debian.org/mirror/list.en.html points http""://deb.debian.org,
   it should be https.
 - deb.debian.org says "deb http://", it should be "deb https://"
 - In release notes, it should be https://security.debian.org, at least
   See https://www.debian.org/releases/stable/amd64/release-notes/ch-information.html#security-archive


*) https://lists.debian.org/debian-devel/2021/08/msg00269.html

#992692#10
Date:
2021-09-01 09:15:55 UTC
From:
To:
Control: tags -1 + moreinfo

I believe that the discussion has later identified that doing so would
break squid-deb-proxy-client and auto-apt-proxy. Given that the security
benefits are not strong (beyond embracing good habits), I think the
reasonable thing to do is keep preferring http.

Caching packages and transport level encryption are fundamentally
incompatible. Possibly it would make more sense to offer users a choice
between performance and privacy on installation?

Helmut

#992692#17
Date:
2021-09-01 09:50:39 UTC
From:
To:
That is an opt-in choice which likely only a small number of users use.
People wanting to use a caching proxy can just switch to http as part
of this choice; it doesn't seem a good reason to not use https by
default for all other users.

No. You can explicitly configure apt to use a local caching mirror or
use a trusted TLS certificate for the mirror the proxy impersonates.


Ansgar

#992692#22
Date:
2021-09-01 14:46:07 UTC
From:
To:
Ansgar <ansgar@43-1.org> writes:

Completely agreed.

Yes.  For example, the approach used by apt-cacher-ng works fine.
Explicitly opting in to a local cache seems desirable.

#992692#27
Date:
2021-09-02 01:22:15 UTC
From:
To:
Hi,

 Providing "default secure setting" is good message to users.


 Some users want proxy but they can configure their settings.
 So just change "default setting for {deb,security}.debian.org"
 is not so harmful, IMO.

 - Users can choose other mirror than https://deb.debian.org
 - Caching .debs from security.debian.org is not so huge, I guess
   (maybe except linux-image).

#992692#32
Date:
2021-09-02 16:27:34 UTC
From:
To:
In this context, it might make sense to describe using HTTPS as the
transport for APT operations is providing "default confidentiality".

Regards,

#992692#37
Date:
2021-09-02 16:08:37 UTC
From:
To:
On 2021-09-02 10:22:15 +0900 (+0900), Hideki Yamane wrote:
[...]
[...]

As previously covered, I'd suggest steering clear of referring to
this as adding "default security." That implies APT wasn't already
effectively secure over plain HTTP, and may give a false impression
that HTTPS is addressing gaps in the existing apt-secure design.

This change is more about recognizing HTTPS as the primary transport
protocol for the modern Web, not sending mixed signals regarding the
general security risks posed by plain HTTP when used for unrelated
purposes, and no longer needing to repeatedly explain to users that
Debian has gone to great lengths to implement package distribution
security which doesn't really depend at all on transport layer
encryption.

#992692#42
Date:
2021-09-02 16:57:29 UTC
From:
To:
On 2021-09-02 12:27:34 -0400 (-0400), Roberto C. Sánchez wrote:
[...]

Which, as similarly discussed, it really doesn't do either (because
of deterministic blob sizes for publicly served files, current SNI
implementations leaking hostnames, general state of the CA and CDN
industries...), so suggesting that may also give users a false
impression. If they really do need confidentiality of their package
installation, they're probably better off doing updates over Tor or
a some VPN which does batching/interleaving of bulk transfers with
some cover traffic, or keeping a private local package mirror.

But to a great extent the degree of confidentiality they can expect
really depends on who they're trying to hide their traffic from. If
their concern is that a company headquartered in the USA might be
tracking the packages they're downloading from deb.debian.org, then
that's already a possibility even with HTTPS: the site is currently
fronted by the Fastly CDN service which terminates TLS encryption
for those HTTPS requests in order to be able to cache them globally.
Of course, without a CDN, mirror site operators can track what
packages you're downloading from them over HTTPS as well.

More generally, what I'm saying is don't try to paint this change as
actually implementing any significant amount of new security or
privacy for Debian users, that would be disingenuous. Just say the
default is switching to HTTPS because that's what users, by and
large, expect today.

#992692#47
Date:
2021-09-02 19:26:11 UTC
From:
To:
Hi,

The TLS layer is not part of the security model, so we'd be teaching
users to look for the wrong thing, kind of like the "encrypted with SSL"
badges on web pages in the 90ies.

We have our own PKI that is decoupled from the X.509 certificate
infrastructure, and neither ascribes any trust in them nor depends on
the availability of an external service.

As it is now, I can install a Debian system where no X.509 certificate
authorities are trusted.

  - If I deselect all CAs in the configuration dialog of the
ca-certificates package, what mechanism will allow apt to work?
  - Do we want to pin the certificate provider for Debian mirrors, in
the knowledge that we want to be bound to this provider for several
years, do we want any "root" CA to be able to provide a trust anchor?
  - Is there a revocation mechanism by which we can mark "root" CAs as
untrustworthy?
  - What does the UI look like if OSCP verification fails?
  - How do mirror operators get a signed certificate?

I think we're adding a lot of complexity and external dependencies to
the system here, which adds a lot of burden to mirror operators that
aren't large CDNs. That may be acceptable for an entity like Ubuntu, who
aren't dependent on donations, but we would be tied to the goodwill of
CDN operators here, so:

  - do we wish to communicate that the existing mirrors outside
deb.debian.org are somehow less "secure"?
  - do we have a contingency plan if deb.debian.org hosting on Fastly is
no longer feasible?

    Simon

#992692#52
Date:
2021-09-02 21:02:33 UTC
From:
To:
That doesn't change with the proposal?

If people intentionally detrust them, they have to deal with the
fallout. People can also detrust Debian's OpenPGP signing keys; it's
not much different.

Accessing www.debian.org will also not work on such systems (and unlike
deb.d.o that does not even offer non-https). It's not Debian's problem.

Probably not?

If we don't have one, shouldn't we worry more about that given the
widespread use of TLS?

Do we have a revocation mechanism by which we can mark OpenPGP signing
keys as untrustworthy (say for apt)?

Not all mirror operators have a TLS certificate. I don't think that was
proposed to be changed (see subject of the mail).

See above.

Is replacing deb.d.o by a non-CDN feasible? If no, what does use of
https change?

As far as I know there is also at least https://cdn-aws.deb.debian.org/
if you don't like Fastly.

Ansgar

#992692#57
Date:
2021-09-03 02:42:29 UTC
From:
To:
The Tor onion services offer alternatives to the https PKI:

https://onion.debian.org/

security.d.o mirrors were getting overwhelmed after Linux kernel
updates, which is why that switched to the Fastly CDN, so probably
not. Also, the entire point of the deb.d.o domain is that it be backed
by a CDN.

httpredir.d.o was an alternative CDN-like thing that was based on HTTP
redirects to the mirror network. It had lots of problems, but now that
we have a mirror checker and zzz-dists, perhaps it could work better.
The mirror:// method in apt has probably improved since then too.
Maybe http redirects to local mirrors could be feasible again, but it
would take a lot of work.

https://mirror-master.debian.org/status/mirror-hierarchy.html

And there are other CDNs that could potentially be used.

#992692#62
Date:
2021-09-03 11:11:46 UTC
From:
To:
Hi,

So this introduces a policy that users need to mark X.509 CAs as trusted?

The Debian signing keys have separate trust setup, and are trusted for
nothing but APT updates.

If we wanted the same for X.509, we'd need /etc/apt/ca-certificates or
something such, and apt to configure the list of accepted CAs explicitly
in the TLS layer rather than using the default settings.

There are a lot of systems out there that have no need to access
www.debian.org, but do need to access deb.debian.org.

So what do we want then? A list of root CAs that users have to mark as
trusted, possibly with an "are you sure?" dialog in ca-certificates?

This isn't a simple change of default, because this simple change pulls
in a lot of dependencies. That users can override the default means
adding another work step for users, either a manual step that needs to
be performed after a manual installation, or an automated step that
needs to be integrated into users' deployment processes.

We have a big hammer, shipping a new ca-certificates package. If we want
something that only affects apt, but not other packages, that mechanism
doesn't exist yet.

Yes, by shipping an update.

It's not about what I like, but on what external services we want to depend.

    Simon

#992692#67
Date:
2021-09-03 11:29:48 UTC
From:
To:
No.

People who only want to trust specific CAs for APT can do that. APT has
a CaPath setting.

And nothing stops them from doing so.

We already have that.

ca-certificates should already be installed by default (it is Priority:
standard and recommended by apt).

I don't see the problem? Currently we add another work step for users
that want to use https.

If a CA is untrustworthy, I don't think we would only want to detrust
it for apt's https method. So I see no problem.

So your concern is about Debian providing the deb.debian.org service at
all? That seems unrelated to the https or not question.

Ansgar

#992692#72
Date:
2021-09-03 11:27:22 UTC
From:
To:
Hi,

On 03.09.21 13:11, Simon Richter wrote:
[Revocation mechanism]
forces pushing for more agility, switching out roots of trust more
frequently. So for very old releases, you usually had the signing key of
the next release on disk, so you could move to the next release. In this
case you sort of risk not having the TLS authority on disk to make that
happen. And of course we need to track what the authorities are doing
that our frontends are using (e.g. [1] around how to deal with old
Android devices).

But then I'm not sure how much we need to care about ancient releases
that are out of security support. We would need to commit to regularly
update the certificate bundle, though.

To your other point: I don't think managing trust into individual CAs
will scale. We cannot really anticipate which CAs we are going to use in
the future.

Kind regards
Philipp Kern

[1] https://letsencrypt.org/2020/12/21/extending-android-compatibility.html

#992692#77
Date:
2021-09-04 20:12:46 UTC
From:
To:
 Is there any strong reason to use HTTP than HTTPS now?
 Should we teach all our users (including non-tech) about "Secure APT"
 mechanism?


 And I said about only deb.debian.org and security.debian.org, and
 just "default" - it means it does provide http access too.

#992692#82
Date:
2021-09-05 10:27:52 UTC
From:
To:
combine both approaches as in: Have apt request a list of mirrors to use
via mirror(+https) and have the server generate that list based on the
requester as that gives you the "regional" mirrors as did httpredir while
solving the major grip it had by having a list of mirrors to use, rather
than one potentially non-working slightly outdated partial mirror (and
the httpredir service is contacted by each client once rather than for
each individual file to then be redirected elsewhere).

Obviously, that approach is only workable if you are actually using
libapt tools. Most debootstrap implementations couldn't really use that
which might or might not be a problem for a given use case. Such
a service would also have a hard time to 'redirect' you to a local
mirror in your network (compared to an 'official' region one).


So that isn't really what seems to be the main worry here:
https prevents MitM attacks including the friendly MitM ones like the
local network at home/at DebConf telling my laptop that there is an
on-site mirror or not telling at all and just proxy transparently the
entire network.

The later seems done for in a https-world, but the former might be
somewhat salvageable: We will have to get the Release² file(s) from the
repo defined in the sources, but the index files and debs after that are
a fairer game to get from elsewhere as they are either identical with
what the defined source would have provided or a hard error.
That still violates the privacy guarantees https has (assuming it does),
so that would still need to be opt-in/out, but that is a one time choice
per machine and could be similar in style to auto-apt-proxy.

Anyway, implementation wise apt could tell $MAGIC which repo it is
interested in (by Origin & Label) and would in return get a list of
mirrors as apt-transport-mirror would. apt would then add the original
source as least priority fallback and proceed with that list for this
source.
I say $MAGIC as I don't want apt to hard code the specifics of how to
get the list, similar to how it is agnostic to how a proxy is currently
picked up, as I could envision different implementations depending on
use cases.

That is different to just using apt-transport-mirror directly in the
sources in so far as the provider of the list remains untrusted (beside
that nobody is constantly editing their sources to adapt to the local
network the machine currently resides in).


Relatively quickly thought up, probably full of holes and not implemented
at all in apt so far, but if someone thinks that might work feel free to
report as a feature request against apt and I will see what I can do
from the apt side. It at least seems slightly more workable than hoping
to prevent https – which might have just as dubious a chance to succeed
as https has to factually improve security in terms of apt. 😜

fwiw: apt does not allow https to http redirects (some https repos
ran into this in the past like those hosted on sourceforge until they
fixed their https 'everywhere' configuration). In this regard apt is
stricter than a normal webbrowser (a mirror list acquired via https can
redirect to http mirrors though, but see the man pages for details).


Best regards

David Kalnischkies


¹ which deb.d.o sort of is just that it is nowadays done via SRV instead
  of an explicit HTTP redirect – and that only one mirror is in the list
  rather than multiple httpredir had picked one to redirect to.

² The main security benefit of https for apt is that you can't fiddle
  with the Release file, mostly in terms of sending an older one (in
  the limits of Valid-Until if used). It is also minor in size compared
  to the indexes and especially the debs, so caching them is not much of
  a concern (if a cacher was even doing it, it probably shouldn't).

#992692#87
Date:
2021-09-08 11:09:13 UTC
From:
To:
Hi,

I fear you are putting this upside down. In reality, some sites (not
users) want their users to use their local cache (transparently or not).

As far as I can tell, most users don't want to make a choice here. They
want downloading packages to just work. Preferably fast. It is the
"fast" thing that you are breaking here.

Not sure why security.d.o is singled out here. The switch is only
reasonable on the whole or not at all. And there the whole volume of
packages counts.

Enabling https by default quite simply breaks the simple recipe of
installing auto-apt-proxy. Would you agree with auto-apt-proxy's
postinst automatically editing your sources.list to drop the s out of
https? The answer repeatedly given in this thread to do so manually is
very unsatisfying.

So I actually argue for installing auto-apt-proxy by default and inside
d-i. That is in direct conflict with the proposed change here.

Unfortunately, I don't see consensus for this, but at the same time I
neither see consensus for enabling https by default. It's a matter that
keeps popping up and people disagreeing on over and over again. The one
thing that we have clearly understood at this point is that one size
does not fit everyone. Either way makes some people unhappy.

Helmut

#992692#92
Date:
2021-09-08 11:37:37 UTC
From:
To:
Then have the users install the site's CA authority that allows
inspecting and caching HTTPS traffic.

Maybe we should just find out who is responsible for this decision and
reassign the bug to them.  The installer team maintaining d-i and
debootstrap or the mirror team seem reasonable choices?

Ansgar

#992692#97
Date:
2021-09-08 11:53:12 UTC
From:
To:
transition is miserable. Let's not repeat that mistake.

It's the same pattern actually:
 * People propose a change that does have positive effects, though some
   find the positive effects unimportant.
 * Other people disagree and raise concerns.
 * Concerns are ignored. <- This is where we are with https-default.
 * Change is being implemented anyway.
 * Stuff breaks. <- This is where we are with /usr-merge.

This is frustrating.

I do see the advantages of using https. I do not see how to not make it
happen without breaking relevant use cases. Same with the /usr-merge. I
do see the advantages. I've stopped counting the things that broke. Most
recent one is the uucp FTBFS. Change has a cost. I do not want to pay
the cost for either of these changes.

Helmut

#992692#102
Date:
2021-09-08 12:01:03 UTC
From:
To:
So what do you suggest then? Tech-ctte as with merged-/usr? Or a GR? Or
something else?

It's also where we are with keep-http-as-default.

To keep up with merged-/usr: keeping non-merged-/usr also has a cost.
Nobody wants to pay the cost for it.

Ansgar

#992692#107
Date:
2021-09-08 12:20:49 UTC
From:
To:
If you have access to the private key, you can request the CA to revoke
the certificate:

+---
| 4.9.1.1 Reasons for Revoking a Subscriber Certificate
|
| The CA SHALL revoke a Certificate within 24 hours if one or more of
| the following occurs:
| [...]
| 3. The CA obtains evidence that the Subscriber’s Private Key
| corresponding to the Public Key in the Certificate suffered a Key
| Compromise
+---[ https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.8.0.pdf ]

So that would not be helpful.

Ansgar

#992692#112
Date:
2021-09-08 12:13:58 UTC
From:
To:
This is a bit tongue in cheek, but how about these sites where the .debs
are downloaded from publish their *private* key? They openly accept that
anyone can MITM them.

That way people who want to see "https" can see it. And people who want
the benefits of http can, with a bit of work, simulate that.

It also nicely addresses my concern which is that the next demand will
be to drop http (because when you visit the site with a webbrowser users
start getting a warning that the site is also available over http or
something like that because the google/firefox dream seems to be that
you cannot use http even where https doesn't add anything.)

#992692#117
Date:
2021-09-08 13:41:58 UTC
From:
To:
I propose that the proponents pay the cost. In this case, it is a bit
unclear what that means precisely (which likely is the reason they
haven't done it already). At the very least though, apt install
auto-apt-proxy should continue to work on a default installation in a
sensible way.

I don't think https resolves any concerns. It's merely best-practice. In
the absence of reason not to use https, https should be preferred. As it
happens, we figured a reason not to use https.

Untrue. You get to choose which changes you want to pay the cost for.
For instance, I want Debian to be cross buildable and bootstrappable.
Holger, Mattia and a few others want Debian to be reproducible. You
don't get to pay the cost for those changes. Change is possible in a way
that limits cost for uninterested people. The contentious changes are
the ones where the initiators fail to pay the cost.

That is very true. With merged-/usr, I suppose most grief arises from
the way the transition was (not) planned and only a minority takes issue
with the goal.

Helmut

#992692#122
Date:
2021-09-08 13:56:14 UTC
From:
To:
saying

  Acquire::https::Verify-Peer false;

That clearly makes it work again: you ask for auto-apt-proxy users to
have connections that can be impersonated by a man in the middle by
default. The above setting does that.

Not verifying certificates for some users seems better than having all
users not verify certificates (as no https is used at all).

I can find a reason not to use https for any protocol (some sites want
to inspect/cache all traffic) :-)


Ansgar

#992692#127
Date:
2021-09-08 23:12:18 UTC
From:
To:
Why not simply automate setting it at install time using preseed? I'm
honestly not sure who the target audience for auto-apt-proxy
is--apparently someone who has an infrastructure including a proxy,
possibly the ability to set dns records, etc., but can't change defaults
at install time or via some sort of runtime configuration management?

#992692#132
Date:
2021-09-09 06:24:44 UTC
From:
To:
2021, സെപ്റ്റംബർ 9 4:42:18 AM IST, Michael Stone <mstone@debian.org>ൽ എഴുതി

Why can't auto-apt-proxy ask this as a debconf question? I also like auto-apt-proxy but I agree with  this, someone needing auto-apt-proxy should be able to change the default as well.

#992692#137
Date:
2021-09-09 06:36:28 UTC
From:
To:
* Michael Stone <mstone@debian.org> [2021-09-08 19:12]:
The same reason you might want to deploy WPAD instead of preconfiguring
proxy settings in web browsers: flexibility. For example, we use
auto-apt-proxy for laptops which roam between different networks.
It's simple to configure, has virtually no maintenance overhead and
degrades gracefully.

Cheers
Timo

#992692#142
Date:
2021-09-09 12:32:04 UTC
From:
To:
None of that speaks to whether an organization that deploys such a thing
has the ability to manage other configuration settings, even if for
some settings there's a desire for additional flexibility.

#992692#147
Date:
2021-09-09 12:54:21 UTC
From:
To:
* Michael Stone <mstone@debian.org> [2021-09-09 08:32]:
I don't understand your point.

You asked for a target audience for auto-apt-proxy. I gave you a
legitimate (and real-world) example for such a deployment. Why
should it matter whether or not an organization has other
configuration facilities?

As another example, nobody says: "the target audience for DHCP is an
organization who has an infrastructure with full control over a
network, including IP address management, but apparently can't
configure IP addresses at install time or via some sort of runtime
configuration management" and concludes that DHCP is useless.

auto-apt-proxy (or DHCP for that matter) *is* the runtime
configuration management, and a quite efficient one at that.

Cheers
Timo

#992692#152
Date:
2021-09-09 13:05:40 UTC
From:
To:
Because the controversy concerning changing the default is over whether
it's reasonable for someone using auto-apt-proxy to have to manage
additional configuration settings. If the target audience is someone who
has the ability to manage the infrastructure required by auto-apt-proxy
but not the ability to manage anything else then I guess it's a valid
criticism (but I have trouble envisioning that). If the auto-apt-proxy
target audience can manage both the proxy infrastructure *and*
sources.list (either at install time or run time) then the criticism is
basically academic and not generally a real-world issue.

#992692#157
Date:
2021-09-09 13:19:46 UTC
From:
To:
* Michael Stone <mstone@debian.org> [2021-09-09 09:05]:
Ah, I understand your point now and I agree.
It would be an inconvenience, yes, not but much more.

Cheers
Timo

#992692#162
Date:
2021-09-09 17:58:35 UTC
From:
To:
Hi,

The strongest reason (IMO) in favor of unencrypted transmission is that
it doesn't introduce a policy decision. The package "ca-certificates"
must be installed and a checkmark set for the "mozilla/ISRG_Root_X1.crt"
certificate, otherwise updates will break.

If we want to have HTTPS as default, we need additional logic to make
sure certificates are installed and cannot be deinstalled (so Essential
or a strong dependency chain from an Essential package) and that the
certificate cannot be deactivated, or apt needs its own repository of
trusted certificates.

With the current Docker images:

$ docker run --rm -it debian:bullseye
root@32529bf86cd3:/# sed -i -e s/http/https/ /etc/apt/sources.list
root@32529bf86cd3:/# apt update
Err:1 https://deb.debian.org/debian bullseye InRelease
   Certificate verification failed: The certificate is NOT trusted. The
certificate issuer is unknown.  Could not handshake: Error in the
certificate verification. [IP: 199.232.138.132 443]
Err:2 https://security.debian.org/debian-security bullseye-security
InRelease
   Certificate verification failed: The certificate is NOT trusted. The
certificate issuer is unknown.  Could not handshake: Error in the
certificate verification. [IP: 151.101.66.132 443]
Err:3 https://deb.debian.org/debian bullseye-updates InRelease
   Certificate verification failed: The certificate is NOT trusted. The
certificate issuer is unknown.  Could not handshake: Error in the
certificate verification. [IP: 199.232.138.132 443]

So changing the default is not sufficient here, we'd need a lot of
additional work and testing to ensure this works for everyone, not just
the desktop users.

Another important argument is that it creates a dependency on
third-party commercial CDNs, and their *continued* sponsorship.

Debian is very conservative when spending money and generally shies away
from recurring expenses because we do not want to find us in a situation
where we are dependent on an external entity making a timely donation in
order to keep operations running, so I wonder why we are that accepting
of it in one of our core services, and I certainly don't think we should
be adding additional roadblocks should we ever need to find an alternative.

We have a (crude) load-balancing framework in infrastructure we control
that can point requests towards a set of untrusted mirrors, and while
it's nice that we don't *need* to use this fallback plan, it is
reassuring it is there.
misconception that anything with an "s" in it is secure and can be trusted.

Anyone who configures an additional source needs to know how the
authentication mechanism works anyway, so we're not gaining anything
there either.

    Simon

#992692#167
Date:
2021-09-09 23:46:36 UTC
From:
To:
This dependency on external providers is unavoidable, Debian
definitely cannot afford to run our own CDN at the scale needed to
support our existing userbase. For example the security mirrors
struggled with Linux kernel security updates, so security.d.o switched
to a commercial CDN. Also, we are dependent on continued sponsorship
for all of our infrastructure, paying for all of our hosting is likely
not feasible.

https://wiki.debian.org/ExternalEntities

DSA setup the CDN provider solution to give the Debian userbase a
better experience than having to choose a mirror and a better
experience than httpredir.d.o's redirect method. We have multiple CDN
providers to mitigate the dependency, and other providers who we
aren't yet using that are offering service. So, as much as I dislike
CDNs as a concept, I recognise that we currently need them and think
that we are able to handle loss of a CDN provider or two.

httpredir.d.o no longer exists, it points at deb.d.o, so it would have
to be rebuilt if we were to need to switch away from CDNs.

Personally I'd like to see a larger variety of Debian delivery
mechanisms; copy Debian/snapshot to archive.org, create a multi-distro
FLOSS CDN, bring back httpredir, DebTorrent and apt-p2p, add an i2p
mirror, use IPFS and content oriented networking etc. Michael Stone's
apt://debian idea seems like a good way to move in that direction
while adding protocol agility.

The volume of questions about missing https means that it is more
efficient to just use https than to have to reply to new questions
about it.

#992692#172
Date:
2021-09-10 07:33:56 UTC
From:
To:
I believe that the target audience is non-tech end-users. Most
orgnizations already optimized their way of downloading .debs via some
way (e.g. auto-apt-proxy) or another. As you point out, organizations
are easily able to deviate from defaults. They're not our primary target
with defaults.

Laptops of end-user systems are the target, but also developers. When
people gather at a place (conference, hackspace, private meetup, etc.)
downloading of .debs should just work quickly by default. Many such
sites could easily provide a local cache and a number even do. BSPs tend
to have a blackboard with information including the local mirror to use.
Seriously, how many people change their mirror when they go to a BSP? If
we installed auto-apt-proxy by default, much of the local caching would
just work.

The thing we seem to be disagreeing on is what is more important? https
by default or quick and efficient downloads? Some may think that our
CDN can handle the load just fine and the effects of caching are
peanuts. I can attest that those peanuts are 4TB/year (equivalent to 8
days waiting for downloads) for me.

I see that we've given up on a global network of independently-managed
mirrors and that CDNs are way easier at this time. While initially CDNs
had bad reliability issues, those have completely vanished in my
experience. On the other hand, local caching still outperforms CDNs by a
factor of 10 or so. I'd love to make it the default.

Helmut

#992692#177
Date:
2021-09-10 08:13:43 UTC
From:
To:
If you push for a local caching method to be used by default, apt
should always request (In)Release.gpg from a regular mirror (not auto-
discovered local cache), preferably via HTTPS; for subsequent data
(which apt can verify via (In)Release) a local mirror can be used,
falling back to the regular mirror when the data provided by the local
cache is not correct for any reason.

Especially at BSPs where people are likely to bootstrap new
environments (via debootstrap, for example for building packages) we
would allow downgrade attacks otherwise: (In)Release for stable
releases has no Valid-Until, so any initial (In)Release file can be
substituted by the cache operator for an older one which then refers to
known-vulnerable packages. (And I'm not sure debootstrap even checks
Valid-Until.)

Ansgar

#992692#182
Date:
2021-09-10 12:07:02 UTC
From:
To:
I think you'd get a lot of pushback on installing auto-apt-proxy by
default. If that's a proposal, make it seperately and not in this
thread.

I use a cache out of habit and to be a good netizen, but my internet
connection is fast enough these days that it's basically a noop at best
and a slight slowdown at worst. I had to move the cache from slow/cheap
spinning disk to reasonably fast SSD to get to that point. If you're
downloading the same stuff over and over (e.g., for testing or somesuch)
it can be a big win, especially for VMs on a virtualized network
connection, or if you're managing a large infrastructure, but
for normal use with a couple of instances? It's just not worth the
trouble.

#992692#187
Date:
2021-09-10 14:48:56 UTC
From:
To:
Hi,

Yes -- and mirrors have traditionally been provided by third parties.
What is new about the CDNs is that there are rather few of those, and
this centralization aspect is what worries me.

Yes, absolutely -- and we especially want to have a better solution for
containers, because my expectation is that a large fraction of our
traffic is just people not bothering to set up caching.

    Simon

#992692#192
Date:
2021-12-09 11:11:19 UTC
From:
To:
I fully support the idea that HTTPS should become the default for apt repos.
 From what I gather, the open question is how best to handle auto-apt-proxy
configuration.  There seems to be a number of reasonable proposals:

* Make auto-apt-proxy set "Acquire::https::Verify-Peer false;"
* automate setting http at install time using preseed with auto-apt-proxy
   asking this as a debconf question.
* Users can always later edit the sources.list.  In the context of a BSP or
   DebConf, that is a very reasonable thing to ask.

auto-apt-proxy sounds like a nice feature, but it also adds security risks.  We
also need to consider that.  Users should get best practice security without
thinking about it at all.  That's HTTPS these days, despite its imperfections.
Not defaulting to HTTPS means people have to be aware that HTTP is the default,
then consider using HTTPS.  We should of course make it as easy as possible to
use caching proxies, that also comes with a responsibility in making the sure
aware that it adds small but present security risks.  So a debconf question in
auto-apt-proxy seems like a good place for that.

For those who think that apt's GPG verification is enough, consider these CVEs:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-1358
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-1829
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-3587
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-1252
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-0501
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-3462

For more on this whole topic, I wrote up a blog post based on my previous
research and these ongoing discussions:
https://guardianproject.info/2021/12/08/debian-over-https/

#992692#197
Date:
2022-03-30 07:44:34 UTC
From:
To:
https://salsa.debian.org/cloud-team/debian-vagrant-images/-/merge_requests/15

There are already many Debian mirrors that support HTTPS, not just CDNs.  Here's
a script to find HTTPS mirrors
https://gist.github.com/HacKanCuBa/e3a998d68a82f81dbf11f2cce4f26d04

I think all mirrors should support HTTPS (this is a requirement for f-droid.org
mirrors), so I filed a bug to track that:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1008652

#992692#202
Date:
2022-03-30 10:15:49 UTC
From:
To:
There is also discussion about making the official Debian Docker images use HTTPS:

https://github.com/debuerreotype/docker-debian-artifacts/issues/15

#992692#207
Date:
2022-03-30 10:40:43 UTC
From:
To:
iHTH https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1008652