pbuilder fails to detect MIRRORSITE if /etc/apt/sources.list
includes only https entries.

Patch attached.

regards,
-mika-

Well, that's not enough.
I haven't tried, by I'd say having https lines in /etc/apt/sources.list
requires apt-transport-https.
I don't want to install apt-transport-https by default on chroots, so if you
really want https being automatically detected and used then you also want to
add some conditional things that install apt-transport-https if needed.


Then I've never understood why apt-transport-https is on a different package
not in the main apt binary, but that's another story.

* Mattia Rizzolo [Fri Jul 03, 2015 at 07:44:19AM +0000]:

Yes, apt-transport-https is indeed needed and that's what I'm doing
to set up the build envs:

| /usr/sbin/cowbuilder --create [,,,] --debootstrapopts --include=apt-transport-https,ca-certificates

ca-certificates isn't explicitely needed because it seems to be
pulled in anyway, but maybe we should add it explicitely as well,
what do you think?

Which I can understand (though it's not nice that we throw so many
pitfalls to users that care about security, but that's related to
the issue of the separate apt-transport-https package as you noted).

Would it be an option to check for usage of https in $MIRRORSITE
in /usr/lib/pbuilder/pbuilder-createbuildenv and then extend the
--include=apt option with apt-transport-https accordingly?

Yeah :-/

regards,
-mika-

ca-certificates is a recommends of libcurl3-gnutls which is in turn a
dependency of apt-transport-https. the chroots created by pbuilder disable the
automatic installation of recommends, so you explicitly need it, yes.

I'm not super happy about having ca-certificates (and that means openssl) in
chroots, though I guess nobody is going to manually install single certificates
for every host he's going to connect to, and ssl without trusting certs is
useless. What a pain.
Until this is not the default I'm ok, though.

not only -createbuildenv, but also -updatebuildenv. There are already a couple
of cases where the installed packages are extended.
And I think we also want to check for https in the chroot's
/etc/apt/sources.list in -updatebuildenv, since a user might have add entries
by hand and now he wants to use them.

But, umh, this is going to be a bit tricky because the first `apt-get update`
is going to fail due to the missing apt-transport-https, and the EXTRAPACKAGES
check is done after that.

Only now I see that you're explicitely installing them in the debootstrap
phase, and not after, e.g. adding them to the EXTRAPACKAGES conf entry. umh.
And as you can read in the comment above the debootstrap invocation (even if
that would mean ignoring the --update use case), adding packages with --include
is not safe from our pov, so that's not really as easy as I first thought.


Please have a look at those two scripts and try to see if you can think of a
clean solution for this :)

* Mattia Rizzolo [Fri Jul 03, 2015 at 09:12:46AM +0000]:

[MIRRORSITE setup with https]
of libcurl3-gnutls so it gets automatically pulled in anyway, as I
just verified).

Agreed.

Oh right, thanks for mentioning that.

Right.

Ok, that's what I was afraid of. :-/

My my main concerns for the current handling of pbuilder WRT https
is, that even with DEBIAN_FRONTEND=noninteractive and only https
entries present in sources.list its installation fails with "Default
mirror not found" and prompts for interactive usage, which I
consider a pity.

Only via preseeding I manage to get pbuilder installed without
failing/prompting for mirror selection. My patch prevents the
failing pbuilder installation and leaves the apt-transport-https
handling to the user. As a first step we could maybe include my
current patch and clarify the usage of https WRT apt-transport-https
in pbuilder's documentation? This at least slightly improves
situation for users of https-only sources.list and we later on we
can further improve the situation. What do you think?

PS: Interestingly with
# echo "pbuilder mirrorsite select https://debian..../debian" | debconf-set-selections
I still end up with "MIRRORSITE=http://cdn.debian.net/debian" in
/etc/pbuilderrc, didn't investigate closer though.

regards,
-mika-

Hi!

* Mattia Rizzolo [Fri Jul 03, 2015 at 07:44:19AM +0000]:
a thing (was deprecated with apt 1.6~alpha1 and is a transitional
package since then).

All what's needed to properly handle https nowadays is
ca-certificates (with its openssl dependency), as I just verified on
a Debian/trixie system.

PS: in the meanwhile the problem get worse though, because APT uses
the DEB822 source format nowadays (so no longer having
/etc/apt/sources.list.d/debian.list but
/etc/apt/sources.list.d/debian.sources), so pbuilder's auto
detection of the default mirror no longer works even with only http
on a current (trixie or newer) Debian system.

regards
-mika-