#642159 debian-installer preseed broken with apt-cacher-ng mirror

#642159#5
Date:
2011-09-19 20:44:47 UTC
From:
To:


Copied from an Ubuntu bug report, since it resembles my problem as well:


In my preseed file I set "d-i mirror/http/proxy string http://apt-cacher:3142/", this is an apt-cacher-ng proxy. This works correctly for installing the system.

I expect "d-i mirror/http/proxy" to only apply to the debian installer http proxy (apt proxy), and all other http applications, e.g., wget, curl, lynx to not use that setting.

Instead when I run applications from the Kickstart %post section, all HTTP connections are proxied; curl, wget, lynx, hg (mercurial), everything. Once the system installs and reboots all is good, the only remanence of the mirror/http/proxy setting is in /etc/apt/apt.conf, which is fine (though I think that should be configurable too). mirror/http/proxy is setting the environment variable http_proxy.

Repo:
Set "d-i mirror/http/proxy string http://apt-cacher:3142/" in a preseed (use your local mirror).

Use a Kickstart post section like this:
%post --interpreter=/bin/bash
exec < /dev/tty3 > /dev/ttyS0
chvt 3
(
echo "## http_proxy = $http_proxy"
echo "## lynx dump head"
lynx -dump -head http://example.com
echo "## wget"
wget -O /root/test http://example.com
) 2>&1 | /usr/bin/tee /var/log/post_install.log
chvt 1

After install check /var/log/post_install.log and it will look like:
## http_proxy = http://apt-cacher:3142/
## lynx dump head
HTTP/1.1 403 Forbidden file type or location
Date: Thu Apr 22 16:29:10 2010
Server: Debian Apt-Cacher NG/0.4.6
## wget
--2010-04-22 09:29:10-- http://example.com
Resolving apt-cacher... 10.0.0.10
Connecting to apt-cacher|10.0.0.10|:3142... connected.
Proxy request sent, awaiting response... 403 Forbidden file type or location
2010-04-22 09:29:10 ERROR 403: Forbidden file type or location.

The workaround for me it to reset $http_proxy at the top of %post:
export http_proxy=""
Then all the applications that use http behave normally.

Suggestion:
Don't set $http_proxy with "d-i mirror/http/proxy", maybe set $apt_http_proxy if d-i needs an environment variable. For users that need a general proxy a setting like "d-i http/proxy" could be created.

#642159#10
Date:
2013-02-20 17:34:50 UTC
From:
To:
This debian-installer bug is perhaps more serious than might seem from
the original bug report. It affects preseed/run and preseed/include. Not
clear how to adapt the work-around from the original bug report.

The bug also affects fetching in preseed/late_command, as well as
subsequent attempts to re-fetch the original preseed file (which comes
up if you have priority=low or some other installer error to bring up
the installer menu, and as you proceed through the steps it will direct
you to the preseed fetch menu option). Particularly confusing is that
Ctrl-Alt-F4 shows a wget 403 error yet tcpdump on port 80 on the preseed
server shows no activity, and furthermore if you go into BusyBox in the
installer and do a manual wget there is no error.

I can't determine a work-around, which might for example be to tell
apt-cacher-ng to serve up (and not cache) local files, but no luck so
far. I considered replacing apt-cacher-ng with two copies of approx (one
for debian, one for ubuntu), but approx seems to be a more apt specific
cache that won't serve up arbitrary local files.

Consequently I avoid fetching additional preseed files entirely, with a
growing "one-liner" script in preseed/late_command. It seems that much
of the additional power of preseeding is unavailable right now, in the
context of an apt proxy cache, until this debian-installer bug can be
addressed.

#642159#15
Date:
2013-02-20 18:12:12 UTC
From:
To:
(had problems with proxy setting before, tho i can't remember what)

### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string 192.168.1.74:3142
d-i mirror/http/directory string /ftp.uk.debian.org/debian
#d-i mirror/http/proxy

ofc this leaves the proxy in /etc/apt/sources.list after install
but i didn't have a problem with that
you could just sed it out or something in the late_command

#642159#30
Date:
2013-12-10 18:20:07 UTC
From:
To:
I've dug a little bit in the code and agree with Matt that probably we require
and extra option (or even more if we want to control when to leave the mirror in
/etc/apt/apt.conf), it seems for me that http_proxy (mirror/http/proxy) is being
abused. It's set|retrieve several times during installation, and used for
several purposes.

/usr/lib/fetch-url/http (debian-installer-utils)
  23    # use the proxy for wgets (should speed things up)
  24    if db_get mirror/$proto/proxy; then
  25            export ${proto}_proxy="$RET"
  26    fi

/bin/choose-mirror (choose-mirror.c, choose-mirror)
  656 static int set_proxy(void) {
  657   char *px = add_protocol("proxy");
  658   char *proxy_var;
  659
  660   proxy_var = xasprintf("%s_proxy", protocol);
  661
  662   debconf_get(debconf, px);
  663   if (debconf->value != NULL && strlen(debconf->value)) {
  664           if (strstr(debconf->value, "://")) {
  665                   setenv(proxy_var, debconf->value, 1);

/usr/lib/apt-setup/generators/50mirror (apt-setup)
  225  if [ "$protocol" = http ]; then
  226     db_get mirror/$protocol/proxy
  227     proxy="$RET"
  228     if [ -n "$proxy" ]; then
  229         if ! grep -iq "Acquire::$protocol::Proxy" $ROOT/etc/apt/apt.conf.new; then
  230            echo "Acquire::$protocol::Proxy \"$proxy\";" >> $ROOT/etc/apt/apt.conf.new

This make the installer to fail on several situations, let's suppose
mirror/http/proxy is set on a PXE APPEND environment this way:

APPEND DEBCONF_DEBUG=5 priority=critical locale=en_US interface=auto mirror/http/proxy=http://10.99.88.1:3142 url=http://domain.com/file.preseed initrd=initrd.gz

The installer will fail while fetching http://domain.com/file.preseed. This is
because /usr/lib/fetch-url/http set 'export http_proxy=http://10.99.88.1:3142
before retrieving the preseed file.

Now, if export ${proto}_proxy="$RET" is commented on /usr/lib/fetch-url/http,
the installer will go one step further, it'll retrieve the preseed file, but
still will fail when downloading other files on preseed/run, preseed/include and
preseed/late_command. This is due to /bin/choose-mirror setting again http_proxy
on setenv(proxy_var, debconf->value, 1);

Finally, if that line is commented as well, the installer will finish
successfully while still using the mirror because apt-setup use the variable to
configure apt-get before base-installer get into the action. The proxy will also
be available on next boot after installation.

I think this can be fixed, if we add a mirror/apt/proxy debconf variable and use
it on /usr/lib/apt-setup/generators/50mirror while leaving choose-mirror and
/usr/lib/fetch-url/http untouched. Then people could use mirror/http/proxy to
control connections for general applications and mirror/apt/proxy for
controlling debian repositories.