#914798 approx doesn't work behind httpS_proxy (using 'HTTP CONNECT' method)

Package:
approx
Source:
approx
Description:
caching proxy server for Debian archive files
Submitter:
Alex Lutay
Date:
2019-11-14 09:51:03 UTC
Severity:
normal
#914798#5
Date:
2018-11-27 13:14:06 UTC
From:
To:
Hi,

We have tested approx behind http_proxy and it works well.
To make it work we have added the following override config:

Out proxy is a default Squid installation which supports the "HTTP CONNECT" method.

When we tested connections via httpS_proxy we got 'apt' errors:
...

The issue happens because the file 'InRelease' in the cache is corrupted (the file contains some proxy headers).

Whenever approx is sending the request to the server, on reply approx strips the proxy headers before storing the content into the cache.
See "iter_headers" code here https://salsa.debian.org/ocaml-team/approx/blob/master/url.ml#L64

Unfortunately it doesn't strip httpS_proxy headers properly/fully.
In case of httpS_proxy we have one more section of headers for the TCP connect to the proxy server
(I have to create a curl wrapper to dump the reply from proxy server to approx):
which further confuses apt a lot as there is no GPG section => error "Clearsigned file isn't valid, got 'NOSPLIT'".

The first header is the proper proxy reply in case of HTTP CONNECT:
https://en.wikipedia.org/wiki/HTTP_tunnel
In this case we need to strip both proxy headers.

I am not sure about the proper fix here, as striping headers is a bit tricky already.
Also I didn't find any 'curl' options to mute proxy headers into STDOUT.

I am using Debian GNU/Linux 9.6 (stretch) with default approx 5.7-3 and curl 7.52.1-5+deb9u8.

Thank you for the possible fix here!

#914798#10
Date:
2019-11-04 15:28:12 UTC
From:
To:
Hi,

Please find the proposed fix/workaround for approx behind
HTTPS proxy which is using "HTTP CONNECT" method.

Is it possible to include it into default approx codebase?

P.S. the patch has been taken from GitHub:

Thank you!

#914798#17
Date:
2019-11-04 21:34:31 UTC
From:
To:
curl 7.54+ has "--suppress-connect-header" option. Using latest curl and a
smaller patch might be a better choice.

#914798#22
Date:
2019-11-14 09:49:30 UTC
From:
To:
Hi,

Thank you Vlad for the nice hint here!

I was searching for such curl option one year ago in Debian stretch
but it was missing there as curl there was only 7.52.
Sure, the fix is much nicer in Debian buster with new curl nowadays.

See attached fix #2 for approx behind HTTPS proxy which
is using "HTTP CONNECT" method.

Please consider to include it into default approx codebase.

P.S. the patch has been taken from GitHub:
P.P.S. I have also attached the Dockerfile for the test environment
where you can reproduce the issue. It contains stock approx and basic
squid proxy. Please see the steps to reproduce at the end of Dockerfile.

P.P.P.S If someone cannot wait for the fix here, you can
workaround the problem adding to /etc/approx/approx.conf:
It will work for curl 7.54+ only (Debian buster+).

Thank you!