when downloading file via ssl connection, the results are of size 0 and I'm
getting "error" message:
"netin: Success"

according to strace and tcpdump output, the file is written in plain form
back to the FTP server vi athe data connection instead of output file on
local disk.

I can provide strace and tcpdump output, if needed.

Fredag den 16:e Oktober 2015, klockan 10:22, skrev Matus UHLAR - fantomas detta:

I can not reproduce this. A short session like

  $ ftp -z ssl,secure,verbose localhost
  ... logging in as myself ...
  ftp> lcd /tmp
  ftp> get .dput.cf _dput.cf
  ftp> bye

produces a tcpdump file where the only clear text parts are
"AUTH SSL", "AUTH SSL OK", and the plain text contained in
the certificate, which the server sends to the client.

Can you repeat your exchanges, clearly stating what options
you are feeding to the client. Do you observe a statement
fairly early containing an encryption selector? This shows
that encryption is active.

Hope to hear from you again, best regards
  Mats E Andersson

did you also look at data connection? For both incoming and outgoing data?

I got personal reply from a person confirming the same problem and that
he'll look at the debian patch.

When I looked at the sources, they seemed to be OK, however the stored file
is of zero size.

I have re-tested the session again and the same happened
- the control connnection is encrypted.
- the data fetched through the data connection are encrypted
- the data are not written to the local file, but sent through the data
   connection back to server unencrypted.

what size was the downloaded file on your machine?

Onsdag den 28:e Oktober 2015, klockan 09:31, skrev Matus UHLAR - fantomas detta:

tcpdump -r ftpsession -i lo  port ftp or port ftp-data

Please do state your call. I specifically want to know if you
have any debugging turned which might upset socket use.
Understood?

Identical sizes and empty respons from 'diff'.

/MEA

Hello,

I know where in source code the problem is :

in file ftp.c, line 1203
#ifdef USE_SSL
    if (ssl_data_active_flag) {
         while ((c = SSL_read(ssl_data_con, buf, bufsize)) > 0) {     <--- pb is here
                            fprintf(stderr, "c  %d\n",c);
                            if ((d = write(fileno(fout), buf, c)) != c)

I have changed few lines after : "if ( c < -1 )"  by "if ( c <= -1 )
then I get the output from :
sprintf(errbuf,"ftp: SSL_read DATA error %s\n",
                                    ERR_error_string(ERR_get_error(),NULL));

$ ./ftp mysite-ftp-ssl.xx

220 FTP SSL mysite-ftp-ssl.xx
234 AUTH SSL exécuté avec succès
[SSL Cipher DHE-RSA-AES256-SHA]
331 Mot de passe requis pour abcdef
230-
230-- FTP TLS SSL -
230-- Only Passive Mode -
230-
230 Utilisateur abcdef authentifié
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> passive
Passive mode on.
ftp> ls
227 Entering Passive Mode (12,33,44,55,252,31).
150 Ouverture d'une connexion de données en mode ASCII pour file list
-rw-------   1 abcdef   FTP     1141 Oct 23 19:38 netkit-ftp-ssl.txt
226 Téléchargement terminé
ftp> get netkit-ftp-ssl.txt
local: netkit-ftp-ssl.txt remote: netkit-ftp-ssl.txt
227 Entering Passive Mode (12,33,44,55,250,129).
150 Opening BINARY mode data connection for netkit-ftp-ssl.txt (1141 bytes)
ftp: SSL_read DATA error error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number     <--- here is the problem
226 Téléchargement terminé
ftp> quit
221 Au revoir.

So, I know where the problem is, but I don't know HOW to solve it (for the moment).

I will try.


Raphael
-------

Le mardi 27 octobre 2015, à 16:24:15 +0100, Matus UHLAR - fantomas (uhlar@fantomas.sk) a écrit :

Hello,

I have made some investigations.
I'm stuck, but I give status of these.

1) My server is proftpd, ProFTPD Version 1.3.3a (Debian squeeze).
My conf relative to tls is :



TLSProtocol             SSLv3 TLSv1
#TLSProtocol            SSLv23     <-- I have tested with that, or only SSLv3, or only TLSv1, no change

<VirtualHost IP>
...
  TLSEngine             on
  TLSRequired           auth+data   <-- I have tested with :   'on', 'ctrl', 'data', 'auth', no changes

with maximal relaxed other parameters :
  TLSOptions                             NoCertRequest AllowClientRenegotiations NoSessionReuseRequired
  TLSVerifyClient                        off
  TLSRenegotiate                         required off
...

- under linux :

a) lftp with :
set ftp:ssl-allow true
set ftp:ssl-force true
set ftp:ssl-protect-data yes
set ftp:ssl-protect-list yes
set ssl:check-hostname no
set ssl:verify-certificate no
set passive-mode yes
set ftp:use-feat/IP off
set ftp:ssl-force/IP on

b) curl -k -u user --ftp-ssl ftp://IP/some-file

- under windows :

a) FileZilla
b) WinSCP (recent version for support TLS)


3) I use the Debian sources from netkit-ftp-ssl-0.17.23+0.2.

A) I have noticed that when I set type to A (ftp> ascii), the download is correct and complete.

$ ./ftp/ftp -p -d -z verbose IP
(my .netrc is filled with login and password)

Connected to IP.
220 FTP SSL eliot IP
ftp: setsockopt: Bad file descriptor
---> AUTH SSL
234 AUTH SSL exécuté avec succès
SSL_connect:UNKWN  before/connect initialization
SSL_connect:3WCH_A SSLv3 write client hello A
SSL_connect:3RSH_A SSLv3 read server hello A
SSL_connect:3RSC_A SSLv3 read server certificate A
SSL_connect:3RSKEA SSLv3 read server key exchange A
SSL_connect:3RSD_A SSLv3 read server done A
SSL_connect:3WCKEA SSLv3 write client key exchange A
SSL_connect:3WCCSA SSLv3 write change cipher spec A
SSL_connect:3WFINA SSLv3 write finished A
SSL_connect:3FLUSH SSLv3 flush data
SSL_connect:3RFINA SSLv3 read finished A
[SSL Cipher DHE-RSA-AES256-SHA]
---> USER test
331 Mot de passe requis pour test
---> PASS XXXX
230-
230-- FTP HubV2 TLS SSL -
230-- Only Passive Mode -
230-
230 Utilisateur test authentifié
---> SYST
215 UNIX Type: L8
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ascii
---> TYPE A
200 Type paramétré à A
ftp> ls
ftp: setsockopt (ignored): Permission denied
---> PASV
227 Entering Passive Mode (11,22,33,44,247,56).
---> LIST
150 Ouverture d'une connexion de données en mode ASCII pour file list
===>START SSL connect on DATA
SSL_connect:UNKWN  before/connect initialization
SSL_connect:3WCH_A SSLv3 write client hello A
SSL_connect:3RSH_A SSLv3 read server hello A
SSL_connect:3RFINA SSLv3 read finished A
SSL_connect:3WCCSA SSLv3 write change cipher spec A
SSL_connect:3WFINA SSLv3 write finished A
SSL_connect:3FLUSH SSLv3 flush data
===>DONE SSL connect on DATA 7
-rw-------   1 test  FTP     1141 Oct 23 19:38 netkit-ftp-ssl.txt
226 Téléchargement terminé
ftp> get netkit-ftp-ssl.txt
local: netkit-ftp-ssl.txt remote: netkit-ftp-ssl.txt
ftp: setsockopt (ignored): Permission denied
---> PASV
227 Entering Passive Mode (11,22,33,44,234,200).
---> RETR netkit-ftp-ssl.txt
150 Opening ASCII mode data connection for netkit-ftp-ssl.txt (1141 bytes)
===>START SSL connect on DATA
SSL_connect:UNKWN  before/connect initialization
SSL_connect:3WCH_A SSLv3 write client hello A
SSL_connect:3RSH_A SSLv3 read server hello A
SSL_connect:3RFINA SSLv3 read finished A
SSL_connect:3WCCSA SSLv3 write change cipher spec A
SSL_connect:3WFINA SSLv3 write finished A
SSL_connect:3FLUSH SSLv3 flush data
===>DONE SSL connect on DATA 7
226 Téléchargement terminé
1170 bytes received in 0.02 secs (52.1 kB/s)
ftp> quit
---> QUIT
221 Au revoir.

B) With type binary (I) or ascii (A) the upload is correct and complete (SSL_write).

C) In ftp/ftp.c, function recvrequest, the problem is with :

while ((c = SSL_read(ssl_data_con, buf, bufsize)) > 0)      (l. 1206)

the SSL_read function fails.

I have put just before : ssl_data_active_flag=0; if (ssl_data_active_flag) {...
to make the "else" in action, that is :
while ((c = read(fileno(din), buf, bufsize)) > 0) {...

and then I receive something : the file netkit-ftp-ssl.txt plus some extra bytes inside (from a second packet I have seen in tcpdump,
I suppose there are bytes relative to SSL that the server is sending).
I see the file netkit-ftp-ssl.txt in clear with tcpdump on data port, this should not be...

D) As it works with TYPE A (ssl encrypted, not in clear), I have tried to replace just after
 case TYPE_I:
 case TYPE_L:
exactly the same code from case type A
(that is : DATAGETC --> SSL_read just one byte).

But SSL_read fails again.

So, I don't know what I can do to make this SSL_read working.
I don't understand.

Raphael
-------


Le mercredi 28 octobre 2015, à 12:19:19 +0100, Raphael Astier (raphael.astier@eliot-sa.com) a écrit :

Hello again,

I wonder if this problem persists? The first report and all follow-ups
were made prior to the binary rebuild of the package ftp-ssl, which
happened on November 3rd, when version 0.17.33+0.2-1+b1 was made
available. This update was caused by a version step in libopenssl,
seemingly related to this problem becaus SSL3 was depreciated.

Attached are two patches that are pending for my next upload.
I hope to hear from you, and would like to express my appreciation
to both of you for your investigation.

I have yet to identify why and how the control socket becomes the
channel on which to transfer a copy of the received data file.

Best regards,
  M E Andersson

Tisdag den 3:e november 2015, klockan 10:20, skrev Raphael Astier detta:

Hello,

I apparantly should note that I am using wheezy - I have encountered the
same behaviour with versions 0.17.23+0.2-1+b1 in wheezy and 0.17.33+0.2-1
compiled manually. I have also added provided patches, no change.

I also have the same result when fetching from both remote and local proftpd
servers.

Seems my first observation was wrong. The plaintext data were sent from
remote host to local one, so the data session apparently wasn't encrypted.
I need to check with different servers...

Hello,

I have tried to compile netkit-ftp-ssl_0.17.33+0.2 (obtained from debian stretch packages), with debian patches + yours 2 patches.

With same configuration server than before.

First I see the same results :

ftp> get file.bin
local: file.bin remote: file.bin
227 Entering Passive Mode (11,22,33,44,248,39).
150 Opening BINARY mode data connection for file.bin (1141 bytes)
ftp: SSL_read DATA error error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
netin: Input/output error
226 Transfer complete
ftp> quit

It seems there is incompatibility between netkit-ftp-ssl code and libssl that I use on my system :

ldd netkit-ftp-0.17/ftp/ftp

        linux-gate.so.1 =>  (0xf770f000)
        libssl.so.0.9.8 => /usr/lib/i686/cmov/libssl.so.0.9.8 (0xf76a6000)
        libcrypto.so.0.9.8 => /usr/lib/i686/cmov/libcrypto.so.0.9.8 (0xf754e000)
        libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xf73ea000)
        libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0xf73e6000)
        libz.so.1 => /usr/lib/libz.so.1 (0xf73d2000)
        /lib/ld-linux.so.2 (0xf7710000)

I have tried also under wheezy :

ldd ftp/ftp
        linux-vdso.so.1 =>  (0x00007ffe851df000)
        libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f34a1f1e000)
        libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f34a1b26000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f34a179a000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f34a1596000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f34a137f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f34a2189000)

before update libssl1.0.0  --> it does not work,

after update libssl1.0.0 (and libssl-dev for compile) : it works.

http://security.debian.org/ wheezy/updates/main libssl1.0.0 amd64 1.0.1e-2+deb7u18
(replace libssl1.0.0:amd64 1.0.1e-2+deb7u14 with libssl1.0.0 1.0.1e-2+deb7u18_amd64.deb)


It also works with the default ftp-ssl package under wheezy AFTER updating libssl.

So, to my opinion :

1) ftp-ssl (get binary files) does not work with libssl prior to a certain version, (whereas lftp, curl, etc,  does...),

2) ftp-ssl (get binary files) works under wheezy, jessie, (...)  after updating libssl.


Then, THANKS M. Andersson !
(are you Neo ?).

Regards,

Raphael Astier
--------------


Le lundi 30 novembre 2015, à 15:03:42 +0100, Mats Erik Andersson (mats.andersson@gisladisker.se) a écrit :