When using an ssh:// apt source, apt-get behaviour is a little awkward - the prompt for the passphrase gets hidden in the apt output but does work, except if the particular SSH key does not actually use a passphrase. apt halts, asking for the passphrase and refuses to accept 'Enter', printing a "Permission denied: please try again" error. apt appears to be only accepting the real login password when the SSH key passphrase is empty. (i.e. with an empty passphrase, apt falls back to the configured login) - this will fail if the remote server is configured to allow only SSH login.