#794063 ssh with ControlMaster and ControlPath hangs on 2nd session in same terminal

Package:
openssh-client
Source:
openssh
Description:
secure shell (SSH) client, for secure access to remote machines
Submitter:
Uwe Kleine-König
Date:
2015-07-30 19:21:06 UTC
Severity:
normal
#794063#5
Date:
2015-07-30 08:39:28 UTC
From:
To:
Hello,

with two terminals a hang can be reproduces as follows:

 - In terminal 1 do:

 	ssh -o "ControlMaster auto" -o "ControlPath ~/.ssh/sshctrl-%l-%h-%p-%r" localhost

 - In terminal 2 do the same:

 	ssh -o "ControlMaster auto" -o "ControlPath ~/.ssh/sshctrl-%l-%h-%p-%r" localhost

 - Then back in terminal 1 again:

 	logout

   This doesn't terminate the ssh-command to keep the control socket
   open. Then pressing Ctrl-Z and run bg to send the ssh into the
   background.

 - The "remote" shell in the 2nd terminal is still working.

 - Now repeat the ssh command in the first terminal:

	ssh -o "ControlMaster auto" -o "ControlPath ~/.ssh/sshctrl-%l-%h-%p-%r" localhost

Both sessions don't react any more. The only way I found to resolve up
to now is to kill all related ssh sessions with kill(1).

Note that just exiting the first shell after sending the ssh process
that created the control socket in the background works fine. Also
adding new sessions in a third terminal works then.

My bet is on ssh being confused on two commands with the same
controlling terminal.

Best regards
Uwe

#794063#10
Date:
2015-07-30 09:11:36 UTC
From:
To:
Hello,

when logging system calls with strace on the process that created the
control socket the following can be seen (when the 2nd ssh on the same
terminal is started):

[select(12, [3 4 8 9], [], NULL, {15, 0}]
) = 1 (in [4], left {11, 431772})
clock_gettime(CLOCK_BOOTTIME, {12450, 924219877}) = 0
accept(4, {sa_family=AF_LOCAL, NULL}, [2]) = 5
getsockopt(5, SOL_SOCKET, SO_PEERCRED, {pid=26544, uid=1000, gid=1000}, [12]) = 0
getuid()                                = 1000
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
fcntl(5, F_GETFL)                       = 0x2 (flags O_RDWR)
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
fcntl(5, F_GETFL)                       = 0x802 (flags O_RDWR|O_NONBLOCK)
clock_gettime(CLOCK_BOOTTIME, {12450, 924749258}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 924807734}) = 0
select(12, [3 4 5 8 9], [5], NULL, {15, 0}) = 2 (in [5], out [5], left {14, 999994})
clock_gettime(CLOCK_BOOTTIME, {12450, 924941979}) = 0
read(5, "\0\0\0\10", 4)                 = 4
read(5, "\0\0\0\1\0\0\0\4", 8)          = 8
write(5, "\0\0\0\10\0\0\0\1\0\0\0\4", 12) = 12
clock_gettime(CLOCK_BOOTTIME, {12450, 925180653}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 925235379}) = 0
select(12, [3 4 5 8 9], [], NULL, {15, 0}) = 1 (in [5], left {14, 999994})
clock_gettime(CLOCK_BOOTTIME, {12450, 925360439}) = 0
read(5, "\0\0\0\10", 4)                 = 4
read(5, "\20\0\0\4\0\0\0\0", 8)         = 8
clock_gettime(CLOCK_BOOTTIME, {12450, 925536700}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 925590213}) = 0
select(12, [3 4 5 8 9], [5], NULL, {15, 0}) = 1 (out [5], left {14, 999995})
clock_gettime(CLOCK_BOOTTIME, {12450, 925717764}) = 0
write(5, "\0\0\0\f\200\0\0\5\0\0\0\0\0\0ga", 16) = 16
clock_gettime(CLOCK_BOOTTIME, {12450, 925832117}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 925885997}) = 0
select(12, [3 4 5 8 9], [], NULL, {15, 0}) = 1 (in [5], left {14, 999995})
clock_gettime(CLOCK_BOOTTIME, {12450, 926010243}) = 0
read(5, "\0\0\0@", 4)                   = 4
read(5, "\20\0\0\2\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0~"..., 64) = 64
recvmsg(5, {msg_name(0)=NULL, msg_iov(1)=[{"\0", 1}], msg_controllen=24, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, {6}}, msg_flags=0}, 0) = 1
recvmsg(5, {msg_name(0)=NULL, msg_iov(1)=[{"\0", 1}], msg_controllen=24, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, {7}}, msg_flags=0}, 0) = 1
recvmsg(5, {msg_name(0)=NULL, msg_iov(1)=[{"\0", 1}], msg_controllen=24, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, {12}}, msg_flags=0}, 0) = 1
ioctl(6, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(6, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(7, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(12, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
fcntl(7, F_SETFD, FD_CLOEXEC)           = 0
fcntl(12, F_SETFD, FD_CLOEXEC)          = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 926826653}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 926905547}) = 0
select(13, [3 4 8 9], [3], NULL, {15, 0}) = 1 (out [3], left {14, 999995})
clock_gettime(CLOCK_BOOTTIME, {12450, 927034328}) = 0
write(3, "\0\0\0 \276\263|\344\300\250\304\25\330\325F\2076hL\t\4\241B`\217\271`\357\376\255N\311"..., 44) = 44
clock_gettime(CLOCK_BOOTTIME, {12450, 927205118}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 927270057}) = 0
select(13, [3 4 8 9], [], NULL, {15, 0}) = 1 (in [3], left {14, 999995})
clock_gettime(CLOCK_BOOTTIME, {12450, 927408029}) = 0
read(3, "\0\0\0 \357\333\222yS\305js\305\372\273wc\242\32\355A\210\310\315\305\315EHE\306\35\261"..., 8192) = 44
ioctl(6, TIOCGWINSZ, {ws_row=24, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 927644303}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 927700923}) = 0
select(13, [3 4 5 8 9], [3 5], NULL, {15, 0}) = 2 (out [3 5], left {14, 999994})
clock_gettime(CLOCK_BOOTTIME, {12450, 927830272}) = 0
write(5, "\0\0\0\f\200\0\0\6\0\0\0\1\0\0\0\4", 16) = 16
write(3, "\0\0\1@\372\241\227{\232\247F\"\211!\364\273\373\265p\272\346\245K\320\325\262Ey\10\322\315\375"..., 436) = 436
clock_gettime(CLOCK_BOOTTIME, {12450, 928059300}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 928120850}) = 0
select(13, [3 4 5 8 9], [], NULL, {15, 0}) = 1 (in [3], left {14, 998954})
clock_gettime(CLOCK_BOOTTIME, {12450, 929408518}) = 0
read(3, "\0\0\0\20\372\347jh\242I\33\233\214\275\261}&4\274\300\207\2209\216\347CD\320\0\0\0\20"..., 8192) = 176
clock_gettime(CLOCK_BOOTTIME, {12450, 929613655}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 929675983}) = 0
select(13, [3 4 5 6 8 9], [7], NULL, {15, 0}) = 1 (out [7], left {14, 999991})
clock_gettime(CLOCK_BOOTTIME, {12450, 929828373}) = 0
write(7, "Last login: Thu Jul 30 11:03:05 "..., 54) = 54
clock_gettime(CLOCK_BOOTTIME, {12450, 929950095}) = 0
clock_gettime(CLOCK_BOOTTIME, {12450, 930005636}) = 0
select(13, [3 4 5 6 8 9], [], NULL, {15, 0}) = 1 (in [3], left {14, 727921})
clock_gettime(CLOCK_BOOTTIME, {12451, 202284805}) = 0
read(3, "\0\0\0000\353\245\246uri\357m\220>\210\244\304\313\354\327R\3535\320\266d\202^:i\221\371"..., 8192) = 60
clock_gettime(CLOCK_BOOTTIME, {12451, 202472937}) = 0
clock_gettime(CLOCK_BOOTTIME, {12451, 202540369}) = 0
select(13, [3 4 5 6 8 9], [7], NULL, {15, 0}) = 1 (out [7], left {14, 999992})
clock_gettime(CLOCK_BOOTTIME, {12451, 202688970}) = 0
write(7, "\33]0;uwe@perseus: ~\7uwe@perseus:~"..., 34) = 34
clock_gettime(CLOCK_BOOTTIME, {12451, 202811407}) = 0
clock_gettime(CLOCK_BOOTTIME, {12451, 202882596}) = 0
select(13, [3 4 5 6 8 9], [], NULL, {15, 0}) = 1 (in [6], left {13, 639311})
clock_gettime(CLOCK_BOOTTIME, {12452, 563761064}) = 0
read(6, 0x7ffdc9f16de0, 16384)          = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGTTIN {si_signo=SIGTTIN, si_code=SI_KERNEL} ---
--- stopped by SIGTTIN ---

with fd 6 being the terminal of the process (that is also connected to file
descriptors 0 1 and 2).

Best regards
Uwe

#794063#15
Date:
2015-07-30 17:16:14 UTC
From:
To:
I think that's basically as it should be.
The manpage's documentation is unfortunately a bit unclear here, I once
sent some patches upstream that should clear things up much more
better:
https://lists.mindrot.org/pipermail/openssh-unix-dev/2015
-February/033471.html

But I guess that was never merged for whatever reasons...
That's kinda interesting though, I can reproduce that.
Even ~. or so doesn't work anymore, and it really seems to require
SIGKILL.

Interestingly it doesn't happen when ControlPersist is e.g. set to 1s.

Sounds like an upstream issue, could you perhaps forward this?




Cheers,
Chris.

#794063#20
Date:
2015-07-30 18:50:39 UTC
From:
To:
I didn't want to wail about this one. I understand that this is (more or
less) necessary.
My plan was, that this is forwarded by the ssh maintainers as they for
sure now better than me where and how to report this :-)

Best regards
Uwe

#794063#25
Date:
2015-07-30 18:55:40 UTC
From:
To:
The upstream bugtracker would be at https://bugzilla.mindrot.org/

Best wishes,
Chris.