#605398 bash: pipeline in an input-reading while loop causes premature termination

Package:
bash
Source:
bash
Description:
GNU Bourne Again SHell
Submitter:
"Panu A. Kalliokoski"
Date:
2010-12-06 01:48:05 UTC
Severity:
normal
#605398#5
Date:
2010-11-29 15:43:42 UTC
From:
To:
I'm not really sure whose fault this is, because the circumstances where
the problem manifests itself are so specific.  This is the shortest
example I could find to reproduce the problem:
--- clip ---
while read line; do ssh somemachine echo "'$line'" | tr a a; done < somefile
--- clip ---

This command will _not_ echo all lines in koe.c, and the number of lines
it manages to echo seems to vary indeterministically:
--- clip --- [atehwa@aulis ~]$ while read rivi; do ssh oiva echo "'$rivi'" | tr a a; done < tmp/koe.c void f( char ** const w ) {}; int main( int argc, char * argv[] ) { char foo[10]; char * p = &foo[0]; [atehwa@aulis ~]$ while read rivi; do ssh oiva echo "'$rivi'" | tr a a; done < tmp/koe.c void f( char ** const w ) {}; [atehwa@aulis ~]$ --- clip --- All of these ingredients seem to be necessary to trigger the problem: while loop with read, making a pipe within the while loop, and using ssh in the pipe. That is, if I replace "ssh oiva echo" with just "echo", all lines get echoed; if I leave off "| tr a a", all lines get echoed. I couldn't find any other program except ssh that I could get to reproduce the problem. This seems very peculiar. regards, Panu
#605398#10
Date:
2010-12-06 00:02:31 UTC
From:
To:
Hi!

This is not a bug, at least, not a bug of bash... ssh does as it always does
unless instructed otherwise: hogs the STDIN for itself. (However, because it
has a command given on the command line to execute, it doesn't actually do
anything for the lines in STDIN, just reads them.) The varying amount of lines
output depends, I think, on how many lines from tmp/koe.c bash had time to
buffer before calling the first instance of ssh (which then hogs all the rest
of the lines - the later instances but echo the lines in the buffer and do
nothing else).

(I was able to replicate the 'bug' without the pipeline part. Probably the
pipeline used caused that there was time to buffer less lines.)

There are a couple of obvious 'workarounds':

while read rivi; do ssh -n oiva echo "'$rivi'" | tr a a; done < tmp/koe.c
(the one recommended in ssh manual)

while read rivi; do ssh oiva echo "'$rivi'"</dev/null | tr a a; done < tmp/koe.c
(does exactly the same thing as the line before)

while read rivi; do ssh oiva echo "'$rivi'"</dev/stdin | tr a a; done < tmp/koe.c
(you can actually get ssh to get its stdin from almost anywhere, as long as it
isn't the stdin of the subshell invoked by the while statement - yes, from even
the same file - different FD. But I don't see much point in other options
besides /dev/null, since they just cause the ssh to read the files without
doing anything with the content)

This won't work - as I said, ssh throws away the contents of STDIN when it has
a command to execute - but it goes to show that it does read it:

#605398#15
Date:
2010-12-06 01:43:54 UTC
From:
To:
I got into thinking about this example...

One could be tempted to think that this '</dev/stdin' here does completely
nothing (we're redirecting STDIN from wherever it happened to be in the first
place). But the point is, this way we don't end up reading it out. We actually
have a copy of the STDIN of the parent shell. One way to confirm this: