#288323 zsh: doesn't handle suspension of commands in conditional lists correctly

Package:
zsh
Source:
zsh
Description:
shell with lots of features
Submitter:
Branden Robinson
Date:
2012-12-20 22:39:09 UTC
Severity:
important
#288323#5
Date:
2005-01-03 01:54:31 UTC
From:
To:
Things like:

foo && bar && baz

are a basic POSIX shell feature, and Bash doesn't handle them right.

Try:

echo one && sleep 10 && echo two

While, in the sleep, background the command with CTRL-Z.

The 'echo two' will run immediately.  This is wrong.  The sleep has no exit
status yet because it has not exited, and the && and || connectives must
only be evaluated once the preceding command has exited.  Until then, the
command in question *has* no exit status.  The box with Schroedinger's Cat
in it has not yet been opened.

ash, dash, pdksh, and zsh are also buggy, but instead they never run "echo
two" at all.  It appears that of Debian's allegedly POSIX-compliant shells
have this problem, except for posh.

Here are some speculations/argument from #debian-devel as to what may be
going on:

08:42PM|<asuffield> Overfiend: what you have in bash is bloody broken
   conditionals. I can't see how to fix it, and I can't stand looking
   at bash any longer to figure it out
08:42PM|<asuffield> it passes WUNTRACED to wait() when job control is
   enabled, so that it can spot jobs which have been sent SIGSTOP
08:43PM|<asuffield> somewhere in the pipeline logic is a missing check
   for WIFSTOPPED on the status code, to see if the process is really
   dead yet or not
08:49PM|<asuffield> look, WSTOPCODE() and WEXITCODE() are the same
   macro. zsh is calling WEXITCODE() and treating it as the exit code,
   and this is *INCORRECT*, because WIFEXITED() is false and WIFSTOPPED()
   is true
08:51PM|<Keybuk> and zsh documents that it won't continue a pipeline if
   the process is terminated by an unhandled signal
08:52PM|<Keybuk> so zsh is being correct, just different to bash
08:52PM|<asuffield> zsh has incorrectly interpreted the result from wait()
   as if the process had been terminated
08:52PM|<asuffield> the process has been stopped. this is a different
   event

#288323#14
Date:
2005-01-03 04:15:12 UTC
From:
To:
Should zsh wait for a suspended process to exit before continuing along
the sublist?
----- Forwarded message from Branden Robinson <branden@debian.org> ----- Things like: foo && bar && baz are a basic POSIX shell feature, and Bash doesn't handle them right. Try: echo one && sleep 10 && echo two While, in the sleep, background the command with CTRL-Z. The 'echo two' will run immediately. This is wrong. The sleep has no exit status yet because it has not exited, and the && and || connectives must only be evaluated once the preceding command has exited. Until then, the command in question *has* no exit status. The box with Schroedinger's Cat in it has not yet been opened. ash, dash, pdksh, and zsh are also buggy, but instead they never run "echo two" at all. It appears that of Debian's allegedly POSIX-compliant shells have this problem, except for posh. Here are some speculations/argument from #debian-devel as to what may be going on: 08:42PM|<asuffield> Overfiend: what you have in bash is bloody broken conditionals. I can't see how to fix it, and I can't stand looking at bash any longer to figure it out 08:42PM|<asuffield> it passes WUNTRACED to wait() when job control is enabled, so that it can spot jobs which have been sent SIGSTOP 08:43PM|<asuffield> somewhere in the pipeline logic is a missing check for WIFSTOPPED on the status code, to see if the process is really dead yet or not 08:49PM|<asuffield> look, WSTOPCODE() and WEXITCODE() are the same macro. zsh is calling WEXITCODE() and treating it as the exit code, and this is *INCORRECT*, because WIFEXITED() is false and WIFSTOPPED() is true 08:51PM|<Keybuk> and zsh documents that it won't continue a pipeline if the process is terminated by an unhandled signal 08:52PM|<Keybuk> so zsh is being correct, just different to bash 08:52PM|<asuffield> zsh has incorrectly interpreted the result from wait() as if the process had been terminated 08:52PM|<asuffield> the process has been stopped. this is a different event
#288323#17
Date:
2005-01-03 04:15:12 UTC
From:
To:
Should zsh wait for a suspended process to exit before continuing along
the sublist?
----- Forwarded message from Branden Robinson <branden@debian.org> ----- Things like: foo && bar && baz are a basic POSIX shell feature, and Bash doesn't handle them right. Try: echo one && sleep 10 && echo two While, in the sleep, background the command with CTRL-Z. The 'echo two' will run immediately. This is wrong. The sleep has no exit status yet because it has not exited, and the && and || connectives must only be evaluated once the preceding command has exited. Until then, the command in question *has* no exit status. The box with Schroedinger's Cat in it has not yet been opened. ash, dash, pdksh, and zsh are also buggy, but instead they never run "echo two" at all. It appears that of Debian's allegedly POSIX-compliant shells have this problem, except for posh. Here are some speculations/argument from #debian-devel as to what may be going on: 08:42PM|<asuffield> Overfiend: what you have in bash is bloody broken conditionals. I can't see how to fix it, and I can't stand looking at bash any longer to figure it out 08:42PM|<asuffield> it passes WUNTRACED to wait() when job control is enabled, so that it can spot jobs which have been sent SIGSTOP 08:43PM|<asuffield> somewhere in the pipeline logic is a missing check for WIFSTOPPED on the status code, to see if the process is really dead yet or not 08:49PM|<asuffield> look, WSTOPCODE() and WEXITCODE() are the same macro. zsh is calling WEXITCODE() and treating it as the exit code, and this is *INCORRECT*, because WIFEXITED() is false and WIFSTOPPED() is true 08:51PM|<Keybuk> and zsh documents that it won't continue a pipeline if the process is terminated by an unhandled signal 08:52PM|<Keybuk> so zsh is being correct, just different to bash 08:52PM|<asuffield> zsh has incorrectly interpreted the result from wait() as if the process had been terminated 08:52PM|<asuffield> the process has been stopped. this is a different event
#288323#18
Date:
2005-01-03 18:19:34 UTC
From:
To:
Trouble is, I can't recall if it was on zsh-workers or austin-group or
somewhere else, and searching the zsh archives isn't helping me.

The short answer is, no, zsh can't wait for the suspended process to exit.

Given "one && two && three", if "two" stops, the shell has three choices:
(1) pretend the command was "{ one && two && three }" and suspend the
    entire sublist; or
(2) pretend that "two" has returned a status and continue the junction; or
(3) stop the entire shell until "two" is resumed.

Choice (1) is undesirable because it subverts the user's intent (if he
meant there to be braces, he should have typed them) and it puts "three"
into a separate process when it might better have been run in the current
shell.  Choice (3) is impossible in an interactive shell.  That leaves
(2), which is what zsh does, using the signal number as the status.

If there's a bug, it's that zsh returns $? == 20 rather than $? == 148.
I forget why that's done -- maybe it's a compromise because $? > 127 would
indicate the signal caused the child to exit, which is not the case here.

#288323#21
Date:
2005-01-03 18:19:34 UTC
From:
To:
Trouble is, I can't recall if it was on zsh-workers or austin-group or
somewhere else, and searching the zsh archives isn't helping me.

The short answer is, no, zsh can't wait for the suspended process to exit.

Given "one && two && three", if "two" stops, the shell has three choices:
(1) pretend the command was "{ one && two && three }" and suspend the
    entire sublist; or
(2) pretend that "two" has returned a status and continue the junction; or
(3) stop the entire shell until "two" is resumed.

Choice (1) is undesirable because it subverts the user's intent (if he
meant there to be braces, he should have typed them) and it puts "three"
into a separate process when it might better have been run in the current
shell.  Choice (3) is impossible in an interactive shell.  That leaves
(2), which is what zsh does, using the signal number as the status.

If there's a bug, it's that zsh returns $? == 20 rather than $? == 148.
I forget why that's done -- maybe it's a compromise because $? > 127 would
indicate the signal caused the child to exit, which is not the case here.

#288323#26
Date:
2008-12-29 20:46:07 UTC
From:
To:
Hi all,

I am triaging bugs in Debian's BTS [1] and the first two things that are
still valid are (both have been on zsh-workers, the first in 2004, the
second in 2005):

1) Possible regression/setting change[2]:
It seems that with zsh 4.07 and

autoload -U compinit
compinit -C
compinit -u
zstyle ':completion:*' menu select interactive

You were able to hit tab twice, get the menu and then use tab to cycle
through all menu options. With 4.3.6 and 4.3.9, you need to use the
cursor keys. Was that on purpose? Does the user need to set something?
At least the garbage chars seem to have disappeared.

2) Unexpected behaviour when stopping a job in a command chain[3]

Consider this:

echo one && sleep 10 && echo two

When stopping `sleep 10`, `echo two` will never be executed, no matter in
what way you revive `sleep 10`. That is OK as backgrounding `sleep 10`
will set $? to 20. Yet, with

echo one ; sleep 10 ; echo two

the same thing happens. As Bart pointed out[4]:

Personally, I think 1) would meet most users' expectations, but any of
the three are OK. Not executing the third command at all is not, imo. Of
course, if the third command is a rm, mv or some other potentially
destructive command, it's best to err on the save side, so I can see why
that was done. If that is a design decission, I will accept that and
close the bug accordingly. But keep in mind that 1) would be a save
solution, as well ;)


Richard

[1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?src=zsh
[2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=276187
[3] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=288323
[4] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=288323#18

#288323#31
Date:
2008-12-29 21:19:51 UTC
From:
To:
On Mon, Dec 29, 2008 at 09:46:07PM +0100, Richard Hartmann wrote:
[...]
[...]

That'd be a feature. All the other shells except bash behave
like that. You'll find plenty of places where people complain
about bash behavior.

#288323#36
Date:
2008-12-29 23:16:49 UTC
From:
To:
Please don't CC 276187@bugs.debian.org any more -- my fault for mixing
two bugs into one email. Won't happen again.

doing the 'right' thing and suspending {1;2;3} :
ksh93

swallowing 3 :
bash, csh, fish,

running 3 the second 2 is suspended:
ash, dash

freezing the whole shell:
Solaris 9 /bin/sh

not allowing you to suspend within a command chain at all:
ksh88

not catching ^Z and suspending themselves; when stopped from
another shell, they do:
posh :
  runs 3 the second 2 is suspended
sash :
  does the 'right' thing and suspends {1;2;3}

being so weird i had to kill the shell from outside:
scsh

Richard

#288323#41
Date:
2008-12-30 08:54:55 UTC
From:
To:
[...]

Sorry, I read too quickly, I thought you meant <Ctrl-C> instead
of <Ctrl-Z>.

#288323#46
Date:
2008-12-30 09:40:01 UTC
From:
To:
On Mon, Dec 29, 2008 at 09:46:07PM +0100, Richard Hartmann wrote:
[...]

I don't observe that with 4.3.9-dev-1-cvs1210

And I can see all the shells behaving the same. Am I missing
something again?

Well, it does return a status, but it also leaves a job stopped
in background.
[...]


When one runs:

cmd1; cmd2

he may intend that to be a little script of two commands.

If one expects <Ctrl-C> to terminate that little script (which zsh,
ksh, pdksh, ash do), one may expect (1) above as well, that is
<Ctrl-Z> to stop that little script.

On the other end, 2 makes even more sense because you'd expect

cmd1; cmd2
to be the same as
cmd1
cmd2

And according to SUSv4, those are two jobs.

According to SUSv4, a job is a shell pipeline. And:

SUSv4> pipeline         :      pipe_sequence
SUSv4>                  | Bang pipe_sequence
SUSv4>                  ;
SUSv4> pipe_sequence    :                             command
SUSv4>                  | pipe_sequence '|' linebreak command
SUSv4>                  ;
SUSv4> command          : simple_command
SUSv4>                  | compound_command
SUSv4>                  | compound_command redirect_list
SUSv4>                  | function_definition

So, in the case of { cmd1; cmd2; }, we've got one job, so one
would expect <Ctrl-Z> to stop the whole job.

But then, in "cmd1; cmd2", according to SUSv4 again, <Ctrl-C>
should only terminate cmd1 as bash does and run cmd2... so
there's a consistency problem here.

#288323#51
Date:
2008-12-30 09:42:08 UTC
From:
To:
[...]

Could you please clarify what you tried? I get different
behaviors from you whether I try

{ echo 1; sleep 10; echo 2; }
or
echo 1; sleep 10; echo 2

#288323#56
Date:
2008-12-30 15:29:09 UTC
From:
To:
stop and restart as a whole. The interesting bit is to see how
the latter is handled. Quite differently, it turns out.


Richard

PS: Are all questions in your previous mails answered? If not,
please tell me.

#288323#61
Date:
2009-01-21 16:53:01 UTC
From:
To:
Bump email

There is an actual thread below this email. See
http://www.zsh.org/mla/workers/2008/msg01850.html

I still think choice 1 meets users expectations best and is the safest
thing to do. Maybe offer this as an option?


Richard

#288323#66
Date:
2012-01-30 23:47:26 UTC
From:
To:
Still valid
#288323#71
Date:
2012-02-20 13:56:57 UTC
From:
To:
I disagree with choice 1, even though zsh behaves like that for
functions (unlike other shells), so that side effects no longer
affect the current shell when the function has been suspended.
There's at least a documentation problem.

I've reported the following bug about functions:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660630

This is the kind of problem I've observed with a suspended function.

I don't see why (3) is impossible (posh does that or ignore the ^Z,
depending on how it has been started: from another shell or directly
by the terminal). Note that the terminal can be able to send signals
to the process group of the process running under the terminal (for
instance, xterm can do that).

For ^Z, there's also:

(4) ignore the TSTP signal. Or handle it only when it is "safe".
[...]

Not if there's a side effect, such as setting a variable.