#850202 dash: test with multiple conditions cannot handle variable whose expanded value is same as an operator

Package:
dash
Source:
dash
Description:
POSIX-compliant shell
Submitter:
J G Miller
Date:
2023-01-05 13:21:15 UTC
Severity:
normal
Tags:
#850202#5
Date:
2017-01-04 23:53:57 UTC
From:
To:
In Bourne shell script with dash, using either [ ] or test,

   test -n "${variable}" && echo "variable is set to -->${variable}<--"

works as expected even if the value of variable is to a value the same
as a test operator eg -ne or -nt

   variable="-ne" ; test -n "${variable}" && echo "variable is set to -->${variable}<--"
   variable is set to -->-ne<--

However the addition of another test condition with -a or -o results in an
unexpected behavior in that the expanded variable is now treated as an operator
and not just as the string as should be the case.

    variable="-ne" ; test -n "${variable}" -a -n "${DISPLAY}" && echo "variable is set to -->${variable}<--"
    dash: 1: test: Illegal number: -n

Applying quoted parentheses makes no difference.

    variable="-ne" ; test \( -n "${variable}" \) -a \( -n "${DISPLAY}" \) && echo "variable is set to -->${variable}<--"
dash: 2: test: Illegal number: -n

Similarly bad behavior is seen with a variable set to "-nt", which is how I stumbled upon this problem.

    variable="-nt" ; test \( -n "${variable}" \) -a \( -n "${DISPLAY}" \) && echo "variable is set to -->${variable}<--"
dash: 3: test: closing paren expected

So if the test is consists of a single condtion, the expanded variable does not cause
problems with the test, and should not affect the test if it is part of a multiple condition test.

Incidentally bash also has this problem.

So the short term kludge in a shell script is to do

     if [ -n "${variable_with_problem_value}" ]
     then
          if [ \( other_condition1 \) -a \( other_condition2 \) ]
          then
               ...

#850202#10
Date:
2022-12-17 14:57:28 UTC
From:
To:
Hi!

Yes, the argument after -n is literal. As-expected so far.
Notably, -a, -o, (, and ) are all killed in POSIX Issue 8 (Draft 2.1),
via Austin Group Defect 1330:
https://www.austingroupbugs.net/view.php?id=1330
They were marked obsolescent and optional in Issue 7 and earlier
(because they're awful and underspecified, you should never use them).

There are a few ways of parsing this:
if you're using a full-blown per-spec precedence-obeying parser,
like voreutils test or ksh, then you'll parse it like
  ( -n '-ne' ) && ( -n ':0' )
which resolves to true:
  $ ~/code/voreutils/out/cmd/test -n -ne -a -n :0; echo $?
  0
  $ ksh -c 'test -n -ne -a -n :0; echo $?'
  0
(well, idk if ksh uses a parse tree, but it produces the same result
that voreutils test does, and that does use one).

If you parse it in compatibility mode, like dash, bash,
and GNU coreutils test, then you'll parse it like
  ( '-n' -ne '-a' ) {garbage}
which is illegal:
  $ bash -c 'test -n -ne -a -n :0'
  bash: line 1: test: -n: integer expression expected
  $ test -n -ne -a -n :0
  ./dash: 1: test: Illegal number: -n
  $ /bin/test -n -ne -a -n :0
  /bin/test: invalid integer ‘-n’
(if you make the -ne arguments integers the garbage -n :0 makes it
 illegal anyway, and all of the above agree).
This is a much more interesting case: the obvious parse here is, indeed,
  ( -n '-ne' ) && ( -n ':0' )
and GNU coreutils test, voreutils test, and ksh are in agreement now:
  $ /bin/test '(' -n -ne ')' -a '(' -n :0 ')'; echo $?
  0
  $ ~/code/voreutils/out/cmd/test '(' -n -ne ')' -a '(' -n :0 ')'; echo
  $?
  0
  $ ksh -c "test '(' -n -ne ')' -a '(' -n :0 ')'; echo \$?"
  0

Nevertheless, bash and dash still parse this differently:
  $ test '(' -n -ne ')' -a '(' -n :0 ')'; echo $?
  ./dash: 9: test: Illegal number: -n
  2
  $ bash -c "test '(' -n -ne ')' -a '(' -n :0 ')'; echo \$?"
  bash: line 1: test: -n: integer expression expected
  2
by substituting the first -n for 0, then the first ) for 0,
then adding a closing paren, yielding:
  $ test '(' 0 -ne ')' -a '(' -n :0 ')'; echo $?
  ./dash: 16: test: Illegal number: )
  2
  $ bash -c "test '(' 0 -ne ')' -a '(' -n :0 ')'; echo \$?"
  bash: line 1: test: ): integer expression expected
  2
  $ test '(' 0 -ne 0 -a '(' -n :0 ')'; echo $?
  ./dash: 18: test: closing paren expected
  2
  $ bash -c "test '(' 0 -ne 0 -a '(' -n :0 ')'; echo \$?"
  bash: line 1: test: `)' expected
  2
  $ test '(' 0 -ne 0 -a '(' -n :0 ')' ')'; echo $?
  1
  $ bash -c "test '(' 0 -ne 0 -a '(' -n :0 ')' ')'; echo \$?"
  1
we can see that they parse it as
  ( ( '-n' -ne ')' ) && ( -n ':0' )
which is unbalanced and puts -n and ) in integer spots.

Similarly, this is test ( -n -nt ) -a ( -n :0 ), which, again,
would make most sense to be
  ( -n '-nt' ) -a ( -n ':0' )
and, similarly, the same suspects agree:
  $ /bin/test '(' -n -nt ')' -a '(' -n :0 ')'; echo $?
  0
  $ ~/code/voreutils/out/cmd/test '(' -n -nt ')' -a '(' -n :0 ')'; echo
  $?
  0
  $ ksh -c "test '(' -n -nt ')' -a '(' -n :0 ')'; echo \$?"
  0
and disagree:
  $ test '(' -n -nt ')' -a '(' -n :0 ')'; echo $?
  ./dash: 26: test: closing paren expected
  2
  $ bash -c "test '(' -n -nt ')' -a '(' -n :0 ')'; echo \$?"
  bash: line 1: test: `)' expected
  2
and through the same reduction process:
  $ test '(' -n -nt ')' -a '(' -n :0 ')' ')'; echo $?
  1
  $ bash -c "test '(' -n -nt ')' -a '(' -n :0 ')' ')'; echo \$?"
  1
we arrive at the same parse:
  ( ( '-n' -nt ')' ) && ( -n ':0' )
which is similarly unbalanced,
and strace confirms they both stat 0 and ).

The correct solution is
  if [ -n "$variable" ] && [ cond ] && [ cond ]
  then
which is, since Issue 8, the only solution, and before Issue 7,
the only one that wasn't a damned underspecified mine-field.

IOW Just Don't Put Multiple Conditions In Test(1);
quoth Issue 7, XCU, test, APPLICATION USAGE:
  The XSI extensions specifying the -a and -o binary primaries and the
  '(' and ')' operators have been marked obsolescent. (Many expressions
  using them are ambiguously defined by the grammar depending on the
  specific expressions being evaluated.) Scripts using these expressions
  should be converted to the forms given below. Even though many
  implementations will continue to support these obsolescent forms,
  scripts should be extremely careful when dealing with user-supplied
  input that could be confused with these and other primaries and
  operators. Unless the application developer knows all the cases that
  produce input to the script, invocations like:
    test "$1" -a "$2"
  should be written as:
    test "$1" && test "$2"
  to avoid problems if a user supplied values such as $1 set to '!' and
  $2 set to the null string. That is, in cases where maximal portability
  is of concern, replace:
    test expr1 -a expr2
  with:
    test expr1 && test expr2
  and replace:
    test expr1 -o expr2
  with:
    test expr1 || test expr2
  but note that, in test, -a has higher precedence than -o while "&&"
  and "||" have equal precedence in the shell.

  Parentheses or braces can be used in the shell command language to
  effect grouping.

  Parentheses must be escaped when using sh; for example:
    test \( expr1 -a expr2 \) -o expr3

  This command is not always portable even on XSI-conformant systems
  depending on the expressions specified by expr1, expr2, and expr3. The
  following form can be used instead:
    ( test expr1 && test expr2 ) || test expr3

Best,
наб

#850202#17
Date:
2023-01-05 13:19:14 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
dash, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 850202@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Andrej Shadura <andrewsh@debian.org> (supplier of updated dash package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Thu, 05 Jan 2023 14:06:02 +0100
Source: dash
Architecture: source
Version: 0.5.12-1
Distribution: unstable
Urgency: medium
Maintainer: Andrej Shadura <andrewsh@debian.org>
Changed-By: Andrej Shadura <andrewsh@debian.org>
Closes: 558607 819829 850202 975326 1017531 1024635
Changes:
 dash (0.5.12-1) unstable; urgency=medium
 .
   * New upstream release (Closes: #1017531, #1024635).
   * Refresh patches.
   * Apply upstream patches for hash, ulimit and manpages
     (Closes: #558607, #819829, #850202, #975326).
Checksums-Sha1:
 97cd1d5275fe8d1f81c036d7c998133674c07f25 1520 dash_0.5.12-1.dsc
 e15444a93853f693774df003f87d9040ab600a5e 246054 dash_0.5.12.orig.tar.gz
 1dd0eee42af5fd7ead7de85a79a9a8699bc5bc13 38520 dash_0.5.12-1.debian.tar.xz
Checksums-Sha256:
 0861850b279fdcd8ff6740e4813896f2ad20365b5995ef175e7d18f6122633c1 1520 dash_0.5.12-1.dsc
 6a474ac46e8b0b32916c4c60df694c82058d3297d8b385b74508030ca4a8f28a 246054 dash_0.5.12.orig.tar.gz
 b022cf6c89c9312fe57e9a767390f022b4dbedb5f6541c06f6a8b20187a3da61 38520 dash_0.5.12-1.debian.tar.xz
Files:
 0f4256c25db24c637966edb041f35fac 1520 shells optional dash_0.5.12-1.dsc
 57222b768b84003ea4b801e5d5e0e52b 246054 shells optional dash_0.5.12.orig.tar.gz
 4b932e195cf79623b4895b3f9a3ae0ff 38520 shells optional dash_0.5.12-1.debian.tar.xz
-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQSD3NF/RLIsyDZW7aHoRGtKyMdyYQUCY7bLcgAKCRDoRGtKyMdy
Ybj4AQDGyhSiHNJ+NT4Tq3TN1OImwvvVhfbyPaaUpOiSq19v3AD/YdpLQHB6Ju7W
/UrNXSUeUsP4ijKpbWe4JYF37hn18Aw=
=uLUG
-----END PGP SIGNATURE-----