--- Please enter the report below this line. ---

Floating point arithmetic no longer works in ksh. Here's how it used to
work:

... and this is how it works now:

The last version which actually supported FP arithmetic was
93u+20120801-1 in Debian 8.
The changelog for Debian 9 lists 4 more updates (93u+20120801-1.1,
93u+20120801-1.2, 93u+20120801-2 and 93u+20120801-3.1), but none of them
mentions any breaking changes.

Debian Release: 9.3
500 stable-updates ftp.ru.debian.org
500 stable security.debian.org
500 stable ftp.ru.debian.org
100 stretch-backports ftp.ru.debian.org
--- Package information. ---
Depends (Version) | Installed
=============================-+-===========
libc6 (>= 2.23) |
binfmt-support |


Package's Recommends field is empty.

Package's Suggests field is empty.

Hello,

The issue is actually locale-dependent:

ksh infers the decimal separator from LC_NUMERIC, and since my locale is
ru_RU.UTF-8,
it expects a comma (,) instead of a dot (.).

Strange thing is, LC_NUMERIC affects neither zsh nor bc -- only ksh.

I believe this behaviour should be at least documented somewhere.

Regards,
Andrey.

Control: retitle -1 ksh: parsing of floating-point constants is broken in locales where the decimal point is not "."

There is documentation, but it does not match the observed behavior.
The man page says under "Arithmetic Evaluation.":

  "Floating point constants follow the ANSI-C programming language
  floating point conventions."

In C, like in many programming languages, the locales do not influence
the syntax of literals, e.g. "1." is the floating-point constant 1
whatever the locale.

However, in CAVEATS, it says:

  "It is a good idea to leave a space after the comma operator in
  arithmetic expressions to prevent the comma from being interpreted
  as the decimal point character in certain locales."

but that's no the main documentation.

I'd rather say that this is a bug in the code (otherwise, hardcoded
constants in scripts would be an issue, in particular), which blindly
use standard library functions in the current locale.

This would also solve the current inconsistency with Turkish locales:

cventin:~> LC_ALL=tr_TR.utf8 ksh
$ echo $((Inf))
inf
$ echo $((ınf))
0

while the lowercase form of "Inf" is "ınf", not "inf", i.e. ksh does
not seem to honor the current locale for infinity.

Thanks for the analysis.

Checking the current state on ksh93u+m (1.0.10-5), the original arithmetic
syntax error is gone, and a '.'  literal in a comma-radix locale produces
an explicit message:

+ ksh -c 'echo $((1./2))'
+ LC_ALL=C
0.5
+ ksh -c 'echo $((1./2))'
+ LC_ALL=de_DE.UTF-8
ksh: 1./2: radix point '.' requires LC_NUMERIC=C
+ ksh -c 'echo $((1,0/2,0))'
+ LC_ALL=de_DE.UTF-8
0,5
+ ksh -c 'echo $((1./2))'
+ LANG=de_DE.UTF-8
+ LC_NUMERIC=C
0.5
+ ksh -c 'echo $((Inf))'
+ LC_ALL=tr_TR.UTF-8
inf
+ ksh -c 'echo $((inf))'
+ LC_ALL=tr_TR.UTF-8
inf
+ ksh -c 'echo $((ınf))'
+ LC_ALL=tr_TR.UTF-8
0

The locale-dependent radix point is intentional and is covered by the
test suite. The workaround is LC_NUMERIC=C.

On the Turkish Inf/NaN point, that recognition happens through ASCII and
is not dependent on LC_NUMERIC and so 'ınf' is not matched by design.
What remains however is a valid documentation gap for which this patch
will be forwarded upstream:
--- a/src/cmd/ksh93/sh.1
+++ b/src/cmd/ksh93/sh.1
@@ -3252,8 +3252,13 @@ double precision floating point
 arithmetic or long double precision floating point for
 systems that provide this data type.
 Floating point constants follow the ANSI C programming language
-floating point conventions.
-The case-insensitive floating point constants
+floating point conventions, except that the radix point character
+is determined by the
+.SM LC_NUMERIC
+locale category; set
+.B LC_NUMERIC=C
+to use a period.
+The ASCII case-insensitive floating point constants
 .B NaN
 and
 .B Inf

 Anuradha

IMHO, this is bad. The interpretation of literals in a source file
(here, a ksh script) should not depend on the current locales.

Agreed with you in principle. To be clear though, this isn't new
or a regression and it has been parsed through the locale-aware
strtold() historically unfortunately, and so changing it would have
backward compatibility concerns.

However, a case could be made whether such behavior could make it
into a portable opt-in without changing the legacy default (POSIX
itself is silent, since shell floating point arithmetic is a ksh
extension), but that would be a discussion that would need to be
had upstream. If you're willing to advocate a case for that, would
you mind opening an enhancement request on
https://github.com/ksh93/ksh/issues?

Also, let me know if you have any concerns with the documentation
fix before I forward it upstream.

thanks,

Anuradha

The bug reporter Andrey ``Bass'' Shcheglov said that there was
a change between Debian 8 and Debian 9. But perhaps he changed
his locale?

Note that this also affects the comma operator when the decimal-point
chracter is the comma (as noted at the end of the ksh93(1) man page,
but a bit incorrectly). However, the comma operator is not specified
by POSIX and is an extension in shells except the basic ones.

See also:
https://www.mail-archive.com/austin-group-l@opengroup.org/msg14429.html

At https://github.com/ksh93/ksh/issues/999 I've mentioned 2 issues
with the ksh93(1) man page:

* "Floating point constants follow the ANSI C programming language
  floating point conventions." This is not correct since C
  floating-point constants do not depend on the current locale.

* "It is a good idea to leave a space after the comma operator in
  arithmetic expressions to prevent the comma from being interpreted
  as the decimal-point character in certain locales." But a space
  *before* the comma operator may also be needed:

$ for s in "1,2" "1, 2" " 1 ,2" "1 , 2" ; do LC_ALL=fr_FR.utf8 ksh93 -c "echo \$(( $s ))" ; done
1,2
ksh93:  1, 2 : arithmetic syntax error
ksh93:   1 ,2 : arithmetic syntax error
2