#1035734 Does not search for terms in latin1 encoding anymore

Package:
translate
Source:
translate
Submitter:
Klaus Ethgen
Date:
2023-05-08 16:15:06 UTC
Severity:
normal
Tags:
#1035734#5
Date:
2023-05-08 12:53:41 UTC
From:
To:
Version: 1.1.3
Severity: normal

I usually have latin1-Encoding everywhere. The output of translate is
good but the input does not allow latin1-umlauts like ä, ö, ü, ... I can
input them but they always returns nothing.

Older versions worked well but I cannot say, when the incompatibility
was implemented. At least version 0.6 worked well.

The strange thing is, that I see no reason for the bug. Even when I do
`bash -x /usr/bin/translate ...` it DOES iconv my input. But if I just
do `/usr/bin/translate ...`, it doesn't. This bug is a full riddle for
me.

As translate is of mayor use for me so please reallow non-UTF-8 input.
#1035734#10
Date:
2023-05-08 13:18:03 UTC
From:
To:
Hoi Axel,

I found the bug. The bug is in the color code.

The replacement of $SEARCH in the new display function can never match
for latin1 chars as it is still UTF-8. So either use the original search
string here or convert it back to local locale.

Gruß
   Klaus
#1035734#15
Date:
2023-05-08 16:03:59 UTC
From:
To:
Hi Mowgli,

Klaus Ethgen wrote:

Interesting, thanks for the bug report.
UTF-8 related changes I see in the git log seemed to have been in
0.6-6 and 0.6-7. Which actually were long ago (2005).

Just a thought: Could it be that this change in Debian's locales could
have caused some unexpected side effect on already configured
non-UTF-8 locales?

  locales (2.31-14) unstable; urgency=low

    * Starting with locales 2.31-14, non UTF-8 locales are deprecated and not
      offered anymore in the debconf dialog, except for the ones already
      configured. Nevertheless users of non UTF-8 locales are encouraged to
      switch their system to an UTF-8 locale.

      Please note that iconv still supports conversion to and from non UTF-8
      charset. For instance reading a file using an ISO-8859-15 charset can be
      done with: iconv --from-code=ISO-8859-15 foobar.txt
[…]

Or could this LC_CTYPE with C.UTF-8 be the reason? Because this would
likely cause $UTF8 to be set in the script:

  UTF8=0
  if locale 2>&1 | grep -E -q "UTF-8?$"; then
      UTF8=1
  fi

Actually I just now noticed a typo in that regexp. It likely should be
"UTF-?8$" and not "UTF-8?$" (i.e. the dash is optional, not the digit
eight). That typo is now fixed in git. (I though don't think that this
typo caused your issue. It could have caused issues the other way
around: Not properly detected UTF-8 locales.)

		Regards, Axel
#1035734#22
Date:
2023-05-08 16:11:34 UTC
From:
To:
Hi,

Klaus Ethgen wrote:

Great. Sorry for my just sent reply then. Saw your second mail only
after sending it.

Oh, ok, that's probably my code then, yes.

Ok, will have a closer look. Don't see the right place for a fix on a
first glance.

		Regards, Axel