Hello dear perl team,

this is a weird one. It appeared while debugging adduser for the
Installer; the Installer people can reproduce that with full adduser as
well. I am presenting here a stripped down example of the issue.

tl;dr: die() hangs with high CPU usage in a certain environment with a
lot of preparation.

I tried this in a debspawn container, but the Installer people,
especially Pascal Hambourg, can see that in their test environments as
well.

Steps to reproduce:

debspawn create --suite sid sid
debspawn login sid
(this is not reproducible on my full system, it must have something to
do with the minimal package set in the container)
copy the attached adduser and AdduserCommon.pm in the container
in the container:
apt purge adduser (just to make sure)
change to the directory where adduser and AdduserCommon.pm were copied
to
mkdir Debian
mv AdduserCommon.pm Debian
export LANG=C.UTF-8
perl -T -I. ./adduser --comment="ÓæÓæßéÀ"
(see it work)
export LANG=C
pull the adduser command line from shell history; the weird characters
will show as some \octals.
see it hang after trace point 4.

This is a minimal reproducer, as minimal as it was possible for me. The
following aspects ARE needed for the bug to show:

- LANG=C
- the weird string must be handed in via command line option, setting the
   string inside the code makes the bug go away
- sanitize_string must be in a module in the Debian namespace, moving the
   function to the main program or moving Debian::AdduserCommon to
   AdduserCommon without the Debian prefix makes the bug go away
- the codeset and charset gymnastics must be present, not setting
   STDERR's binmode makes the bug go away

Control: tag -1 confirmed upstream

Thanks for the report.

I got it down to

  % cat 1104726.pl
  #!/usr/bin/perl -w
  binmode(STDERR, ":encoding(ascii)");
  warn("\x{c3}\x{93}" x 9);

It's quite a heisenbug in that it doesn't hang for me with a shorter
script filename than 10 characters, and it's sensitive to the multiplier
(7 is enough for 5.40.1 in sid, but 9 is required for 5.36.0 in bullseye.)

The encoding can also be "C" or "ANSI_X3.4-1968" (as acquired via
langinfo(I18N::Langinfo->CODESET) in your original reproducer)
and it still hangs.

It happens with current upstream version (bleadperl), and bisecting
one version shows it regressed with

  83c555564918110c20f99f55c77d9dfc27f217fb is the first bad commit
  commit 83c555564918110c20f99f55c77d9dfc27f217fb
  Author: Father Chrysostomos <sprout@cpan.org>
  Date:   Thu Sep 16 14:07:50 2010 +0200

      Fix warn to respect utf8-encoded scalars [perl #45549]

   t/op/warn.t | 26 +++++++++++++++++++++++++-
   util.c      |  4 +---
   2 files changed, 26 insertions(+), 4 deletions(-)

which is quite plausible. I'll report it upstream next.

It seems to me that it's trying to warn about the characters, and then
tripping over encoding them in stderr. It might be a case of "don't do
that, then", but the failure mode is rather awkward.

It also happens with a -DDEBUGGING Perl (including /usr/bin/debugperl
in Debian.)  For reference, below is a backtrace from current upstream
version built with -DDEBUGGING -Doptimize="-g -O0" when interrupted
manually with Ctrl-C.

  Program received signal SIGINT, Interrupt.
  PerlIOBuf_write (f=0x555555c39550, vbuf=0x555555c77370, count=22) at perlio.c:4580
  4580	        PerlIOBase(f)->flags |= PERLIO_F_WRBUF;
  (gdb) bt
  #0  PerlIOBuf_write (f=0x555555c39550, vbuf=0x555555c77370, count=22) at perlio.c:4580
  #1  0x00007ffff7c3bba8 in PerlIOEncode_write (f=0x555555c39550, vbuf=0x555555c77370, count=75)
      at encoding.xs:607
  #2  0x00005555556b38e4 in Perl_PerlIO_write (f=0x555555c39550, vbuf=0x555555c77370, count=75) at perlio.c:2010
  #3  0x000055555594019f in Perl_do_print (sv=0x555555c5afc0, fp=0x555555c39550) at doio.c:2260
  #4  0x00005555559271a5 in Perl_write_to_stderr (msv=0x555555c5afc0) at util.c:1692
  #5  0x0000555555928388 in Perl_vwarn (pat=0x7ffff7c2ba08 "\"\\x{%04lx}\" does not map to %s",
      args=0x7fffffffd470) at util.c:2054
  #6  0x0000555555928ad3 in Perl_vwarner (err=44, pat=0x7ffff7c2ba08 "\"\\x{%04lx}\" does not map to %s",
      args=0x7fffffffd470) at util.c:2233
  #7  0x000055555592876f in Perl_warner (err=44, pat=0x7ffff7c2ba08 "\"\\x{%04lx}\" does not map to %s")
      at util.c:2218
  #8  0x00007ffff7c22c44 in encode_method (enc=0x7ffff7c30bc0 <ascii_encoding>, dir=0x7ffff7c30500 <utf8_ascii>,
      src=0x555555c3daa0,
      s=0x555555c69798 "\303\203\302\223 at /home/ntyni/1104726/1104726.pl line 3.\n\"\\x{00c3}\" does not map to ascii at /home/ntyni/1104726/1104726.pl line 3.\n\"\\x{0093}\" does not map to ascii at /home/ntyni/1104726/1104726.pl line 3.\n\"\\x"..., slen=0, check=2322, offset=0x0, term=0x0, retcode=0x0,
      fallback_cb=0x555555c1a558 <PL_sv_immortals+24>) at Encode.xs:244
  #9  0x00007ffff7c27d00 in XS_Encode__XS_encode (cv=0x555555c81180) at Encode.xs:876
  #10 0x0000555555722deb in Perl_rpp_invoke_xs (cv=0x555555c81180) at inline.h:1176
  #11 0x0000555555742fbe in Perl_pp_entersub () at pp_hot.c:6540
  #12 0x0000555555629f47 in Perl_runops_debug () at dump.c:3003
  #13 0x00005555555e9112 in Perl_call_sv (sv=0x555555c3db78, arg_flags=130) at perl.c:3235
  #14 0x00005555555e8c7a in Perl_call_method (methname=0x7ffff7c3cfa0 "encode", flags=2) at perl.c:3132
  #15 0x00007ffff7c3a17e in PerlIOEncode_flush (f=0x555555c39550) at encoding.xs:429
  #16 0x00005555556b3a60 in Perl_PerlIO_flush (f=0x555555c39550) at perlio.c:2033
  #17 0x00005555556b7bff in PerlIOBuf_write (f=0x555555c39550, vbuf=0x555555c652e0, count=0) at perlio.c:4588
  #18 0x00007ffff7c3bba8 in PerlIOEncode_write (f=0x555555c39550, vbuf=0x555555c652e0, count=71)
      at encoding.xs:607
  #19 0x00005555556b38e4 in Perl_PerlIO_write (f=0x555555c39550, vbuf=0x555555c652e0, count=71) at perlio.c:2010
  #20 0x000055555594019f in Perl_do_print (sv=0x555555c247f8, fp=0x555555c39550) at doio.c:2260
  #21 0x00005555559271a5 in Perl_write_to_stderr (msv=0x555555c247f8) at util.c:1692
  #22 0x0000555555928314 in Perl_warn_sv (baseex=0x555555c47100) at util.c:2032
  #23 0x0000555555776f8e in Perl_pp_warn () at pp_sys.c:580
  #24 0x0000555555629f47 in Perl_runops_debug () at dump.c:3003
  #25 0x00005555555e7bd8 in S_run_body (oldscope=1) at perl.c:2883
  #26 0x00005555555e7267 in perl_run (my_perl=0x555555c222d0) at perl.c:2798
  #27 0x000055555559f142 in main (argc=3, argv=0x7fffffffe538, env=0x7fffffffe558) at perlmain.c:127

Control: forwarded -1 https://github.com/Perl/perl5/issues/14362

and further to

 perl -we 'binmode(STDERR, ":encoding(us-ascii)"); warn "\x{ff}" x 100'

It turns out this (or something very much like it) was already reported
upstream at https://github.com/Perl/perl5/issues/14362 .

I've added information to the upstream issue, but I don't expect a
quick resolution.

I had worked around in adduser anyway, I think the error message is a
bit less helpful now, but *shrug*

I really really appreciate the way you handled this report. It gave me
the impression that the hours I spent (on Minidebconf Hamburg) for a
not quite minimal reproducer were well spent. Thank you for your work!

Greetings
Marc