#879205 MIME::Words::encode_mimewords: double-encodes (produces Mojibake), produces too long lines

Package:
libmime-tools-perl
Source:
libmime-tools-perl
Submitter:
Thorsten Glaser
Date:
2017-10-21 00:03:03 UTC
Severity:
normal
#879205#5
Date:
2017-10-20 12:56:01 UTC
From:
To:
Perhaps in a misguided attempt to fix #879204, this happens:

(sid-amd64)tglase@tglase:~ $ perl -MEncode -MMIME::Words -e 'print MIME::Words::encode_mimewords(Encode::encode("UTF-8", "Re: Bildungsurlaub für CCC-Fahrt? [THD#1424195]"), Charset => "UTF-8", Field => "Subject") . "\n";'
Re: Bildungsurlaub =?UTF-8?Q?f=C3=83=C2=BCr=20?=CCC-Fahrt? [THD#1424195]

Not only decodes this to…
	Re: Bildungsurlaub für  CCC-Fahrt? [THD#1424195]
… but it’s also too long *and* has no space between the encoded MIME part
and the next word (“CCC-Fahrt?”), which might cause trouble in strict
readers.

As in #879204, expected output is something along these lines:

tglase@tglase:~ $ php
<?php
mb_internal_encoding('UTF-8');
echo mb_encode_mimeheader('Subject: Re: Bildungsurlaub für CCC-Fahrt? [THD#1424195]', 'UTF-8', 'Q', "\015\012") . "\n";
Subject: Re: Bildungsurlaub =?UTF-8?Q?f=C3=BCr=20CCC-Fahrt=3F=20=5BTHD=23?=
 =?UTF-8?Q?=31=34=32=34=31=39=35=5D?=

Similar to #787512 and #787513 it saddens me it takes PHP to
provide an example of correct output :|
#879205#12
Date:
2017-10-20 13:31:27 UTC
From:
To:
Right, forwarded upstream as
https://rt.cpan.org/Ticket/Display.html?id=123335


Cheers,
gregor
#879205#17
Date:
2017-10-20 20:19:42 UTC
From:
To:
gregor herrmann dixit:


Dianne Skoll dixit:
[…]

Hi, I believe your test program is not correct.

perl -MEncode -MMIME::Words -e 'print MIME::Words::encode_mimewords(Encode::encode("UTF-8", "Re: Bildungsurlaub für CCC-Fahrt? [THD#1424195]"), Charset => "UTF-8", Field => "Subject") . "\n";'

This will do it. Alternatively (semi-tested) with yours:
my $sample = "Re: Bildungsurlaub f\x{FC}r CCC-Fahrt? [THD#1424195]";

You were missing the “f” and “r” there. This is extremely sensitive
to context.

This looks to me as if the old code (with the bug from Debian
#879204) was called, *then* things are re-read and re-encoded.

bye,
//mirabilos
#879205#22
Date:
2017-10-20 21:54:19 UTC
From:
To:
Dianne Skoll via RT dixit:

Hm, probably. I’d say you just found a bug in OTRS then ;-)
(Just now it’s going to be another tricky thing to figure
out where exactly and how to fix that. Might report this
to the OTRS developers.)

One thing I don’t understand is how this was *not* double-
encoded in the old version of MIME tools?

Thanks,
//mirabilos
#879205#27
Date:
2017-10-20 23:59:29 UTC
From:
To:
Dianne Skoll via RT dixit:

Yes, that was on Debian wheezy. I reported this as bug in Debian
against wheezy (which is still supported-ish) first, then as a
separate bug against sid because I tried to see if it was still
reproducible, and got a different result.

Let me dig out version numbers…

Original system:
otrs2			3.3.18-1~deb7u1
perl    		5.14.2-21+deb7u5
libmime-tools-perl	5.503-1

New system:
perl			5.26.0-8
libmime-tools-perl	5.508-1

In addition to that, OTRS would have the original subject in
a Perl string already, whereas I tried¹ to draft a testcase
until I succeeded reproducing the original bug.

① tried, because I don’t really know Perl — I just can program

bye,
//mirabilos