Dear Maintainer,

I noticed that how-can-i-help fails to display properly bug titles when
they contain non-ASCII characters:



Best,

  nicoo

Hey,

Those non-ASCII characters are getting converted by a script running on
UDD [1] (UDD is a data source for how-can-i-help). Since it is a
deliberate action (force_ascii [2]), I'll ask the original author what
was the reasoning behind it and if I can change that.

Interesting thing - UDD json output also seems to have some issues with
non-ASCII characters [3].

Thanks for the report! :)


Regards,
T. Nitecki

[1] https://wiki.debian.org/UltimateDebianDatabase
[2]
https://anonscm.debian.org/cgit/collab-qa/udd.git/tree/scripts/generate-how-can-i-help
[3]
https://udd.debian.org/bugs/?release=stretch&merged=ign&fnewerval=7&flastmodval=7&dmd=1&packages=piuparts&sortby=id&sorto=asc&format=json#results

Hey Lucas,

Recently, Nicolas filled a bug against how-can-i-help pointing out that
we convert all the non-ASCII characters into '?'. Looking at the UDD
script [1], I can tell that we are performing this conversion
intentionally (force_ascii method) on most title and descriptions
fields. I have two questions:

1. Why was this forced conversion added? To sanitize output and to
prevent formating errors?
2. Shouldn't it be applied to all title and description fields?


Regards,
T.

[1]
https://anonscm.debian.org/cgit/collab-qa/udd.git/tree/scripts/generate-how-can-i-help

Hi,

Yes. The bugs data contains lots of non-UTF-8 characters.

yes probably.

Maybe it should be modified to force-convert to valid UTF-8 instead of
ASCII though.

Lucas