- Package:
- www.debian.org
- Source:
- www.debian.org
- Submitter:
- Paulo Henrique Santana
- Date:
- 2025-04-30 07:21:02 UTC
- Severity:
- normal
Hi all,
Adriano and I were searching why some po files haven't been listed on brazilian status pages for po-debonf [1] and for regular po [2] (in this case, the files are not listed on the first table) and we found this:
On gen-files.pl [3] there is an If, and the last part
($team eq "debian-l10n-$LanguageList{$lang} at lists dot debian dot org")
is comparing $team with "debian-l10n-portuguese at lists dot debian dot org".
But some of our po files have breaking lines on the address, like these:
"Language-Team: Brazilian Portuguese <debian-l10n-"
"portuguese@lists.debian.org>\n"
or
"Language-Team: Brazilian Portuguese <debian-l10n-portuguese@lists.debian."
"org>\n"
So, the "eq" is not matching because the $team variable hasn't the full address of team mailing list address.
We can change "Brazilian Portuguese" for other name from now on, but we have the issue with the po files already updated.
Could someone help to improve the code for $team variable get the line after the break line, or any other idea for the comparison part?
[1] https://www.debian.org/international/l10n/po-debconf/pt_BR
[2] https://www.debian.org/international/l10n/po/pt_BR
[3] https://salsa.debian.org/webmaster-team/webwml/-/blob/master/english/international/l10n/scripts/gen-files.pl?ref_type=heads#L331
Best regards,
Hi, More details about the bug. If you look this page, you will see the first table listing some packages: https://www.debian.org/international/l10n/po/pt_BR Looking some of their po files, we can see the name and the address of the team on the same line: https://sources.debian.org/src/shadow/1%3A4.17.3-1/po/pt_BR.po/ https://sources.debian.org/src/xscreensaver/6.08%2Bdfsg1-1/po/pt_BR.po/ https://sources.debian.org/src/pppconfig/2.3.30/po/pt_BR.po/ "Language-Team: Debian-BR Project <debian-l10n-portuguese@lists.debian.org>\n" There are other variations: "Language-Team: Debian-BR <debian-l10n-portuguese@lists.debian.org>\n" "Language-Team: l10n portuguese <debian-l10n-portuguese@lists.debian.org>\n" As I said before, to build the first table, the script compares the address of the team with this field Language-Team. When we look other packages on the second table, we can see the line has more than 80 collums and the address is broken: https://sources.debian.org/src/cwidget/0.5.18-6/po/pt_BR.po/ "Language-Team: Brazilian Portuguese <debian-l10n-portuguese@lists.debian." "org>\n" https://sources.debian.org/src/debian-security-support/1%3A13%2B2025.01.30/po/pt_BR.po/ "Language-Team: Brazilian Portuguese <debian-l10n-portuguese@lists." "debian.org>\n" On the second table, you can see the collum "Equipe" is empty for these cases. So, the package was translated debian-l10n-portuguese@lists.debian.org team but the script can't deal with broken lines. For the po debconf files, we have similar issue. There are around 80 packages are not listed on this page: https://www.debian.org/international/l10n/po-debconf/pt_BR Some of them: https://sources.debian.org/src/anna/1.96/debian/po/pt_BR.po/ https://sources.debian.org/src/apt-setup/1%3A0.192/debian/po/pt_BR.po/ https://sources.debian.org/src/base-installer/1.222/debian/po/pt_BR.po/ "Language-Team: Brazilian Portuguese <debian-l10n-portuguese@lists.debian." "org>\n" Ideally, we would like the script gets the second line when the address is broken. Or maybe consider only the first part of the address until the @: debian-l10n-portuguese https://salsa.debian.org/webmaster-team/webwml/-/blob/master/english/international/l10n/scripts/gen-files.pl?ref_type=heads#L331 Best regards,
A possible approach would be, to convert all po files before processing, so they don't have line breaks at ~79 characters, as it is common. Instead, they would have all msgid's/msgstr's on one line. That would solve your issue here, I think. I have attached two files, to show the results. [[ Attention: be aware, your mail client may break the view of the files! Prefer to view it in a plain text editor and on a device, that can show very long lines! ]] That can be done with msgattrib: To convert a po file with lines at max 79 chars into 'long lines' which have the msgid's/msgstr's completely on one line: msgattrib --no-wrap -o po-file_without-line-break.po po-file-with-line-break-at-79-chars.po To convert a file with 'long lines' into a file with line breaks at 79 chars: msgattrib -o po-file-with-line-break-at-79-chars.po po-file_without-line-break.po However, since the po files in all the debian packages are in "line-break at 79 chars" mode, you would have to convert them all to the "without line-breaks" mode before this script does its work. And after the script run, you would have to convert the po files back to the "line-break at 79 chars" mode, since this is what translators are expecting. Thus, that means a significant change to this script, which is already heavily complicated as it is now... And it takes a long time to run, since it processes all po files in all debian packages!!! Think about the numbers here: bookworm had more than ~59000 packages in total! Holger
Hi, I don't speak perl, but maybe something like changing: https://salsa.debian.org/webmaster-team/webwml/-/blob/master/english/international/l10n/scripts/transmonitor-check?ref_type=heads#L703 from: open(PO, "< $PO_DIR/$filename"); to: open(PO, "-| msgattrib --no-wrap $PO_DIR/$filename"); could do the trick?
Hello.
The problem is that the regular expression in "transmonitor-check" can't match multiple lines. To fix that you have to use the "/s" modifier. But now "(.*)" will match everything after "Language-Team:" until the last "\n", that is, until the end of the po file header. To fix that you have to use a question mark, in which case ".*" will make the shortest match. You also have to remove quotes and line breaks from the language team address with `$langteam =~ s/"\n"//g`.
I used this for testing.
----------
use strict;
use warnings;
my $langteam = "";
my @headers;
$headers[0] = '"Project-Id-Version: project\n"
"Language-Team: Brazilian Portuguese <debian-l10n-portuguese@lists.debian.org>\n"
"Content-Type: text/plain; charset=UTF-8\n"';
$headers[1] = '"Project-Id-Version: project\n"
"Language-Team: Brazilian Portuguese <debian-l10n-"
"portuguese@lists.debian.org>\n"
"Content-Type: text/plain; charset=UTF-8\n"';
$headers[2] = '"Project-Id-Version: project\n"
"Language-Team: Brazilian Portuguese <debian-l10n-"
"portuguese@lists.debian."
"org>\n"
"Content-Type: text/plain; charset=UTF-8\n"';
foreach (@headers) {
if (m/^"Language-Team:\s*(.*?)\\n.*"$/ms) {
$langteam = $1 || '';
$langteam =~ s/"\n"//g;
}
print("$langteam\n");
}
----------
This is a patch.
----------
--- a/transmonitor-check
+++ b/transmonitor-check
@@ -710,9 +710,10 @@
$lasttrans =~ s/!//g;
$lasttrans = '' if $lasttrans =~ m/EMAIL\@ADDRESS/;
}
- if (m/^"Language-Team:\s*(.*)\\n.*"$/m) {
+ if (m/^"Language-Team:\s*(.*?)\\n.*"$/ms) {
$langteam = $1 || '';
$langteam =~ s/!//g;
+ $langteam =~ s/"\n"//g;
$langteam = '' if $langteam =~ m/<LL\@li.org>/;
}
if (m/^"Content-Type:.*charset=(.*)\\n.*"\s*$/m) {
----------
BTW, the "Last-Translator" and "Content-Type" fields are also affected by this problem. Fix is similar.