#965947 lintian: Extract string constants from interpreter scripts and modules

#965947#5
Date:
2020-07-21 10:33:16 UTC
From:
To:
Dear Maintainers,

Until recently lintian emitted a warning that python3-reportbug uses the dpkg
database directly. The corresponding code in the reportbug source package has
not changed, but the warning is gone. Does this mean that the warning was a
false positive in the first place? Or is the lack of warning now a false
negative?

Thanks!

#965947#10
Date:
2021-09-04 14:08:54 UTC
From:
To:
The lack of warning is a false negative.

In lib/Lintian/Index/Item.pm, `mentions_in_operation` uses `is_script`
to decide whether to check the file content of non-binary files. Only
the main reportbug and querybts python scripts are identified as
scripts. The content of the python module files is not checked.

Should all .py files be considered scripts? The issue probably affects
other scripting languages as well.

#965947#15
Date:
2021-09-22 12:39:44 UTC
From:
To:
Hi Nis,

Sorry I took so long. Your bug is real, but I am not clear about the
best solution.

The change in behavior was probably introduced when I rewrote the file
indices to reside in memory. Among other things, I made the code more
"literate" in the sense that routines now have names that resemble
more closely what they do. Twenty-two years of collective patching had
led to a bit of a jumble.

From other, more prominent parts of the code I had decided that
scripts were executable files with a hashbang. We devote a whole check
to them. It is called 'scripts'.

For your package python3-reportbug, the change resulted in a loss of
functionality. I am not sure, however, that the best fix is to simply
add *.py files—without an executable flag or a hashbang—to the
definition of a script.

For ELF executables, Lintian looks at the equivalent of 'nm' strings.
It is not perfect but avoids confusion with documentation or code
comments.

We struggle with the issue of script parsing in other parts of
Lintian, too. (See open bugs.) For this bug, I would really like to
use a Python parser that extracts string constants from Python modules
under /usr/lib/python3/dist-packages/. I would also like to find
similar solutions for other scripting languages—especially for Perl.

Thank you for your diligent maintainership, and especially for this report!

Kind regards
Felix Lechner