#992823 takes long time to analyse perl processes

#992823#5
Date:
2021-08-23 21:42:14 UTC
From:
To:
The time needrestart takes to analyse processes is to a great amount
influenced by the existence and count of perl processes:
real    0m0,716s
user    0m0,479s
sys     0m0,233s
# systemctl start spamassassin.service
# time needrestart -lp
OK - Services: none, Containers: none, Sessions: none|Services=0;;0;0 Containers=0;;0;0 Sessions=0;0;;0

real    0m2,668s
user    0m2,314s
sys     0m0,335s
----8<----8<----8<----

On another system (3.4-6 on buntu) the times are comparable but rise to
2 minutes when the sympa daemons are running as well.

This delay is a major inconvenience when installing packages or refreshing
in apt-dater. When used for monitoring standard timeouts are exceeded.

#992823#10
Date:
2023-02-21 08:54:02 UTC
From:
To:
On the server with sympa package, it takes more than 2 minutes when the
services are up !
-------
# time /usr/sbin/needrestart
Scanning processes...


Scanning linux images...



Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

real	2m23.706s
user	2m10.592s
sys	0m12.930s
# service sympa stop
# time /usr/sbin/needrestart
Scanning processes...


Scanning linux images...



Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

real	0m1.430s
user	0m1.299s
sys	0m0.125s
---------

Could we raise the Severity ?

#992823#15
Date:
2023-03-06 05:40:50 UTC
From:
To:
I just profiled needrestart using NYTProf and found that the
problem is actually caused by the Module::ScanDeps Perl module:

   sudo apt install libdevel-nytprof-perl
   sudo env NYTPROF=trace=2:start=init:file=`pwd`/nytprof.out perl -d:NYTProf /usr/sbin/needrestart
   sudo chown "$USER"
   nytprofhtml --open

I found that on my system there are a couple of Perl processes
and one of them takes a lot more time to scan than the other one.

   $ ps auxf | grep perl | grep -oE '/usr/s?bin/[^ ]*' | sort -u
   /usr/bin/parcimonie
   /usr/bin/perl
   /usr/sbin/spamd

   $ time scandeps /usr/sbin/spamd > /dev/null

   real	0m1.430s
   user	0m1.350s
   sys	0m0.080s

   $ time scandeps /usr/bin/parcimonie > /dev/null
   # Use of runtime loader module Module::Runtime detected.  Results of static scanning may be incomplete.
   # Use of runtime loader module Module::Implementation detected.  Results of static scanning may be incomplete.

   real	0m8.155s
   user	0m7.006s
   sys	0m0.999s

Researching the landscape of such tools, I found these other ones:

   Perl::PrereqScanner
   Perl::PrereqScanner::NotQuiteLite
   Perl::PrereqScanner::Lite

https://stackoverflow.com/questions/358891/how-do-i-find-the-module-dependencies-of-my-perl-script

These all complete a *lot* faster but do not produce the same
results and so they are probably *not* useful for this situation.

#992823#20
Date:
2023-03-10 01:20:36 UTC
From:
To:
Some ideas from a discussion on #debian-perl about this:

 * Parse the Perl code using PPI instead of Module::ScanDeps
    - Change Module::ScanDeps to use PPI instead?
 * Parse the Perl code using Perl's parsing instead
    - Would require getting Perl upstream to export a module
    - or possibly the libperl might have something?
 * Add a Perl module that exports data about imported code
    - PERL5OPT=-MNeedRestart
    - divert /usr/bin/perl to a script that adds -MNeedRestart
 * Add code to Perl upstream that exports data about imported code

Ultimately though, needrestart shouldn't have to parse code (as root?)
in order to figure out what code a particular interpreter process,
for *any* language (including Perl, Python, shell etc) has loaded.

That will require defining a standard for how the processes should
export that information and then sending patches for all interpreters.