#571926 popularity-contest: popcon doesn't track usage of python modules in a consistent manner

#571926#5
Date:
2010-02-28 11:40:24 UTC
From:
To:
Hi!

popcon currently uses the regexp:

  m{/bin/|/sbin/|/lib/.+/|^/usr/games/|\.[ah]$|\.pm$|\.php$|^/boot/System\.map-}

to detemine if the atime/ctime etc for a file should be checked in order to
decide if the package has been used recently.

This means that a python module that has no C extensions (i.e. a collection
of .py files only in /usr/share/pyshared) always shows up as being "nofiles",
whereas a python module that has a C extension and so installs .so files into
/usr/lib/pyshared/python2.X/package/ will report recent usage etc.

Given the explicit inclusion of .pm files to indicate usage of perl modules
and .php to pick up php snippets/modules/scripts/whatever, it would seem more
useful to collect python data in a similar fashion. That the data for a python
module also depends on whether or not it has a C module or not (something that
is entirely opaque to the end user!) is also a bit odd.

Perhaps add |\.py$ to that regexp?

cheers
Stuart

#571926#10
Date:
2010-02-28 11:52:55 UTC
From:
To:
We cannot do that because the .py files are read by the python compiler at
package installation time to generate the .pyc file, and never after, so the
atime of the .py file is useless (it is always equal to the installation time).

See also bug #265360.

Cheers,

#571926#15
Date:
2010-02-28 12:14:38 UTC
From:
To:
Hi,

duh... of course.

Can we come up with some way of making popcon more useful for python module
packages then? (Other than just number of installations) The discussion of
how widely used a package is (i.e. is it low popcon and a candidate for
removal because it looks orphaned etc) is harder to have if you don't know if
the X installation reports are actually using module or if it just happened
to be installed once back on the box when it was running potato and left to
rot there...

As a (probably poorly-conceived) idea, would it be too much hard work for the
popcon scripts to special-case \.py$ to actually take the atime of the
corresponding \.pyc$ file? Popcon would need to know where
pysupport/pycentral is sticking these files of course.

cheers
Stuart

#571926#20
Date:
2010-02-28 14:56:08 UTC
From:
To:
Ah potato!

I think at this point if someone would commit themself to write a
reasonnably-sized patch to deal with python *and* maintain it, we would accept
it. But I agree with Petter that if popcon tried to handle every special case,
then the popcon script would be too large and too complex.

Cheers,