Currently, the popcon script on user computer sends the list of all
installed packages.
However, it would the nice to have a configurable option to limit it
to the official package because today, there is only two choices if
the user have sensitive packages:

 - refuse popcon to not publish them
 - accept popcon and be unpleased because it's published.

If they can choose what they publish: {all | only official packages | none}
it could fit more needs.
(And perhaps to have more contributors because it could fit the users
using currently the first choice?)

Quoting Stéphane Blondon (stephane.blondon@gmail.com):


Is there a point in reporting non official packages?

/me would be OK with the "yes" choice to only report official packages

Hello Stéphane,

The issue is how popcon can reasonnably known whether a package is "official"
or not.

Also the list of official packages installed which are dependencies of the
sensitive packages might provide too string an hint that the sensitive package
is installed anyway.

Cheers,

16 juillet 2012, Christian PERRIER:

I have been told it's a way to know other used packages so it could
help to know what is needed by debian' users.
The fact that every used package are send is documented in
/usr/share/doc/popularity-contest/FAQ.gz:
   "Unofficial and local packages are reported. This can be an issue
   due to 2) above, especially for custom-build kernel packages.
   We are evaluating how far we can alleviate this problem."


2012/7/16 Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
good start to me.
For example for an AMD64 arch processor:
 -  http://ftp.fr.debian.org/debian/dists/squeeze/main/binary-amd64/Packages.bz2
 - the same archive for contrib and non-free (?)

I don't think it's a problem because the dependancy pckages stats are
summed up to all other users' stats.

Yes, it is useful to find packages that are somehow missing in Debian proper.

But how do you perform the check client-side ?

Only on the server side, not in the report that are sent to the server.

If really your packages are sensitive, you do not want them to be transmitted in
clear text to the server, I suppose.

Cheers,

2012/7/17 Bill Allombert <Bill.Allombert@math.u-bordeaux1.fr>:
 - download the Packages.bz2 from http ou ftp server, uncompress and
keep only the package names.
or
 - use files in /var/lib/apt/lists to rebuild the packages names. On
my computer, the file
ftp.fr.debian.org_debian_dists_testing_main_binary-amd64_Packages
seems to be a good start. I don't know exactly how it works (with
IndexDiff) but they are text files so I think it's doable.

Then, only stats from packages listed in the previous generated list
are sended.


The first way needs a network connection but popCon needs network to
send the stats, so I don't think it's a problem.



root@foehn:/var/lib/apt/lists# LANG=C; ls -l
total 75152
-rw-r--r-- 1 root root   198615 Jul 17 22:24
ftp.fr.debian.org_debian_dists_testing_InRelease
-rw-r--r-- 1 root root 29021698 Jul 17 16:15
ftp.fr.debian.org_debian_dists_testing_main_binary-amd64_Packages
-rw-r--r-- 1 root root     7876 Jul 17 16:15
ftp.fr.debian.org_debian_dists_testing_main_binary-amd64_Packages.IndexDiff
-rw-r--r-- 1 root root 18902880 Jul 17 04:14
ftp.fr.debian.org_debian_dists_testing_main_i18n_Translation-en
-rw-r--r-- 1 root root     7876 Jul 17 04:14
ftp.fr.debian.org_debian_dists_testing_main_i18n_Translation-en.IndexDiff
-rw-r--r-- 1 root root  3167707 Jul 17 16:15
ftp.fr.debian.org_debian_dists_testing_main_i18n_Translation-fr
-rw-r--r-- 1 root root     7819 Jul 17 16:15
ftp.fr.debian.org_debian_dists_testing_main_i18n_Translation-fr.IndexDiff
-rw-r--r-- 1 root root 25521688 Jul 17 16:18
ftp.fr.debian.org_debian_dists_testing_main_source_Sources
-rw-r--r-- 1 root root     7876 Jul 17 16:18
ftp.fr.debian.org_debian_dists_testing_main_source_Sources.IndexDiff
-rw-r----- 1 root root        0 Jul 26  2010 lock
drwxr-xr-x 2 root root     4096 Jul 18 01:02 partial

#681721 looks like a duplicate of the bug report I filed earlier
#632438. I've manually solved this on my own systems by modifying the
popcon cron job to remove sensitive packages from the output. In #632438
I mentioned a few ideas that could be used to exclude packages.

Well, I am not sure I like the idea to help users to remove packages from the
list. If you are afraid to leak information, the only safe course is not to
report to popcon. I do not want popcon to be held responsible for leaking
information it was told to protect.

Beside, I am afraid this will skew popcon results because some packages will be
under-reported.

Cheers,

The main use of the feature would be to not report sensitive packages,
especially metapackages not in Debian.

I would suggest that lack of this feature will reduce the amount of
people willing to report to popcon.d.o and skew the results.

What kind of metapackages ?

But in a more obvious way.

Maybe such packages could have a control field 'X-Popcon-report: no' that would prevent
popcon from reporting them. This way it would not be a per-user decision.

Cheers,

The usual:

organisation-site-purpose

For example:

google-nycdc-crawler
amazon-sydney-ec2storagenode

I don't think it would be enough and the decision should be with the
owner of the machine running popcon not with the apt repos they use.

Who create them ? What information is leaked by the release of this package name ?

If the APT repos is public, what is the security concern ?

Cheers,

These two bugs seem identical:

  #632438: popularity-contest: a way to exclude certain packages
  #681721: popularity-contest: option to limit the list of packages sended to popcon

Hopefully you'll agree.

live well,
  vagrant

It is not me who should agree but the submitters, which you did not CC.

I am sure you meant well, but as far as I am concerned this is a distraction.
There is too much diverging discussion to handle them as a single report.

What I am more interested is what are the proposed use cases for the feature
and whether this is something popularity-contest should support.

I offered 'X-Popcon-report: no' but the reporters do not seem interested.

Cheers,

It is not me who should agree but the submitters, which you did not CC.

I am sure you meant well, but as far as I am concerned this is a distraction.
There is too much diverging discussion to handle them as a single report.

What I am more interested is what are the proposed use cases for the feature
and whether this is something popularity-contest should support.

I offered 'X-Popcon-report: no' but the reporters do not seem interested.

Cheers,

Dear popular people,

I plan to add support for 'XB-Popcon-Reports: no' to popularity-contest.
This allows to build packages with private names that will not be
reported to popcon, by adding XB-Popcon-Reports: no to debian/control.
This must not used by packages in the debian archive, however that can
be used by packages generators that create packages with "random" names,
or by organizations that use custom packages whose name include the
organization name.

The rationale is that the only info really leaked is the package name,
so it only make sense to hide a package if every system that have it
installed are also hiding it, so it is better to make it a property
of the package than of the system.

Any comment ?

Cheers,

Please file a bug against lintian asking to add a check that this new
field is not present for packages being checked using the Debian
profile. Probably using this field should be an auto-reject.

It seems reasonable to me, but I still think that popcon itself needs a
way for the local sysadmin to ignore packages. Personally I have
modified the popcon cron job to `grep -v` the file before upload.

We believe that the bug you reported is fixed in the latest version of
popularity-contest, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 681721@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Bill Allombert <ballombe@debian.org> (supplier of updated popularity-contest package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Mon, 18 Jul 2022 12:57:02 +0200
Source: popularity-contest
Architecture: source
Version: 1.74
Distribution: unstable
Urgency: medium
Maintainer: Popularity Contest Developers <debian-popcon@lists.debian.org>
Changed-By: Bill Allombert <ballombe@debian.org>
Closes: 681721 999319 1001956
Changes:
 popularity-contest (1.74) unstable; urgency=medium
 .
   * debian/rules: add missing targets. Closes: #999319 Thanks Lucas Nussbaum
   * debian-popcon.gpg: use new submission key.
     The key is back to RSA4096. Closes: #1001956.
   * popularity-contest: New feature: skip private packages that declare
     XB-Popcon-Reports: no in debian/control. This is to be used solely
     by tools that generate packages with unique names, to avoid the
     unique name to leak. Closes: #681721.
   * debian/control:
     - Build-Depends: debhelper-compat (= 13)
Checksums-Sha1:
 f2feaf044d0ec3602a00430ad2fb6980dd234d02 1731 popularity-contest_1.74.dsc
 2f16cc110a6ae92e8b99d13a22c2e4790c2b21d4 79544 popularity-contest_1.74.tar.xz
 33acc558ea1b20156279208b5126792f7f31d881 5624 popularity-contest_1.74_source.buildinfo
Checksums-Sha256:
 15652667dbeb527326b0420cd9a7a2c024c6e7e7d99fc0d298341298aacd5599 1731 popularity-contest_1.74.dsc
 4b2d7db55a84d100c1b5995a881971cf604eeb3a6d9562cc9570e8caed035069 79544 popularity-contest_1.74.tar.xz
 66ffa05b6eeced877eec44ddac05b3f2bb9caaf9b6b8f44959f358679fffff7e 5624 popularity-contest_1.74_source.buildinfo
Files:
 d2e880f2c63eb86a3fcbcf78571a97da 1731 misc optional popularity-contest_1.74.dsc
 0ba66137a2ae1b90dd8a0371b5772665 79544 misc optional popularity-contest_1.74.tar.xz
 4ef62e75641f36e4d8bc757d51a9e7b9 5624 misc optional popularity-contest_1.74_source.buildinfo
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEQgKOpASi6dgKxFMUjw58K0Ui44cFAmLV4kcACgkQjw58K0Ui
44d/AhAAn6llSJyy0tMa6rUwrkDr53oECOyUcs+SKT6M9ztVFohAw/NiHB+Ja/ho
gYgeQq4P9KAoFwPAK/sHQL7gp3ZHKxOlZd4w2ZPCmDKu1Mzocs0tQ7y5TRutyOzc
QVgyIWFBZQ/z/aERmy7+WD5+F7/h4+ZxGjMNDOgKVuSPs4b5pZeQTf7xI3/NTEk5
PzNAS+7SiNhQTsrcldtVmBjgMTxSG8tiyfbKYNjmEs78PFpxqc49wudp5ftwXEf7
VWKYTsnFlUSF3oKhz2HEvEw7ulL9C21yEEMbZFIHvXSAVH3mBrS6W+wzKvQTcoCS
ETeJGm0Leg4+Y/RetE4FXynRMemhgqWqX7NJof6NYvw+CgM/bhQnnh5qzO/b+Ed+
obZrEzCXcPRtN0VTPcHOZjYBEaFrYumn2RRvfOVhP4geJ9Oqfxw8alecEvePQ7KV
QDpYKN6bbAO7LAHksRnRuoE195YTK8pxjZpdXOGZQRydXJ7o9euoGqG/5ACVU7MB
uf6VRuCt22t9gFIktSwMaAlqWrmbF1jU2+uZsmcr54JkY3kvk52L6GfcmCgQxf1x
z7YuTBcNZ+ax7aKtYn0VQAYQLiG9brUHGYaPGEQMH2D+Iaq4vTAgOEpwZIoTcKVp
rYHmDH7FjWmaq6fwX+fFVFC1W0foBDQ9tTbk5IFm40MODs2Phfk=
=Ny6J
-----END PGP SIGNATURE-----