#570650 popcon.debian.org: please provide raw popcon time data

#570650#5
Date:
2010-02-20 14:15:07 UTC
From:
To:
  Hello,

  Often, I would be interested to know more than just "how many
percents of the people have this package ?": it would be great if one
could have more information, such as correlations "how many people
have this and that packages ?" installed at the same time, or "how many
people still use the buggy 1.0.1-2 version of this software ?".

  You definitely have the information somewhere (except for the
version information, it seems, but it wouldn't be too difficult to
get). The question then is how to store/disclose this information,
without losing anonimity.

  Maybe it would be interesting to publish the raw emails (without the
mail envelope, of course), or would that be too big ? (around 100k *
90 000 submitters is one gigabyte, but I guess it should compress
really well). Other formats could make it much more compact.

  My guess is that using fully this data would enable us to know much
more than just "which package is the most popular ?".

  Cheers,

	Vincent, who can't help wanting more information when looking
at his popcon graphs

#570650#10
Date:
2010-02-23 16:03:58 UTC
From:
To:
Hello Vincent,

This has been discussed a lot of time, but we cannot provide such data
because that would break popcon submitters privacy expectations.

Certainly. For example, you could reach conclusion like 'every submitter
that use foo, bar, and baz also use wilma and fred'. Unfortunately this
is a major provacy issue: if you guess that a popcon submitter is using
foo, bar, and baz (because the submitter run a web service that use
foo, bar and baz, because the submitter is the maintainer of foo, bar, and baz,
etc.) you can conclude that the submitter is also running wilma and fred,
which break the privacy expectation.

Cheers,

#570650#15
Date:
2010-03-18 21:48:46 UTC
From:
To:
retitle 570650 popcon.debian.org: please provide raw popcon time data
thanks

  Hello,

Bill Allombert wrote:

  This is exactly what I was afraid you would answer ;-)...

  Although for the latter case, you're likely to pick up dependencies or
otherwise related packages, so it won't provide too much information in
the end.

  I think I have a proposition that would allow people like me to do
more quantitative analysis of popcon data while not requiring more than
what is publicly available now: could you please provide somewhere the
raw time data for the popcon graphs, ie, the raw date/popcon series for
each package ? This would allow one to manipulate the data, compute
correlation between different packages (though with less certainty than
the raw popcon data, which is good too for the privacy of our users),
correlate with periods of open bugs...

  What do you think about this ?

  Cheers,

	Vincent

#570650#22
Date:
2010-03-22 14:37:40 UTC
From:
To:
Well, since you are a Debian developer, you should be able to access this
data from popcon.debian.org in
/org/popcon.debian.org/popcon-mail/all-popcon-results

At least, I hope this directory is accessible to Debian developers. If not
please tell me.

Cheers,