Hi,

I would love to see more statistics about packages sizes in debsources.

For example, I wonder if the average package size (in LOC) decreases or
increases. It could increase because programs are getting bigger (more
bloated? :-) ), or decrease because software we are packaging for the
first time is probably smaller (the bigger bits are likely to already be
in Debian).

The average can be computed by hand, but it would also be interesting to
provide quartiles or deciles information in a graph.

Lucas

Hello,

I ve been playing around with this bug :)

At first I thought I'd calculate the quartiles and create some tables
and then plot the data using boxplots. I managaed to generate the tables
but then i found out that pyplot's boxplot uses as input the data itself
and not the quartiles in question.

So I am wondering how should I proceed.
Right now the calculated quartiles are saved in the stats dict during
the statistics update run and saved in the cache file stats.data. Then I
found out that we can extract the generated quartiles from the pyplot
boxplot. So i see 4 options:

- extract the data from boxplot and save in cache. This means reading
the current stats from the file and updating its values although this is
not supposed to be done on a charts update and if somebody disables
charts then that table won't be available.

- Let pyplot do its thing and calculate elsewhere the quartiles for a
table representation.The complexity for the calculation is :
	* database queries to get the metrics for each suite or one group by query.
	* sub-setting a list
	* calculating 3 medians (one for the actual median and the other ones
for the lower_half and upper_half set of values) one min and one max.

- Do not create any summary table with these values and keep only the
boxplot. Lucas what do you think on that? You mentioned generating a
graph so does this mean that a summary table wouldn't be of much use?

- hack into boxplot to generate custom boxplots. Already done by
somebody [1]. IMHO it looks pretty clean since it just overwrites some
values and keeps all the functionalities of the boxplot intact.


I attach you a print screen of how the table and the boxplot look like.

Cheers,

Orestis


[1]
http://stackoverflow.com/questions/27214537/is-it-possible-to-draw-a-matplotlib-boxplot-given-the-percentile-values-instead

Hi,

I think that the summary table is also useful, especially given that the
first quartile value is really small. It would be hard to say from the
graph if it's increasing or decreasing.

I have no particular comment about the best strategy here, but given the
SLOC values are pre-computed, I wouldn't worry too much about the
performance for computing/generating those stats.

Lucas

Hey,
[...]

Ok good :)

I did some more digging mostly since i discovered that my computation
for quartiles was giving sometimes different results for the Q1 and Q3
than the one in pyplot.
Long story short I compared them with the help of R and i found out in
the documentation [1] that there are many ways to calculate them. I used
the type 2 and pyplot, if i am correct (results coincide), uses type 7.

If i understand properly the doc type 7 makes the assumption that the
sample is continuous whereas type 2 considered to be discontinuous.
Since my knowledge in statistics is fairly minimal i am not really sure
what we have in this case :p

In any case if we consider that the sample is discontinuous then i think
the best option, in order to have same results both in the table and the
graph is to hack into pyplot to insert pre - calculated quartiles in
boxplot. I am thinking this since i couldn't find any way to tell pyplot
to calculate quartiles in another way.
If however the sample is considered continuous then i ll have to
implement another algorithm to calculate quartiles so that the results
are the same.

Orestis

[1] https://stat.ethz.ch/R-manual/R-patched/library/stats/html/quantile.html