Hi,

It would be good if ben had support for working with partial suites
(e.g. experimental). One could then stage transitions without having to
host a full repository.

I'm interested in this because we've started using a partial suite for
Ubuntu uploads (series-proposed) before migrating to the release proper,
and it's good to be able to carry on using ben as expected in this
situation. It's in -proposed where transitions will happen.

My current solution is to externally concatenate and deduplicate
Packages/Sources files. The deduplication is required not for ben but
for edos-debcheck (it's interested in finding solutions in the whole
repository but for the transition case we only want to know about
solutions involving the newest packages). It would be good if we could
somehow specify multiple suites in a .ben file and have ben DTRT itself.

Cheers,

Le 19/11/2012 13:23, Iain Lane a écrit :

ben should deduplicate packages before piping them into edos-debcheck.
Can you provide a test case where it does not?


Cheers,

I'm using a slightly older version, 0.6.1. So if deduplication is a part
of newer versions then that's great (I'll try to find some time to check
it out soon). I don't believe partial suite support is, but that becomes
a simpler matter if you already have the deduping code. :-)

Cheers,

Le 2012-11-20 15:31, Iain Lane a écrit :

IIRC, the dedupe code was there since ever (at least, before the first
upload to Debian).

Cheers,

Hello,

Edos-debcheck will vanish soon. The replacement is dose-debcheck. And
dose-debcheck has an option --latest which seem to do what you need :

% man dose-debcheck

[...]

   OPTIONS FOR FILTERING THE INPUT
       --latest
               Only the latest version of each package is kept, older versions
               of packages are ignored.

dose-debcheck is distributed in the debian package dose-distcheck.

I lied, we're running a pre-package version (it's proving hard to get
this updated because of having to go through sysadmins) which doesn't
pass the deduped stuff to debcheck (just gives the filename):

http://anonscm.debian.org/gitweb/?p=collab-maint/ben.git;a=blob;f=lib/benl_utils.ml;h=4c58182e7e97020188fb5b1a6d1f549167bb0402;hb=78fe204f2a51f362d088f29507b3f372167fb8e4#l33

but later versions do it right, at least by source inspection. So that's
cool. Don't worry about that part of my initial mail. :-)

Le 19/11/2012 13:23, Iain Lane a écrit :

As said elsewhere in this bug report, the deduplication is already done
in ben.

However, after thinking about it, I am planning to redesign the plumbing
between commands. What I am planning to do is the following:

 * "download" will only download files;
 * a new "import" command will take a list of Sources and Packages
   files (or query projectb or some other kind of database), and create
   a "ben.cache" file. This command would do the canonicalization phase
   (i.e. add source and source-version fields to all binary packages,
   etc.);
 * a new "filter" command will add the uninstallable (previously
   edos-debcheck) field, and any other additional massaging we'll want
   in the future (like keeping only the last version or the
   "is-in-testing" virtual field);
 * the "query" command will be able to output "ben.cache" files as
   well. It will basically act as the dual of "import";
 * "monitor" will only read a "ben.cache" file.

Basically, it promotes "ben.cache"-like files (which are binary dumps of
an OCaml datastructure and efficiently parsed) as first-class citizens,
and the canonical format passed between commands, with bridges to the
text control file format via import and query).

The logic for partial suite support would be put in "import", but
concatenating files, then using "import" and "filter" looks simple
enough to be done externally in a shell script.

What is your opinion on that plan?


Cheers,

Hi,

not quite sure why I was CC'ed but I do have some comments for Iain :)

Quoting Stéphane Glondu (2013-08-19 14:51:47)

You should really not use edos-debcheck. Edos has been superseeded by dose3 in
2011 I think and there is no reason to use the deprecated edos anymore. With
     dose3 you are interested in using dose-builddebcheck

One feature of dose-builddebcheck is for example the --latest option which I
guess does what you want: reduce the input to the latest package versions.

In botch we use the python module apt_pkg to do all kind of package operations
(union, intersection, difference). It is very fast and the scripts are super
short. Maybe you want to have a look at them (the packages-*.py scripts):

https://gitorious.org/debian-bootstrap/botch/trees/master/tools

cheers, josch

Hi,

Quoting Stéphane Glondu (2013-08-19 15:15:54)

Ah right, I didnt think that was the reason because your newly proposed
strategy works just fine for me because it retains the ability to create and
process the ben.cache file. :)

Thanks for notifying me!

cheers, josch

Le 19/08/2013 15:03, Johannes Schauer a écrit :

I've put you in CC because my proposal will basicaly revert your patch
from #714703 and offer an alternative solution. Actually, I should have
put this bug in CC... done.


Cheers,

Hi Stéphane,

Yes. I can't quite remember the details but it might be that I had to do
this for edos-debcheck's purposes, or maybe we're just running a too-old
version of ben. It's not been updated for quite some time (the new level
calculation stuff you landed seems to be a good reason for us to
modernise though; thanks for that!).

Sounds fine, but having things in custom shell scripts makes reproducing
production environments a bit harder. Ideally (as your description of
"import" says) we'd be able to download, import and filter things for
multiple (partial + base) suites entirely inside of ben. It's not hard
to do it externally but a bit more messy.

Seems like it'd meet my needs. And thanks for working on ben :-). I'll
try to update our instance soon.

Thanks,

Hi Johannes,

Just quickly, as it's OT for this bug really, we aren't using that as we
are using ancient versions of things (the Ubuntu environemnt we're
running in is Lucid with some backported OCaml packages). I'll be
updating to a more modern system soon and we'll get dose with that. Not
sure if ben already passes --latest in its current codebase but if not
it should be easy enough to add in the mean time, before Stéphane's
refactorings.

Cheers,