Hello,

thank you for sbuild.

When used by DDs to build packages to upload, sbuild has the unexpected
behaviour of ignoring the distribution set in the changelog.

Of course this behaviour is what is needed in build daemons, but it
would be useful if sbuild could be able to do it both ways.

An option would be to use the distribution set in the changelog when one
specifies "-d auto", but that may have a rare failure scenario if
someone having their own local distribution with a suite called "auto".

Alternatively, one could use another option like --auto-distribution or
--no-auto-distribution, and set the defaults from $sbuild_mode.

This was roughly the result of the brainstorming we just had on IRC.


Ciao,

Enrico

This has come up in the past, because it's an obviously useful
feature.  When we had our conversation on IRC about this, I forgot
about one major problem blocking this:

- if building a package downloaded from a mirror,
  ("sbuild package_version" i.e. no .dsc extension since there's
  no local file), we need to download the package *inside* a
  chroot.  I.e. we are forced to choose the chroot (distribution)
  *before* downloading the package.  Until the package is
  downloaded, we can't see which distribution was in the changelog.

- if building using local files,
  ("sbuild /path/to/package_version.dsc") we can easily parse the
  DSC.  However, the distribution isn't put in the DSC either!
  It only does into the .changes.

In consequence, we can only realistically do this when using local
files.  However, this would require unpacking the sources in order
to read debian/changelog.  For large packages, this would be very
wasteful of time and diskspace because we would have to then
delete the unpacked sources before copying the DSC/orig/diff.gz
into the build chroot where they will be unpacked again for the
real build.


I still this this is a useful feature.  This issue does need
resolving before it can be implemented, however.


Regards,
Roger

tags 529281 + moreinfo wontfix
thanks

Tagging wontfix until these technical problems have a solution.
I'm doubtful this can be resolved for these usecases, but I'll
be happy to hear any suggestions.


Regards,
Roger

tag 529281 - moreinfo wontfix
retitle 529281 sbuild: Accept .changes on the command-line and auto-detect distribution in that case
severity
thanks

I would like to resurrect this bug. I understand the .dsc doesn't have the
distribution, however a .changes file has it.

When I generate the source package, I do debuild -S and this generates me
a .changes files with the correct distribution.

So please improve sbuild to accept a .changes listing a source package
and don't require the distribution in that case, instead extract it
from the .changes file.

I don't like this. Source package migrate and can be reuploaded in
multiple places (including distributions of derivatives) without being
rebuilt.

The Distribution is part of the .changes file and it's enough there IMO.
I don't see a good reason to change this.

I have already been bitten by this and I made a concrete suggestion
here in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=529281#22

And in fact you're mixing two concepts, the chroot to use for the build
and the target distribution.

You can still use "-d <dist>" to infer the default chroot to use but you
should not override the target distribution set in the changelog unless
the users sets another flag "--force-distribution". If the flag is
missing and the two values differ, you should just abort with an
error.

I mean dpkg-genchanges uses the value from the changelog by default and
you should not overwrite the value generated unless you're in a context
where you know that you want to overwrite it. In that case, supplying one
more parameter is not an issue (I'd rather have the buildd config provide
one more parameter than requiring the casual users to change their
habits).

I'll let some time pass for the discussion but I will probably close
#630472 or tag it wontfix.

Cheers,

Hi,

my thoughts on this:

= sbuild side =

Apparently, sbuild is used by buildds and developers, with differing
needs. There should be a switch to sbuild, which buildd calls but
developers don’t, that enables Distribution overriding (for example
for binNMUs to packages in testing that were uploaded to unstable in
the first place), and the default should be that it doesn’t override
the field.

That is, specifying the distribution (e.g. unstable) to sbuild would
still be mandatory (so this wouldn’t entirely solve #529281), but by
default (unless buildd), this value would NOT be copied into the Di-
stribution field of the .changes file – instead it would behave like
cowbuilder (which uses dpkg-genchanges) and copy the field from the
package’s changelog.
This is especially useful for developers building _local_ packages
in e.g. a sid or stable chroot, to prevent them from accidentally
uploading them to Debian, but also to avoid the case of forgetting
to change away UNRELEASED before uploading.

= lintian side =

I also propose an ftpmaster autoreject (via lintian) if:
a) Distribution=unstable, Changes=experimental
b) Distribution=experimental, Changes=unstable
c) Distribution=*, Changes=UNRELEASED

All other mismatch cases should probably be a W, but for these three,
I think not just an E but an autoreject could, maybe, be justified.

= social side =

Just use cowbuilder ;-)

https://www.mirbsd.org/cvs.cgi/contrib/hosted/tg/deb/pbuilderrc?rev=HEAD
contains a sample ~/.pbuilderrc which does multi-arch (i386 and amd64 on
an amd64 host, for example – otherwise of course just one) multi-distro
(debian, ubuntu, debian-ports) multi-suite (from dapper and sarge up to
sid and (currently) precise) cowbuilder handling, plus “multi-client ca-
pability” by setting $CUSTOM. (It also contains some “optimisations” for
the chroots, such as dropping debconf-i18n (for Perl transitions; I did
the 5.12 one on m68k with that) and file-rc, which you may or may not
want; one can also locally optimise by putting fakeroot and even debhel-
per into EXTRAPACKAGES. These are of course not suitable for when buildd
could use cowbuilder, as suggested below.)

Of course, be aware of differences between cowbuilder and sbuild, but I
have had few problems with it (as long as cowbuilder, pbuild and deboot-
strap are taken from sid on the build machine, e.g. due to architecture
wildcards) and would love to see the buildd software support to use both
so buildds could be set up with either. (It’s easier to set up.)

= What now? =

#542747 could contain discussion about the lintian checks I suggested in
the second chapter of this mail.

We could clone #559659 and discuss the changes from the first chapter of
this mail there, as they would obviously not fix #559659 but still ad-
dress the issue of packages uploaded to the archive being not rebuildable
(with cowbuilder) due to their debian/changelog last entry being UNRELEA-
SED instead of unstable. I’ve found some on snapshot.d.o this week, and
buxy had the issue today, too.

Flames as followup-to-poster please. Improvements to pbuilderrc welcome.

bye,
//mirabilos

We certainly have an existing mechanism which may be set in .sbuildrc
or sbuild.conf:

$sbuild_mode = "buildd|user"

This does not currently encompass Distribution handling, but could
be made to do so.

In the last few releases, I've refactored local building to allow
this.  sbuild itself only operates on source packages; previously
local building required generating the source package and then
operating on the .dsc.  We now (internally) only deal with .dscs.
Local building is done at the top level prior to creating and running
a build job.  The implication of this change is that it's now possible
to run dpkg-parsechangelog and change the distribution default when
building locally.  This is the case right now.

This still doesn't solve the problem of building a local .dsc,
which would require an unpack of the source package to determine
the distribution, but does solve it for unpacked sources.
Adding support for using a .changes file in place of a .dsc is
still on my TODO, list.  Once a .changes is accepted, we could
only permit setting the distribution in buildd mode.


Regards,
Roger

(Dropping #647028 from CC. Roger's message, and most of Thorsten's
message had nothing to do with lintian...)

* Roger Leigh <rleigh@codelibre.net>, 2011-10-29, 18:59:

The idea of running sbuild on *.changes sounds very weird to me, but as
long as I don't have to use it, I don't really mind.

I understand that if you have only .dsc at hand, you need to know which
chroot to use. So make -c mandatory if -d is not given, and the problem
is solved.

Why? Auto-detecting distribution is good, but what's wrong with an
option to set it manually?

This would indeed solve the problem.  The main issue preventing this
would be the breaking of interface backward compatibility for sbuild
users.
Debian a package where the distribution in the .changes does not
match the one in debian/changelog?  Or even when building for local
use?

The -d option currently serves two roles:
• overriding of the distribution in the .changes
• selecting the build chroot

I would argue that the former is only required on the buildds,
but nowadays we're using schroot and even this is tenuous--it's
easy to add aliases for e.g. stable-proposed-updates for stable
etc.  The selection of the build chroot just provides a default
chroot to use if --chroot it's used to manually specify one.

It would certainly be possible to alter -d to only do the latter
when not in buildd mode, and potentially disable entirely should
the consensus be to do so.


Regards,
Roger