There are merge and rebase workflows in use for the Debian changelogs of
backports.  Some backport maintainers merge the new version in testing
into their backports branch, so that the previous backport's entry
remains in the changelog (the 'merge' workflow).  Others base each
backport on the version in stretch, discarding the changelog entries for
older backported versions (the 'rebase' workflow).

For a backport using the rebase workflow, dgit refuses to push:

    Checking package changelog for archive version 2.13.0-1.1~bpo8+1 ...
    dpkg-parsechangelog: warning: 'from' option specifies non-existing version
    dpkg-parsechangelog: warning: use oldest entry that is later than the one specified
    dpkg-parsechangelog: warning: 'to' option specifies non-existing version
    dpkg-parsechangelog: warning: use newest entry that is earlier than the one specified
    dgit: missing field Version in parsed changelog
    dgit: Perhaps debian/changelog does not mention 2.13.0-1.1~bpo8+1 ?

dgit has correctly identified the problem -- the changelog does indeed
fail to mention 2.13.0-1.1~bpo8+1.  But this is a perfectly legitimate
approach to backporting (personally, I prefer the merge workflow, but a
sponsee of mine uses the rebase workflow, which is how I ran across
this).

Thanks.

Sean Whitton writes ("Bug#857490: dgit: shouldn't fail when previous backport not mentioned in changelog"):
in current master, this message only happens if you say --overwrite.

The documentation for --overwrite says (now, but 4.0 was similar):

       If  previous-version  is not specified, dgit will check that the
       version in the archive is mentioned  in  your  debian/changelog.
       (This  will  avoid losing changes, even with --overwrite, unless
       someone committed to git a finalised changelog entry,  and  then
       made  later  changes  to  that version.)  If previous-version is
       specified, it ought to be the version currently in the archive.

So that is precisely what ought to be expected.


Stepping back a bit: with a rebasing workflow such as you describe,
dgit's sanity check strategy for overwriting-pseudomerges (which are
basically like force pushes) cannot work.

I am not sure if there is a better sanity check strategy that could
work.  It seems difficult.  If the user is in this situation their
branch is not ff from the suite branch *and* their changelog doesn't
mention the previous upload.

It is difficult to see how to distinguish this from the case where
someone (for example) simply prepares a fresh backport of a package
without regard to any previous work.  And of course doing such a fresh
backport without looking at the previous work risks introducing a
regression, if a naive backport would introduce a bug which is in fact
fixed, right now, by extra change(s) in the current backport.

So I think the right answer is perhaps indeed to require that the user
says
  --overwrite=2.13.0-1.1~bpo8+1
or whatever, so forcing them to at least prove that they knew there
was a previous backport.


Does that make some kind of sense ?  How can we better guide the user
who finds themselves in this situation ?

Ian.

Ian Jackson writes ("Re: Bug#857490: dgit: shouldn't fail when previous backport not mentioned in changelog"):

#913451 "dgit manpages: discuss --overwrite in more detail, esp. diff.
 with --overwrite=X.Y.Z"

seems relevant.

control: tag -1 -moreinfo

Hello,

Suggesting that the user use --overwrite=2.13.0-1.1~bpo8+1, or `git
merge -s ours dgit/dgit/stretch-backports`, is indeed what we should do.

I think that official backports is the only case where this is likely to
come up often.  So I suggest that we start a dgit-maint-bpo(7) with tips
and techniques for maintaining official backports (also see #898494).
In that page we can say that if you are using what in this bug I've
called a rebase workflow, you will need to use --overwrite=X.Y.Z for
each upload.

(Calling it dgit-maint-bpo(7) rather than dgit-maint-backport(s)(7) is
meant to make explicit reference to backports.debian.org.)

I also think that dgit could emit a recommendation to look at that
manpage in the kind of case that prompted me to file this bug.  It can
just look for ~bpoNN in the version number that is missing from
d/changelog, and output a hint.

Sean Whitton writes ("Re: Bug#857490: dgit: shouldn't fail when previous backport not mentioned in changelog"):

SGTM.

Are you volunteering to write the (7) manpage ? :-)

Thanks,
Ian.

Hello,

Yes, probably.

control: tag 857490 +patch

Hello,

Here is a patch.

This does not completely resolve #898494, as I haven't modified dgit to
output the hint to look at the manpage.

I took a look at this.

I think the merging workflow you describe will not go well if the sid
package develops changes to upstream files (through new upstream
version, or through changes to the delta queue).  Typically the
package is `3.0 (quilt)', so if there are any changes to the delta
queue, dgit patch linearisation will be needed, and it may well fail
after `git merge'.

Possible solutions may include:

 - being the maintainer in sid and knowing what your git workflow
   and branch format is and doing something workflow-specific

 - git-debcherry (blocked by #930881)

 - changing the source format

 - git-ffqrebase (does not exist yet, #869993)

 - git-debrebase convert-from-dgit-view may help in some cases;
    I have used it for security updates with moderate success,
    but I think repeated conversions may be troublesome

Ian.

We discussed IRL the handling of backport and -v versions.

There are *two* things we are trying to do with the previous version
informatiion:

 * Calculate the -v option to pass to dpkg-parsechangelog.
   (With a normal upload this is the version currently present in the
   target suite.)

 * Perform the overwrkte check: we normally insist (with
   --trust-changelog eg in t2u) that the version currently present
   in the target suite, is mentioned in *this* upload's changelog.

Backports may be based on the previous backport (merge workflow) or
may be re-created afresh from the parent suite (restart workflow).

We think this can be supported with the following behavioural change
in dgit:

 * Backports suites are special (and known via the configuration).

 * When uploading to backports, check our changelog (of the thing
   we're uploading) for *either*
    - the current version in the parent suite
    - the current version in the backport suite

 * Whichever of these exists and is most recent in our changelog,
   should be used for dpkg-parsechangelog -v.  It should appear
   because otherwise we'll be failing the --trust-changelog check.

 * With --overwrite=SPECIFIC-VERSION, it is ok to specify either of
   those versions as SPECIFIC-VERSION.  We might the user (who isn't
   using t2u) to specify -v.

Ian.