Hi. I'm pretty sure this isn't a bug, and that I'm doing something
wrong. But I can't tell by reading the docs what I should be doing
instead. Help appreciated.

I want to take an existing gbp repo and convert it to a dgit repo. To be
clear, by "gbp repo" I mean: tracking debian/ in git, merging squashed
upstream tarball with each release. By "dgit repo" I mean using the
upstream git repo, and merging in the debian/ commits as needed.

I want to preserve the existing history of debian/. So I did a "git
filter-repo" on the old tree to make a tree that just has the history of
debian/. Then I "git rm debian/patches". And I merged that to my
upstream. Looks like this:

  M─┐ Merged previous non-dgit debian/
  │ o removed patches; these will be handled by dgit
  │ o changelog
  │ o .....
  o {upstream/master} python-cameramodel-converter.c doesn't use PyTuple macros
  o Added test-python-cameramodel-converter.py
  o minor comment, docs
  o docs
  o I ship mrcal-show-model-resolution,mrcal-show-stereo-pair-diff
  o When loading data from optimization_inputs, I match the endianness
  o loosened tests to make stuff pass on more architectures
  o build notes
  o no more buster builds
  o <v2.5> test-save-load-image works in old numpy
  o tests work on ancient numpy
  ....

The branch on the right is the debian/ history with the patches removed.
The branch on the left is upstream. I'd like dgit to figure out that
v2.5 is the upstream release, and that everything from v2.5 to
upstream/master should be made into a patch. This doesn't work. I see
this:

  $ dgit --dry-run --clean=git quilt-fixup
  DRY RUN ONLY
  Format `3.0 (quilt)', need to check/update patch stack
  # git clean -xdf
  examining quilt state (multiple patches, linear mode)
  dgit: base trees orig=0a6b7feaacba15276a34 o+d/p=0a6b7feaacba15276a34
  dgit: quilt differences: src:  ## orig ==     gitignores:  == orig ==
  dgit: quilt differences:      HEAD ## o+d/p               HEAD == o+d/p
  starting quiltify (multiple patches, linear mode)

  dgit: error: quilt fixup cannot be linear.  Stopped at:
  dgit:  a922e354..41f06c89: changed debian/source/format
  dgit:  1f6d0344..80c69b69: changed debian/patches/0001-loosened-tests-to-make-stuff-pass-on-more-architectu.patch,debian/patches/0002-When-loading-data-from-optimization_inputs-I-match-t.patch,debian/patches/series

  dgit: This might be a patches-unapplied branch.
  dgit: Maybe you need one of --[quilt=]gbp --[quilt=]dpm --quilt=unapplied ?
  dgit: Warning: Tree has .gitattributes.  See GITATTRIBUTES in dgit(7).
  dgit: Maybe orig tarball(s) are not identical to git representation?

  dgit: error: quilt history linearisation failed.  Search `quilt fixup' in dgit(7).
  dgit:

So it's looking in the branch that has debian/, seeing that I mucked
with the patches in there, and gets confused. It says

  orig=0a6b7feaacba15276a34 o+d/p=0a6b7feaacba15276a34

Note these are the same. The orig is correct: it's the upstream release
+ debian/. The o+d/p is NOT correct: it should be seeing all the extra
patches in upstream/master, but it doesn't.

It feels like this should be normal and simple, but I can't figure it
out. How should this be done? And a very related question: how should
migrations to dgit be done, if not like this?

Thanks!

Hello Dima,

Dima Kogan [30/Dec 12:00pm -08] wrote:

I'm not clear what workflow you think you are aiming for, but I would
first say that what you call a "gbp repo" is fully compatible with dgit,
and also compatible with switching to merging upstream Git history
instead of squashed tarballs.  You would just start merging it.  It
should work fine.

It sounds though that you might like dgit-maint-debrebase(7).
That instructs you to start with 'git debrebase convert-from-gbp'.
Perhaps you'd like to try that out.

Sean Whitton <spwhitton@spwhitton.name> writes:

Hi. Thanks for replying. I did read the docs, and the "maint-merge"
workflow is what I'm pretty sure I want.

There's so much complexity here that it's hard to know if I'm doing
something wrong or I'm hitting a bug. I just looked into it again just
now, and I'm pretty sure that this is either a bug or a case that dgit
doesn't handle well (yet?).

It looks like dgit looks through all of the git history, and if it sees
any commits that it doesn't know about that touch debian/patches, it
gives up. Right? Here I tried to manage this by removing all the
debian/patches commits from my debian branch before merging it. However,
as I was just reminded, many years ago the upstream git tree had the
debianization in it. This was removed years ago, when mrcal was uploaded
to Debian, but it's still in the history and dgit still sees it and
barfs. Since this is in the upstream tree, I cannot just remove those
commits. dgit should probably not react to the debian/patches commits
prior to those files being deleted.

Suggestions?

Thank you.

Dima Kogan writes ("Bug#1124338: dgit: dgit is confused by merged branch with debian/"):

dgit-maint-merge(7) works with a patches-applied branch.
Your existing "gbp" branch is presumably patches-unapplied.

You need to convert it.  The easiest way is probably to use `gbp pq`
to make what it calls a "patch queue" branch with, err I think `gbp pq
import`.  Then, switch back to the main branch, and git merge the
patch queue branch.  Delete the patch queue branch and delete the
patches in d/patches.  You'll need to change the source formt to 1.0
native, too.

But, I wonder if this is the right workflow for this pakage.
dgit-maint-merge is good if there are usually no patches, or only a
very small delta.  If there were no patches then your tree would
*already* be patches-applied since they'd be equivalent.

dgit by default expects your branch to be patches-applied, but with
possible local changes to upstream files added on top in commits.
It is trying to find those commits so that it can turn them into
patches in d/patches.

Your existing branch would work with --quilt=gbp, probably.
The default, --quilt=linear, is simply wrong in this case.

Don't try to "remove commits".  You don't want to be running the
commit linearisation algorithm at all.  It's wrong in your situation.

Ian.

Ian Jackson <ijackson@chiark.greenend.org.uk> writes:

Hi. I want to move from patches-unapplied gbp to a patches-applied
plain-git thing. I've already done this for a few of my packages with
great success, but this one doesn't work. I thought I understood why,
but I just poked at it again, and I don't know why. I am also upstream
here. Which means that most of the time there will be no patches, or a
very small number. RIGHT NOW there are a few patches: I fixed a few
small things since the last release, but haven't made a new release yet.

That's exactly what I want.
merge of rebase of whatever) I will have a patches-applied repo, so
--quilt=gbp is 100% not the right thing. Right? Is --quilt=linear also
not the right thing?

If you have a moment to look at the specific failing thing, could you do
that? Would clarify the situation for everybody, and would really help
me. Recipe:

  gbp clone --debian-branch=dgit-test-deb-patches-merged git@salsa.debian.org:science-team/mrcal.git
  cd mrcal
  git checkout dgit-test-deb-patches-merged
  origtargz

This is a merge of the latest upstream git tag (v2.5) and a debian
branch. Then there's a commit to remove all the patches. This should be
the nominal case: there are no debian/patches and the tree and clean and
no extra commits exist. dgit should be fine, with nothing to do:

  dgit --clean=git --quilt=linear quilt-fixup

This is as expected: looks at stuff, and happily does nothing. Same with
--quilt=gbp. I merge in the latest upstream. This isn't released yet,
but has a few small patches I'd like to be quiltified:

  git remote add upstream https://github.com/dkogan/mrcal.git
  git fetch upstream
  git merge upstream/master

All good. Here --quilt=gbp fails since this is patches-applied.
--quilt=linear squashes the commits into one, however:

  dima@shorty:/tmp/mrcal$ dgit --clean=git --quilt=linear quilt-fixup
  Format `3.0 (quilt)', need to check/update patch stack
  examining quilt state (multiple patches, linear mode)
  dgit: base trees orig=e949c82c22e8bc6e803a o+d/p=e949c82c22e8bc6e803a
  dgit: quilt differences: src:  ## orig ==     gitignores:  == orig ==
  dgit: quilt differences:      HEAD ## o+d/p               HEAD == o+d/p
  starting quiltify (multiple patches, linear mode)
  quiltify linearisation planning successful, executing...
  Commit Debian 3.0 (quilt) metadata

  [dgit (12.8) quilt-fixup]

  [master 6b56ef91] Commit Debian 3.0 (quilt) metadata
   2 files changed, 424 insertions(+)
   create mode 100644 debian/patches/merge-remote-tracking-branch-upstreammas.patch
   create mode 100644 debian/patches/series

Other dgit trees I've used figure out how to do one-commit-per-patch. I
think this is the normal way to be doing this, right? I'm not trying to
do anything weird and non-standard.

I had another earlier attempt that was complaining about the old debian/
stuff in the upstream tree, hence the title fo this bug. I'm now
thinking that's probably a red herring.

Thanks for the help.

Dima Kogan writes ("Bug#1124338: dgit: dgit is confused by merged branch with debian/"):

dgit-maint-merge says to use the 1.0 native source format.  So you
need to change the source format.  Then you don't need to specify a
quilt mode and dgit won't need to do anything with patches.

The other patches-applied format we recommend is
dgit-maint-debrebase.  Please refer to the manpage intros for guidance
on which to choose.

Ian.

I'm sorry to butt in like this, but I happened to be browsing dgit's bugs
and I noticed that Ian wrote:

I don't think that's correct.  At least as of
c99f8a83e0d5a1e454b37978f2f89dd7dde4f230, the only mention of "1.0" in
dgit-maint-merge.7.pod is to tell you that you don't need to create
debian/source/options if you're using a 1.0 package format.  And in my
very limited experience, dgit-maint-merge does indeed work perfectly well
at least with 1.0-with-diff, which is what I use for bedstead.

Dima Kogan writes ("Bug#1124338: dgit: dgit is confused by merged branch with debian/"):

I went to look at this bug again, and reread dgit-maint-merge(7).
It says to do this (and also to create a debian/source/options):

    git config dgit.default.quilt-mode single

https://manpages.debian.org/trixie/dgit/dgit-maint-merge.7.en.html#SOURCE_PACKAGE_AND_GIT_CONFIGURATION

This is explicitly requesting --quilt=linear, which won't work with a
merging git branch.  That's why dgit-maint-merge(7) says to use
--quilt-single (via a git config option).

So I think this was probably the root cause of the problem.

Since we last spoke about this I wrote a blog post:

  Adopting tag2upload and modernising your Debian packaging
https://diziet.dreamwidth.org/20851.html

Although (as Ben correctly points out) dgit-maint-merge can be used
with "3.0 (quilt)", in that post I recommend using 1.0 native.

You're right that this is all quite confusing.  Especially, it can get
very confusing if you try to do the transition and only get halfway
through it.

Please let us know if we can help any further.

Regards,
Ian.