Hi, As discussed on IRC, I had a slightly bad experience using dgit to prepare an NMU for f2fs-tools. paul@mulciber ~/packages/bugs $ dgit clone f2fs-tools canonical suite name for unstable is sid fetching existing git history last upload to archive: NO git hash using existing f2fs-tools_1.11.0.orig.tar.gz using existing f2fs-tools_1.11.0-1.1.debian.tar.xz dpkg-source: info: extracting f2fs-tools in f2fs-tools-1.11.0 dpkg-source: info: unpacking f2fs-tools_1.11.0.orig.tar.gz dpkg-source: info: unpacking f2fs-tools_1.11.0-1.1.debian.tar.xz synthesised git commit from .dsc 1.11.0-1.1 Version actually in archive: 1.11.0-1.1 (older) Last version pushed with dgit: 1.12.0-1 (newer or same) Perhaps the upload is stuck in incoming. Using the version from git. HEAD is now at fa204f3 Declare fast forward from 1.11.0-1.1 dgit ok: ready for work in f2fs-tools So apparently the previous upload was done with dgit, cool.... Unless that previous upload included a new upstream version and the upload went through new and the upload was rejected... paul@mulciber ~/packages/bugs $ cd f2fs-tools paul@mulciber ~/packages/bugs/f2fs-tools $ dch -i "bla" paul@mulciber ~/packages/bugs/f2fs-tools $ dch -r "" paul@mulciber ~/packages/bugs/f2fs-tools $ git commit -m"Commit changelog" debian/changelog [dgit/sid 003cb83] Commit changelog 1 file changed, 7 insertions(+) paul@mulciber ~/packages/bugs/f2fs-tools $ dgit build-source Format `3.0 (quilt)', need to check/update patch stack gzip: warning: GZIP environment variable is deprecated; use an alias or script dgit: split brain (separate dgit view) may be needed (--quilt=unapplied). examining quilt state (multiple patches, unapplied mode) dpkg-source: error: missing orig.tar or debian.tar file in v2.0 source package dgit: failed command: sh -ec 'exec dpkg-source --no-check --skip-patches -x fake.dsc >/dev/null' dgit: error: subprocess failed with error exit status 255 The new upsteam tar ball is missing, and as the upload was rejected, I don't know where to obtain that tar ball. Once I discovered the problem (I had initially missed the "Perhaps the upload is stuck in incoming. Using the version from git." part), it would have helped if it was clear to me where to start working if I wanted to have the version in unstable. In the end I resorted to paul@mulciber ~/packages/bugs $ dgit clone f2fs-tools testing as unstable and testing have the same version, but that doesn't work if unstable and testing don't have the same version. Paul
Hello, In this situation what it seems we want to achieve is a) get the version you want to hack on into dgit as if `dgit clone` had given it you b) make it easy to `dgit push` your new version, which is based on the result of (a). How dgit could automate this is if it could detect that the upload on dgit-repos never made it to the archive, it could then prepare a new repo for you as if there was nothing on dgit-repos, and then pseudomerge that on top of the contents of dgit-repos, so a push would just work. However, detecting whether an upload made it to the archive would require incorporating a lot of idiosyncratic knowledge about dak into dgit, I think, with a fair bit of guessing? Or is the way that dak keeps track of such things amenable to adding a new ftp-master API query to find out? In the meantime what I would have done is `apt-get source` followed by `dgit import-dsc` followed by pseudomerging in the result of `dgit fetch unstable`. What do you think about the error message suggesting that for this sort of situation?
Sean Whitton writes ("Bug#977845: dgit: unhelpful behavior in case previous upload contained new upstream release"):
The problem is that in this situation the person who uses dgit clone
does not get the orig tarball and has no systematic way to find it.
dgit clone already knows this situation has arisen, because it can see
that the version in git is newer than the version in the archive.
That wouldn't work either, because apt-get source doesn't have access
to the orig tarball.
The original uploader had the orig tarball. dgit push sent it to the
archive. But I think if the upload was REJECTed (or dcut or
something), the archive doesn't have it any more. I don't think
ftpmaster keep uploaded things that don't end up in the archive.
(Please correct me if I'm wrong.)
I think the solution has to involve dgit push messing about with
pristine-tar to send a pristine-tar branch to dgit-repos :-/.
Ian.
Hi Ian, Sean, Indeed. And the fact that I didn't expect to get a version that's not in the archive. Yes, there's a message, but it's buried in other output. Also, for NMU's, I think it's more natural to work on the version in the archive, not a version that's uploaded but didn't appear (yet) in the archive. But I think Sean is suggesting to make sure I also get something like "dgit clone XXX unstable", i.e. I can tell dgit to make sure I get something that reflects unstable, even though a later upload with dgit happened. Without knowledge, yes, a fair bit of guessing I think. I have no knowledge of dak on this front. Exactly, it even tells me so (after I went back and read the output). However, what dgit could do in such a case is create a local situation that has both the situation in the archive I'm getting around with git, but you'd have to explain quite literally how to get there, as it's unclear to me what "pseudomerging" means. I'm not sure, but I think you're right. Paul
Hello, It can't distinguish between stuck in some dak queue vs. actually rejected, though? I must be missing something here, because I don't see why the rejected orig.tar is needed. apt-get source will get you the orig.tar for the version actually in the archive, which is what you want to base your work off?
Sean Whitton writes ("Bug#977845: dgit: unhelpful behavior in case previous upload contained new upstream release"):
dgit doesn't currently distinguish that. Mostly because I thought the
information is not available.
situation you should start from the version in git rather than the
version in the archive.
This is in large part because if you do "dgit clone" in the period
between "dgit push" and archive publication, you probably didn't want
to revert what the last uploader did.
I hesitate to put it like this, but: it is difficult to represent the
Debian archive in something moderately sensible like the git data
model, because the Debian archive is slow[1], opaque[2],
capricious[3], and racy[4].
Ideally the situation I would like is that if someone does dgit push,
and has the package REJECTed for some reason, their co-maintainer
should be able to fix it without having to do more than "dgit fetch",
hack hack hack, "dgit push". For this the co-maintainer needs the new
orig, not the one in the archive.
If a dgit-clone-using NMUer wants to upload from the last ACCEPTed,
either the git history has to diverge (and then what?), or the NMUer
has to make a pseudomerge which effectively reverts the
new-source-package upload.
This is all quite a mess. I think it's a worse mess to try to go
backwards, than to treat the uploaded-but-REJECTed version as the
baseline.
Ian.
[1] Several hours from upload to visibility in ftpmaster data API, in
the best case. Significant fractions of a year in the worst.
[2] As I understand it, you can't tell what's queued and whether
things are held back, or rejected, or what.
[3] Software can only do a very poor job of predicting whether a
particular package will go via the fast path, or the slow path of
NEW, or the very slow path of sitting in NEW for months while
people procrastinate and/or argue.
[4] Proper use of the Debian archive data model requires external
synchronisation on a per-source-package basis, to avoid lost
updates. This is because the update operation does not guarantee
that of multiple concurrent updates, at most one will return
"success" for the synchronous result of the upload attempt. In
dgit's data model, this synchronisation is provided by the
dgit-repos git server. In the trad Debian data model this is
provided by unstructured and manual human coordination.