Hi,
after a "dgit push" of the package src:pdfrw I got a REJECT from
ftpmaster because "Source-only uploads to NEW are not allowed". Since
the source package is not yet in Debian I guess this talks about an
upload to binary-NEW.
Here are two issues I have with this (close this bug as you see fit):
1. dgit is Debian specific so I guess it might be reasonable to assume
that it encodes Debian specific rules like the above such that I can
never do a "dgit push" of a source-only .changes file if there are
new binary packages in the upload compared to the last
2. I do not see documentation of how to fix this situation. Is the only
way around to bump my Debian revision to -2, rebuild with binaries
and then do "dgit push" again? Or is there a way to mangle my
source-only .changes file file I already have such that it contains
the new binary packages while at the same time doesn't break any of
dgits expectations?
Thanks!
cheers, josch
Johannes Schauer writes ("Bug#801435: dgit: avoiding/fixing "Source-only uploads to NEW are not allowed""):
...
Looking at the ftpmaster db I think you misspoke. You said "the
source package is not yet in Debian" but in fact it appears that 0.1-3
is in sid.
By definition a source-only upload doesn't involve dgit seeing the
binary packages. I'm not even sure that it is possible to determine
which binary packages ought to be produced without doing the actual
build. So this would be tricky. (But how does the dak tell: does it
process debian/control?)
Also it would be possible for dgit to see that the package is entirely
new (which I think it is in your case) and to know to reject a
source-only upload in that case.
dgit is not Debian-specific, but it does have the ability to apply
different rules to different archives. OTOH the more policy
information like this I encode in dgit, the more I have to track
ftpmaster's policy changes. I don't know if this ftpmaster rule is
likely to change.
Ideally you should bump the revision, yes, because the old tag and
signed .dsc and .changes for the old package are out in the wild with
your signature on.
You will need to tell dgit --deliberately-include-questionable-history
because it doesn't know why the package was REJECTed and it wants to
be told that it's not because of copyright problems (in which case
ideally you would provide a differnet git history as well as a
different package).
In some circumstances you might need to say --dpkg-genchanges:-sa I
think, to tell dgit to tell dpkg-genchanges to include the original
source, or --dpkg-genchanges:-sd to tell dgit tell dpkg-genchanges not
to. I'm not 100% sure which the archive requires but given that 0.2-1
is in new I guess it can reuse the old .orig, in which case the
default should DTRT. Please let me know what you discover. TBH I
think dgit should figure this out for itself but I'm not sure of the
exact rules.
Ian.
Hi, Quoting Ian Jackson (2015-10-10 18:07:02) whoops, that sentence indeed had a *not* too many. Yes, the package was in sid and my upload added two more binary packages, meaning it is now in binary-new. As far as I remember, even source packages which generate debian/control from a debian/control.in must ship their generated debian/control. So it should be possible to use debian/control together with dpkg-genchanges to retrieve the list of binary packages. I do not know either what dak does. Maybe it uses the Binary field of the .changes file which lists the binary packages of the uploaded source package even for a source-only upload? That field is also part of the .dsc. Is there a "not" missing above? dgit could compare the Binary field of the previous .dsc file with the one of the new .dsc file and check if there are any changes in its value. Sorry me neither. I just know that it was bothersome for me that I only noticed my error after the "dgit push" and then had no idea how to fix the problem because I cannot just rebuild and dput again as I'd otherwise do. But they didn't reach the archive yet, so they've been deleted. Not even I could retrieve the stuff I uploaded from there anymore. Aha, so --deliberately-include-questionable-history is the knob I have to turn. That wasn't clear to me from the documentation before you told me to use it, and reading the documentation again it is still not clear to me how that option works, why it exists and how it fixes my problem. From the context I assume that this option lets me do a `dgit push` even though I already pushed that exact same version? If so, then maybe this should be mentioned in the description of the option? I was also wondering about why the mentioning of copyright and redistributability reasons is important there. I can think of many other reasons why there could be a REJECT after a dgit push including the one we talked about in #800060 which was REJECTED because of a hash sum mismatch. So is --deliberately-include-questionable-history indeed the right option I want to do a `dgit push` of a fixed .changes or .dsc file after a REJECTED upload (assuming I didn't change anything of the packaging but just did a rebuild with different options)? So what I ended up doing (which might've been totally wrong but in that case it'll serve as a funny experiment) was to do a rebuild of the package with `dgit sbuild`, then added a field Dgit to the .dsc file containing the git commit hash of my top commit, then adjusted the .changes file with the changed size and hashes of the .dsc, then used debsign to sign both and then dput-ed the .changes file. Clearly, the upload now ended up in binary-new but was doing this also okay from the dgit side? Thanks for all your help! cheers, josch
Hi, Quoting Johannes Schauer (2015-10-10 20:34:40) I just so happened to have made the same mistake and used `dgit push` with a source-only .changes file even though the values of the Binaries field of the .changes and .dsc changed compared to the last version. Thus I got, again, a reject. So I tried using --deliberately-include-questionable-history when doing `dgit push` but this just results in: josch@hoothoot> dgit push --deliberately-include-questionable-history canonical suite name for unstable is sid downloading http://ftp.debian.org/debian//pool/main/f/fuzzylite/fuzzylite_5.1+dfsg-1.dsc... last upload to archive has NO git hash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1221k 100 1221k 0 0 2038k 0 --:--:-- --:--:-- --:--:-- 2042k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 12364 100 12364 0 0 105k 0 --:--:-- --:--:-- --:--:-- 106k dpkg-source: info: extracting fuzzylite in fuzzylite-5.1+dfsg dpkg-source: info: unpacking fuzzylite_5.1+dfsg.orig.tar.xz dpkg-source: info: unpacking fuzzylite_5.1+dfsg-1.debian.tar.xz dpkg-source: info: applying pkgconfig_installdir Format `3.0 (quilt)', checking/updating patch stack Format `3.0 (quilt)', checking/updating patch stack synthesised git commit from .dsc 5.1+dfsg-1 HEAD is now at f327338 release 5.1+dfsg-2 Version actually in archive: 5.1+dfsg-1 (older) Last allegedly pushed/uploaded: 5.1+dfsg-2 (newer or same) Perhaps the upload is stuck in incoming. Using the version from git. Format `3.0 (quilt)', checking/updating patch stack nothing quilty to commit, ok. checking that fuzzylite_5.1+dfsg-2.dsc corresponds to HEAD dpkg-source: warning: extracting unsigned source package (../../../../fuzzylite_5.1+dfsg-2.dsc) dpkg-source: info: extracting fuzzylite in fuzzylite-5.1+dfsg dpkg-source: info: unpacking fuzzylite_5.1+dfsg.orig.tar.xz dpkg-source: info: unpacking fuzzylite_5.1+dfsg-2.debian.tar.xz dpkg-source: info: applying pkgconfig_installdir Format `3.0 (quilt)', checking/updating patch stack Format `3.0 (quilt)', checking/updating patch stack You need a passphrase to unlock the secret key for user: "Johannes Schauer <josch@mister-muffin.de>" 4096-bit RSA key, ID 8FBD83E1, created 2013-07-04 (main key ID CF4D3EB4) gpg: Signature made Mon 26 Oct 2015 11:07:58 AM CET using RSA key ID 8FBD83E1 gpg: Good signature from "Johannes Schauer <josch@mister-muffin.de>" gpg: aka "Johannes Schauer <j.schauer@email.de>" gpg: aka "Johannes Schauer <josch@debian.org>" To git+ssh://dgit@push.dgit.debian.org/dgit/debian/repos/fuzzylite.git ! [rejected] debian/5.1+dfsg-2 -> debian/5.1+dfsg-2 (already exists) error: failed to push some refs to 'git+ssh://dgit@push.dgit.debian.org/dgit/debian/repos/fuzzylite.git' hint: Updates were rejected because the tag already exists in the remote. dgit: failed command: git push 'git+ssh://dgit@push.dgit.debian.org/dgit/debian/repos/fuzzylite.git' HEAD:refs/dgit/sid 'refs/tags/debian/5.1+dfsg-2' dgit: subprocess failed with error exit status 1 ! Push failed, while updating the remote git repository - see messages above. ! If you want to try again, you should use a new version number. So this doesn't seem to have worked. What is the right fix then? Thanks! cheers, josch
Johannes Schauer writes ("Bug#801435: dgit: avoiding/fixing "Source-only uploads to NEW are not allowed""):
I think I must have confused myself and you when I wrote my earlier
advice.
In Debian
dgit push --deliberately-include-questionable-history
is applicable only for a completely-NEW source package.
Ian.
Hi, Quoting Ian Jackson (2015-10-26 12:39:17) okay, I see. What do I do after I did `dgit push` and got a reject because binary-NEW doesn't allow source-only uploads? Thanks! cheers, josch
Johannes Schauer writes ("Bug#801435: dgit: avoiding/fixing "Source-only uploads to NEW are not allowed""):
Bump the package version.
Ian.
As I wrote in my initial response: But: I think this is the only really actionable work item in this bug report. Retitling it accordingly. Thanks, Ian.
We believe that the bug you reported is fixed in the latest version of
dgit, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 801435@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Ian Jackson <ijackson@chiark.greenend.org.uk> (supplier of updated dgit package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Thu, 05 Jul 2018 15:02:21 +0100
Source: dgit
Binary: dgit git-debrebase dgit-infrastructure
Architecture: all source
Version: 5.7
Distribution: unstable
Urgency: medium
Maintainer: Ian Jackson <ijackson@chiark.greenend.org.uk>
Changed-By: Ian Jackson <ijackson@chiark.greenend.org.uk>
Closes: 801435 878443 903006 903007
Description:
dgit - git interoperability with the Debian archive
dgit-infrastructure - dgit server backend infrastructure
git-debrebase - rebasing git workflow tool for Debian packaging
Changes:
dgit (5.7) unstable; urgency=medium
.
New feature:
* dgit checkout: new subcommand. Closes:#878443.
* dgit: Check that entirely-new uploads to Debian are not
source-only-uploads, as those are REJECTed. Closes:#801435.
.
Bugfixes:
* dgit(7): Mention git-debrebase and gbp pq alongside git-dpm,
in the comment about handling patch stacks.
* dgit update-vcs-git: Honour --package properly.
* test suite: Always pass LC_COLLATE=C to sort(1). Closes:#903006.
* test suite: Fix trustingpolicy-replay & dput-ng. Closes:#903007.
* test suite: Test dput-ng compatibility.
Checksums-Sha1:
6d794001bfb0c6dbaa3f74fa599814197fbdb527 1633 dgit_5.7.dsc
3c444f279c57e4eea53572c551873f8db7a4bcf1 457975 dgit_5.7.tar.gz
12fda31f1b98952fe9a2e0cec25df274f22f859a 54120 dgit-infrastructure_5.7_all.deb
2c5ec9116b59e1b4eab6f4ed51d7ff2cc988c5c4 155120 dgit_5.7_all.deb
b579480e707880d16c8f1fa7d5a81d9ff0c28407 5662 dgit_5.7_amd64.buildinfo
a88bb8de9e8da5939a6107486e936abdd30ad709 64180 git-debrebase_5.7_all.deb
Checksums-Sha256:
c1e3fd16d3755388c0242593bd0358c61d5fb2d7803507a4d5db64061c261bc6 1633 dgit_5.7.dsc
c3b2c8c2f93acb2c80cf88bbba59e2946ad3ddd5b0e861364206eb88c00e80fb 457975 dgit_5.7.tar.gz
3b432c2270661286fadeb81839739a5296f675bf5fca8092bb698132ca7e0cd8 54120 dgit-infrastructure_5.7_all.deb
35e4b45db61d756951de38c410e7dd5466a06f774138364a5a332d0b73b02f30 155120 dgit_5.7_all.deb
2e6c551ac2511a7299f8b77911c1331b44180118e4a51c65dfd94c799df6fbac 5662 dgit_5.7_amd64.buildinfo
7a40076c74b19b2bbdf6e9d6b6c6c26fc16f2f50079241059b4daaf58879667c 64180 git-debrebase_5.7_all.deb
Files:
cb1a8586fd764bb6b01fa4fe37a0df41 1633 devel optional dgit_5.7.dsc
b3a9c88e20686fa49f2b517e89c230ee 457975 devel optional dgit_5.7.tar.gz
00b1f6f2985ce4b6bcc3bc0dd9b9b8b5 54120 devel extra dgit-infrastructure_5.7_all.deb
2a0bbe2d639da404600e02ac58ba5f09 155120 devel optional dgit_5.7_all.deb
d157592958e9a85c4dbfca483bf0453c 5662 devel optional dgit_5.7_amd64.buildinfo
c8b07a0bcfa8ac601c07c41ca7a73c65 64180 devel optional git-debrebase_5.7_all.deb
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEVZrkbC1rbTJl58uh4+M5I0i1DTkFAls+JU8ACgkQ4+M5I0i1
DTk88wgAkk5tNdQO02t2TKyUDzXnLtMhYPyfJs8Gg0c9XeM5woE8E5G6ZL88UxJ4
++nzZpmpvu7Nos9jLLSHVK5Y0PFcbhV4cF6CPUp03w+LmQkMx83uHWgrFCPv8DlP
8CKg7xF1+p6e8lfoeHZnmExFF7AOhgtA83MWm9zclv0IQkU7HbwuEZb1vGPhHr+e
BXmOq7anAjKJ2/pttzmb++SEICvZ27psqke96HUE60eiDRuq6eW7wfHqEiM1DGvM
Xy6GKSVRw4zitDp9mJAoW63fD2cMioEZfYqgFf5oEuwBAgo90d1n7BJANCgMtCv9
+PdSO8k2dnutnP5lmOV8yg+CAEgXmA==
=kePz
-----END PGP SIGNATURE-----
Hi, Quoting Debian Bug Tracking System (2018-07-05 16:30:04) I didn't investigate further, but this sounds as if dgit only takes care that one cannot do a source-only upload for entirely new packages. But my initial message was talking about uploads with new binary packages that have to first go to binary-NEW. Thus, maybe this bug is actually not fixed by this version? Thanks! cheers, josch
Johannes Schauer writes ("Bug#801435: closed by Ian Jackson <ijackson@chiark.greenend.org.uk> (Bug#801435: fixed in dgit 5.7)"):
Arguably you are right. The problem is that detecting binary-NEW is
not anywhere near as easy as detecting source-NEW.
I think in principle it may be possible, but it would involve dgit
parsing your debian/control to extract the list of binary packages
(which I think is required to be accurate in Debian?) and correlating
it with the archive. If you think that would be valuable then please
say and I will clone this bug. I'm afraid it won't get done any time
soon though...
Ian.
Hi, Quoting Ian Jackson (2018-07-05 17:04:44) I ran into this issue *twice* so personally I would love it if dgit could stop me from making this mistake. I understand that it's a hard problem to fix but if you don't mind this bug hanging around in the bts for a while, then I'd appreciate you cloning it. :) Thanks for dgit! cheers, josch
Johannes Schauer writes ("Bug#801435: closed by Ian Jackson <ijackson@chiark.greenend.org.uk> (Bug#801435: fixed in dgit 5.7)"):
NP.
You're welcome.
Ian.
For the purposes of tracking interest, I just ran into this problem with a package that introduced a new binary package, and would also like dgit to be able to detect this situation and warn me before doing a push-source. Parsing debian/control to get a list of built binaries would work for my use case. For what it's worth, I also always follow a practice of running dgit sbuild before dgit push-source, so dgit could have detected the binary packages that way in theory, but I'm not sure if that's common.
Hello Russ, Wouldn't dgit have to connect to the Internet in order to perform the detection? I'm not sure we want a build command performing any connections.
Sean Whitton <spwhitton@spwhitton.name> writes: My thought process was more that since I've run dgit sbuild, there's a *.changes file that lists all the binary packages that the source builds. But there probably isn't any reasonable way for push-source to be able to use that *.changes file to know the binary package list, since it doesn't know that any *.changes file is from the same package that one is calling push-source on.
Hello, I see what you mean. If there is a single pkg_*.changes containing binaries in the build-products-dir, it is probably safe for dgit to use that to get a list of the binary packages built by 'pkg'. After all, this is only a check to prevent mistakes, not a process which determines any of the contents of uploads.