Hi,

this is from the big tag2upload thread in #debian-devel. I'll just quote
an exchange between myself an rra here:
[...]
[...]

Since noone chipped in in a week, the feature seems to be indeed
missing.

Greetings
Marc

Marc Haber writes ("Bug#1128520: please implement dgit.default.tarball-dir"):

Thanks for recording this information in the BTS!

You're right, we don't have a feature that puts just orig tarballs
somewhere else.  We only have--build-products-dir (which I think is
analagous to gbp's export-dir) which puts *everything* there.

Marc, do you set both export-dir and tarball-dir ?  We (dgit) should
probably align our behaviour with gbp's and there will be corner cases
to consider.

Ian.

Ian Jackson <ijackson@chiark.greenend.org.uk> writes:
does. If build-products-dir is set, dgit does use that directory as an
*input* as well as an *output* get the orig.tar.gz file from that
directory. This is not the semantics of git-buildpackage as I recall (it's
been a few years), but it sort of makes sense.

The logging is a little weird in a way that makes that somewhat inobvious:

dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building libnet-ldapapi-perl using existing ./libnet-ldapapi-perl_3.0.7.orig.tar.gz

I assume this is because dgit has already changed working directories to
the build-products-dir. But I guess I never actually tried it; sorry about
that!

A common pattern with git-buildpackage if you don't want to use
pristine-tar (or that those of us who were using it before pristine-tar
existed always did) is to have a ../tarballs directory where you keep all
the upstream tarballs for all your various Debian packages, and then you
set build-dir to ../build-dir and tarballs-dir to ../tarballs. That way,
you can always rm * in ../build-dir after you finish uploading without
losing anything of value.

Marc of course should weigh in on what he was expecting, but that may be
what he was hoping for.

I don't really care (I can just run origtargz to get the tarball again in
places where I use pristine-tar), so now that I know build-products-dir
works, I'm mostly happy. The missing piece is telling origtargz to always
use that directory as well, but that's not a dgit problem.

Yes, you should. I set both. To my understanding, export-dir is only
written to¹, and tarball-dir is only read from. There is some
inconsistency even in the "legacy" tools regarding the tarball, and with
dgit having such an option, I have more "standing" with a wishlist
bug against other software to also have a tarball dir. Sadly, at the
moment I don't remember which other software insists on the tarball
being in ..

¹ I'd love to have export-dir templatable (for example, writing build
artifacts to ../build-area-<package>-<version>-<arch>), but gbp upstream
was not excited about that.

Greetings
Marc

Russ Allbery writes ("Re: Bug#1128520: please implement dgit.default.tarball-dir"):

Yes.

I set dgit.default.build-products-dir to ../bpd soon after I
implemented the feature (after someone else's request) and haven't
looked back.

Sadly git-deborig and origtargz don't use it.

In fact dgit is running dpkg-source in a secret hidden working area it
creates inside your .git directory.  I don't think there is any way to
improve this output from dpkg-source.

Oh there are *three* configurable directories!  I'm concerned that the
number of available corner cases is rising exponentially.  Maybe we
can satisfy most people by understanding their use cases.

Separating out the orig tarballs makes sense for the reason you give.
If we allowed you to do that then you could indeed always safely nuke
your bpd.
part is deciding on a common location where tools like this get their
information.

origtargz is in devscripts which doesn't really have a common
configuration scheme and I doubt trying to institute one there would
be either a good idea, or fun.

git-deborig is in src:dgit nowadays, so we *can* do what we feel best,
but what is best?

dgit uses git for its configuration and this has a lot of really
compelling advantages.  You get host-global, user-global and
tree-local config settings.  You get a taxonomy, a set of file formats,
can easily add a coherent commnad-line override facility.  You get the
config file reader and only have to interpret the values.  The data
model is pretty good.

I think it's a shame that gbp doesn't read git-buildpackage.whatever
things from git config.

Maybe we should invent a debian.* namespace for git config options and
attempt to run an internal registry for it.  git-deborig could use
that and dgit could fall back to debian.* if it doesn't find
dgit-distroy.debian.build-products-dir and
dgit.default.build-products-dir.

Marc Haber writes ("Re: Bug#1128520: please implement dgit.default.tarball-dir"):

We could institute a Debian-wide default for orig tarballs by putting
it in devscripts and making the tools there honour it.  It would want
some negotiation with the gbp maintainers but it doesn't seem
impossible.

I think we should prioritise getting all these different programs to
agree and if that means the functionality is more restricted then
that's better.

Personally I wouldn't object to such a scheme in principle but I am
already foreseeing difficulties with it: for example, dgit
doesn't always *know* what the architecture is.

We could probably define a substitution syntax, and some variables,
and tell programs that don't want to implement it (or a particular
substitution) that they should fail.  We might be able to persuade the
gbp maintainer that it would be OK to reject configurations
containing % with a message saying substitutions aren't implemented.
(That would be the minimum, because we wouldn't want to set ourselves
up for some programs to write to un-substituted templates as if they
were actual paths.)

Ian.

Are there bug reports requesting that?

"Changing work directory to ..."

#include <zugschlus-default-rant-about-no-technical-leadership-in-debian.md>

Somebody should write a tool to generate .gbp.conf from git config.

Sounds good to me.

That's what I would try to do (in the scope of my limited
possibilities). Having dgit walk ahead would make that considerably
easier.

Yes.

I'd basically go with the distinction between "source" and "not source"
builds. It's a pain that doing a binary buildpackage (for example, to
run Lintian) over an already built source package will render the source
package unuploadable since the .dsc gets overwritten.

Greetings
Marc

Marc Haber writes ("Re: Bug#1128520: please implement dgit.default.tarball-dir [and 1 more messages]"):

I don't know.  I thought I had filed at least one such *somewhere* but
I can't seem to find it.  I looked in the archived bugs for
git-deborig in devscripts (where it was for a lot of its life), too.
I found this
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=848816
which is kind of related and mentions a possible -devel conversation.

dgit changes working directory a *lot*.  Maybe we could have it print
the working directory explicitly before running important dpkg-source
commands, or other commands that have a tendency to print relative
pathnames.

Oooh!  Very good idea!  Are you volunteering? :-)
the devscripts thing would need to be an env var.

How about DEBIAN_ORIG_TARBALLS_DIR ?  It doesn't need to be short.

devscripts usually has a relatively relaxed maintenance approach so I
doubt we'd get any serious pushback, if we do it in devscripts first.
That would give folks a chance to bikeshed the env var name.

Also the implementation in dgit is nontrivial.

To be clear I think the semantics of DEBIAN_ORIG_TARBALLS_DIR ought to
be: programs that just process orig tarballs (like git-deborig and
origtargz) should *create* any *only* there.  They might look for them
only there, or in .. .

dgit looks for origs in .. as well as the bpd, and copies them into
the bpd if it finds them in .., to work around the lack of dir support
in git-deborig and origtargz.  Should it do the same with
DEBIAN_ORIG_TARBALLS_DIR: that is, if it finds a .orig in .., should
it copy it to the tarballs dir ?

Programs that generate .dscs still need to put the origs next to the
.dsc, which means that origs now might end up in more places.
Specifically, they would end up in the bpd *and* in
DEBIAN_ORIG_TARBALLS_DIR.  Should we use hardlinks or something or
does everyone already just expect them to be copied about?

(How do other tools that invoke dpkg-source cope with the fact that
dnpkg-source insists on expecting origs next to the .dsc .. ?)

Hmm, yes.  I still think this is a more complicated quesstion.

Ian.