#743617 Option for pure chaining

#743617#5
Date:
2014-04-04 08:42:26 UTC
From:
To:
Let's say A chains to B, then A has a stanza [B], which usually also
includes a checkout command.

Within B, if . should also be controlled by mr, then it needs
a stanza for [.]. Now, if mr is run in the context of A, the B root
repository gets processed twice:

  mr status: /home/madduck/debian/debconf/team/pub-data

  mr status: /home/madduck/debian/debconf/team/pub-data/.

which is mostly cosmetical until you start using -j and potentially
have two processes work on the same repo at the same time, which is
just asking for trouble.

I think this could be done in two ways:

  1. Either chain==true repos should only ever be processed for
     "checkout", but nothing else, assuming that the sub-repo
     specifies [.];

  2. One could introduce chain==pure as a keyword to indicate this
     condition.

#743617#10
Date:
2014-09-09 03:49:02 UTC
From:
To:
also sprach martin f krafft <madduck@debian.org> [2014-04-04 10:42 +0200]:
[…]

I think there may be a third way:

    3. If mr is run in a subdirectory, which is specified in
       a parent .mrconfig file, this should be honoured, i.e. the
       subdirectory managed according to a parent directory
       .mrconfig stanza.

So let's say ~/some/path/.mrconfig specifies

  [subdir]
  checkout = …
  chain = true

and ~/some/path/subdir/.mrconfig specifies

  [another]
  checkout = …

then, with the status quo, running mr in ~/some/path will work on
two repos, as it should.

However, running mr in ~/some/path/subdir will only run over
"another", not the repo in the CWD itself, and I don't really see
why it should do that.

The manpage specifies:

    mr is configured by .mrconfig files, which list the
    repositories. It starts by reading the .mrconfig file in your
    home directory, and this can in turn chain load .mrconfig files
    from repositories. It also automatically looks for a .mrconfig
    file in the current directory, or in one of its parent
    directories.

So when run in ~/some/path/another, it should read ./.mrconfig, but
it should also read ../.mrconfig (according to the manpage) and
hence process the current directory as well, according to the stanza
in the parent .mrconfig.

Am I overlooking something?

#743617#15
Date:
2014-09-09 03:51:04 UTC
From:
To:
The hack to work around this weirdness seems to be to write

  [.]
  skip = test $PWD = $OLDPWD

into ~/some/path/subdir/.mrconfig. This has the side effect that the
counter of skipped repositories is +1 on runs in ~/some/path, which
may be confusing, but when run in ~/some/path/subdir, the current
directory is included.

#743617#20
Date:
2014-09-09 05:04:12 UTC
From:
To:
also sprach martin f krafft <madduck@debian.org> [2014-09-09 05:49 +0200]:

After some more debugging on this, I am now fairly certain that the
code does not do this, at least not in "bottom-up" fashion.

Here's what I mean with "bottom-up" vs. "top-down":

  Top-down is when ~/.mrconfig chains to ~/subdir/.mrconfig, which
  might chain to ~/subdir/another/.mrconfig. If I now run mr in
  ~/subdir/another, ~/subdir/.mrconfig is sourced.

  Bottom-up is when ~/.mrconfig does not know about
  ~/subdir/.mrconfig or ~/subdir/another/.mrconfig. If I now run mr
  in ~/subdir/another,

    - ~/.mrconfig and ~/subdir/another/.mrconfig *are* being read
    - but ~/subdir/.mrconfig is *not* read

I think this is an inconsistency. When run in bottom-up fashion,
then either ~/.mrconfig should also be ignored, or all parent config
files need to be included.

If I had a vote, I'd say that all parent config files should be
included.

#743617#25
Date:
2014-09-09 05:37:11 UTC
From:
To:
also sprach martin f krafft <madduck@debian.org> [2014-09-09 07:04 +0200]:

The loadconfig() function does something prefixed by a comment:

  # copy in defaults from first parent

in which it walks up the tree until it finds a config file that
defines a DEFAULT section. It then loads this section into the
current config space, and ends the loop, i.e. it only copies from
the first parent, as specified in the comment.

I do not understand why it does this. Can you enlighten me? Why are
you loading ~/.mrconfig explicitly and unconditionally, and merge
DEFAULT from the first parent, rather than just backtracking the
tree and loading all configs it finds? You can still load
~/.mrconfig first and explicitly, as the code has provisions to
prevent double-loading.

Thanks,

#743617#30
Date:
2014-09-09 19:06:21 UTC
From:
To:
martin f krafft wrote:

This is already how mr behaves, as long as ~/.mrconfig chains to
~/some/path/.mrconfig.

If ~/.mrconfig does not chain to ~/some/path/.mrconfig,
then mr is left looking for the first .mrconfig file it
can find at or under the current directory. So, when run in
/some/path/subdir/, it finds that .mrconfig, and ~/.mrconfig, but not
any other .mrconfigs you may have scattered around, and behaves as you
describe.

The solution, then, is to use chaining from ~/.mrconfig ...


Regarding the original message in this bug report:

  mr status: /home/madduck/debian/debconf/team/pub-data

  mr status: /home/madduck/debian/debconf/team/pub-data/.

Yes, it can sometimes make sense to have a [.] entry in a
.mrconfig. d-i does this for example. And then you get a duplicate
entry for that. A solution might be for mr to do path normalization
and throw out such duplicates. But then that leaves the knotty question
of which one to throw out, the [.] one or the one in the parent repository
that chains to it? Both could concevably have different update or other
actions defined..

Perhaps, then, it's best to leave this to the mr user to decide.
It's easy enough to configure mr to not update in one of the duplicates
if desired. For example, after noticing I had this problem in my configuration,
I now have:

[d-i]
checkout = svn co svn+ssh://joeyh@svn.debian.org/svn/d-i/trunk d-i
chain = true
update = : # handled by d-i's mrconfig

#743617#35
Date:
2014-09-09 19:14:32 UTC
From:
To:
martin f krafft wrote:

This would mean that if you cloned d-i into /tmp and followed the
instructions to use mr in that repo, /tmp/.mrconfig would also be read,
possibly quite unexpectedly.

#743617#40
Date:
2014-09-09 19:10:35 UTC
From:
To:
martin f krafft wrote:

You make it sound like it's walking the filesystem tree, but it's not,
it's messing around in its data structures from the configs it's already
loaded.

I have not touched this code since it was written, but I belive this is
how DEFAULT stanzas get inherited around to be available by other
stanzas.

#743617#45
Date:
2014-09-09 21:47:19 UTC
From:
To:
also sprach Joey Hess <joeyh@debian.org> [2014-09-09 21:06 +0200]:

Is there a reason why it shouldn't backtrack up the tree?

Nice!

So saying something like chain=pure is really the same, except it
would handle status/log/diff etc. as well, and implicitly. Maybe
there is also a better way to do it, e.g. chain=true and then
something like skip_all_but_checkout=true.

#743617#50
Date:
2014-09-09 21:51:00 UTC
From:
To:
also sprach Joey Hess <joeyh@debian.org> [2014-09-09 21:10 +0200]:

It's actually using dirname($dir) in a while loop, but it uses the
dirname to index into the $config structure. So: both. ;)
first parent's $config{'DEFAULT'} into the current DEFAULT, then
is this first parent's DEFAULT guaranteed to contain its parent's
DEFAULT at the time of the merge? I cannot really gather this from
the code, maybe because I am looking at it with a different lense
(cf. my interpretation of backtracking without the need for chaining
in #743576).

#743617#55
Date:
2014-09-09 21:54:09 UTC
From:
To:
also sprach Joey Hess <joeyh@debian.org> [2014-09-09 21:14 +0200]:

This is true. Good point.

All I am trying to do is lowering the barrier of entry for
debconf-team and possibly similar situations:

https://wiki.debconf.org/wiki/GitMigration#Help.21_Too_many_repositories.21

And I think requiring people to create ~/.mrconfig or adding the
checkout to their existing config is such a barrier.

But maybe I am also seeing pink elephants…

#743617#60
Date:
2014-09-10 00:44:03 UTC
From:
To:
martin f krafft wrote:

skip = ! lazy will work, although it may not be very obvious.

#743617#65
Date:
2014-09-10 04:03:21 UTC
From:
To:
also sprach Joey Hess <joeyh@debian.org> [2014-09-10 02:44 +0200]:

This will also skip the initial checkout, unfortunately.

#743617#70
Date:
2014-09-10 20:25:35 UTC
From:
To:
martin f krafft wrote:

No, like I said it's not very obvious, but it works. Runs initial
checkout and then skips any operations of the repository once it exists.
If you think about it, that is the logical opposite of lazy, which
skips the initial checkout but runs everything once the repository exists.

I'd be fine with adding a better name for ! lazy to fix this bug.