Hello Goswin,
Attached is a rough patch to allow more fine grained mirroring than is
currently possible. The gist of the patch logic is that as
arch/dist/section tuples are parsed, on duplicate option or end of line,
defaults are filled in. the data structure is stored as nested hashes
instead of arrays, so that it becomes possible to request something like:
$VAR1 = {
'sid' => {
'amd64' => {
'non-free' => 1,
'contrib' => 1,
'main' => 1
},
'i386' => {
'contrib' => 1,
'main/debian-installer' => 1,
'main' => 1
}
},
'etch' => {
'mips' => {
'non-free' => 1,
'contrib' => 1,
'main/debian-installer' => 1,
'main' => 1
}
},
'sarge' => {
'powerpc' => {
'non-free' => 1,
'contrib' => 1,
'main' => 1
},
'sparc' => {
'non-free' => 1,
'contrib' => 1,
'main/debian-installer' => 1,
'main' => 1
}
}
};
(Data::Dumper output from a debug run)
As you can see, it allows more fine grained mirroring than is currently
possible. I would appreciate if you could apply this patch or something
like it. I have not chased all the corner cases yet, so I can't say
with absolute confidence it introduces no regressions, but it seems sane
from the tests I've run so far.
Thanks,
This one time, at band camp, Stephen Gran said: Hey, Have you had a chance to look at this? Do you mind if I NMU it? I would really like to see this feature make it into etch. Thanks,
Hello Stephen and Goswin,
Wow, great! This is exactly what I was going to request! --dry-run shows
that the expected Packages have been downloaded. I'll test this more.
This is definitely a useful feature!
What I'd like to see:
1. Include --nosources in the tuple.
2. Documentation.
With kind regards,
Baurzhan.
P.S. My first mail was rejected by spamhaus. Where should I direct
requests about b.d.o mail infrastructure?
While this patch tries to address a real issue (also covered by wishlist bug #282636 and worked around by #369061 for the special case of sarge and amd64), I find it unsatisfactory for the following reasons: 1. It isn't backwards-compatible, in that it changes the semantics of previously-valid invocations of debmirror. 2. If I read it correctly, --dist=etch --arch=i386 --arch=amd64 is not equivalent to --dist=etch --arch=i386,amd64 as I at least would expect (and, in this particular case, as backwards compatibility would require) but to (etch,,i386) (sid,,amd64) where I'm using a (dist,section,arch) tuple notation to avoid confusion with the current debmirror semantics. Currently, debmirror takes three sets (@dists, @sections, @arches) and tries to mirror all tuples in the cross-product (except for a few that do not exist in the official archive). I don't see any way to do more fine-grained mirroring without supporing more set arithmetic. At a minimum one would like a union operator (say, an --or switch on the command line that does more or less the same thing as &add_data in the proposed patch). This could also be achieved by multiple invocations of debmirror, if the cleanup step could be taught not to delete valid files from other invocations. A set substraction operator (--except ?) may also be useful, but then you'd start needing parentheses. So maybe the simplest way to rescue this patch is to add a --or switch that invokes &add_data, and to let --dist, --section and --arch simply accumulate into @dists, @sections and @arches as before.
Hi,
I added a new option to debmirror (--localclean) to allow multiple
debmirror runs with different arguments to be run. This allows fine
grained mirroring by splitting the mirroring into multiple runs. E.g:
debmirror -a i386 -s contrib -h ftp.debian.org --localcleanup /mirror/debian
debmirror -a amd64 -s main -h ftp.debian.org --localcleanup /mirror/debian
Make a mirror of i386 contrib and amd64 main.
I believe that this is a far simpler fix for the problem and does not
destroy backward compatibility. It also allows mirroring
ftp.debian.org and security.debian.org into a single local mirror and
as such is superior to the proposed patch.
Is that sufficient for you?
MfG
Goswin
Hi,
I added a new option to debmirror (--localclean) to allow multiple
debmirror runs with different arguments to be run. This allows fine
grained mirroring by splitting the mirroring into multiple runs. E.g:
debmirror -a i386 -s contrib -h ftp.debian.org --localcleanup /mirror/debian
debmirror -a amd64 -s main -h ftp.debian.org --localcleanup /mirror/debian
Make a mirror of i386 contrib and amd64 main.
I believe that this is a far simpler fix for the problem and does not
destroy backward compatibility. It also allows mirroring
ftp.debian.org and security.debian.org into a single local mirror and
as such is superior to the proposed patch.
Is that sufficient for you?
MfG
Goswin
This one time, at band camp, Goswin von Brederlow said: That's great, thanks very much.
This one time, at band camp, Goswin von Brederlow said: That's great, thanks very much.
This one time, at band camp, Goswin von Brederlow said: That's great, thanks very much.
Sadly, there is still no such option in the current version of debmirror (20070123), and the proposed patch didn't get applied either. I would like to see more fine grained mirroring too, specifically I need a way to mirror three combinations: lenny-i386, squeeze-i386 and squeeze-amd64 - in one directory tree.
In 2007 you wrote: I guess this --localclean was not in the released verison of debmirror when Frans took it over, and so was lost. I'm not clear on what it did. Do you have the patch or remember? It must be different than --nocleanup somehow, but I don't understand how. Using --nocleanup in this scenario wouldn't quite work, because while the multiple sources could be combined, old files would never be expired.