#1089816 myrepos: order not respected in parallel mode

Package:
myrepos
Source:
myrepos
Submitter:
Alper Nebi Yasak
Date:
2024-12-18 18:30:02 UTC
Severity:
normal
Tags:
#1089816#5
Date:
2024-12-13 08:51:43 UTC
From:
To:
Hello,

I use git with git-worktree, so wanted to order those after the main
repositories they are part of, using the 'order' setting. However, I
see them processed out of order with e.g. jobs = 10.

A simple reproducer:

    $ w="$(mktemp -t)"
    $ mkdir -p "$w/10" "$w/11" "$w/12"
    $ cat >"$w/.mrconfig" <<EOF
    [10]
    order = 10
    sleep = sleep 5; echo 10

    [11]
    order = 11
    sleep = sleep 5; echo 11

    [12]
    order = 12
    sleep = sleep 5; echo 12
    EOF

    # Make sure you don't have 'jobs' set in ~/.mrconfig
    $ time mr -t -c "$w/.mrconfig" -d "$w" -j1 sleep
    mr sleep: /tmp/tmp.j3fE35tdDM/10
    10

    mr sleep: /tmp/tmp.j3fE35tdDM/11
    11

    mr sleep: /tmp/tmp.j3fE35tdDM/12
    12

    mr sleep: finished (3 ok)
    mr -t -c "$w/.mrconfig" -d "$w" -j1 sleep  0.29s user 0.09s system 2% cpu 15.382 total

Runs commands one after the other, so takes 15s.

    $ time mr -t -c "$w/.mrconfig" -d "$w" -j3 sleep
    mr sleep: /tmp/tmp.j3fE35tdDM/11
    11

    mr sleep: /tmp/tmp.j3fE35tdDM/10
    10

    mr sleep: /tmp/tmp.j3fE35tdDM/12
    12

    mr sleep: finished (3 ok)
    mr -t -c "$w/.mrconfig" -d "$w" -j3 sleep  0.32s user 0.09s system 7% cpu 5.398 total

Runs everything in parallel, done in 5s and prints out of order. I
would expect this to behave like in the previous section instead.
I think it should be able to run things with the same order in parallel,
and await all running jobs before switching to a higher order.

#1089816#10
Date:
2024-12-18 18:27:10 UTC
From:
To:
Control: forwarded -1 https://myrepos.branchable.com/todo/config___39__order__39___is_ignored_when___39__jobs__39_____62__1/
Control: tag -1 patch

Well, I should've checked the upstream todo page.
Attaching my attempt at this, but it's lightly tested.