Hi,

Starting with vtk9 9.6.2+ds-1, the package use guess-concurrency to
limit parallel workers, which is a good move. However on some build
daemons, it significantly increased the build time.

One of the reason is bug#1149922. The other reason is that hardware or
VMs usually use an integer number of GiB for the RAM (and often a power
of 2). There is always a bit of RAM lost (kernel code or data, reserved
memory, ...) and this is enough to get just below the 4G per core.

Would you consider reducing the required memory to 3.8G (guess-concurrency accepts
floating numbers) to account for that?

Thanks
Aurelien

Control: severity -1 serious
This change has increased the build time on the slower riscv64 buildds
from 4 days to 6 days, it has to be reverted.
same i386 buildd where this guess-concurrency use reduced the number of
cores used by a factor of 3, even 2G would likely still harm build time
on some architectures.

It is harmful to use guess-concurrency unless:
1. it is fixing an actual problem that has been observed, and
2. the numbers have been properly measured and not just guessed

The only good option is to drop this bogus usage of guess-concurrency in
vtk9, there is no visible reason why it was added at all and adding it
with guessed numbers is just harmful.

cu
Adrian

¡Hola Adrian!

El 2026-10-11 a las 17:26 +0300, Adrian Bunk escribió:

I wasn't aware that a build taking longer than before is an RC bug, but
I agree that we should work on this issue and fix it as soon as
possible.

I only added guess-concurrency after running into OOM issues over and
over while working on the vtk9 package. This happened both on my local
machine and on the debusine builders I was using to test, because in
both cases there were a high number of processors and not as much memory
as the build required (in my case it was 16 processors and 64 GB of
RAM).

So, it is indeed fixing an actual problem.

You can see that the package that was uploaded to experimental didn't
manage to build on riscv64 and it's almost certainly for this reason
(when the build dies due to OOM there are no logs left behind to
actually verify this).

Now, regarding the measuring, it's true that in the package I uploaded
to unstable I had added a ballpark estimation and that I didn't take
into account that it would be a very different number for 32bits. I have
now taken a much more accurate measure and I've found that:

- In 64bit architectures, the maximum amount of memory that a single
   process takes is slightly over 6 GB.
- In 64bit architectures, the time-weighted average amount of memory is 1 GB.
- In 32bit architectures, the maximum amount of memory is slightly over 2 GB
- In 32bit architectures, the time-weighted average amount of memory is 550 MB.
- In both cases, the memory consumption is quite skewed, some processes
   take very little memory and very little CPU time, while others take a
   lot of memory and time.

I've added a script to the vtk9 package, that allows to (optionally)
obtain these and other metrics while building the package. I'm currently
building the package on several architectures to obtain all of these
metrics and compare them across architectures.

I can see how my terse changelog entry and similarly terse paragraph in
the debian/README.source file might not have explained why I added the
use of guess-concurrency. When I did the change, I was focused on
solving many other problems that the package had (like 4 CVEs and 5
other bugs), and didn't expect that this change would cause problems so
I didn't spend a lot of time documenting it.

I'm now working on a new revision of the package, incorporating the
changes that were uploaded to experimental plus some tweaking of the
values of guess-concurrency to adapt it to work correctly on both 64bit
and 32bit architectures.

I will also expand the entry in the debian/README.source file to explain
why guess-concurrency was added and how the metrics were chosen.

Happy hacking,

Hi,

Thanks for doing that. The above changes combined to the fix done on the
guess-concurrency side (#1149922), 4 jobs are used again on the riscv64
build daemons. There is still less jobs than before used on amd64 and
i386, but the resulting build time is comparable.

I am therefore closing the bug.

Regards
Aurelien