#1000336 numba: FTBFS with Python 3.10

Package:
src:numba
Source:
numba
Submitter:
Stefano Rivera
Date:
2022-06-15 13:33:03 UTC
Severity:
serious
Tags:
#1000336#5
Date:
2021-11-21 19:55:55 UTC
From:
To:
numba isn't compatible with Python 3.10, yet.
Upstream is working on it, see https://github.com/numba/numba/issues/7562

setup.py immediately fails with:
RuntimeError: Cannot install on Python version 3.10.0; only versions >=3.6,<3.10 are supported.

See e.g. https://buildd.debian.org/status/fetch.php?pkg=numba&arch=amd64&ver=0.52.0-5%2Bb1&stamp=1637478629&raw=0

SR
#1000336#10
Date:
2021-11-21 21:01:03 UTC
From:
To:
Thanks for the bug report and the forward,

I'd managed to spot the build failures last night but only read the log
files about why today. I subscribed to the upstream bug report.

Diane
#1000336#15
Date:
2021-12-10 07:56:49 UTC
From:
To:
Hi,

I've commited version 0.54.1 to Git since I assume that we have better
chances for Python3.10 support in the more recent upstream version.
Unfortunately the build stops with

dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:237: python3.10 setup.py clean
...
  from distutils import sysconfig
Traceback (most recent call last):
  File "/build/numba-0.54.1/setup.py", line 51, in <module>
    _guard_py_ver()
  File "/build/numba-0.54.1/setup.py", line 48, in _guard_py_ver
    raise RuntimeError(msg.format(cur_py, min_py, max_py))
RuntimeError: Cannot install on Python version 3.10.1; only versions >=3.7,<3.10 are supported.
E: pybuild pybuild:354: clean: plugin distutils failed with: exit code=1: python3.10 setup.py clean


Hoping for an upstream fix of #7562 in their tracker now.

I think pushing for Python3.10 in Debian before such packages with lots
of rdepends (specifically python3-pandas) are ported is quite demanding.

Kind regards

       Andreas.
#1000336#22
Date:
2021-12-22 16:54:46 UTC
From:
To:
Hi all,

Upstream has mentioned that it has been fixed upstream. Diane, could you
please do the needful and upload?

Actually because of the current state of numba, several reverse depends are FTBFS so it's
bit urgent to push. Apologies for getting on your nerves, though.

Regards,
Nilesh
#1000336#27
Date:
2021-12-23 01:09:35 UTC
From:
To:

I tried. but numba needs tbb version >= 2021. I tried to update tbb but
ran into problems trying to build it.

I pushed the compatibility patch to a python-3-10-compatibility branch
on salsa for the moment.
#1000336#32
Date:
2021-12-23 05:57:24 UTC
From:
To:
Hi,

Am Wed, Dec 22, 2021 at 05:09:35PM -0800 schrieb Diane Trout:

I've checked tbb Git and have read:


onetbb (2021.4.0-1~exp1) UNRELEASED; urgency=medium

  * Source rename following upstream. The new upstream Git URL is
https://github.com/oneapi-src/oneTBB
#1000336#37
Date:
2021-12-23 09:24:11 UTC
From:
To:

Diane is testing a python3.10-compatibility branch for us in numba.

At the same time numba upstream has released 0.55.0rc1 which contains
their python3.10 fix.  Should we just jump straight to it (and not wait
for the final 0.55 release)?  I don't know how it goes with tbb though.

Drew
#1000336#42
Date:
2021-12-23 09:58:42 UTC
From:
To:
Actually I guess 0.55.0rc1 won't help so easily. It needs llvmlite
0.38.0rc1, and we've only just got 0.37 packaged. numba is a kind of
ouroboros, can never get to the end of it.

Drew
#1000336#47
Date:
2021-12-23 16:03:56 UTC
From:
To:
Hi all,

I'm back.

I've just finished my final exams so I could do something during
the holiday. That TBB repository is still work-in-progress and
FTBFS from the master branch is something expected. I will finalize
it soon. Andreas said in previous posts that we prefer a faster
NEW queue process. I understand that but we cannot bypass NEW process
this time as upstream has bumped the SONAME. So I'm changing the
source name as well following the upstream since NEW is inevitable.

As for llvmlite, the latest upstream RC release v0.38.0rc1 seems
to support python 3.10 . Should I upload the RC release?

BTW, what else should I do? I've been out of sync from the mailing
list for a long while.
#1000336#52
Date:
2021-12-23 16:19:45 UTC
From:
To:
Hi,

Am Thu, Dec 23, 2021 at 11:03:56AM -0500 schrieb M. Zhou:

Great.  Hope you finished the exams successfully. ;-)

In this case for sure it sounds sensible to change name together
with SONAME.

I'm not very deeply involved in this but from my gut feeling I'd
say go for it if it bears the chance to move away some blockers.

May be python-sklearn could need some helping hands[1] which I
did not managed despite I had put it on my agenda.

Kind regards

      Andreas.

[1] https://lists.debian.org/debian-science/2021/12/msg00006.html
#1000336#57
Date:
2021-12-30 07:27:30 UTC
From:
To:

Have you managed to make much progress?

I fiddled with the packaging and got it to build and trying to run the
autopkgtests with 2021.4.0-1

What'd help me is to have a package I could build locally and test
numba against. If you got it working could you commit what you have to
a salsa branch and let me know where it is?

Thanks,
Diane
#1000336#62
Date:
2022-01-09 01:30:57 UTC
From:
To:
Hi all,

The good news is that I managed to upgrade onetbb. It 
is in the NEW queue now:
https://ftp-master.debian.org/new/onetbb_2021.4.0-1~exp1.html
All changes have been pushed onto salsa (master branch).

SOVERSION was bumped from 2 to 12 so NEW is inevitable.
There are also some non-trivial API changes. So I guess the
transition won't be easy.


Have you managed to make much progress?

I fiddled with the packaging and got it to build and trying to run
the
autopkgtests with 2021.4.0-1

What'd help me is to have a package I could build locally and test
numba against. If you got it working could you commit what you have
to
a salsa branch and let me know where it is?

Thanks,
Diane
#1000336#67
Date:
2022-01-09 05:49:08 UTC
From:
To:
Hi Mo,

thanks a lot.  I asked on IRC for priorisation of this
package.

Kind regards

      Andreas.

Am Sat, Jan 08, 2022 at 08:30:57PM -0500 schrieb M. Zhou:
#1000336#76
Date:
2022-02-08 21:49:45 UTC
From:
To:
Hi,

After Andreas pointed it out I looked through some of the build
failures for onetbb and talked to upstream about the i386 failure.
https://github.com/oneapi-src/oneTBB/issues/370#issuecomment-1030387116

They have a patch.
https://github.com/oneapi-src/oneTBB/commit/542a27fa1cfafaf76772e793549d9f4d288d03a9

I've tested it against a checkout of the 2021.5.0-1 version of onetbb
on i386 and it does build with it applied. Once there was a test
failure, and once all tests ran successfully

I see that you've made some more progress for the memory alignment bugs
so I didn't commit "Detect 32 bit x86 systems while adding -mwaitpkg
option" i386 patch but could if you want.

Diane
#1000336#81
Date:
2022-02-08 22:59:52 UTC
From:
To:
Hi Diane,

Thank you. I have added that patch in the git repository.
#1000336#86
Date:
2022-02-19 10:02:24 UTC
From:
To:
onetbb is now building (in experimental) for all arches except
armel and armhf.

Is it worth at this point to make an upload of numba 0.55 to
experimental to check that it's building and passing tests for the
other arches?

Drew
#1000336#91
Date:
2022-02-21 03:37:43 UTC
From:
To:
It would... except numpy 1.22 just hit experimental on the 18th. and
numba isn't compatible with numpy 1.22. I tried adding this to
d/control,

python3-numpy (<< 1.22),

but my sbuild resolver still picked up numpy from experimental and
refused to build.

Upstream has a meta bug with the numpy 1.22 issues.
https://github.com/numba/numba/issues/7754
#1000336#96
Date:
2022-02-21 07:47:15 UTC
From:
To:
Hi Diane

That doesn't sound right.  You should be targeting experimental, but
still building against packages in unstable.  Something like this:

sbuild -d experimental -c unstable-amd64-sbuild ...

Regards
Graham
#1000336#101
Date:
2022-02-23 05:31:07 UTC
From:
To:
Hello guys. Finally it's all green on our release architectures
https://buildd.debian.org/status/package.php?p=onetbb&suite=experimental

I shall request the slot for transition once finished the rebuild
of its reverse dependencies and filed FTBFS bugs if any.
#1000336#106
Date:
2022-02-23 05:53:29 UTC
From:
To:
Am Wed, Feb 23, 2022 at 12:31:07AM -0500 schrieb M. Zhou:

Sounds good - thanks a lot for your work on this

    Andreas.
#1000336#111
Date:
2022-02-23 07:20:00 UTC
From:
To:
Wonderful! That is great news.

Thank you!

Diane
#1000336#116
Date:
2022-03-13 20:00:17 UTC
From:
To:
Hi Mo,

Did you get a chance to do this yet?
As we _really_ need numba at this point.

Regards,
Nilesh
#1000336#121
Date:
2022-03-13 20:50:32 UTC
From:
To:
Hi,

Recently I'm not able to test the build of libtbb-dev's reverse dependencies
as my build machine was out of access. That blocks my submission of the
transition bug and hence I'm stalled at this point.
According to some archlinux developers, this transition breaks a lot of
reverse dependencies since some of the core APIs have been changed.
Please expect a relatively negative rebuild result.

Help is welcome.
#1000336#126
Date:
2022-03-14 00:13:50 UTC
From:
To:
I've built both mathicgb and macaulay2 in unstable against TBB 2021 from
experimental and they're both ready to go for the transition.

Doug
#1000336#131
Date:
2022-03-14 06:21:01 UTC
From:
To:
Hi,

Am Mon, Mar 14, 2022 at 12:13:50AM +0000 schrieb Torrance, Douglas:

If you ask me we should simply start the transition and see what needs
fixing ... may be asking release team for temporary removal from
testing.  We are in a quite early process of the release cycle.  So we
are not really in danger to loose any important package in the next
stable release.  Even if we are running any tests now the fact that
something is broken will not increase forces to fix these breaks and
we might wait for ages.

Kind regards

      Andreas.
#1000336#136
Date:
2022-03-15 11:05:14 UTC
From:
To:
I just came to know a method to run salsa-CI for reverse-dependencies.
Probably we can use it

https://bzed.de/post/2021/01/building_reverse_build_dependencies_in_salsa_ci/

But since the reverse-deps might be large in number, we need to open an issue similar to this
to ask the CI admins once about it

https://salsa.debian.org/salsa/support/-/issues/291

Hope that helps.

Regards,
Nilesh