#1006008 python-cryptography: FTBFS with OpenSSL 3.0

Package:
src:python-cryptography
Source:
python-cryptography
Submitter:
Sebastian Andrzej Siewior
Date:
2022-05-18 16:51:05 UTC
Severity:
serious
Tags:
#1006008#5
Date:
2022-02-18 21:13:59 UTC
From:
To:
Your package is failing to build using OpenSSL 3.0 with the
following error:

| =================================== FAILURES ===================================
| ______________________ TestPoly1305.test_vectors[vector0] ______________________
|
| self = <tests.hazmat.primitives.test_poly1305.TestPoly1305 object at 0x7f1d0612e6b0>
| vector = {'key': b'0000000000000000000000000000000000000000000000000000000000000000', 'msg': b'00000000000000000000000000000000...0000000000000000000000000000000000000000000000000000000000000000000000000', 'tag': b'00000000000000000000000000000000'}
| backend = <cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f1d11008c10>
|
|     @pytest.mark.parametrize(
|         "vector",
|         load_vectors_from_file(
|             os.path.join("poly1305", "rfc7539.txt"), load_nist_vectors
|         ),
|     )
|     def test_vectors(self, vector, backend):
|         key = binascii.unhexlify(vector["key"])
|         msg = binascii.unhexlify(vector["msg"])
|         tag = binascii.unhexlify(vector["tag"])
|         poly = Poly1305(key)
|         poly.update(msg)
| >       assert poly.finalize() == tag
|
| ../../../tests/hazmat/primitives/test_poly1305.py:51:
| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
| cryptography/hazmat/primitives/poly1305.py:34: in finalize
|     mac = self._ctx.finalize()
| cryptography/hazmat/backends/openssl/poly1305.py:56: in finalize
|     self._backend.openssl_assert(res != 0)
| cryptography/hazmat/backends/openssl/backend.py:242: in openssl_assert
|     return binding._openssl_assert(self._lib, ok, errors=errors)
| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
| lib = <module 'lib'>, ok = False
| errors = [<cryptography.hazmat.bindings.openssl.binding._OpenSSLError object at 0x7f1d0612e980>]
|
|     def _openssl_assert(lib, ok, errors=None):
|         if not ok:
|             if errors is None:
|                 errors = _consume_errors(lib)
|             errors_with_text = _errors_with_text(errors)
|
| >           raise InternalError(
|                 "Unknown OpenSSL error. This error is commonly encountered when "
|                 "another library is not cleaning up the OpenSSL error stack. If "
|                 "you are using cryptography with another library that uses "
|                 "OpenSSL try disabling it before reporting a bug. Otherwise "
|                 "please file an issue at https://github.com/pyca/cryptography/"
|                 "issues with information on how to reproduce "
|                 "this. ({0!r})".format(errors_with_text),
|                 errors_with_text,
|             )
| E           cryptography.exceptions.InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([_OpenSSLErrorWithText(code=50331803, lib=6, reason=155, reason_text=b'error:0300009B:digital envelope routines::buffer too small')])
|
| cryptography/hazmat/bindings/openssl/binding.py:77: InternalError

Full build log:
https://people.debian.org/~bigeasy/openssl-rebuild-3/attempted/python-cryptography_3.4.8-1_amd64-2022-02-15T03:33:24Z

For more information see:
https://www.openssl.org/docs/man3.0/man7/migration_guide.html

Sebastian

#1006008#18
Date:
2022-05-16 05:14:08 UTC
From:
To:
Hi,

On Fri, 18 Feb 2022 22:13:59 +0100 Sebastian Andrzej Siewior  <sebastian@breakpoint.cc> wrote:

 > Source: python-cryptography
 > Version: 3.4.8-1
 > Severity: important
 > Tags: bookworm sid
 > User: pkg-openssl-devel@lists.alioth.debian.org
 > Usertags: ftbfs-3.0
 > control: forwarded -1 https://github.com/pyca/cryptography/pull/6000
 >
 > Your package is failing to build using OpenSSL 3.0 with the
 > following error:

Looks like an upgrade to at least v35.0.0 is needed to fix this issue:
https://github.com/pyca/cryptography/issues/7039#issuecomment-1088566628=

Bests,

Agata.

#1006008#23
Date:
2022-05-17 06:43:59 UTC
From:
To:
Hi!

Not necessarily. One of the Python core developers, Christian Heimes, actually
backported fixes for Python3.10 and OpenSSL 3.0.0 for Fedora [1].

I have extracted the patches and I'm attaching them to this bug report. I will
test whether they fix the build on Debian.

Adrian

#1006008#28
Date:
2022-05-17 06:50:44 UTC
From:
To:
Hi!

I just noticed the patches for OpenSSL 3.0 support have already been added to the
Debian package [1]. I also verified that the package builds fine in unstable
with OpenSSL 3.0.

Therefore closing this bug report.

Adrian

#1006008#33
Date:
2022-05-17 07:23:45 UTC
From:
To:
Control: reopen -1

=========================== short test summary info ============================
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:192: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:233: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:240: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:251: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:262: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:270: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:285: Requires OpenSSL with ENGINE support and OpenSSL < 1.1.1d
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:425: Requires OpenSSL without rsa_oaep_md (< 1.0.2)
SKIPPED [1] ../../../tests/hazmat/backends/test_openssl.py:441: Requires OpenSSL without rsa_oaep_md (< 1.0.2)
SKIPPED [3] ../../../tests/hazmat/backends/test_openssl.py:612: Requires OpenSSL without EVP_PKEY_DHX (< 1.0.2)
SKIPPED [2] ../../../tests/hazmat/backends/test_openssl.py:642: Requires OpenSSL without EVP_PKEY_DHX (< 1.0.2)
SKIPPED [2] ../../../tests/hazmat/backends/test_openssl.py:664: Requires OpenSSL without EVP_PKEY_DHX (< 1.0.2)
SKIPPED [1] ../../../tests/hazmat/primitives/test_aead.py:41: Requires OpenSSL without ChaCha20Poly1305 support
SKIPPED [1] ../../../tests/hazmat/primitives/test_aes.py:258: AES in dummy-mode mode not supported
SKIPPED [1] ../../../tests/utils.py:30: 256-bit DH keys are not supported in OpenSSL 3.0.0+ (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/hazmat/primitives/test_dh.py:432: DH keys less than 512 bits are unsupported
SKIPPED [1] ../../../tests/utils.py:30: Requires OpenSSL without Ed25519 support (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Requires OpenSSL without Ed448 support (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/hazmat/primitives/test_ed448.py:60: ed448 contexts are not currently supported
SKIPPED [1] ../../../tests/utils.py:30: Does not support IDEA ECB (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Does not support IDEA CBC (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Does not support IDEA OFB (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Does not support IDEA CFB (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [480] ../../../tests/hazmat/primitives/utils.py:432: KBKDF does not support algorithm: cmac_aes128
SKIPPED [480] ../../../tests/hazmat/primitives/utils.py:432: KBKDF does not support algorithm: cmac_aes192
SKIPPED [480] ../../../tests/hazmat/primitives/utils.py:432: KBKDF does not support algorithm: cmac_aes256
SKIPPED [480] ../../../tests/hazmat/primitives/utils.py:432: KBKDF does not support algorithm: cmac_tdes2
SKIPPED [480] ../../../tests/hazmat/primitives/utils.py:432: KBKDF does not support algorithm: cmac_tdes3
SKIPPED [800] ../../../tests/hazmat/primitives/utils.py:438: Does not support counter location: middle_fixed
SKIPPED [1] ../../../tests/utils.py:30: Requires OpenSSL without poly1305 support (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Requires backend without RSA OAEP label support (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [4] ../../../tests/hazmat/primitives/test_serialization.py:1910: Requires bcrypt module
SKIPPED [1] ../../../tests/utils.py:30: Requires that bcrypt exists (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Requires OpenSSL without X25519 support (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Requires OpenSSL without X448 support (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [23] ../../../../../../usr/lib/python3/dist-packages/_pytest/config/__init__.py:1473: no 'wycheproof_root' option found
SKIPPED [1] ../../../tests/utils.py:30: Requires OpenSSL < 1.1.0f (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
SKIPPED [1] ../../../tests/utils.py:30: Requires LibreSSL (<cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f39e0d34d90>)
========== 17 failed, 2726 passed, 3261 skipped in 203.25s (0:03:23) ===========

So, we need to ignore these failures for the package to build with OpenSSL.

Ignoring the failures should be safe as it's just the tests that assume OpenSSL to be version <= 1.1.1.

Adrian

#1006008#40
Date:
2022-05-17 16:48:15 UTC
From:
To:
Hi,

17/05/2022 08:43, John Paul Adrian Glaubitz :

I do not know when that was done, but the two latest Fedora releases
have been using >=35 versions which properly support OpenSSL 3.0 [1]. I
have opened #1011155 in order to discuss why we cannot just update to
latest upstream versions, if that is the case, and to not pollute this
thread.

[1] https://src.fedoraproject.org/rpms/python-cryptography

[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1011155

Best regards,

Agata.

#1006008#45
Date:
2022-05-17 17:02:32 UTC
From:
To:
Hello Agathe!

At least for Debian Ports, updating to python-cryptography 35 or newer would mean that the package
becomes BD-Uninstallable, i.e. not buildable as the Rust compiler is not available on all architectures
yet.

Rust support is slowly coming to more architectures with the rustc_codegen_gcc backend and gccrs,
so this problem will be eventually resolved. However, this work is not completed yet.

Adrian

#1006008#50
Date:
2022-05-17 19:13:37 UTC
From:
To:
Hi,

17/05/2022 19:02, John Paul Adrian Glaubitz :
Thanks for this explanation. I totally forgot about architectures other
than x86_64 and arm64. Makes sense. You may want to repost this to bug
#1006008 [1] so that we can keep track (or I’ll do it if you want).

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006008

Bests,

Agata.

#1006008#53
Date:
2022-05-18 12:56:14 UTC
From:
To:
Hello,

Bug #1006008 in python-cryptography reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/python-team/packages/python-cryptography/-/commit/f9110887a2b8dbb53f1e28fc39421f2e5863e947

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1006008

#1006008#60
Date:
2022-05-18 16:49:21 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
python-cryptography, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1006008@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Stefano Rivera <stefanor@debian.org> (supplier of updated python-cryptography package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Wed, 18 May 2022 12:22:15 -0400
Source: python-cryptography
Architecture: source
Version: 3.4.8-2
Distribution: unstable
Urgency: medium
Maintainer: Tristan Seligmann <mithrandi@debian.org>
Changed-By: Stefano Rivera <stefanor@debian.org>
Closes: 1006008
Changes:
 python-cryptography (3.4.8-2) unstable; urgency=medium
 .
   * Team upload.
   * Add support for OpenSSL 3.0.1 (Closes: #1006008)
Checksums-Sha1:
 8c1419a06fdecee1d33efb530af8f7cdcd2c66d5 2432 python-cryptography_3.4.8-2.dsc
 ef091046b5f422ae9b66c385a435e31ebcb0eb16 23404 python-cryptography_3.4.8-2.debian.tar.xz
 a9ffac939d63477007c277cd9e206c0830dd3949 7932 python-cryptography_3.4.8-2_source.buildinfo
Checksums-Sha256:
 1344aed0cb26c95b06804732afd1d375356585eb0686b9d8e3a80b8527c9db2c 2432 python-cryptography_3.4.8-2.dsc
 4a71dff41cb3ff0e582ec7da1d23fb78fe368f5fec4f0467353c6b3689333a03 23404 python-cryptography_3.4.8-2.debian.tar.xz
 92648af7899c5f6eda4d5cc428ef0a0a4dd8e57e5b13b69e73218c6ba6621eb3 7932 python-cryptography_3.4.8-2_source.buildinfo
Files:
 ffc8e30ef0c02e7ea7fb887873fd3743 2432 python optional python-cryptography_3.4.8-2.dsc
 a0509b79389a2f6a50a02062d7384050 23404 python optional python-cryptography_3.4.8-2.debian.tar.xz
 221982807421a5ad1d9c5ce4a0376751 7932 python optional python-cryptography_3.4.8-2_source.buildinfo
-----BEGIN PGP SIGNATURE-----

iIoEARYKADIWIQTumtb5BSD6EfafSCRHew2wJjpU2AUCYoUdWhQcc3RlZmFub3JA
ZGViaWFuLm9yZwAKCRBHew2wJjpU2BLmAQDVVYShTiPF7qD1xVlFxxHbzAzY9nJf
DkH+dT/tfa9C0QD+MWpHf1yIMnonzzQ3Sz0lKUKbWCbA/fg3Yx1WIpgtpgw=
=bDmT
-----END PGP SIGNATURE-----