Hi,

While rebuilding libmatio 1.5.23 against hdf5 1.12.0, 3 testcases failed with:

- --- /dev/null   2022-04-25 15:38:39.000000000 +0000
+++ /<<PKGBUILDDIR>>/test/testsuite.dir/at-groups/2581/stderr   2022-04-25 17:23:52.456966636 +0000
@@ -0,0 +1,32 @@
+-E- test_mat: HDF5 error #000 in H5Fopen()
+      file : ../../../src/H5F.c:793
+      major: File accessibility
+      minor: Unable to open file
+-E- test_mat: HDF5 error #001 in H5VL_file_open()
+      file : ../../../src/H5VLcallback.c:3500
+      major: Virtual Object Layer
+      minor: Can't open object
+-E- test_mat: HDF5 error #002 in H5VL__file_open()
+      file : ../../../src/H5VLcallback.c:3465
+      major: Virtual Object Layer
+      minor: Can't open object
+-E- test_mat: HDF5 error #003 in H5VL__native_file_open()
+      file : ../../../src/H5VLnative_file.c:100
+      major: File accessibility
+      minor: Unable to open file
+-E- test_mat: HDF5 error #004 in H5F_open()
+      file : ../../../src/H5Fint.c:1622
+      major: File accessibility
+      minor: Unable to open file
+-E- test_mat: HDF5 error #005 in H5FD_lock()
+      file : ../../../src/H5FD.c:1675
+      major: Virtual File Layer
+      minor: Can't update object
+-E- test_mat: HDF5 error #006 in H5FD_sec2_lock()
+      file : ../../../src/H5FDsec2.c:959
+      major: File accessibility
+      minor: Bad file ID accessed
+-E- test_mat: HDF5 error #000 in H5Fclose()
+      file : ../../../src/H5F.c:880
+      major: Invalid arguments to routine
+      minor: Inappropriate type
- --- expout      2022-04-25 17:23:52.440966612 +0000
+++ /<<PKGBUILDDIR>>/test/testsuite.dir/at-groups/2581/stdout   2022-04-25 17:23:52.436966606 +0000

The related testcases are:
- - mat73_compressed_read_le
- - mat73_uncompressed_read_le
- - mat73_read_be

Please find attached the full build log.

Thanks,

_g.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEoJObzArDE05WtIyR7+hsbH/+z4MFAmJnr2IACgkQ7+hsbH/+
z4NB3Af+Pxfab/hymm3B0loRr7235GsdeFmbPqy+ZpI/OKej2NLirNuivD/sEg7l
8cvA5LjnUjq0IVC4STCWlOx7H0zuzY1JYyFpjeJjL2Qh18lTqpumoFJRaBMqj59Y
PC9EHAkJTxIR1fG+u6pWb/qFyj8EYVYYwUQk2COjHHY1ApKj9qgXkro/8Vqq3yNw
j1dEDwIZ28To8e3VnmnGEGrndcM8FF3RwvzWpT+Tkuj3G9S+jrqof4aag1Xxtu3G
+MFD1ANGbqTg1Q4Q8i5IomT7b25/t5C2IciA1KE4rqAJlqbFPFouEEwvy/vaXuW2
+SHE0kAOYjqwjsztP5sBkNnL2jkjqA==
=haiC
-----END PGP SIGNATURE-----

Hi Gilles,

Le mardi 26 avril 2022 à 10:38 +0200, Gilles Filippini a écrit :
[…]

I had an exchange with upstream, who says that libmatio 1.5.23 should
pass all tests against hdf5 1.12.2.

Is there any reason why you’re sticking to hdf5 1.12.0 instead of the
latest stable release 1.12.2?

Best,

Hi Sébastien,

Sébastien Villemot a écrit le 29/04/2022 à 11:45 :

No, no reason. I wanted to transition 1.12.0 before switching to 1.12.2,
but the other way should be fine as well. I'll give it a try.

Thanks,

_g.

Hi,

Gilles Filippini a écrit le 29/04/2022 à 14:10 :

I've just tested a rebuild against HDF5 1.12.2, and the same 3 tests
fail mosly the same way:

2272. mat73_compressed_read_le.at:475: testing Read directory ...
./mat73_compressed_read_le.at:478: cp $srcdir/results/dir_le.out expout
          $builddir/test_mat directory
$srcdir/datasets/matio_test_cases_compressed_hdf_le.mat
--- /dev/null   2022-04-25 15:38:39.000000000 +0000
+++ /<<PKGBUILDDIR>>/test/testsuite.dir/at-groups/2272/stderr
2022-05-02 08:55:13.515220334 +0000
@@ -0,0 +1,40 @@
+-E- test_mat: HDF5 error #000 in H5Fopen()
+      file : ../../../src/H5F.c:620
+      major: File accessibility
+      minor: Unable to open file
+-E- test_mat: HDF5 error #001 in H5VL_file_open()
+      file : ../../../src/H5VLcallback.c:3501
+      major: Virtual Object Layer
+      minor: Iteration failed
+-E- test_mat: HDF5 error #002 in H5PL__path_table_iterate()
+      file : ../../../src/H5PLpath.c:578
+      major: Plugin for dynamically loaded library
+      minor: Iteration failed
+-E- test_mat: HDF5 error #003 in H5PL__path_table_iterate_process_path()
+      file : ../../../src/H5PLpath.c:620
+      major: Plugin for dynamically loaded library
+      minor: Can't open directory or file
+-E- test_mat: HDF5 error #004 in H5VL__file_open()
+      file : ../../../src/H5VLcallback.c:3351
+      major: Virtual Object Layer
+      minor: Can't open object
+-E- test_mat: HDF5 error #005 in H5VL__native_file_open()
+      file : ../../../src/H5VLnative_file.c:97
+      major: File accessibility
+      minor: Unable to open file
+-E- test_mat: HDF5 error #006 in H5F_open()
+      file : ../../../src/H5Fint.c:1898
+      major: File accessibility
+      minor: Unable to lock file
+-E- test_mat: HDF5 error #007 in H5FD_lock()
+      file : ../../../src/H5FD.c:1625
+      major: Virtual File Layer
+      minor: Unable to lock file
+-E- test_mat: HDF5 error #008 in H5FD__sec2_lock()
+      file : ../../../src/H5FDsec2.c:1002
+      major: Virtual File Layer
+      minor: Unable to lock file
+-E- test_mat: HDF5 error #000 in H5Fclose()
+      file : ../../../src/H5F.c:707
+      major: Invalid arguments to routine
+      minor: Inappropriate type

Best,

_g.

Hi Sébastien,

Gilles Filippini a écrit le 02/05/2022 à 11:04 :

Running each failed test alone doesn't fail. Hence I've just tried adding:

override_dh_auto_test:
	dh_auto_test --no-parallel

and tada! no more failure.

Best,

_g.

Hi,

Thomas Beutlich a écrit le 19/05/2022 à 22:36 :

Setting the above environment variable fixes the problem when running
the testsuite in parallel mode.

Best,

_g.

Le jeudi 19 mai 2022 à 23:39 +0200, Gilles Filippini a écrit :

Thanks to both of you for your help on this issue.

Actually, it turns out that the problem is not related to HDF5 1.12.0,
but to a new debhelper version. More specifically, debhelper 13.7
enabled parallelization of autotest testsuites by default, which
exposed the underlying issue related to concurrency in HDF5.

In particular, the problem is also present in unstable (see #1011805).
Incidentally, this debhelper release also enabled verbose output of the
testsuite, which unfortunately makes the output look rather bad.

I am going to set HDF5_USE_FILE_LOCKING=FALSE when running the
testsuite.

Cheers,

We believe that the bug you reported is fixed in the latest version of
libmatio, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1011805@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Sébastien Villemot <sebastien@debian.org> (supplier of updated libmatio package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Wed, 01 Jun 2022 11:36:29 +0200
Source: libmatio
Architecture: source
Version: 1.5.23-2
Distribution: unstable
Urgency: medium
Maintainer: Debian Science Team <debian-science-maintainers@lists.alioth.debian.org>
Changed-By: Sébastien Villemot <sebastien@debian.org>
Closes: 1011805
Changes:
 libmatio (1.5.23-2) unstable; urgency=medium
 .
   * Set HDF5_USE_FILE_LOCKING=FALSE when running the testsuite.
     Since debhelper 13.7, the testsuite is run in parallel by default, which
     exposes an HDF5 concurrency issue. (Closes: #1011805)
   * Bump S-V to 4.6.1
Checksums-Sha1:
 2554954ecca302f17036174a6f0f34b8990fae45 2182 libmatio_1.5.23-2.dsc
 0c602237ab0ec69287e6a0376cdab633ed82b1ef 8140 libmatio_1.5.23-2.debian.tar.xz
 72c3e962393e4fedbc5252e4d9f35bc0d5b9e28a 9958 libmatio_1.5.23-2_amd64.buildinfo
Checksums-Sha256:
 0896a25e7062a4f55a3bed511bac8cc6a443bef7d8e99669957b40d947dde7a3 2182 libmatio_1.5.23-2.dsc
 84c386971cc1ccae89517dd3212611ff2ae4d30763b427573996c53e8a795ffe 8140 libmatio_1.5.23-2.debian.tar.xz
 1742f4014bc7f5e49a98dd5fbdfa408a1f104290ffa89850f80ef134f1992094 9958 libmatio_1.5.23-2_amd64.buildinfo
Files:
 d251f0169dbd0e1f8f5081bd490aaef3 2182 libs optional libmatio_1.5.23-2.dsc
 2cb146eec3c6da13d86002a3c0f279d3 8140 libs optional libmatio_1.5.23-2.debian.tar.xz
 93612d0a3033207dccf716b7d25dff54 9958 libs optional libmatio_1.5.23-2_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEU5UdlScuDFuCvoxKLOzpNQ7OvkoFAmKXNcUACgkQLOzpNQ7O
vkqgJg//WjYe8AIP2wd81D4CXI1EgS4pLKQ42yl51oHS5xROmXQuDI48KhS5zbFV
+O2/JbSNnWOpYo9pqMue4fEmB6oQKeVTCma4br4Jek06f2zxgY9vGBpoirMF/I1Q
/QhhbH77o0WhiYyhWrgM/mjySat3sktypmlx+7U6T+LZc+hN+tvj7G/WdLY3Gwur
d7kU++GHtpbqLLsoa0UKl21PwXxsokPAe0kEPHPshsztFqnp62yVzcoj6kPmdN1B
vKuGFYXUpgVh+GUICgLgQLT7CrfV89sGbFcVSK15xEWudjCU1l2VWAhNd5I+7JZo
Lx7js/UoS2DUKkSLJaN62nBIJmwKwYM7CtXRugK/t4cWFRcV+HhDKzBGlzxOkxgc
wmUoh87Z3NLZRlquVxrTAjQIZNEkL5N1kLZtaiQHmx5Mo3M+W5WTPowb97X0fKqp
Xet9VxMeAUvVRO12dLFYKNXN38tI8XESidX2J/u72R1E9tn3TgiaBZ9PL/3YWw5V
hWgyw1B+L4MlZ7gemWMPgqZYxVaSquH5pI6epYDjMJ50nxoTcgZA8WnH/TQbB30K
yW0uJuiVQkVXKi38GihgZ/IHukTSZ5BZStOSOaas7o43H+44cB/dXkCoZ0Qpu2bK
ce39oU2Bhe4LlOmksU87Vrc/3TbUHu9jj+WHaUCRmVchgsjcrno=
=I0Bn
-----END PGP SIGNATURE-----