parsinsert fails it's tests when built with -O3, seen at least on amd64 and ppc64el.

[...]
Taxonomies List:
 [root] 957
    [Bacteria] 957
       [Actinobacteria] 957
          [Actinobacteria] 957
             [Acidimicrobidae] 957
                [Acidimicrobiales] 957
                   [Acidimicrobineae] 957
                      [Acidimicrobiaceae] 957
                         [Acidimicrobium] 957
Writing Newick tree file: [set1000.tree]

Rank Matches:
                     _________Precision________    __________Recall__________
            Domain:      957      938 ( 98.01%)          938      957 ( 98.01%)
            Phylum:      957       83 (  8.67%)           83      957 (  8.67%)
             Class:      957       83 (  8.67%)           83      957 (  8.67%)
             Order:      915        0 (  0.00%)            0      915 (  0.00%)
            Family:      876        0 (  0.00%)            0      876 (  0.00%)
             Genus:      706        0 (  0.00%)            0      706 (  0.00%)
           Species:       53        0 (  0.00%)            0       53 (  0.00%)
Insert Time = 13, 265015 per hour
Process Completed: 15 sec
incorrect result
make[1]: *** [debian/rules:21: override_dh_auto_test] Error 1

Control: tags -1 unreproducible
Control: tags -1 moreinfo

I admit I failed to reproduce this at least on amd64.  I explicitly
tried -O3 to use the same optimisation flag as upstream to possibly fix
#976929 since this package was showing issues with different
optimisation flags in the past.  Can you be more verbose how to
reproduce this issue?

Kind regards

        Andreas.

Hi there,

By exporting DEB_CXXFLAGS_MAINT_APPEND = -O3 explicitely in the
d/rules file, I could reproduce the output on amd64.  In case
the behavior is not consistent accross CPUs, I have the
following:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   43 bits physical, 48 bits virtual
CPU(s):                          6
On-line CPU(s) list:             0-5
Thread(s) per core:              1
Core(s) per socket:              6
Socket(s):                       1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           113
Model name:                      AMD Ryzen 5 3600 6-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         3860.805
CPU max MHz:                     3600.0000
CPU min MHz:                     2200.0000
BogoMIPS:                        7186.72
Virtualization:                  AMD-V
L1d cache:                       192 KiB
L1i cache:                       192 KiB
L2 cache:                        3 MiB
L3 cache:                        32 MiB
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce
cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse
sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm
constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid
aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2
movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm
extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs
skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc
mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibpb stibp
vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap
clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc
cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf
xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save
tsc_scale vmcb_clean flushbyasid decodeassists pausefilter
pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov
succor smca

Kind Regards,

Control: retitle -1 parsinsert fails it's tests when built with -O2
Control: tags -1 - unreproducible
Control: tags -1 - moreinfo

Hi,

Just mentionning that since introduction of gcc-13, parsinsert
saw also its tests failing in #1037816 with optimization -O2 on
amd64.  So we're currently building parsinsert -O1 for now.

Cheers,  :)

please can you identify the object file which is wrongly built?
build two versions, one with -O1, the other with -O2, then combine the object
files from the two builds to identify a specific file?

also there seem to be some warnings, is the package really ready for C++17?

Matthias

Hi Matthias,

Matthias Klose, on 2023-08-24:
faulty object file when built with -O2.  Erasing it with the
equivalent object file built with -O1 fixes the errors.

Something intresting though: when I manually build ParsInsert.o
with `c++ -c ParsInsert.c -o ParsInsert.o -O2`, the error
doesn't appear, so it looks to result from a bad combination
with something else.  After suspecting hardening, it turned out
that the following, when combined with build of ParsInsert.o
with -O2 optimization, is causing the test failure:

	-specs=/usr/share/dpkg/no-pie-compile.specs

The upstream version hasn't changed since the introduction of
the package in the archive, ten years ago, so I guess not?

Have a nice day,  :)

Étienne Mollier, on 2023-08-26:

Rerunning these tests in a properly isolate chroot show me that
the hardening is also contributing to the improper results.
Hardening must be disabled and PIE must be enforced to ensure
the program runs appropriately

	                         -O1  -O2   -O3
	-fPIE + no hardening      OK   OK    OK
	-fno-PIE + no hardening   OK  FAIL  FAIL
	-fPIE + hardening         OK  FAIL  FAIL
	-fno-PIE + hardening      OK  FAIL  FAIL

I'm considering pushing an upload that goes in the direction of
disabling hardening and enforcing PIE in the upcoming week,
unless there are reasons to hold on, or someone is faster than
me in uploading.

Have a nice day,  :)

Hi Étienne,

Am Sun, Aug 27, 2023 at 11:26:32PM +0200 schrieb Étienne Mollier:

Thanks a lot for creating this matrix.

I will probably be not faster than you.  Please make sure you document
in d/rules that hardening breaks the tests.  This should avoid that
someone later might simply switch on hardening since we usually do this.
(may be same for salsa-ci.yaml when you switch of the CI test there.)

Thanks a lot for caring
   Andreas.

Hi Andreas,

Andreas Tille, on 2023-08-28:

The change even fixed LTO builds.  :)
On the other hand, well, the hardening is gone.  :(

The package looks to range on the performance critical side of
the spectrum, so this is probably an acceptable trade-off.  This
is probably symptomatic of a deeper problem in the source code
though, but I don't really expect to pinpoint the exact cause
without help from upstream.

This is documented in d/rules.  I'll also add the necessary
lintian overrides and blhc markers to reduce the noise caused by
the change in automated checks.

Have a nice day,  :)

Hello,

Bug #964082 in parsinsert reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/med-team/parsinsert/-/commit/dcf353dac3d048c265fafe07e3bc5e6647e34392
------------------------------------------------------------------------
d/rules: disable hardening and enforce -fPIE.

This fixes parsinsert self-test failures when a number of optimization
options are applied (e.g. -O2, -O3, LTO) once and for all hopefully;
there have been more than one bug of this kind.

Closes: #964082
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/964082

We believe that the bug you reported is fixed in the latest version of
parsinsert, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 964082@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Étienne Mollier <emollier@debian.org> (supplier of updated parsinsert package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Mon, 28 Aug 2023 22:14:03 +0200
Source: parsinsert
Architecture: source
Version: 1.04-14
Distribution: unstable
Urgency: medium
Maintainer: Debian Med Packaging Team <debian-med-packaging@lists.alioth.debian.org>
Changed-By: Étienne Mollier <emollier@debian.org>
Closes: 964082
Changes:
 parsinsert (1.04-14) unstable; urgency=medium
 .
   * Team upload.
   * d/rules: disable hardening and enforce -fPIE.
     This fixes parsinsert self-test failures when a number of optimization
     options are applied (e.g. -O2, -O3, LTO) once and for all hopefully;
     there have been more than one bug of this kind. (Closes: #964082)
   * d/rules: instruct blhc about the missing flags.
   * d/lintian-overrides: override missing hardening flags.
Checksums-Sha1:
 a056b0743c413f8eb44c7705836206b6db4e63fc 2219 parsinsert_1.04-14.dsc
 364ae5c111315256eba2882b43c798c7e33b1da5 2841504 parsinsert_1.04-14.debian.tar.xz
Checksums-Sha256:
 d55d643bb0efcfe6cda942aeba8b3b843b11555cda43994afe9e97f0b208d0c8 2219 parsinsert_1.04-14.dsc
 d465338012595fa7aed55ae5b23573a94fe17b3d1104b21b807b0acc907e2769 2841504 parsinsert_1.04-14.debian.tar.xz
Files:
 32a34287f2735523d25825c3badc3b56 2219 science optional parsinsert_1.04-14.dsc
 b86dd794cf5a88268c72711cbde002ab 2841504 science optional parsinsert_1.04-14.debian.tar.xz
-----BEGIN PGP SIGNATURE-----

iQJIBAEBCgAyFiEEj5GyJ8fW8rGUjII2eTz2fo8NEdoFAmTtD8cUHGVtb2xsaWVy
QGRlYmlhbi5vcmcACgkQeTz2fo8NEdpgHQ/9EYxHCMp5mYmWKhDpf4jilftGoGlu
PL/8PCrCAB563gN87WowVF3wJCfqfZ1CIK0r9J5/CuBE4d+TtcTfiSQdnRXQYOlE
mI8sLBIZk6GzWGxD8BUcJaag1/iXo8OlpGPRJ5oLTOd6AmgaF8nADTtK8ekrmUO7
ZeiGdGKEWwO6GDnPGonqAwzdeo7kNyDv+S9Ef2g7OWGmn8huMRiEDXIO0oSwRDBc
ZUJ3pawg2qn+UbYUb7XAKhOaLXPLqpqaOfPii+cj7rwnX6erLbmYl6aXO6UCYVEV
AG9Pihc0RGihhCCJC1wrvatE9qcNoqN27g2v2obcEKP1uQSJcrTt5wDR/xysVw3z
+2/uhiyL0fUgoZzQRY3hZUDSEiT29Q/TpqAeKbmSubhKJg14O9ZK5ijnbVbnWGB/
MSVJFWmdknVnulmntSKZg62cuYKWQ70S2FmIh4fIFVgWn6GRBQkhbr9B31a3HI5J
Wm33hlHgDSpgU09W4S9U+dQ5MkF0uysGUFCgg/ZFLiam6Ai2BMn7i+e+crJ4HRkK
C3htLLc2IbMR86VtSGDNYQdgCrDAOZROz9mhJj6To9metOJ87Y6SaFNxuiknT4As
KCmjAvZgx9gleLS1YvhuqxQKqfyFXcfH04CfS5kuX4I0vjtQuOWkV/Ibf2Qott7A
iU/ObPGU4nUV13w=
=1TWy
-----END PGP SIGNATURE-----

Hi Étienne,

Am Mon, Aug 28, 2023 at 10:09:42PM +0200 schrieb Étienne Mollier:

Nice.

Definitely.  We do not have resources to dive that deeply in "random"
packages.

Thanks a lot for the tough work
  Andreas.