Hi,

I have the following test:

Then I run

$ adt-run -B -s -d --unbuilt-tree . --timeout-copy=10000 --- null

from the *source package*.

(there's no build needed, as junit-subsequentcheck already does built what it
needs. copy timeout choosen as "randomly big", the default for sure is not
enough for LO. Note this:

(jessie)rene@frodo ..Office/libreoffice/libreoffice-5.1.1.3 % du -hs
3.1G	.
)

So let's see.

adt-run: DBG: Parsed options: Namespace(apt_pocket=[], auto_control=True, copy=[], env=[], gainroot=None, logfile=None, output_dir=None, set_lang=None, setup_commands=[], shell=False, shell_fail=True, summary=None, timeout_build=None, timeout_copy=10000, timeout_factor=1.0, timeout_install=None, timeout_short=None, timeout_test=None, user=None, verbosity=2)
adt-run: DBG: Remaining arguments: ['-B', '--unbuilt-tree', '.']
adt-run: DBG: Interpreted actions: ['--no-built-binaries', '--unbuilt-tree', '.']
adt-run: DBG: Virt runner arguments: ['null']
adt-run: DBG: testbed init
adt-run [00:35:16]: version 3.19.3
adt-run [00:35:16]: host frodo; command line: /usr/bin/adt-run -B -s -d --unbuilt-tree . --timeout-copy=10000 --- null
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: testbed open, scratch=None
adt-run: DBG: sending command to testbed: open
adt-run: DBG: got reply from testbed: ok /tmp/adt-run.c6BSZ8
adt-run: DBG: sending command to testbed: print-execute-command
adt-run: DBG: got reply from testbed: ok env
adt-run: DBG: sending command to testbed: capabilities
adt-run: DBG: got reply from testbed: ok isolation-machine downtmp-host=/tmp/adt-run.c6BSZ8
adt-run: DBG: testbed capabilities: ['isolation-machine', 'downtmp-host=/tmp/adt-run.c6BSZ8']
adt-run: DBG: testbed command ['dpkg', '--print-architecture'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run [00:35:17]: testbed dpkg architecture: amd64
adt-run: DBG: testbed command ['which', 'eatmydata'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 1
adt-run: DBG: testbed command ['which', 'dpkg-query'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: testbed command ['sh', '-ec', "dpkg-query --show -f '${Package}\\t${Version}\\n' > /tmp/adt-run.c6BSZ8/testbed-packages"], kind short, sout raw, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.c6BSZ8/testbed-packages /tmp/adt-run.output.4ufg40yp/testbed-packages
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: testbed command ['uname', '-srv'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run [00:35:17]: testbed running kernel: Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u4 (2016-02-29)
adt-run: DBG: Binaries: initialising
adt-run [00:35:17]: @@@@@@@@@@@@@@@@@@@@ unbuilt-tree .
adt-run: DBG: blame += .
adt-run: DBG: testbed reset: modified=False, deps_installed=[](r: False), deps_new=[](r: False)
adt-run: DBG: testbed command ['mkdir', '-p', '/tmp/adt-run.c6BSZ8'], kind short, sout raw, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: sending command to testbed: copydown ./ /tmp/adt-run.c6BSZ8/ubtree-./


Here it copies the source tree to /tmp/. OK. I assume ubtree means
"unbuilt tree". #1

adt-run: DBG: got reply from testbed: ok
adt-run: DBG: testbed command ['sh', '-ec', 'exec 3>&1 >&2; set -x; cd /; builddir=$(mktemp -d /tmp/adt-run.c6BSZ8/build.XXX); cd $builddir; cp -rd --preserve=timestamps -- "/tmp/adt-run.c6BSZ8/ubtree-." real-tree; chmod -R a+rX .; cd [a-z0-9]*/.; pwd >&3; sed -n "1 {s/).*//; s/ (/\\n/; p}" debian/changelog >&3; set +e; grep -q "^Restrictions:.*\\bbuild-needed\\b" debian/tests/control 2>/dev/null; echo $? >&3'], kind build, sout pipe, serr raw, env []
+ cd /
+ mktemp -d /tmp/adt-run.c6BSZ8/build.XXX
+ builddir=/tmp/adt-run.c6BSZ8/build.Pdm
+ cd /tmp/adt-run.c6BSZ8/build.Pdm
+ cp -rd --preserve=timestamps -- /tmp/adt-run.c6BSZ8/ubtree-. real-tree

Here it copies the source tree from #1 to _an other_ "real" tree. Why? This is
cp #2.

+ chmod -R a+rX .
+ cd real-tree/.
+ pwd
+ sed -n 1 {s/).*//; s/ (/\n/; p} debian/changelog
+ set +e
+ grep -q ^Restrictions:.*\bbuild-needed\b debian/tests/control
+ echo 1
adt-run: DBG: testbed command exited with code 0
adt-run [00:52:46]: testing package libreoffice version 1:5.1.1~rc3-2
adt-run [00:52:46]: build not needed
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.c6BSZ8/build.Pdm/real-tree/ /tmp/adt-run.output.4ufg40yp/tests-tree/

What on earth? Another copy? From "real-tree" to "tests-tree"?

adt-run [00:52:46]: testing package libreoffice version 1:5.1.1~rc3-2
adt-run [00:52:46]: build not needed
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.c6BSZ8/build.Pdm/real-tree/ /tmp/adt-run.output.4ufg40yp/tests-tree/
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: processing dependency libreoffice
[...]

AH, FINALLY.

Copying 3.1GB three times? Especially as there's no build-needed etc.
At least "real-tree" to "test-tree" can be avoided, if you already have a
"ubtree"? And if you have "ubtree" and don't need a build, why copy it over
again? It's not as you run into the dir you started it from.

1 copy to /tmp should be enough here. Takes already enough time and disk.

Or maybe

01:01 < terceiro> if should be probably mounted inside the testbed instead of
                  copied

Regards,

Rene

Hi,

and when I run the tests they run in *real-tree*:

From the same run as the initial report:

[...]
adt-run: DBG: testbed command ['test', '-e', '/tmp/adt-run.tQVlcc/build.G5O/real-tree'], kind short, sout raw, serr raw, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: copydown: tb path /tmp/adt-run.tQVlcc/build.G5O/real-tree already exists
adt-run [01:36:25]: test junit-subsequentcheck: [-----------------------
adt-run: DBG: testbed command ['bash', '-c', 'set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/adt-run.tQVlcc/build.G5O/real-tree"; mkdir -p -m 1777 -- "/tmp/adt-run.tQVlcc/junit-subsequentcheck-artifacts"; export ADT_ARTIFACTS="/tmp/adt-run.tQVlcc/junit-subsequentcheck-artifacts"; export ADTTMP=$(mktemp -d --tmpdir adttmp.XXXXXX); export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=$(grep -c ^processor /proc/cpuinfo | sed \'s/^0$/1/\'); unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/adt_test_script_pid; set -C; echo $$ > /tmp/adt_test_script_pid; set +C; trap "rm -rf $ADTTMP /tmp/adt_test_script_pid" EXIT INT QUIT PIPE; chmod 755 $ADTTMP; cd "$buildtree"; chmod +x /tmp/adt-run.tQVlcc/build.G5O/real-tree/debian/tests/junit-subsequentcheck; touch /tmp/adt-run.tQVlcc/junit-subsequentcheck-stdout /tmp/adt-run.tQVlcc/junit-subsequentcheck-stderr; /tmp/adt-run.tQVlcc/build.G5O/real-tree/debian/tests/junit-subsequentcheck 2> >(tee -a /tmp/adt-run.tQVlcc/junit-subsequentcheck-stderr >&2) > >(tee -a /tmp/adt-run.tQVlcc/junit-subsequentcheck-stdout); '], kind test, sout raw, serr raw, env []
====== Patching the tree to only build Java-based unittests against an existing installation ======
patching file jurt/Module_jurt.mk
patching file postprocess/Module_postprocess.mk
patching file solenv/gbuild/JavaClassSet.mk
patching file solenv/gbuild/JunitTest.mk
Hunk #1 succeeded at 55 (offset 1 line).
patching file solenv/gbuild/Module.mk
patching file solenv/gbuild/extensions/post_SkipBuild.mk
patching file solenv/gbuild/platform/unxgcc.mk

====== Generating configuration =======
[...]
====== Cleaning tree ======
rm -fr /tmp/adt-run.tQVlcc/build.G5O/real-tree/test-install
rm -fr /tmp/adt-run.tQVlcc/build.G5O/real-tree/instdir
rm -fr /tmp/adt-run.tQVlcc/build.G5O/real-tree/workdir

====== Enabling core dumps ======

====== Starting subsequentcheck with  job against path:/usr/lib/libreoffice/program/soffice ======
Automatic fetching of external tarballs is disabled.
mkdir -p /tmp/adt-run.tQVlcc/build.G5O/real-tree/instdir
/tmp/adt-run.tQVlcc/build.G5O/real-tree/solenv/bin/install-gdb-printers -a /tmp/adt-run.tQVlcc/build.G5O/real-tree/instdir -c
make -j 3  -rs -f /tmp/adt-run.tQVlcc/build.G5O/real-tree/Makefile.gbuild    subsequentcheck
[...]
[build JUT] chart2_unoapi
[build JUT] comphelper_complex
[build JUT] configmgr_unoapi
[build JUT] dbaccess_complex
[build JUT] dbaccess_unoapi
[build JUT] extensions_unoapi
[build JUT] forms_unoapi_1
[build JUT] forms_unoapi_2
[build JUT] forms_unoapi_3
[build JUT] forms_unoapi_4
[build JUT] forms_complex
[build JUT] framework_complex
[build JUT] framework_unoapi
[build JUT] juh
[build JUT] linguistic_unoapi
[build JUT] qadevOOo_unoapi
[build JUT] ridljar_typedesc
[build JUT] ridljar_uno
[build JUT] ridljar_util
[build JUT] sc_complex
[build JUT] sc_unoapi_1
[build JUT] sc_unoapi_2
[build JUT] sc_unoapi_3
[build JUT] sc_unoapi_4
[build JUT] sc_unoapi_5
[build JUT] sc_unoapi_6
[build JUT] sc_unoapi_7
[build JUT] sd_unoapi
[build JUT] sfx2_complex
[build JUT] sfx2_unoapi
[build JUT] sot_complex
[build JUT] starmath_unoapi
[build JUT] svl_complex
[build JUT] svtools_unoapi
[build JUT] svx_unoapi
[build JUT] sw_complex
[build JUT] sw_unoapi_1
[build JUT] sw_unoapi_2
[build JUT] sw_unoapi_3
[build JUT] sw_unoapi_4
[build JUT] toolkit_complex
[build JUT] ucb_complex
[build JUT] ucb_unoapi
[build JUT] unotools_complex
[build JUT] unoxml_complex
[build JUT] unordf_complex
[build JUT] xmloff_unoapi
[...]
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.tQVlcc/junit-subsequentcheck-stdout /tmp/adt-run.output.bkjzczum/junit-subsequentcheck-stdout
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.tQVlcc/junit-subsequentcheck-stderr /tmp/adt-run.output.bkjzczum/junit-subsequentcheck-stderr
adt-run: DBG: got reply from testbed: ok
adt-run [02:05:41]: test junit-subsequentcheck:  - - - - - - - - - - results - - - - - - - - - -
junit-subsequentcheck PASS
[...]
junit-subsequentcheck PASS
pyuno-import         PASS
adt-run: DBG: testbed stop
adt-run: DBG: testbed close, scratch=/tmp/adt-run.tQVlcc
adt-run: DBG: sending command to testbed: close

so what is the cp to tests-dir for?

Regards,

Rene

Hi,

so this is even more severe than only "annyoing".

Especially for machines with slow or spinning disks, even on machines
with an ssd where the ssd is slow for whatever reason.

1. i386 autopkgtest of libreoffice does a libreoffice build (because of
#1002477 !) and then times out when copying the built tree

2. I lately did armhf and arm64 autopkgtests in preparation of
experimental->sid.

$ grep test.*\-\-\- tmp/autopkgtest.log
autopkgtest [03:40:51]: test testtools-bridgetest: [-----------------------
autopkgtest [03:43:42]: test testtools-bridgetest: -----------------------]

(copy... around 2 hours)

autopkgtest [05:49:17]: test pyuno-import: [-----------------------
autopkgtest [05:49:18]: test pyuno-import: -----------------------]
autopkgtest [05:49:39]: test uno-import: [-----------------------
autopkgtest [05:49:40]: test uno-import: -----------------------]
autopkgtest [07:49:38]: test access2base-import: [-----------------------
autopkgtest [07:49:40]: test access2base-import: -----------------------]

(copy... around 2 hours). For tests basically doing "python3 -c 'import
<module>')

autopkgtest [09:50:59]: test test-extension: [-----------------------
autopkgtest [09:51:11]: test test-extension: -----------------------]
autopkgtest [09:51:28]: test test-extension-shared: [-----------------------
autopkgtest [09:51:39]: test test-extension-shared: -----------------------]

(copy... around 2 hours)

autopkgtest [12:05:15]: test uicheck-cui: [-----------------------
autopkgtest [12:06:33]: test uicheck-cui: -----------------------]
autopkgtest [14:22:25]: test uicheck-sc: [-----------------------
autopkgtest [15:55:23]: test uicheck-sc: -----------------------]
[...]

You get the idea. And this _is_ a ssd. Even though a rpi4 so
usb-attached and thus not fast.

(The above only works because I wanted the results and exaggerated with
--timeout-factor=20)

In any case, I consider only *5 MINS* to be a ridicuously small timeout.

Regards,

Rene