From: Kienan Stewart Date: Tue, 15 Oct 2024 18:19:54 +0000 (+0000) Subject: Tests: Convert tests readme to adoc format X-Git-Url: https://git.lttng.org./?a=commitdiff_plain;h=4d2d016041cef0a35f4d43aa29b69b8d82694df2;p=lttng-tools.git Tests: Convert tests readme to adoc format New documentation should be in adoc, and it's not a big hassle to change this one. Change-Id: I43b82749db0795b488d936925683aaa3a6a4f95b Signed-off-by: Kienan Stewart Signed-off-by: Jérémie Galarneau --- diff --git a/tests/Makefile.am b/tests/Makefile.am index 01babf7da..e3aa85e52 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -16,7 +16,7 @@ endif dist_noinst_SCRIPTS = run.sh long_regression root_regression -EXTRA_DIST = run.sh long_regression root_regression README +EXTRA_DIST = run.sh long_regression root_regression README.adoc all-local: @if [ x"$(srcdir)" != x"$(builddir)" ]; then \ diff --git a/tests/README b/tests/README deleted file mode 100644 index fc8630cda..000000000 --- a/tests/README +++ /dev/null @@ -1,63 +0,0 @@ -* Test Anti-Patterns - -OK, there are a few patterns that have been found over and over in the -testing code base which makes the tests flaky. Here is an incomplete -list. Don't do that. - -1) Using pidof to wait for a background application (by name) to - disappear. - - Why is it flaky ? - - The application may be delayed after being forked, but not executed - yet. Therefore, pidof will not find it. Use "wait" instead. - -2) Using sleep as delay-based optimistic synchronization technique. - - Why is it flaky ? - - Everything that needs to happen before/after other things need to - be explicitly synchronized using e.g. a file (used as a flag). - Sleep is just an indicator of a minimum arbitrary delay, but - machine load and scheduling can actually mess up the real delay - between applications. Use explicit synchronization points. Never - sleep. - -3) Using killall on a background application. - - Why is it flaky ? - - Similarly to pidof, killall may run before the background application - executes, thus failing to find it. Store the application PID after it - it launched in background into a temporary variable for later use - by kill and wait. - -4) Using wait ${!} to wait for completion of many background - applications. - - Why is it flaky ? - - It just waits for the last application put in background. Use - "wait" to wait for all background applications. - -5) Forgetting wait at the end (or error return path) of a test phase - that has background applications. - - Why is it flaky ? - - Those application may interact with the following testing phases, - thus skewing the results. - -6) Not grepping into the entire code base for similar patterns. - - When you find a problematic coding pattern, chances are it appears - elsewhere in the testing code base. Please fix it everywhere! - -7) Introducing a utility abstraction without changing all open coded - similar code path. - - When an abstraction for e.g. starting and stopping the session daemon - is introduced as a utility (e.g. utils.sh), future changes will - assume that all the testing code base is using this abstraction. - Leaving a few custom open-coded sites of duplicated code around is a - good way to make it a pain to update the abstraction in the future. diff --git a/tests/README.adoc b/tests/README.adoc new file mode 100644 index 000000000..64b037ac6 --- /dev/null +++ b/tests/README.adoc @@ -0,0 +1,66 @@ +// Render with Asciidoctor + += LTTng-tools Tests + +:toc: + +== Test Anti-Patterns + +OK, there are a few patterns that have been found over and over in the +testing code base which makes the tests flaky. Here is an incomplete +list. Don't do that. + + +=== Using pidof to wait for a background application (by name) to disappear + +Why is it flaky ? + +The application may be delayed after being forked, but not executed yet. +Therefore, pidof will not find it. Use "wait" instead. + +=== Using sleep as delay-based optimistic synchronization technique + +Why is it flaky ? + +Everything that needs to happen before/after other things need to +be explicitly synchronized using e.g. a file (used as a flag). +Sleep is just an indicator of a minimum arbitrary delay, but +machine load and scheduling can actually mess up the real delay +between applications. Use explicit synchronization points. Never +sleep. + +=== Using killall on a background application + +Why is it flaky ? + +Similarly to pidof, killall may run before the background application +executes, thus failing to find it. Store the application PID after it +it launched in background into a temporary variable for later use +by kill and wait. + +=== Using wait ${!} to wait for completion of many background applications + +Why is it flaky ? + +It just waits for the last application put in background. Use +"wait" to wait for all background applications. + +=== Forgetting wait at the end (or error return path) of a test phase that has background applications + +Why is it flaky ? + +Those application may interact with the following testing phases, +thus skewing the results. + +=== Not grepping into the entire code base for similar patterns + +When you find a problematic coding pattern, chances are it appears +elsewhere in the testing code base. Please fix it everywhere! + +=== Introducing a utility abstraction without changing all open coded similar code path + +When an abstraction for e.g. starting and stopping the session daemon +is introduced as a utility (e.g. utils.sh), future changes will +assume that all the testing code base is using this abstraction. +Leaving a few custom open-coded sites of duplicated code around is a +good way to make it a pain to update the abstraction in the future.