Clean-up: unify comment style

Some comment blocks use /** in lieu of /* on the first line. Since the
latter style is prevalent in the code base, replace all instances to
attain One True Style.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: Ica1aee472bdbeaaa40ee32f67274e3c43f7f9850

Bump required clang-format version to 16

Bump the required clang-format version from 14 to 16. For reference,
this is the version the Babeltrace project uses [1], and it is checked
by the LTTng CI [2], so I think it makes sense to go to that version.
More recent versions might not be readily available on the CI machines
(mostly running Debian 12 at the moment).

FWIW, I think that the code changes generated by this version change are
improvements.

[1] https://github.com/efficios/babeltrace/blob/d16ccfd174984c3d18f4f4427e4e438b1a64730e/tools/format-cpp.sh#L7
[2] https://ci.lttng.org/view/Babeltrace/job/babeltrace_master_lint/

Change-Id: I188f04928357ac7562b1a2dce3238a622189b04a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: relayd: leaked socket for live connections

Observed issue
==============

While exploring an issue where Python programs using the `bt2` module
would crash on shutdown, it was noticed that the same case also
highlighted a leaked file descriptor.

The following Python program may be used to demonstrate the leak.

```

import os
import socket
import time

import bt2

def is_connected():
    ctf_live_cc = bt2.find_plugin("ctf").source_component_classes["lttng-live"]
    q = bt2.QueryExecutor(ctf_live_cc, "sessions", params={"url": "net://localhost"})
    connected = False
    try:
        for x in q.query():
            print(x)
            if x['session-name'] == 'test' and x['client-count'] >= 1:
                connected = True
                break
    except Exception as e:
        print(e)
    return connected

os.system("lttng create test --live")
os.system("lttng enable-event -u --all")
os.system("lttng start")

ctf_live_cc = bt2.find_plugin("ctf").source_component_classes["lttng-live"]
iterator = bt2.TraceCollectionMessageIterator(bt2.ComponentSpec(ctf_live_cc, {'inputs': ["net://localhost/host/{}/test".format(socket.gethostname())], 'session-not-found-action': 'end'}))

data = []
while not is_connected():
    try:
        data.append(next(iterator))
    except Exception as e:
        print(e)
    time.sleep(0.1)

os.system("lttng stop")
os.system("lttng destroy")
os.system("killall lttng-sessiond")
os.system("killall lttng-relayd")
```

Cause
=====

During the clean-up at the end of the live worker thread, all remaining
viewer connections have their references put (which should cause the
connection to be released).

When releasing connections, `close` is never called on the socket's file
descriptor. Furthermore, the release doesn't modify `the_fd_tracker`.

Solution
========

Explicitly close and stop tracking the sockets for viewer connections
just prior to the final put.

Known drawbacks
===============

None.

Change-Id: I5eb0a3e5b9cb14dc1199be463bc312cbc72d8244
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Document common environment variables used to control tests

Change-Id: I4837d506de0a366d847fd529e0c747b48701ccf1
Signed-off-by: Kienan Stewart <kstewart@efficos.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Convert tests readme to adoc format

New documentation should be in adoc, and it's not a big hassle to
change this one.

Change-Id: I43b82749db0795b488d936925683aaa3a6a4f95b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add test to cover live viewer hangs with an early inactive app

Refs: https://bugs.lttng.org/issues/1406

Change-Id: Iefe796d39ff39a04055acc5f95fc9440d5ee244a
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add environment variables in tests to attach gdbserver

When debugging tests, the current infrastructure in both the python and
bash test harnesses require that the user start the sessiond or relayd
beforehand, and run the tests with environment variables to stop the
spawning of the respective programs.

To facilitate the process, new environment variables are added to allow
gdbserver to be spawned and attach to the relayd or sessiond. The user
may then connect with gdb, for example: `gdb -ex "target
localhost:1001"`.

Change-Id: Id4d1b446c7d6682c011ef27682198fb4a503f5f4
Signed-off-by: kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add wait_until_disconnected() to python test LiveViewer

Change-Id: I70cf5e46a08c658b3fecbcea9317e1b2d9f769fb
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add _LiveViewer.is_connected()

Change-Id: I3adf51ceea08d62e3d23598b57460427cf5561ea
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Stop draining LiveViewer when iterator is inactive

Change-Id: Ic67cea7c3fa57ed5c48bbab7d21d206a827852ad
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

doc: Specify that string memory must be freed

Specify that the caller of `get_session_name()` must explicitly free the
memory associated with the returned string.

Historically this software was written in C. C programmers have the
reflex to assume they are responsible for freeing memory. However, now
that this project is mixed C/C++ and is transitioning towards C++ the
assumption that developers will automatically know to free memory
(according to C programming conventions) does not hold as well. For this
reason, clarify that memory must be explicitly freed by the function
caller.

There are other C functions in this software that return variables that
must explicitly be freed by the caller. However, these changes have not
been applied consistently to all these cases. The assumption is that
this individual clarification still reduces confusion even if not
applied consistently.

Change-Id: I72d3568b5c1a7d8b7ff9b3d241bca1c5c55e47c1
Signed-off-by: Erica Bugden <ebugden@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: relayd: viewer_stream leak causes assertion failure on exit

Observed issue
==============

Running the test proposed in change #11584[1], the relay daemon aborts
when destroying the viewer_streams_ht as it is not empty.

Cause
=====

A viewer stream reference is leaked when sending streams to the live
client causing them to remain published in the viewer_streams_ht beyond
the lifetime of the viewer_connection.

The send_viewer_streams() function operates in two phases. First, it
iterates over the viewer_streams_ht to find streams that belong to the
target session and have not been sent yet.

In the second phase, it iterates over the session's unannounced stream
list. The commit message of 98b82dfa2 gives more background on the role
of the unannounced stream list.

When a viewer stream is created, two references are acquired:
  - one belongs to the global viewer_streams_ht,
  - the other belongs to the unannounced stream list.

When the viewer stream is eventually sent to the client, it is removed
from the unannounced stream list and that reference must be dropped.

Unfortunately, the reference is not dropped during the first phase.

Solution
========

Put the reference of the viewer streams that are sent during the first
phase of send_viewer_streams().

Known drawbacks
===============

None.

[1] https://review.lttng.org/c/lttng-tools/+/11584

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I7d5d5d3a77f6f08744b712d9ef1233a1d19a7124

Use compiler-agnostic defines to silence warning

g++ emits warnings that it can't recognize the clang-specific diagnostic
pragmas. They are replaced by the internal compiler-specific macros so
that nothing is emitted when g++ is used.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I543987a861d2322aa2ef3b7d631f280d2ac999bf

Disable clang warning for injected class name ambiguity in non_copyable_reference

clang raises a warning (-Winjected-class-name) due to ambiguity between
a constructor name and a type within the non_copyable_reference code.

Since clang could not infer the correct type context, this commit uses
`#pragma clang diagnostic` to disable the specific warning in the
affected area of the code.

The `push` and `pop` pragmas ensure that the warning is disabled only
where needed, preventing it from affecting other parts of the codebase,
and allowing us to maintain clean and clear code without unnecessary
compiler warnings.

A static_assert enforces that CustomDeleter::deleter is indeed a type,
although interpreting it as a constructor would be non-sensical here.

Change-Id: Ic0aac06d7af4272438f6f3d0275f29dc57a32194
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

lttng: remove use of variable length array

Use fmtlib to format the session attribute string when saving
the current session to .lttngrc. This eliminates a warning
emitted by clang (VLAs are not standard in C++).

Change-Id: Icdb8c1cc47adcbdfd82eefa8d2f1bf37a042a028
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add and set new log-file-d for the tap driver

This adds a new option `--log-file-d` to the tap-driver, which will
create a `*.log.d` folder for each test when running `make check` and
the `LTTNG_TEST_LOG_DIR` accordingly.

Doing so allows the tests to be run in verbose and create logs in a
predictable location. These log folders are removed when running `make
clean`.

Change-Id: Ibcf7e2cb54098a3e9ccd828ca76df6efcf33431d
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Use _run_babeltrace_cmd where possible

Change-Id: I5b279201a1806e8975de39fa85e666da3d0c5204
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add environment variables for verbosity and log directory

Observed issue
==============

When working locally with test failures, changes to `utils.sh` are often
required to produce verbose output.

Solution
========

By adding environment variables to allow running the various tools
with higher verbosity and potentially outputting to either stderr or
temporary files in a given directory test runners now have the option
to quickly get more information.

Known drawbacks
===============

Some tests depend on parsing either stderr or stdout, and these global
defaults may potentially make developing robust tests more
complicated.

Change-Id: I4128c421cdf9ce12827adc017dba5a298b62b6de
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Clean-up: Remove duplicate babeltrace calls

Change-Id: Iec2471dcc97c8a2d73df6afe1b93db95b6316c4f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Clean-up: fix string truncation warning

Some version of g++ emits the following warning:
  'char* strncpy(char*, const char*, size_t)' output may be truncated
  copying 255 bytes from a string of length 255 [-Wstringop-truncation]

Using the internal strncpy wrapper, which checks for truncation,
fixes the problem.

Change-Id: I8ab0f2cca0247eee329b137f547d3dfed32c995f
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: unload all kernel modules on sessiond exit

Stopping a root lttng-sessiond that has loaded kernel modules currently
leaves some modules loaded, add them in the correct order to allow
unloading them all.

Change-Id: I71f25c798f8c42737d295f32a5e3708287168bc6
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

clang-format: run clang-format on the tree

It appears re-running clang-format produces additional changes after
7c8d0f41c. This state now seems stable for the moment.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: If46d7cab85fa6a5df41453b3ea3aa0e80782768d

Tests: test_session: reduce session count in unit test

The destruction of the sessions takes 45 secondes to complete and
I don't see what testing 10 000 iterations tests that is not
achieved by 1 000 iterations.

That step now completes in 32ms on my development machine 🌪

Change-Id: I84e807750678144045974f27d084dd45f0f9713b
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: test_session: fix conditionally-supported offsetof warning

g++ produces the following warning when building test_session on the CI:

  /home/jenkins/workspace/dev_review_lttng-tools_master_linuxbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/stable-0.14/platform/deb12-amd64/deps/build/include/urcu/list.h:160:49: warning: ‘offsetof’ within non-standard-layout type ‘ltt_session’ is conditionally-supported [-Winvalid-offsetof]
     160 |         for (pos = cds_list_entry((head)->next, __typeof__(*(pos)), member); \
   /home/jenkins/workspace/dev_review_lttng-tools_master_linuxbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/stable-0.14/platform/deb12-amd64/deps/build/include/urcu/list.h:126:49: note: in expansion of macro ‘caa_container_of’
     126 | #define cds_list_entry(ptr, type, member)       caa_container_of(ptr, type, member)
         |                                                 ^~~~~~~~~~~~~~~~

Replace the problematic liburcu macros with the C++-ified internal
replacement wrappers.

Change-Id: I1396e9f6e1d99710150e03cabc33d07df3d8558a
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

vendor/argpar: sync with upstream repository

Sync with argpar, commit 88d1c8ae5c47 ("tap: import some changes").

Change-Id: I2338119ffa55391f836ed2bacc09360fc3f47217
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Update fmtlib to 11.0.2 and use the built library

Change-Id: I51c80c7d1829604a035e84dac7d87a48bb6fe05b
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Clean-up: sessiond: logging typo hast -> hash

The comment refers to hash tables, not The Hate Tables 🤘

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: Idac6e7e5d9b4bebca9db9e3187ec33a4dd335ddf

clang-format: run clang-format on the tree

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: Ifa81e86645655c1e4c1ee81375dad3ba45e850d1

Move argpar to vendor directory

Since this is source copied as-is from another project, I think it
belongs to the vendor directory.  This will make it so it will be
skipped by format-cpp, for instance.

Change-Id: I78892f80c4cbb3a2e863567b0021e895c6489402
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

configure.ac: enable automake's subdir-objects option

From [1]:

    subdir-objects

        If this option is specified, then objects are placed into the
        subdirectory of the build directory corresponding to the
        subdirectory of the source file. For instance, if the source file is
        subdir/file.cxx, then the output file would be subdir/file.o. See
        Program and Library Variables.

This will allow reducing the number of Makefiles, but placing rules in
Makefiles in parent directories, instead of having Makefiles in every
single directory with something that needs to be built.

[1] https://www.gnu.org/software/automake/manual/html_node/List-of-Automake-options.html

Change-Id: Ic0c6ee5d809dee6b9e560239abd15408b28d9695
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Remove unused helper `default_pipe_size_getter`

Change-Id: If72cb1456edbf26c340fcffdf93c8fb874ab030e
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Make notifier discard count test more robust

Observed issue
==============

In the CI, this test would intermittently fail. During failures,
the calculated pipe size from the `default_pipe_size_getter`
application was 8192, while in other cases it was 65536.

```
ERROR: tools/notification/test_notification_notifier_discarded_count
====================================================================

1..41
ok 1 - Add trigger my_trigger
PASS: tools/notification/test_notification_notifier_discarded_count 1 - Add trigger my_trigger
  ---
    duration_ms: 1323.966137
  ...
ok 2 - No discarded tracer notification
PASS: tools/notification/test_notification_notifier_discarded_count 2 - No discarded tracer notification
  ---
    duration_ms: 22.021590
  ...
ok 3 - Generating 390 tracer notifications
PASS: tools/notification/test_notification_notifier_discarded_count 3 - Generating 390 tracer notifications
  ---
    duration_ms: 154.790871
  ...
not ok 4 - Discarded tracer notification number non-zero (0) as expected
FAIL: tools/notification/test_notification_notifier_discarded_count 4 - Discarded tracer notification number non-zero (0) as expected
  ---
    duration_ms: 24.323759
  ...
```

Cause
=====

The initial size of pipes in linux may have different values:

1) `16 * PAGE_SIZE` (as documented in `man 7 pipe`) (since Linux 2.6.11)
2) When a user has many pipes open and is above a soft limit:

  * `2 * PAGE_SIZE` (undocumented, see[1]), as of Linux 5.14[2]
  * `1 * PAGE_SIZE` since linux 2.6.35[3]

As the program `default_pipe_size_getter` opened a pipe to check it's
size, there could be times in a system where a user has many pipe
buffers open beyond the soft limit and the lower value would be
returned; however, the previously opened sessiond may have had a pipe
opened with the larger default pipe size.

Solution
========

Use the maximum page size (on Linux, from
`/proc/sys/fs/pipe-max-size`) for the estimated pipe size rather than
opening a pipe and checking it's size.

Known drawbacks
===============

When the maximum pipe size value is much larger than the actual size
of the notification pipe, many more events are emitted than is
necessary to complete the test.

References
==========

[1]: https://gitlab.com/linux-kernel/stable/-/blob/3e9bff3bbe1355805de919f688bef4baefbfd436/fs/pipe.c#L809
[2]: See upstream commit 46c4c9d1beb7f5b4cec4dd90e7728720583ee348
[3]: See upstream commit 6a6ca57de92fcae34603551ac944aa74758c30d4

Change-Id: Id547a1d772b5a7f9b18ffa686ff6644afca4ab15
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

docs: Update relayd architecture

Observed issue
==============

The sessiond sessions do not map one-to-one with relay
sessions. Rather, there can be one relay session associated with
each of the active consumers, e.g. ustconsumerd64, ustconsumerd32,
and kconsumerd of each lttng-sessiond sessions.

Solution
========

The phrasing of the the relay session has been updated to
"per-consumer". An additional mention is added to say that
attaching a viewer session to multiple lttng-sessiond
sessions is not supported.

Change-Id: I1df18c4e97c0ee9ec4ee17b3bf35c6e74c90774f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

tests: Add test for missing short lived applications

Change-Id: I691b9fb2cfae7603e40ca4c668acaff752ba3f28
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: relayd: live: Catch short lived applications for attached viewers

Observed issue
==============

When a live viewer is attached to a session and a new application
starts, emits events, and exits the viewer may not see the produced
events.

With per-UID buffer allocation, the application needs to run as a new
user that hasn't had streams allocated before. With per-PID buffers,
spawning a new traced application is sufficient.

Cause
=====

When the new relay streams are created, associated viewer streams are
not immediately created. As a result, there is a gap between in which
the session may start being destroyed and/or the relay streams
unpublished and the time at which the live viewer sends a GET_NEW_STREAMS
command. When the relay streams are unpublished for any reason, the
reference to the relay stream in the ctf_trace is removed. The new
and unsent streams iterate over the relay streams in each ctf_trace.
Therefore, relay streams that were created and unpublished while
the live viewer was already attached to the session can be completely
missed.

Solution
========

The solution has three main aspects:

1. When new relayd streams are published and a viewer is attached for the
corresponding relay session or when a live viewer session attaches to
an existing relay session the viewer streams are created immediately.

2. The unsent viewer streams are tracked in a per-viewer session
list so that there continues to be a reference (via the
viewer_stream->stream backreference) held for the relay stream, and that
unpublished relay streams can be found without iterating over the
entire relay streams hashtable.

3. To cover cases where a relay stream has been closed but there are
still known trace chunks available, an additional check has been added
to the `get_next_index` viewer stream transition checks. When the
seen rotation count and relay stream rotation count are the same and
that the relay stream no longer has an active trace chunk, the
viewer stream is not forcibly rotated. This stops the final drop to
the trace chunk reference (via
viewerstream->stream_file->trace_chunk). Later, when the relay stream
is fully closed, there is a final rotation that is performed.

Known drawbacks
===============

The current implementation adds a global hash table which holds
references to created viewer sessions. When searching to determine if
new viewer streams should be created, the search is O(N*M) where N is the
number of viewer sessons and M is the number of relay sessions.

A different approach to recording references from relay sessions to
viewer sessions (if any exist) could reduce the search space.

Change-Id: Ie8f00697a4dafd5c9b0bfe60a872d1c1882f6944
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add controls to run python tests with verbose output

When running failing tests, it can be useful to get verbose output
immediately without trying to run an environment with a separate
sessiond and/or relayd.

Setting `LTTNG_TEST_VERBOSE_RELAYD` or `LTTNG_TEST_VERBOSE_SESSIOND`
environment variables will cause the corresponding application to be run
in it's most verbose configuration.

Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Change-Id: Ic2dd84a36f61837dfbca99d06d6a438ae884f782
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Clean-up: Tests: Use lttngtest to run live viewer for test_live_hang.py

Change-Id: Iadc40684c8cd5f0ce64e45e3c78747ca54f5bc89
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: Tests: Use wait_before_exit_file_path in WaitTraceTestApplication

The `_WaitTraceTestApplication`'s `__init__` method proposed the
`wait_before_exit` and `wait_before_exit_file_path` parameters; however,
the parameters weren't then passed onwards to the trace test
application's invocation.

Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Change-Id: I9055aa206a8fd943012bacfa49d6ff152f2dfbde
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Add LiveViewer to lttngtest environment

Drawbacks
=========

With the current python bindings, the relayd seems to leak a file
descriptor; however, this doesn't stop the tests from working.

E.g.

```
ok 1 - BT2 live viewer exited successfully

Change-Id: I13994f7c8b0b6cffcee0d0ea0f8fca22538de651
---
  duration_ms: 1097.302968
...

 Killing session daemon (pid = 3340512)
Session daemon killed
lttng-relayd: Error: A file descriptor leak has been detected: 1
tracked file descriptors are still being tracked
```

Change-Id: Ie4294dd7238d4b6074af2d4cf193e1ca9949a741
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Allow the creation of dummy users in the lttngtest environment

There are tests that need other user accounts created (e.g. to exercise
per-UID buffers with more than one user). Those accounts may be created
by the test environment and cleaned up on deletion.

An option has been added to the _WaitTraceTestApplication to run the
application as another using `su`.

Change-Id: Ie003e628258fdfbea1972f1f8825c4466fc2792b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Tests: Do not remove interrupted test log files by default

Observed issue
==============

During CI runs, builds may timeout or be killed for another reason.
Those tests logs are deleted and cannot be checked for diagnostic
information, warnings, or errors.

Cause
=====

By default, the test log for the currently running test is deleted by
automake so that subsequent invocations of `make check` will re-run the test.

Solution
========

Add a disable flag `--disable-precious-tests` and set
`PRECIOUS_TESTS` to true by default when configuring lttng-tools. When
`PRECIOUS_TOOLS` is set, all test logs in `tests/regression` will be
marked as `.PRECIOUS` and subsequently not deleted when interrupted.

Known drawbacks
===============

This could make interrupting a test and re-running during test
development more of a hassle.

References
==========

[1]: https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
[2]: https://automake.gnu.narkive.com/1TjEGbH2/delete-on-error-test-suite-log-and-precious

Change-Id: I08b4a1bb29eb609827cc1c047f141f94b210effe
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

lttng: add-trigger: clarify terminology for log levels

To eliminate ambiguity in the code, the terminology for log levels has
been updated. The previous terms "min" and "max" log levels have been
replaced with "least_severe" and "most_severe" respectively.

This change addresses the varying conventions across different logging
domains, where numerical values for severity can either increase or
decrease with severity. The new terminology provides clarity, making it
easier to understand the severity levels regardless of the logging
domain's convention.

Change-Id: Ie90bcc8e4c07b8b7437d9580e166141fae5c6d2f
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: inverted logic in loglevel_parse_range_string_common function

The mapping of numerical severity levels to their corresponding names
varies across different logging domains. Some domains, like
Java Util Logging, use higher numerical values for more severe logging
levels, while others, like Log4j2, use lower values for the same
purpose.

To accommodate this variation, the `loglevel_parse_range_string_common`
function has been updated. It now accepts the numerical value
representing the most severe logging level in a given domain. This
change ensures that log level specifications in the format `TRACE..` are
parsed correctly, regardless of the domain's convention.

Change-Id: Idbc3949ac33b69c71fce484a6d8912f59cdbe08d
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Add 'log4j2' domain event-rule unit tests

Change-Id: I8a075516de8cb0b682791e4bf44d9a5d7780680e
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Add Log4j 2.x agent tests for the 'log4j2' domain

Add integration tests for the new Log4j 2.x agent in its native mode
using the new 'log4j2' domain, the new configure switch
'--enable-test-java-agent-log4j2' to enable it or
'--enable-test-java-agent-all' to enable all Java agents tests.

To run only this new test, use this command :

  cd tests/regression && make check TESTS="ust/java-log4j2/test_agent_log4j2_domain_log4j2"

Change-Id: Idfac151d2e523b5ac109f2dae2f182b0bc9415d8
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Add 'log4j2' domain to the documentation

Change-Id: Ie76c686583f10bc09b9769db66e8e079b8472a37
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Add 'log4j2' domain to zsh completion

Change-Id: Ic9121630022981909962a0b5943a20dbe5240558
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Add 'log4j2' domain to common test code

Change-Id: I84961823eb875c673e525d83d8096291506c1edb
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Add a Log4j 2.x agent specific domain 'log4j2'

The initial version of the new LTTng-UST Log4j 2.x agent only operated
in a compatibility mode making use of the existing 'log4j' tracing
domain currently implemented in LTTng-Tools.

While this is useful when migrating existing Log4j applications using
the compatibility bridge it does require converting the log levels from
the new Log4j 2.x values to the old Log4j 1.x standard. This results in
hiding the actual log level values from the users for applications
natively using Log4j 2.x.

Exposing the native Log4j 2.x log level values requires a new domain
since the changes are significant:

  * The same list of standard log levels and names
  * Each standard log level has a new integer value
  * The log levels scale is reversed and shortened from
    'int32_max -> int32_min' to '0 -> int32_max'
  * The interval between standard log levels has changed

This new 'log4j2' domain is basicaly a straight copy of the current
'log4j' domain with minor adjustements for the reversed and shortened
scale.

Change-Id: I89f9c0a428ffe1d0bd26f7af547e9e21503de653
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

clang-format: run clang-format on the tree

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I45c2a1f375e5b19b649ee5d11fb299ccd77c6845

clang-format: ignore generated files

Two auto-generated files cause clang-format < v17 to hang when
they are being formatted. I have not looked into the root-cause,
but formatting them is useless anyhow.

Adding them to .clang-format-ignore works around the problem for the
moment.

Since clang-format 14 does not support ignore files, their support is
crudely emulated here using grep to filter out find's results.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I1cd10349ef1a66b1d595105b0ec1a4beef9dcc9a

Fix: Do not null out lttng_consumer_stream channel on deletion

Change-Id: Ic98a27e6704d2683d24b8645d345955cee8b038c
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: Crash when unregistering UST apps during shutdown

Observed issue
==============

The following crash has been observed in v2.12.2:

```
function=0x55ac7c4c9600 <_ PRETTY FUNCTION .12873> "lttng_ustconsumer_close_metadata") at assert.c:92
function=0x55ac7c4c9600 <_ PRETTY FUNCTION .12873> "lttng_ustconsumer_close_metadata") at assert.c:101
```

The underlying cause is applicable in the current master branch as
well.

Cause
=====

There is a potential race between the threads the consumerd control
thread which handles commands coming from the sessiond and the main
thread when shutting down a consumerd.

Is it possible that the following happens:

1. `destroy_metadata_stream_ht` has the locks on `consumer_data`,
`channel`, `stream`
2. `lttng_ustconsumer_close_all_metadata` looks up the channel and starts to try and acquire a channel lock (`stream->chan->lock`)
3. `destroy_metadata_stream_ht` sets `stream->chan` to `null`
4. `destroy_metadata_stream_ht` releases the `stream`, `channel`, and `consumer_data` locks
5. `lttng_ustconsumer_close_all_metadata` now has the channel lock, and looks up `stream->chan` again to call `destroy_metadata_stream_ht`, and that member is now null

Solution
========

Acquire the stream lock after acquiring the channel lock.

part 2 follows: don't set stream->chan to null.

Known drawbacks
===============

None.

Change-Id: I1d27ea6ac08f3e7ed4624a8921cffb675be649d2
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: Compilation failure deducing type of `auto` variables in GCC 4.8

Observed issue
==============

When compiling with GCC 4.8.5 or GCC 5.5.0 on SLES12SP5, the following
error happens:

```
save.cpp: In function 'int save_agent_events(config_writer*, agent*)':
save.cpp:1185:43: error: use of 'agent_event' before deduction of 'auto'
       lttng::urcu::lfht_iteration_adapter<agent_event,
                                           ^                                                                 save.cpp:1185:43: error: use of 'agent_event' before deduction of 'auto'
save.cpp:1185:43: error: use of 'agent_event' before deduction of 'auto'
save.cpp:1187:26: error: template argument 1 is invalid
        &agent_event::node>(*agent->events->ht)) {
                          ^
save.cpp:1187:26: error: creating pointer to member of non-class type '<type error>'
save.cpp:1187:26: note: invalid template non-type parameter
In file included from ../../../src/vendor/fmt/core.h:3316:0,
                 from ../../../src/common/format.hpp:20,
                 from ../../../src/common/error.hpp:13,
                 from ../../../src/common/common.hpp:12,
                 from snapshot.hpp:13,
                 from consumer.hpp:12,
                 from session.hpp:11,
                 from kernel.hpp:13,
                 from save.cpp:10:
```

Cause
=====

This appears to be a limitation in older versions of GCC. I did not
find specific commit(s) or bugs which hilight the issue, but
compilation of this code works as of GCC 6.5.0 on SLES12SP5. Previous
point releases of GCC 6.x were not tested.

Solution
========

Explicitly define the type of the pointer and the type passed to
`lttng::urcu::lftht_iteration_adapter` so the compiler does not have
to perform type deduction.

Known drawbacks
===============

None.

Change-Id: I71c5937a38336756ece4f396ea5ba7af7f3d36c3
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: Compilation failure in session_not_found_error with GCC 4.8

Observed issue
==============

When compiling with gcc 4.8.5, the compilation fails with the
following erorr:

```
session.hpp:577:2: error: function 'lttng::sessiond::exceptions::session_not_found_error::session_not_found_error(lttng::sessiond::exceptions::session_not_found_error&&)' defaulted on its first declaration with an exception-specification that differs from the implicit declaration 'lttng::sessiond::exceptions::session_not_found_error::session_not_found_error(lttng::sessiond::exceptions::session_not_found_error&&)'
session.hpp:577:2: error: function 'lttng::sessiond::exceptions::session_not_found_error::session_not_found_error(lttng::sessiond::exceptions::session_not_found_error&&)' defaulted on its first declaration with an exception-specification that differs from the implicit declaration 'lttng::sessiond::exceptions::session_not_found_error::session_not_found_error(lttng::sessiond::exceptions::session_not_found_error&&)'
```

Cause
=====

This is due a bug in GCC which is fixed as of GCC 5.0[1]

Solution
========

Do not explicitly define the move_assignable for
`lttng::sessiond::exceptions::session_not_found_error` as
`noexcept`. The function should be implicitly generated as `noexcept`.

Known drawbacks
===============

None.

References
==========
[1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59526

Change-Id: I3368633ce3b45627f2e67f7d2def361e662eec3d
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: main.cpp: iterate on list using list_iteration_adapter

Change-Id: I492b597b70040c0e1f3eb826aadb66ca44550fb5
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: agent-thread.cpp: iterate on list using list_iteration_adapter

Change-Id: Ibd02f3e2c8d91fc8aa09097be2ea7b563001b1da
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: trace-ust.cpp: iterate on list using list_iteration_adapter

Change-Id: I20a5549e8a93b7fe0d111b72548af8e74d80531a
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: ust-app.cpp: iterate on list using list_iteration_adapter

Change-Id: I77d7ecb33f297561ec9c887495c7a798fa7f73ce
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: client.cpp: iterate on list using list_iteration_adapter

Change-Id: Ibb45513080329e805c757cfc69a99eaa14387ac0
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

relayd: ctf-trace.cpp: iterate on rcu list using rcu_list_iteration_adapter

Change-Id: I5c16d02d44fc90b9bf9336ac6cd795398d4ab4f5
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: manage-kernel.cpp: iterate on list using list_iteration_adapter

Change-Id: Icf0e10d675e1d0ba116c09b92d9426309b7cb606
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: dispatch.cpp: iterate on list using list_iteration_adapter

Change-Id: Ie8a45753922b0a5dd476be06ce15a1f7d6883c08
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

relayd: viewer-session.cpp: iterate on rcu list using rcu_list_iteration_adapter

Change-Id: Ie8110d36a9c59e687366309e6ee399e6a3f93bbc
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

relayd: session.cpp: iterate on rcu list using rcu_list_iteration_adapter

Change-Id: I9cfca29e54873c696ef6b8c84454e77e299ddd10
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

relayd: live.cpp: iterate on rcu list using rcu_list_iteration_adapter

Change-Id: I89824eb36bb317a424880f34dc962cf7b1eca1ed
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

relayd: main.cpp: iterate on rcu list using rcu_list_iteration_adapter

Change-Id: Id3070b39458b3e44185875c87dca069c2dbb6ed6
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

urcu: add rcu_list_iteration_adapter

Like its list_iteration_adapter counterpart, this utility allows
the iteration over cds_list structures and provides ranged-for
semantics.

This utility differs in that it provides an RCU-protected iteration
over the list.

Change-Id: I810126ee5e099a6e6bb15509d00efa0ebfa56e73
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: save.cpp: iterate on list using list_iteration_adapter

Change-Id: Ib413d1bf87c4c557c049c3ee329e9255525f7666
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: cmd.cpp: iterate on list using list_iteration_adapter

Change-Id: I79eeaad0b57da7d7650ca77c26a781d44b3a8152
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: buffer-registry.cpp: iterate on list using list_iteration_adapter

Change-Id: I30a861585a5c5c398f9999ab2efa687a133e09e6
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: session.cpp: iterate on list using list_iteration_adapter

Change-Id: Ie138974d18f25db7e449a856c59bfb3be87ac06a
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: context.cpp: iterate on list using list_iteration_adapter

Change-Id: I2e9d80dcf167d28c33d78bc269cd3e179c22f299
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: ust-consumer.cpp: iterate on list using list_iteration_adapter

Change-Id: I90d06ef8bbe674bc91fc19f489758a8c7c07cad1
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: kernel-consumer.cpp: iterate on list using list_iteration_adapter

Change-Id: I4464898e56cc88baa9a9b2b4c56461d95f342ac0
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: consumer.cpp: iterate on list using list_iteration_adapter

Change-Id: Iad57a1bfdce8b2b3f14702e93f2eceb4ff535f6a
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

urcu: add list_iteration_adapter

Add a type-safe cds_list iteration adapter. Like those provided for
the lfht, this adapter provides type-safe range-for semantics for
cds_list structures.

Change-Id: I03d3b3c31911d495c9c6187000400563a3d05ad8
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: ust-consumer.cpp: iterate on lfht using lfht_filtered_iteration_adapter

Change-Id: If4f54d8e79a8eb68e2214d3182150b974b806398
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: consumer.cpp: iterate on lfht using lfht_filtered_iteration_adapter

Change-Id: I6f7e66d8446e1ebc05679a61fd190053f252902f
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: consumer-timer.cpp: iterate on lfht using lfht_filtered_iteration_adapter

Change-Id: I3fc6eab351d901379e2ebcbf81e98d7b54c09586
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: consumer-stream.cpp: iterate on lfht using lfht_filtered_iteration_adapter

Change-Id: I625c1a942fc7aa635d78a62cf468b17651596412
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

urcu: add lfht_filtered_iteration_adapter helper

The urcu lfht macros often make use of caa_container_of (and other equivalent
variations) which use offsetof. Unfortunately, offsetof is conditionally
supported by compilers for non-POD types.

The tree already has lttng::utils::container_of to work around this
problem. This new utils makes it possible to iterate on the
elements of an lfht that match a given key without using those macros. Those iterations are the
main reason such warnings are emitted. The interface of
lfht_filtered_iteration_adapter also allows the use of ranged-for loops.

Change-Id: I9acd3fa9f6523de8006bc9107bcca6b6b654d6fa
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: consumer.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Icecd70d1023847b1489b298e4c04625bac4ebcc9
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

consumerd: ust-consumer.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Idbbfd6be4666bca420faf408241f5f195818981a
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

common: fd-tracker.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Ic28cf4f9a8534540f2e28dd5d69a146367280a9b
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

common: trace-chunk.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Ia05a6b974a9bafac993de76a781a22be487e94cd
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: ust-registry-session.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I0bc5ff2754be9ba417c6fa16535cf20cd23875f3
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: ust-registry-channel.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Ie415952021ef795259b9290e4b16d68051403524
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: tracker.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I96b0cd3bfe84357f9ec09d254a6d7f8757c52c69
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: trace-ust.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I5f87aebb6d8e4fc0ef6a4088faf0b7b1a84f37d0
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: snapshot.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I826575008b4ba2d828ad5ac52dc28d78ccca5e7d
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: session.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I2deae4866e0664e32cc40b891fc9e19b9936d66e
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: save.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I760ad9294584718e308775af65279688c2f58c65
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: rotation-thread.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I00f293b25e8747058c643ba63c07813a8d9a24ec
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: manage-kernel.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Id6a47ca1971f48dec5563085fa9e5affd48a1567
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: lttng-syscall.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Ib4439701c48b64b2244b346c86c7b1abb55503c4
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: notification-thread-events.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I15de31ca184c255b4a7540c2bf15f90ddf633da0
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: kernel.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: Ib129073617fc79c0dc9844129e637cac3d715598
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: event.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I12292f505e9f867898c92bcf6cbd81c10204d5d8
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

sessiond: event-notifier-error-accounting.cpp: iterate on lfht using lfht_iteration_adapter

Change-Id: I8a4486f86e634187439b8e6c13fdd1a52fd051de
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>