Michael Jeanson noticed that f94318611 mistakenly introduces a new
public symbol in liblttng-ctl.so. This change was not intended and is
due to a bad backport of a fix initially developed against the master
branch.
The master branch (and upcoming version) of LTTng-tools is built with
the -fvisibility=hidden. Hence, the initial version of the fix had no
need to hide the utils_create_lock_file symbol.
Since the supported stable releases (2.12 and 2.13) are not built with
those options, that symbol has to be explicitly marked as hidden.
Reported-by: Michael Jeanson <mjeanson@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I874538317617003eb1c58c2d3b7b0a2bdf905ef7
sessiond: open_packets: use user_space_consumer_channel_keys util
Replace the hand-rolled consumer channel key iteration logic of
ust_app_open_packets by an iteration on the user_space_consumer_channel_keys
returned by the ltt_session.
sessiond: clear_session: use user_space_consumer_channel_keys util
Replace the hand-rolled consumer channel key iteration logic of
ust_app_clear_session by an iteration on the user_space_consumer_channel_keys
returned by the ltt_session.
refactor: session: provide an iterator over consumer data channel keys
The code iterating over the various data channel keys of the userspace domain is
relatively complex and repeated in many places. It is refactored to provide an
iterator that can be used in those various places.
sessiond: ust_app: use RAII to manage ust_app_session locking
Like ltt_session, use the non_copyable_reference util to implement weak locked
reference helpers that are used instead of directly manipulating the
ust_app_session's pthread mutex.
hashtable: replace non-const iterator node accessors by a const version
The various lttng_ht_iter_get_node_* functions are not const-correct which makes
their use difficult in some places. This is mostly due to the fact that
cds_lfht_iter_get_node (from liburcu) is, itself, not const-correct.
These functions are replaced by a single templated function that is
const-correct by virtue of inlining the trivial iterator node accessor of
liburcu.
clang-tidy reports:
'NOLINTBEGIN' comment without a subsequent 'NOLINTEND' comment
If a specific warning is mentionned as part of the NOLINTBEGIN clause, it must
also be mentionned in the matching NOLINTEND clause. Otherwise, clang-tidy
doesn't match them.
sessiond: propagate the use of ltt_session::locked_ref
The mix of managed and unmanaged ltt_session references makes their use
potentially confusing. Move to managed locked references for most of the client
request processing paths.
Introduce ltt_session::locked_ref look-up functions and use them at the various
sites performing session look-ups by id or name.
The users of those look-up functions are refactored to make them exception-safe.
The biggest change is that process_client_msg is now exception-safe. The
functions are also moved inside of the ltt_session class as static methods. This
namespaces them implicitly.
A number of functions that expect a locked ltt_session are modified to accept an
ltt_session::locked_ref, thus enforcing the locking assumptions of the session
at compile time.
Exception: make source location print-out optional
The project-specific exceptions are changed to not include the source location
as part of the exception's message (return of what()) since it is unsuitable in
some contexts (the output of the lttng client for example).
Moreover, the exceptions are modified to provide their own message formatting
instead of relying on the user doing it at the site at which it is thrown since
that formatting is often repetitive. The sites at which the exceptions are
thrown can still augment the exceptions with more context.
The intention of our exceptions is that they still contain the requisite context
to change the print-out based on their members; that context should not be baked
into the message string at the site at which the exception is thrown.
sessiond: transition session_list synchronization from pthread to std
In order to allow the use of RAII to manage the locking in the client command
handler (process_client_msg), the session list's synchronization primitives are
changed to use those of libstd.
The locked_ref semantics garantee that an instance of an is locked and that it
exists (it can't be null). It is currently implemented with a unique_ptr, but
that will most likely change in the future to enforce the non-null requirement.
Clean-up: sessiond: client.cpp: list of inclusions
Correct some inconsistencies in the list of inclusions in client.cpp:
- include headers under common using angle-brackets
- only include the top-level lttng.h lttng-ctl header
vscode: export MI_XSD_PATH and LD_LIBRARY_PATH in gdb wrapper
vscode fails to launch a session daemon using the libtool wrapper on machines
that don't have lttng installed since the XSD can't be loaded.
Export the MI_XSD_PATH to point to the file in the source tree to allow the
daemon to launch.
Moreover, launching the session daemon fails since it can't find
liblttng-ctl.so.0. Adding the library's .libs to LD_LIBRARY_PATH works around
the issue.
Fix: consumerd: consumed size miscomputed during statistics sampling
Issue observed
==============
When specifying a size-based automatic rotation, sessions sometimes continuously
rotate after the first size-based rotation is triggered.
Steps to reproduce:
$ ./lttng create the_test
$ ./lttng enable-channel -u the_channel
$ ./lttng enable-rotation --size=32M
$ ./lttng enable-event -ua --channel the_channel
$ ./lttng start
# Produce more than 32MiB of tracing data and stop the application(s), then
# observe a rotation occuring on every expiration of the channel's monitor timer.
Cause
=====
The consumer daemon samples statistics of the various streams it monitors on
every expiration of their channel's "monitor" timer. One of the statistic it
maintains is the channel's total consumed size.
Once the stats are collected, the consumer sends a channel statistics sample
message to the session daemon. One of the fields of this message is the amount
of data that was consumed for the channel since the last statistics sample was
sent. The session daemon relies on that information to aggregate the consumed
size a session based on the consumed sizes of its various channels.
The consumer maintains a per-channel 'last_consumed_size_sample_sent' which is
used to compute the amount of data consumed since the last statistics sample was
sent to the session daemon. The computation is erroneous since the count sent to
the session daemon is 'the channel's total consumed data amount' minus 'the data
consumed since the last statistics sample'.
Solution
========
The counter maintained on a per-channel basis is now
consumed_size_as_of_last_sample_sent: the total amount of data consumed for that
channel as of the last statistics sample. On every expiration of the monitor
timer, the total amount of data consumed for the channel is determined and the
difference from that counter is sent to the sessiond daemon. The counter is then
updated with the latest computed value.
The test has been modified to loop, waiting for the post-destroy shm
count to drop back to the post-start count. In the case of a failure,
the test will hang forever but doesn't fail spuriously.
Known drawbacks
===============
None.
Change-Id: Id3c8a9f6db83fe888e79b8f06cb8308b4d90da87 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Mon, 15 Apr 2024 19:15:28 +0000 (15:15 -0400)]
lttng: Indicate file path and error reason when open_config fails
Observed issue
==============
In a test environment where the configuration file had been chowned to
no longer be writeable by the current user, `lttng` commands would work
but return a non-zero exit code and print `Error: Unable to create
config file`.
This doesn't give the user much information to work with to address the
issue.
Solution
========
The `PWARN` macro is used to print the file_path and reason when the
call to `fopen` fails.
This is done since the `file_path` percolated up to where the existing
error message is may be null for multiple reasons (eg. allocation
failure).
Known drawbacks
===============
The output of `PWARN` is inconsistent with the output used for most of
the `lttng` client.
Eg.
```
$ lttng create
Spawning a session daemon
Session auto-20240415-151450 created.
Traces will be output to /home/lttng-traces/auto-20240415-151450
PWARN - 15:14:50.420628389 [1184515/1184515]: fopen '/home/.lttngrc': Permission denied (in open_config() at conf.cpp:57)
Error: Unable to create config file
```
Change-Id: I5a9253fb02f3a712e7c5c2567311920dc33d36c7 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
docs: lttng-sessiond(8): bitness mix-up in env variable name
The section about the --consumerd64-path parameter erroneously refers to
the 32-bit consumer daemon binary environment
variable (LTTNG_CONSUMERD32_BIN).
The concatenated namespace syntax (e.g. namespace my::ns::woah{}) was
introduced in C++17 while this project is built as C++11. Leaving this
rule it in the .clang-tidy only adds noise (and salt to the wound).
Kienan Stewart [Fri, 10 May 2024 18:41:11 +0000 (14:41 -0400)]
tests: Correct use of taskset in snapshot tests
Observed issue
==============
While investigating potential changes to how buffers are flushed
during snapshots, I discovered that the changes made the small
overwrite test flaky.
After investigation, the issue appears to be a problem with the test
that was exposed by the other changes.
Cause
=====
`taskset` handles its arguments more strictly than many other CLI
programs, and the argument order used is not valid. As the return
of the taskset invocation in the test isn't validated, there was
little indication of the issue save a warning that is easily lost
in the text.
The following example demonstrates the situation:
```
$ bash -x test.sh
+ taskset -p 910843
pid 910843's current affinity mask: f
+ echo 0
0
+ ./tests/utils/testapp/gen-ust-events/gen-ust-events
Process 910845 has 4 cpus in its affinity set
CPU 0 is set? 1
CPU 1 is set? 1
CPU 2 is set? 1
CPU 3 is set? 1
+ taskset -c 0 -p 910843
taskset: failed to execute -p: No such file or directory
+ echo 127
127
+ ./tests/utils/testapp/gen-ust-events/gen-ust-events
Process 910849 has 4 cpus in its affinity set
CPU 0 is set? 1
CPU 1 is set? 1
CPU 2 is set? 1
CPU 3 is set? 1
+ taskset -pc 0 910843
pid 910843's current affinity list: 0-3
pid 910843's new affinity list: 0
+ echo 0
0
+ ./tests/utils/testapp/gen-ust-events/gen-ust-events
Process 910853 has 1 cpus in its affinity set
CPU 0 is set? 1
```
Solution
========
Correct the invocation of taskset and add a check on its return code
when used to set the cpu affinity of the current process.
Known drawbacks
===============
None.
Change-Id: Ia629121624532746431875b2031dd65df207666d Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Michael Jeanson [Tue, 30 Apr 2024 19:17:52 +0000 (15:17 -0400)]
fix: relayd: unaligned access in trace_chunk_registry_ht_key_hash
In 328c2fe7297c941aa9cbcfa4ce944fca1bd7300f, the type of 'lttng_uuid'
was changed from a C array of 16 'uint8_t' to a C++ std::array of the
same type and length.
In 'trace_chunk_registry_ht_key_hash()' we access these 16 bytes as 2
'uint64_t', to do so we used to cast the array to '(uint64_t *)' and
then access index 0 and 1.
When it was converted to C++, an error was introduced where instead we
reinterpret_cast to 'const uint64_t *' the index 0 and 1 of the array
which results in a 'uint64_t' pointer to the first and second bytes of
the array. These values overlap but since they are used as keys for an
hash table it still works. However, on platforms that don't allow
unaligned access, the second pointer being only offset by one byte
results in a 'Bus error'.
Reintroduce the old behavior by applying the index 0 and 1 to the
pointer resulting from the reinterpret_cast.
Change-Id: I2bc287edbe6864a2a870f9de1f3b4dd8f8a90ace Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Fix: syscall event rule: emission sites not compared in is_equal
The emission sites are not compared when comparing two kernel-syscall
event rules. This prevents users from subscribing to notifications of
triggers that use event-rule-matches syscall conditions which differ
only by their emission site (entry, exit, entry+exit).
vscode: Add configurations to run the executables under the debugger
Add tasks.json and launch.json which allow VSCode users to build the
project and run the various binaries (lttng, lttng-relayd,
lttng-sessiond) under the integrated debugger.
For the moment, the configuration assumes the user wants to build
"in-tree" and has setup the tree to build the project (running
./bootstrap and ./configure).
The build job attempts to build a compile database if 'bear' is
available on the system.
To debug the LTTng client, make sure to edit the matching configuration
in .vscode/launch.json to provide your desired arguments (for the
moment, 'help' is passed by default).
Fix: consumerd: leak of tracing buffers on relayd connectivity issue
Observed issue
==============
A leak of the tracing buffers can be noticed when the relay daemon is
terminated following the creation of a live session, but prior to the
initiation of any applications.
The issue can be reproduced with the following steps:
# Create a live session
$ lttng create --live
# Kill the relay daemon before the allocation of the buffers
$ killall lttng-relayd
$ lttng enable-event --userspace --all
$ lttng start
# Launch an instrumented application
$ ./my_app
# Destroy all sessions
$ lttng destroy --all
# List the open file descriptors of the lttng-consumerd process
# and notice how the tracing buffer are still visible.
$ ls -lah /proc/$pid_of_lttng_consumerd/fd
The consumer daemon allocates recording channels and their tracing
buffers in a two-step process.
First, the session daemon emits an `ASK_CHANNEL_CREATION` command, which
results in the allocation of the internal consumer channel structures
and of the actual tracing buffers. The channel's unique key is returned
to the session daemon.
After this command, the channel temporarily holds a list of streams
which are waiting to be sent to the session and relay daemons as a
result of the `GET_CHANNEL` command.
At this moment, the channel's reference count is one over the number of
streams as they all hold a back-reference to their parent channel and
there is a global reference held by the session daemon.
The session daemon uses the key it received to emit the `GET_CHANNEL`
command. When executing this command, the consumer daemon attempts to
send the streams to the relay daemon.
On failure to do so, the session daemon is informed of the connection
error. The consumer daemon then omits a step of the command: the streams
are never handed-off from the channel's internal list to the
consumption/monitoring thread. This hand-off is what is internally
referred-to as making the streams "globally visible".
The session daemon, upon receiving the failure error code of the
GET_CHANNEL command, tears down its internal ust_app channel structures.
As part of that process, it emits the `DESTROY_CHANNEL` command to
reclaim the channel on the consumer daemon's end. This command is
deferred to the channel poll thread as the `CHANNEL_DEL`
internal command.
As part of this internal command, the channel poll thread cleans the
channel's stream list to clean-up any streams that are not "globally
visible": all of them, in our case.
Then, the session daemon's global reference is released which should
normally result in the reclamation of the channel itself.
While reproducing the problem, we noted that channel wasn't reclaimed
and that its reference count matched the number of CPUs on the system at
the time the `CHANNEL_DEL` command completed.
This hinted at the streams holding a reference to the channel even after
the completion of the reclamation command.
Looking at clean_channel_stream_list(), which cleans up the channel's
temporary stream list, we note that the streams' monitor property is
overridden to `false` just before the call to consumer_stream_destroy().
This is strange and a comment (added as part of 212d67a2 in 2014) hints
at a locking problem that was being worked-around. In all likelihood,
this no longer applies as the locking strategies used have evolved quite
a bit since then.
Still, setting the monitor property to `false` is problematic as, in
that mode (say, channels that are used to record a snapshot), streams do
not hold a reference to their parent channel. This causes the clean-up
code to forego the clean-up of the channel, resulting in its leak.
Since the channel ultimately owns the 'stream_fds' which represent the
shared memory files, those files (and associated memory) are also leaked
(they are closed during the execution of lttng_ustconsumer_del_channel()).
Solution
========
We simply remove the stream monitor mode override to leave it in its
appropriate state. The clean-up then proceeds normally, ensuring the
tracing buffers are properly reclaimed.
Philippe Proulx [Tue, 15 Jun 2021 01:41:43 +0000 (21:41 -0400)]
docs: Partially document the liblttng-ctl C API
This patch:
1. Performs the required changes to make the build system able to build
an HTML API documentation using Doxygen.
The way it's done is a replica of what does the Babeltrace 2 project,
which you may be familiar with.
`doc/api` is for all API documentation projects while
`doc/api/liblttng-ctl` is the specific liblttng-ctl API documentation
project.
To build and view the HTML API documentation:
a) Configure the project with the `--enable-api-doc` option.
b) Build and install the project.
c) Open
`$prefix/share/doc/lttng-tools/api/liblttng-ctl/html/index.html`,
where `$prefix` is the installation prefix (for example,
`/usr/local`).
2. Fully documents some modules while not documenting others at all.
Because some liblttng-ctl headers contain functions/types which
conceptually belong to more than one module (unlike in Babeltrace 2),
I decided to put all the Doxygen group (module) definitions and any
"extra" module documentation in `dox/groups.dox`. The latter is a
huge file of which most of the contents was copied from the
LTTng-tools manual pages (and from the online LTTng Documentation)
and adapted to the C API context.
Images are direct copies from the LTTng Documentation.
The complete module tree and its state, as of this patch, is as
follows, where ✅ means it's fully documented and ❌ means it's not
documented at all:
✅ Home page
✅ General API (error codes, session daemon connection,
common definitions)
Includes parts of `lttng.h`, `lttng-error.h`, and
`constant.h`.
✅ Recording session API
Includes parts of `lttng.h`, `channel.h`, `handle.h`,
`domain.h`, and `session.h`.
✅ Recording session descriptor API
Includes all `session-descriptor.h`.
✅ Recording session destruction handle API
Includes all `destruction-handle.h`.
✅ Domain and channel API
Includes parts of `channel.h`, `domain.h`, and `event.h`.
✅ Recording event rule API
Includes parts of `event.h`.
❌ Process attribute inclusion set API
Would include parts of `tracker.h`.
✅ Recording session clearing API
Includes all `clear.h` and `clear-handle.h`.
❌ Recording session snapshot API
Would include all `snapshot.h`.
❌ Recording session rotation API
Would include all `rotation.h` and `location.h`.
❌ Recording session saving and loading API
Would include all `save.h` and `load.h`.
✅ Instrumentation point listing API
Includes parts of `event.h`.
❌ Trigger API
Would include all `trigger/trigger.h`.
❌ Trigger condition API
Would include all `condition/buffer-usage.h`,
`condition/condition.h`, `condition/evaluation.h`,
`condition/session-consumed-size.h`, and
`condition/session-rotation.h`.
❌ "Event rule matches" trigger condition API
Would include all `condition/event-rule-matches.h`.
❌ Event rule API
Would include all headers in `event-rule` as well
as all `kernel-probe.h` and `userspace-probe.h`.
❌ Log level rule API
Would include all `log-level-rule.h`.
❌ Event expression API
Would include all `event-expr.h`.
❌ Event field value API
Would include all `event-field-value.h`.
❌ Trigger action API
Would include all `action/action.h`,
`action/firing-policy.h`, `action/list.h`, `action/path.h`,
`action/rate-policy.h`, `action/rotate-session.h`,
`action/snapshot-session.h`, `action/start-session.h`, and
`action/stop-session.h`.
❌ Notify trigger action API
Would include all `action/notify.h`,
`notification/channel.h`, and
`notification/notification.h`, as well as parts of
`endpoint.h`.
❌ Error query API
Would include all `error-query.h`.
I'm voluntarily not documenting the health API (`health.h`), as I
believe it's not super important for most users. We could document it
on demand.
In `groups.dox`, the groups of the undocumented modules are already
defined, so that the complete tree above is visible in the rendered
"API reference" section. The undocumented modules simply show the
text "To be done". Because there are references to undocumented
modules in `groups.dox` and in the documented headers, this means
that the links at least resolve.
Note that there are non-comment changes in `include/lttng`: I needed
to name some anonymous, nested types so that I could reference their
members, as you can only link to the member of a named type with
Doxygen. For example, the type of the `u` union member of
`struct lttng_event_context` is now `union lttng_event_context_u`;
then you can reference its `probe` member as such:
`lttng_event_context::lttng_event_context_u::probe` (_not_
`lttng_event_context::u::probe`).
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I2783419159f4892a992fe5bc760b6e2cd6d13a60
Xiangyu Chen [Mon, 25 Mar 2024 10:20:14 +0000 (18:20 +0800)]
Fix: rotation-destroy-flush: fix session daemon abort if no kernel module present
Testing rotation-destroy-flush when no lttng kernel modules present, it
would be failed with error message:
Error: Unable to load required module lttng-ring-buffer-client-discard
not ok 1 - Start session daemon
Failed test 'Start session daemon'
not ok 2 - Create session rotation_destroy_flush in -o /tmp/tmp.test_rot ...
...
This because test script that sets the LTTNG_ABORT_ON_ERROR environment
variable. It's this environment variable that causes the sessiond to
handle the kernel module loading failure as an abort rather than a
warning.
Using "check_skip_kernel_test" to detect whether the kernel module fails
to load is expected or not. If the failure is expected, the script won't
set that environment variable any more.
Fixes: 3a174400
("tests:add check_skip_kernel_test to check root user and lttng kernel modules")
Fix: consumerd: type confusion in lttng_consumer_send_error
lttng_consumer_send_error sends an lttcomm_return_code to the session
daemon. However, the size of lttcomm_sessiond_command was used.
This was probably missed since the function accepts an integer instead
of a proper enum type.
The size accepted by the function is changed to use lttcomm_return_code
and the size of a fixed-size type is used to send the error code to the
session daemon.
Clean-up: lttng: utils: missing special member functions
clang-tidy warns:
warning: class 'session_list' defines a move constructor but does not define a destructor, a copy constructor, a copy assignment operator or a move assignment operator [cppcoreguidelines-special-member-functions]
This warning related to the "Rule of Five":
If a class requires a custom destructor, copy constructor, or copy
assignment operator due to manual resource management, it likely needs
to explicitly define all five (including move constructor and move
assignment operator) to correctly manage the resources across all types
of object copying and moving scenarios. This rule helps prevent resource
leaks, double frees, and other common issues related to resource
management.
The coding style mentions that override and virtual should not be used
together (as virtual would be redundant). However, one of the examples
commits that cardinal sin.
Fix: waiter: use std::reference_wrapper instead of a raw reference
clang-tidy warns that
/home/jgalar/EfficiOS/src/lttng-tools/src/common/waiter.hpp:32:9: warning: use '= default' to define a trivial copy-assignment operator [modernize-use-equals-default]
The warning itself is bogus since it is not possible to use the default
copy-assignment operator in this case. Indeed, _state being a reference,
it cannot be rebound after its initialization. The compiler will refuse
to generate a default implementation as it would be ill-formed.
However, this does highlight a problem in the implementation, namely
that the explicit assignment operator does attempt to re-assign _state.
Switching the _state reference to use std::reference_wrapper does make
it legal to re-bind the reference to a different object.
Having the ability to assign a waker is conceptually a bit strange (vs
obtaining another instance from the waiter) and should probably be
disabled, but it is required for the moment as one the of use in a huge
C structure that gets copied using by assignment.
Fix: lttng-destroy: string formating error when default session is unset
Using `lttng destroy` when no default session is set in .lttngrc results
in the following print-out:
Error: Can't find valid lttng config /root/lttng-build/home/.lttngrc
Did you create a session? (lttng create <my_session>)
Error: Failed to format string: string pointer is null
This is because the client attempts to format the following message:
ERR_FMT("Session `{}` not found", spec.value);
When no default session could be found in .lttngrc, spec.value is left
at nullptr and it is assumed that the listing succeeded.
A new CLI-specific exception, no_default_session_error, is added to the
project and thrown when the session listing fails. This allows the
calling code to mark the listing as having failed.
Fix: lttng-destroy: string formating error when default session is unset
Using `lttng destroy` when no default session is set in .lttngrc results
in the following print-out:
Error: Can't find valid lttng config /root/lttng-build/home/.lttngrc
Did you create a session? (lttng create <my_session>)
Error: Failed to format string: string pointer is null
This is because the client attempts to format the following message:
ERR_FMT("Session `{}` not found", spec.value);
When no default session could be found in .lttngrc, spec.value is left
at nullptr and it is assumed that the listing succeeded.
A new CLI-specific exception, no_default_session_error, is added to the
project and thrown when the session listing fails. This allows the
calling code to mark the listing as having failed.
Fix: lttng-destroy: string formating error when default session is unset
Using `lttng destroy` when no default session is set in .lttngrc results
in the following print-out:
Error: Can't find valid lttng config /root/lttng-build/home/.lttngrc
Did you create a session? (lttng create <my_session>)
Error: Failed to format string: string pointer is null
This is because the client attempts to format the following message:
ERR_FMT("Session `{}` not found", spec.value);
When no default session could be found in .lttngrc, spec.value is left
at nullptr and it is assumed that the listing succeeded.
A new CLI-specific exception, no_default_session_error, is added to the
project and thrown when the session listing fails. This allows the
calling code to mark the listing as having failed.
orbea [Fri, 8 Mar 2024 16:17:37 +0000 (08:17 -0800)]
Fix: baddr-statedump: use $(LIBTOOL) --mode=execute
GNU libtool inconsistently places the compiled executable in the source
directory or in the .libs directory where a libtool wrapper script is
placed in the source directory.
While slibtool will always place the compiled executable in the .libs
directory and a wrapper script in the source directory.
This will result with a build error when using slibtool since objcopy
needs the executable and not the shell wrapper script, but this can be
solved for both implementations by using $(LIBTOOL) --mode=execute on all
commands that operate on the libtool compiled executables.
Gentoo issue: https://bugs.gentoo.org/858095
The GNU libtool --mode=excute is documented upstream.
And the GNU libtool behavior of when to create a wrapper script is
documented in the 'Linking Executables' section.
"Notice that the executable, hell, was actually created in the .libs
subdirectory. Then, a wrapper script (or, on certain platforms, a
wrapper executable see Wrapper executables) was created in the current
directory.
Since libtool created a wrapper script, you should use libtool to
install it and debug it too. However, since the program does not depend
on any uninstalled libtool library, it is probably usable even without
the wrapper script."
And the inconsistency between GNU libtool and slibtool is documented at
the Gentoo wiki.
"One difference between GNU libtool and slibtool is that the former will
conditionally place the compiled executable or a shell wrapper script in
the build directory, depending on whether or not the executable depends
on a build-local libtool library (e.g. libfoo.la). Where slibtool will
always place a compatible wrapper script in the build directory where
GNU libtool would have conditionally placed the executable. When the
wrapper script is created both GNU libtool and slibtool will place the
executable in the .libs directory within the build directory.
Consequently build systems, ebuilds, and other users should take care to
avoid scenarios like installing the wrapper script to the system instead
of the executable. In these cases ideally the executable would be
installed by the same libtool implementation that compiled it."
Fix: relayd: live client not notified of inactive streams
Observed issue
--------------
Some LTTng-tools live tests failures appear to show babeltrace2
hanging (failing to print expected events). The problem is intermittent,
but Kienan was able to develop a test case that's reproducible for him.
The test case performs the following steps:
- Start a ust application and leave it running
- Configure and then start an lttng live session
- Connect a live viewer (babeltrace)
- Run a second ust application
- Wait for the expected number of events
- In the failing case, no events are seen by babeltrace
Using per-uid buffers, the test typically completes normally. With
per-pid buffers the test fails, hanging indefinitely if waiting for the
specified number of events. While "hanging", babeltrace2 is polling the
relayd.
This affects for babeltrace2 stable-2.0 and master while using
lttng-tools master.
For more information, see the description of bug #1406[1]
Cause
-----
When consuming a live trace captured in per-PID mode, Babeltrace
periodically requests the index of the next packet it should consume.
As part of the reply, it gets a 'flags' field which is used to announce
that new streams, or new metadata, are available to the viewer.
Unfortunately, these 'flags' are only set when the relay daemon has new
tracing data to deliver. It is not set when the relay daemon indicates
that the stream is inactive (see LTTNG_VIEWER_INDEX_INACTIVE).
In the average case where an application is spawned while others are
actively emiting events, a request for new data will result in a reply
that returns an index entry (code LTTNG_VIEWER_INDEX_OK) for an
available packet accompanied by the LTTNG_VIEWER_FLAG_NEW_STREAM flag.
This flag indicates to the viewer that it should request new
streams (using the LTTNG_VIEWER_GET_NEW_STREAMS live protocol command)
before consuming the new data.
In the cases where we observe a hang, an application is running but not
emiting new events. As such, the relay daemon periodically emits "live
beacons" to indicate that the session's streams are inactive up to a
given time 'T'.
Since the existing application remains inactive and the viewer is never
notified that new streams are available, the viewer effectively remains
"stuck" and never notices the new application being traced.
The LTTNG_VIEWER_FLAG_NEW_METADATA communicates a similar semantic with
regards to the metadata. However, ignoring it for inactive streams isn't
as deleterious: the same information is made available to the viewer the
next time it will successfully request a new index to the relay daemon.
This would only become a problem if the tracers start to express
non-layout data (like supplemental environment information, but I don't
see a real use-case) as part of the metadata stream that should be made
available downstream even during periods of inactivity.
Note that the same problem most likely affects the per-UID buffer
allocation mode when multiple users are being traced.
Solution
--------
On the producer end, LTTNG_VIEWER_FLAG_NEW_STREAM is set even when
returning an inactivity index.
Note that to preserve compatibility with older live consumers that don't
expect this flag in non-OK response, the LTTNG_VIEWER_FLAG_NEW_STREAM
notification is repeated until the next LTTNG_VIEWER_GET_NEW_STREAMS
command that returns LTTNG_VIEWER_INDEX_OK.
The 'new_streams' state is no longer cleared from relay sessions during
the processing of the LTTNG_VIEWER_GET_NEXT_INDEX commands. Instead, it
is cleared when the viewer requests new streams.
On Babeltrace's end, the handler of the LTTNG_VIEWER_GET_NEXT_INDEX
command (lttng_live_get_next_index) is modified to expect
LTTNG_VIEWER_FLAG_NEW_STREAM in the cases where the command returns:
- LTTNG_VIEWER_INDEX_OK (as done previously),
- LTTNG_VIEWER_INDEX_HUP (new),
- LTTNG_VIEWER_INDEX_INACTIVE (new).
Drawbacks
---------
This is arguably a protocol change as none of the producers ever set the
NEW_METADATA/NEW_STREAM flags when indicating an inactive stream.
Clean-up: tests: bt2 plug-ins: modernize the plug-ins
By virtue of their use of the C Babeltrace 2 APIs, the test plug-ins
perform a fair amount of manual resource management.
To make it possible to adopt a more modern C++ style in those plug-ins,
a number of helpers are introduced.
Introduce reference wrappers for the Babeltrace 2 interface:
- value_ref: wraps a bt_value reference using std::unique_ptr
- message_const_ref: wraps a constant message reference using a
unique_ptr
- message_iterator_ref: wraps a message iterator reference using a
unique_ptr
- event_class_const_ref: wraps a constant event class reference using
a unique_ptr
A specialized random_access_container_wrapper is specialized to wrap
bt_value arrays of strings.
In doing so, it is possible to eliminate the use of gotos and manual
reference management on error paths. Some struct/classes are renamed to
eliminate ambiguities that arose over the refactoring.
The changes allow some simplifications of the code flow in places which
are applied directly.
Kienan Stewart [Thu, 28 Sep 2023 20:54:42 +0000 (16:54 -0400)]
tests: Replace babelstats.pl with bt2 plugins
Observed Issue
==============
`tests/regression/tools/filtering/test_valid_filters` is a long running
test, especially when running as root and exercising the tests across
the kernel domain.
I observed that a sizable amount of time was being spent in the analysis
of the results using `babelstats.pl`.
Solution
========
Instead of using a script to parse the pretty output of babeltrace2, I
decided to write two C++ plugins to replicate the behaviour of the
`babelstats.pl` script.
I measured the time using `sudo -E time ./path/to/test`
| Test | Time with `babelstats.pl` | Time with bt2 plugins |
| test_tracefile_count | 13.04s | 11.73s |
| test_exclusion | 22.75s | 22.07s |
| test_valid_filter | 301.04s | 144.41s |
The switch to using babeltrace2 plugins reduces the runtime of the
`test_valid_filter` test (when running with kernel tests) by half. The
runtime changes to the other tests that were modified are not
significant.
Known drawbacks
===============
The field_stats plugin behaviour differs from `babelstats.pl` with
regards to enumeration fields ("container" in `babelstats.pl`). However,
no tests depend on that behaviour to pass.
The field_stats sink plugin doesn't perform a lot of run-time
error-checking of functions it invokes, and doesn't fully clean up all
the references it allocates though the babeltrace2 API. As the intended
usage is for short lived invocations with relatively small traces, the
principal drawback of this approach is that errors in the plugin may be
harder to debug.
Building tests of lttng-tools will now depend on having the babeltrace2
development headers and libraries available.
Change-Id: Ie8ebdd255b6901a7d0d7c4cd584a02096cccd4fb Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Tue, 26 Sep 2023 13:39:41 +0000 (09:39 -0400)]
tests: Run relayd-grouping tests by grouping type
Observed issue
==============
The `relayd-grouping/test_ust` test takes ~2 minutes to run. A
significant amount of that time is statring and stopping the relay and
sesion daemons.
Solution
========
Each test function is run with a different grouping setup for the
relayd. Rather than iterating over each test and then grouping
variations, the iteration can be changed to organize the tests run by
grouping setup. This allows us to start th relay and session daemons
once per grouping setup, rather than twice for each test function.
Further more, each test function is run twice: once with auto-generated
session names, once with user-defined session names. This behaviour can
be cut out to reduce the runtime of the test further.
On my development machine, the test went from running in 113s to 18s.
Known drawbacks
===============
This no longer exercises the automatic session naming. I don't think
that the automatic session naming paths are pertinent with regards to
the grouping settings; however it appears it can impact output
directories (eg. in
`test_ust_uid_streaming_snapshot_add_output_custom_name`).
Change-Id: I89d8cb224e594dd68b7e8f3367d1907ecfa2bf13 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Thu, 7 Mar 2024 20:20:17 +0000 (15:20 -0500)]
tests: Split test_ust_constructor into several tests
Observed issue
==============
TAP parsers fail when parsing a single executable that contains
several plans. Eg.,
```
ok 44 - Found no unexpected events
PASS: ust/ust-constructor/test_ust_constructor.py 44 - Found no unexpected events
1..44
ERROR: ust/ust-constructor/test_ust_constructor.py - multiple test plans
ok 1 - Create a session
ERROR: ust/ust-constructor/test_ust_constructor.py 1 - Create a session # UNPLANNED
```
and
```
14:03:23 org.tap4j.parser.ParserException: Error parsing TAP Stream: Duplicated TAP Plan found.
14:03:23 at org.tap4j.parser.Tap13Parser.parseTapStream(Tap13Parser.java:257)
14:03:23 at org.tap4j.parser.Tap13Parser.parseFile(Tap13Parser.java:231)
14:03:23 at org.tap4j.plugin.TapParser.parse(TapParser.java:172)
14:03:23 at org.tap4j.plugin.TapPublisher.loadResults(TapPublisher.java:475)
14:03:23 at org.tap4j.plugin.TapPublisher.performImpl(TapPublisher.java:352)
14:03:23 at org.tap4j.plugin.TapPublisher.perform(TapPublisher.java:312)
14:03:23 at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
14:03:23 at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
14:03:23 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
14:03:23 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
14:03:23 at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:767)
14:03:23 at hudson.model.Build$BuildExecution.post2(Build.java:179)
14:03:23 at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:711)
14:03:23 at hudson.model.Run.execute(Run.java:1918)
14:03:23 at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
14:03:23 at hudson.model.ResourceController.execute(ResourceController.java:101)
14:03:23 at hudson.model.Executor.run(Executor.java:442)
14:03:23 Caused by: org.tap4j.parser.ParserException: Duplicated TAP Plan found.
14:03:23 at org.tap4j.parser.Tap13Parser.parseLine(Tap13Parser.java:354)
14:03:23 at org.tap4j.parser.Tap13Parser.parseTapStream(Tap13Parser.java:252)
14:03:23 ... 16 more
```
Kienan Stewart [Tue, 23 Jan 2024 15:08:34 +0000 (10:08 -0500)]
tests: Add diagnostic info for kernel bug, warning, and oops
When test_select_poll_epoll fails with an error due to hitting one a new
WARNING, OOPS, or BUG statements in dmesg, the user must go and read the
the logs themselves to try and find the matching statements.
Providing the previous and new messages in diagnostic output will allow
a person reading the test results to more quickly ascertain if the
messages are pertinent to lttng-modules or not. That being said, there
is no guarantee that there are not other WARNINGs, OOPs, or BUGs in
dmesg between before and after that are pertinent.
Change-Id: Ida026dfe852cafdcc55979089c92995949e2ef0d Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
The single test executable gen-ust-events-constructor covers a lot of
different cases in a single executable. This decreases the legibility of
the test results and debuggability of the test application as many
different pieces are in play.
Solution
========
The test functionality covered by the executable is split into two main
parts: one using a dynamically loaded shared object, and the second
using a statically linked archive.
Known drawbacks
===============
Rather than creating a second test script, the same script is re-used to
run multiple TapGenerator sequentially. This could hamper future efforts
to parallelize python-based tests.
Change-Id: I86d247780ce5412570eada6ebadb83a01547f2b0 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Fri, 9 Feb 2024 14:16:26 +0000 (09:16 -0500)]
tests: Ensure `_process` is set in _TraceTestApplications
Observed issue
==============
An exception is thrown when deleting a _TraceTestApplication object that
has thrown an exception during it's `__init__` method. Eg.
```
Exception ignored in: <function _TraceTestApplication.__del__ at 0x7fcbc9a21620>
Traceback (most recent call last):
File "/home/kstewart/src/efficios/lttng/master/src/lttng-tools/tests/utils/lttngtest/environment.py", line 348, in __del__
self._process.kill()
^^^^^^^^^^^^^
AttributeError: '_TraceTestApplication' object has no attribute '_process'
```
Similarly, this can happen to _WaitTraceTestApplication objects.
Cause
=====
The object's `_process` attribute is set during `__init__`; however,
if an exception is thrown during `subprocess.Popen` a value is never
assigned to the attribute.
Solution
========
A default value for the `_process` attribute is set and checked as
part of the condition when executing the `__del__` method.
Known drawbacks
===============
None.
Change-Id: I2220ae764be49fafb3b977a5e723931421485d63 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Fri, 9 Feb 2024 14:08:07 +0000 (09:08 -0500)]
tests: Correct tap_generator skip() when count is greater than 1
Issue observed
==============
Output when skipping multiple was incorrectly printing the test case number,
eg.
```
ok 3 - Start session `session_ldr8cxix`
41
ok 4 # Skip: Test application 'gen-ust-events-constructor/gen-ust-events-constructor-so' not found
ok 6 # Skip: Test application 'gen-ust-events-constructor/gen-ust-events-constructor-so' not found
ok 8 # Skip: Test application
'gen-ust-events-constructor/gen-ust-events-constructor-so' not found
```
Cause
=====
The `test_number` was adding the current index to the already modified
`self._last_test_case_id`.
Solution
========
Use `self._last_test_case_id` with no changes.
Known drawbacks
===============
None.
Change-Id: I8ff16b83619cf6e6db2636eeccd58725cc03d0f8 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Thu, 8 Feb 2024 14:02:48 +0000 (09:02 -0500)]
tests: test_ust_constructor: Use a C-compiled shared object
Similar to the previous change, this change splits the c-style
constructors for the shared object into a separate object which can be
compiled with gcc instead of g++.
This makes it possible to test the constructors are traced even if
LTTng-UST uses the LTTNG_UST_ALLOCATE_COMPOUND_LITERAL_ON_HEAP build
configuration.
Change-Id: Icd96cb30cedc1615951a6fec3c72731776f95d81 Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Kienan Stewart [Thu, 8 Feb 2024 13:46:46 +0000 (08:46 -0500)]
tests: test_ust_constructor: Use a C-compiled static archive
Observed issue
==============
The test output describes the tracepoint as
`constructor_c_provider_static_archive`, which can be a bit misleading.
The tracepoints are indeed emitted inside C-style constructors. However,
as the tracepoints are being compiled inside a C++ translation unit,
they were never traceable when using a heap allocated implementation. If
the static archive is compiled as C and linked against the C++
application, the tracepoints are expected to always be visible.
Solution
========
In splitting the c-style constructors for the static archive into a
separate object the compilation can be made to use gcc instead of g++.
Drawback
========
This change doesn't keep a C-style constructor inside a C++ application
and asserts that it is indeed not traced when compiled using a heap
allocated implementation.
Signed-off-by: Kienan Stewart <kstewart@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I3837fe318b2f8e1d9572ee0bfb6f6bbbd047c5f5
Kienan Stewart [Wed, 7 Feb 2024 20:49:26 +0000 (15:49 -0500)]
tests: Handle test failures for ust-constructors with heap allocation
Observed issue
==============
A number of tests from `ust/ust-constructor/test_ust_constructor.py`
fail when compiled with gcc-4.8 (observed on SLES12SP5). Eg.
```
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 8 - Found
expected event name="tp_a:constructor_c_provider_static_archive"
msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 10 - Found expected event name="tp_a:constructor_cplusplus_provider_static_archive" msg="global - static archive define and provider"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 11 - Found expected event name="tp:constructor_c_across_units_before_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 12 - Found expected event name="tp:constructor_cplusplus" msg="global - across units before define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 13 - Found expected event name="tp:constructor_c_same_unit_before_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 14 - Found expected event name="tp:constructor_c_same_unit_after_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 15 - Found expected event name="tp:constructor_cplusplus" msg="global - same unit before define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 16 - Found expected event name="tp:constructor_cplusplus" msg="global - same unit after define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 17 - Found expected event name="tp:constructor_c_across_units_after_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 18 - Found expected event name="tp:constructor_cplusplus" msg="global - across units after define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 19 - Found expected event name="tp:constructor_c_same_unit_before_provider" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 20 - Found expected event name="tp:constructor_c_same_unit_after_provider" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 21 - Found
expected event name="tp:constructor_cplusplus" msg="global - same unit
before provider"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 34 - Found expected event name="tp:destructor_cplusplus" msg="global - same unit before provider"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 35 - Found expected event name="tp:destructor_cplusplus" msg="global - across units after define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 36 - Found expected event name="tp:destructor_cplusplus" msg="global - same unit after define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 37 - Found expected event name="tp:destructor_cplusplus" msg="global - same unit before define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 38 - Found expected event name="tp:destructor_cplusplus" msg="global - across units before define"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 39 - Found
expected event
name="tp_a:destructor_cplusplus_provider_static_archive" msg="global -
static archive define and provider"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 41 - Found expected event name="tp:destructor_c_across_units_after_provider" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 42 - Found expected event name="tp:destructor_c_same_unit_after_provider" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 43 - Found expected event name="tp:destructor_c_same_unit_before_provider" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 44 - Found expected event name="tp:destructor_c_across_units_after_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 45 - Found expected event name="tp:destructor_c_same_unit_after_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 46 - Found expected event name="tp:destructor_c_same_unit_before_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 47 - Found expected event name="tp:destructor_c_across_units_before_define" msg="None"
12:22:17 FAIL: ust/ust-constructor/test_ust_constructor.py 48 - Found expected event name="tp_a:destructor_c_provider_static_archive" msg="None"
```
Cause
=====
As gcc-4.8 and earlier don't support C99 compound literals, the
lttngust `ust-compiler.h` falls back to using heap allocated
compound literals[1][2].
The probe registration in these cases is done via a C++ object[3].
As C-style constructors are executed before the C++ runtime is
processed, the probe is not yet registered[4].
In a case where g++ <= 4.8 is being used or
`-DLTTNG_UST_ALLOCATE_COMPOUND_LITERAL_ON_HEAP` is defined, the
following tracepoints will not be recorded:
* C-style constructors and destructors in statically linked archives
* C-style constructors and destructors in the application itself
* Some C++ constructors and destructors invoked during the
initialization of the static global variables
* Note: this depends on the initialization order both between translation
units, which is not specified, and the initialization order (usually
lexicographical) within a given translation unit.
This is a known limitation; however, the test does not support
verifying that it's being run in a such a situation.
Solution
========
A small program has been added which returns a different status code
depending on whether `LTTNG_UST_ALLOCATE_COMPOUND_LITERAL_ON_HEAP` is
defined or not.
The test script uses this application to signal that certain events
may fail (in that they may be present, or they may be absent).
Xiangyu Chen [Mon, 12 Feb 2024 14:23:54 +0000 (09:23 -0500)]
tests: add check_skip_kernel_test to check root user and lttng kernel modules
The current tests will run both userspace and kernel testing. Some of
use cases only use lttng for one kind of tracing on an embedded
device (e.g. userspace), so in this scenario, the kernel modules might
not install to target rootfs, the test cases would be fail and exit.
Add LTTNG_TOOLS_DISABLE_KERNEL_TESTS to skip the lttng kernel features
test, this flag can be set via "make":
make check LTTNG_TOOLS_DISABLE_KERNEL_TESTS=1
When this flag was set, all kernel related testcases would be marked as
SKIP in result.
Since the the LTTNG_TOOLS_DISABLE_KERNEL_TESTS was checked in function
check_skip_kernel_test, lots of testcases also need to check root
permission, so merging the root permission checking into
check_skip_kernel_test.
Warning: Failed to produce a random seed using getrandom(), falling back to pseudo-random device seed generation which will block until its pool is initialized: getrandom() is not supported by this platform [getrandom_nonblock() random.cpp:90]
Traceback (most recent call last):
File "/home/jenkins/workspace/lttng-tools_master_elbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/master/platform/el7-amd64/src/lttng-tools/extras/bindings/swig/python/lttng.py", line 24, in swig_import_helper
fp, pathname, description = imp.find_module('_lttng', [dirname(__file__)])
File "/usr/lib64/python3.6/imp.py", line 297, in find_module
raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named '_lttng'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./ust/exit-fast/test_exit-fast.py", line 19, in <module>
from test_utils import *
File "/home/jenkins/workspace/lttng-tools_master_elbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/master/platform/el7-amd64/src/lttng-tools/tests/utils/test_utils.py", line 24, in <module>
from lttng import *
File "/home/jenkins/workspace/lttng-tools_master_elbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/master/platform/el7-amd64/src/lttng-tools/extras/bindings/swig/python/lttng.py", line 34, in <module>
_lttng = swig_import_helper()
File "/home/jenkins/workspace/lttng-tools_master_elbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/master/platform/el7-amd64/src/lttng-tools/extras/bindings/swig/python/lttng.py", line 26, in swig_import_helper
import _lttng
ImportError: /home/jenkins/workspace/lttng-tools_master_elbuild/babeltrace_version/stable-2.0/build/std/conf/std/liburcu_version/master/platform/el7-amd64/src/lttng-tools/extras/bindings/swig/python/.libs/_lttng.so: undefined symbol: _ZNSt13runtime_errorC2EPKc
ERROR: ust/exit-fast/test_exit-fast - missing test plan
```
The link mode can be seen the commands use to do the linking in the CI
node build logs. For example,
Automake chooses the link mode based on the types of files in the
library or executable. Given that the generated bindings are only C
code, automake uses the gcc link mode.
Solution
========
By adding a dummy (non existant) C++ source file to the library,
automake can be 'forced' to the switch the link mode to `g++`.
Fix: relayd: live: dispose of zombie viewer metadata stream
Issue observed
==============
In the CI, builds on SLES15SP5 frequently experience timeouts. From
prior inspections, there are hangs during
tests/regression/tools/clear/test_ust while waiting for babeltrace to
exit.
It is possible to reproduce the problem fairly easily:
# Launch an application that emits a couple of events
$ ./my_app
$ lttng stop
# Clear the data, this eventually results in the deletion of all
# trace files on the relay daemon's end.
$ lttng clear
# Attach to the live session from another terminal
$ babeltrace -i lttng-live net://...
# The 'destroy' command completes, but the viewer never exits.
$ lttng destroy
Cause
=====
After the clear command completes, the relay daemon no longer has any
data to serve. We notice that the live client loops endlessly repeatably
sending GET_METADATA requests. In response, the relay daemon replies
with the NO_NEW_METADATA status.
In concrete terms, the viewer_get_metadata() function short-circuits to
send that reply when it sees that the metadata stream has no active
trace chunk (i.e., there are no backing files from which to read the
data at the moment).
This situation is not abnormal in itself: it is legitimate for a client
to wait for the metadata to become available again. For example, in the
reproducer above, it would be possible for the user to restart the
tracing (lttng start), which would create a new trace chunk and make the
metadata stream available. New events could also be emitted following
this restart.
However, when a session's connection is closed, there is no hope that
the metadata stream will ever transition back to an active trace chunk.
Solution
========
When the metadata stream has no active chunk and the corresponding
consumerd-side connection has been closed, there is no way the relay
daemon will be able to serve the metadata contents to the client.
As such, the viewer stream can be disposed-of since it will no longer be
of any use to the client. Since some client implementations expect at
least one GET_METADATA command to result in NO_NEW_METADATA, that status
code is initially returned.
Later, when the client emits a follow-up GET_METADATA request for that
same stream, it will receive an "error" status indicating that the
stream no longer exists. This situation is not treated as an error by
the clients. For instance, babeltrace2 will simply close the
corresponding trace and indicate it ended.
The 'no_new_metadata_notified' flag doesn't appear to be necessary to
implement the behaviour expected by the clients (seeing at least one
NO_NEW_METADATA status reply for every metadata stream). The
viewer_get_metadata() function is refactored a bit to drop the global
reference to the viewer metadata stream as it exits, while still
returning the NO_NEW_METADATA status code.
Known drawbacks
===============
None.
Note
====
The commit message of e8b269fa provides more details behind the
intention of the 'no_new_metadata_notified' flag.
Docs: relayd: received metadata position is reset on clear
Correct a comment in the relayd documentation that incorrectly mentioned
the 'sent' position being reset by the 'clear' command.
The correct behavior resets the metadata stream's 'received' position to
'0', not the 'sent' position. The relay daemon expects to re-receive the
metadata contents that matches the previous contents up to the previous
'received' position.
The client, however, does not expect to receive the original contents of
the metadata stream a second time.
Note that from the relay daemon's perspective, a "clear" command does
not exist per se. It is implemented as a stream rotation that moves the
streams from a trace chunk that has an associated 'DELETE' close command
to a new one (which may also be a 'nil' chunk).
Philippe Proulx [Wed, 30 Aug 2023 16:33:13 +0000 (12:33 -0400)]
Add Zsh completion files for public LTTng CLI commands
Zsh is an extended Bourne shell with many improvements, including some
features of Bash, ksh, and tcsh. Zsh features a powerful completion
system which makes it possible to improve the interactive user
experience greatly when using an LTTng command.
Those four new files are Very Sophisticated Zsh completion files,
especially `extras/zsh-completion/_lttng`.
Notable features for all commands:
* Support of LTTng 2.5 through LTTng 2.14, with version-specific
completion.
Set `LTTNG_ZSH_COMP_IGNORE_VERSION_LIMIT=1` to disable the upper limit
of the version check. This should be safe most of the time, but if
there's a breaking change in option/argument interaction, the
completions might be wrong.
* Exclusion of options and arguments depending on the current options
and arguments, according to the manual pages.
For example, for `lttng enable-channel`, you cannot specify
`--buffers-uid` if you already specified `--kernel` (and vice versa).
Notable features for the `lttng` command:
* Full support, except for the condition and action specifiers of the
`add-trigger` subcommand: although I may now add "skillful in Zsh
completion" to my resume, the positional design of `--condition` and
`--action` needs event more spicy Zsh wizardry which I didn't explore
yet.
* Custom tags and support for the `verbose` style to customize the
completion behaviour and look with `zstyle`.
* For any dynamic completion (relying on some output of the `lttng`
command), connect to the right session daemon depending on the
selected tracing group (`g`/`--group`).
* User/group ID completion with displayed corresponding Unix user/group
names.
* Dynamic recording session name completion with a summary of properties
(activity and mode).
Only the relevant ones are added to the completion set. For example,
names of active sessions are not part of the completion set for
`lttng start`.
* Current recording session taken into account for subcommands needing
one when you don't specify the dedicated recording session
option/argument.
* Dynamic channel name completion depending on the selected recording
session and tracing domain, with a summary of properties (status,
tracing domain, event record loss mode).
Only the relevant ones are added to the completion set. For example,
names of enabled channels are not part of the completion set for
`lttng enable-channel`.
* Dynamic recording event rule name condition completion for
`lttng disable-event`.
* Dynamic instrumentation point name completion depending on the
selected tracing domain
* Dynamic context field type completion depending on the selected
tracing domain.
* Log level name completion depending on the selected tracing domain.
* Dynamic trigger name completion depending on the selected owner
user ID.
Notable features for the `lttng-sessiond` command:
* LTTng kernel probe module name completion (checks within the
`/usr/lib/modules` directory).
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: If8c2c58a50664f41ecc41ab1df72879127d1cd02
The goal is to be able to only build liblttng-ctl, for example without
needing to build bin/lttng.
Since liblttng-ctl is required when building some of the binaries,
./configure will fail if --disabled (explicitly) unless those binaries
are --disabled too.
Previously, the following would result in liblttng-ctl not getting
built, but it now gets built by default:
Kienan Stewart [Thu, 29 Feb 2024 17:06:49 +0000 (12:06 -0500)]
Misc: add pyproject.toml
This file provides the metadata of what versions of python are required
to run `make check` and tests[1]. The required versions of python will
also inform `black` for choosing which syntaxes and linting formats to
use[2].
An additional section for the `black` linter[3] is provided. While
empty, it allows `blacken-mode`[4] for emacs to use the
`blacken-only-if-project-is-blackened` setting.
Michael Jeanson [Thu, 29 Feb 2024 19:42:41 +0000 (14:42 -0500)]
Tests: standardize TAP_AUTOTIME parsing in python
Disable TAP autotime only when 'TAP_AUTOTIME == 0'. Also remove the
version check as we don't support Python <= 3.3 and there is already an
assertion in the code.
Change-Id: Idf8badb5a27b1a01cbe7c230495eec342b9c3878 Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
projects / lttng-tools.git / log
