Robert Daniels wrote:
> > I'm attempting to use lttng userspace tracing with a C++ application
> > on an ARM platform. I'm using GCC 4.8.4 on Linux 3.14 with the 2.6
> > release of lttng. I've compiled lttng-modules, lttng-ust, and
> > lttng-tools and have been able to get a simple test working with C
> > code. When I attempt to run the hello.cxx test on my target it will
> > segfault.
>
>
> I spent a little time digging into this issue and finally discovered the
> cause of my segfault with ARM C++ tracepoints.
>
> There is a struct called 'lttng_event' in ust-events.h which contains an
> empty union 'u'. This was the cause of my issue. Under C, this empty union
> compiles to a zero byte member while under C++ it compiles to a one byte
> member, and in my case was four-byte aligned which caused my C++ code to
> have the 'cds_list_head node' offset incorrectly by four bytes. This lead
> to an incorrect linked list structure which caused my issue.
>
> Since this union is empty, I simply removed it from the struct and everything
> worked correctly.
>
> I don't know the history or purpose behind this empty union so I'd like to
> know if this is a safe fix. If it is I can submit a patch with the union
> removed.
That's a very nice catch!
We do not support building tracepoint probe provider with
g++ yet, as stated in lttng-ust(3):
"- Note for C++ support: although an application instrumented with
tracepoints can be compiled with g++, tracepoint probes should be
compiled with gcc (only tested with gcc so far)."
However, if it works fine with this fix, then I'm tempted to take it,
especially because removing the empty union does not appear to affect
the layout of struct lttng_event as seen from liblttng-ust, which must
be compiled with a C compiler, and from probe providers compiled with
a C compiler. So all we are changing is the layout of a probe provider
compiled with a C++ compiler, which is anyway buggy at the moment,
because it is not compatible with the layout expected by liblttng-ust
compiled with a C compiler.
Reported-by: Robert Daniels <robert.daniels@vantagecontrols.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Fix: use lttng_secure_getenv to handle env. vars. involving paths
This is a security fix for applications linked against liblttng-ust
which are exposed as setuid binaries.
A malicious user which can run those applications could target those
environment variable paths to locations that would allow it to create
files in various areas of the filesystem.
When performing a library base address statedump, the UST threads can
grab the RCU read-side lock, which touch a TLS. We should "pre-fault"
this TLS in the constructor to eliminate a deadlock involving libc
dynamic loader lock.
When an application traced with per-UID buffers vanishes between its
reserve and commit ring buffer operations (e.g. segfault), it may cause
the consumer daemon to enter an endless loop, awaiting for the
application to complete its commit.
Fix this situation by bounding the number of retry using an adaptative
approach: try actively for a few times, and then try a few more times
with a 10ms delay between attempts.
This issue affects only architectures without efficient unaligned
accesses, only when a context field with larger alignment follows a
context field with smaller alignment. It generates unreadable traces
when such context fields are enabled in this configuration.
std=c99 without #define __USE_BSD does not work for tracepoint header,
due to missing endianness macros. Force their definition in the
ust-endian.h wrapper if they are missing.
In order to correctly handle the use-case where events are enabled
_after_ trace is started, and _after_ applications are already being
traced, the event should be created in a "disabled" state, so that it
does not trace events until its filter is attached.
This fix needs to be done both in lttng-tools and lttng-ust. In order to
keep ABI compatibility between tools and ust within a stable release
cycle, we introduce a new "disabled" within struct lttng_ust_event
padding (previously zeroed). Newer LTTng-UST checks this flag, and
fallback on the old racy behavior (enabling the event on creation) if it
is unset.
Therefore, old session daemon works with newer lttng-ust of the same
stable release, and vice-versa. However, building lttng-tools requires
an upgraded lttng-ust, which contains the communication protocol with
the new "disabled" field.
Fix: preserve example files' timestamps when copying
This fixes an issue where examples were always being rebuilt
when performing an out of tree build since the examples were
being copied to the build directory with a timestamp more
recent than the already-built example objects.
Mutex while updating statedump_pending is not needed
Take care of coverity warning ** CID 1226392: Data race condition (MISSING_LOCK)
/liblttng-ust/lttng-ust-comm.c: 1704 in lttng_ust_sockinfo_session_enabled()
It's not that the lock is missing, it's that we don't need to hold the
lock when doing the other modification.
Paul Woegerer [Mon, 7 Jul 2014 14:04:21 +0000 (16:04 +0200)]
Add lttng_ust_notrace to static inline functions
The static inline functions
__tracepoint_provider_mismatch_<TRACEPOINT_PROVIDER> and
__tracepoint_provider_check_<TRACEPOINT_PROVIDER> are missing the
lttng_ust_notrace attribute to prevent them from getting instrumented
with -finstrument-functions (if e.g. defined in CFLAGS). This patch is
adding the attribute to those functions.
Signed-off-by: Paul Woegerer <paul_woegerer@mentor.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
David Goulet [Wed, 18 Jun 2014 18:08:24 +0000 (14:08 -0400)]
JUL: use root logger to capture events
The JUL agent now uses the root logger ("") to capture all events. This
allows us to remove the Timer thread and cleanup a huge portion of the
code base. It simplifies a great deal the internal structure of the
agent since we don't have to monitor the Logger object anymore.
Since tracepoint filtering is done in UST, we just the LTTng handler to
the root logger and send everything to UST.
Signed-off-by: David Goulet <dgoulet@efficios.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Christian Babeux [Thu, 26 Jun 2014 15:42:28 +0000 (11:42 -0400)]
Fix: Wrong configure check for UST perf event counters context support
When building on non-x86 platforms, the build errors out with the following:
CC lttng-context-perf-counters.lo
lttng-context-perf-counters.c:95:2: error: #error "Perf event counters are only supported on x86 so far."
lttng-context-perf-counters.c: In function 'read_perf_counter':
lttng-context-perf-counters.c:114:4: warning: implicit declaration of function 'rdpmc' [-Wimplicit-function-declaration]
make[2]: *** [lttng-context-perf-counters.lo] Error 1
The configure script checks for the presence of the "perf_event.h" header to enable support for
perf events counters. However, the current implementation is only available on x86,
hence the build fails on platform where the perf header is available.
Fix this issue by detecting the architecture we are currently building and wether
to enable or not the support for perf event counters context.
Signed-off-by: Christian Babeux <christian.babeux@efficios.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
If we have a C file in an application that is meant to contain both the
instrumentation and the provider, build fails if we have more than one
provider.
Fixed by putting back definitions to the state they were when defined by
tracepoint.h after a probe provider is created. This allows the next
tracepoint provider to define its tracepoints.
David Goulet [Wed, 7 May 2014 15:33:41 +0000 (11:33 -0400)]
Fix: remove JUL loglevel filtering from the agent
The loglevel filtering is now done on the UST side setup by
liblttng-ctl. To achieve that, the agent has a new structure to support
two semantic concepts being a LTTng event and a JUL logger that have a
relation of N:1. This addition does not impact the current interfaces of
the agent.
Signed-off-by: David Goulet <dgoulet@efficios.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Fix: Move generated headers to the BUILT_SOURCES target
This fixes the out of tree build when enabling Java support. The
problem in using an LTTngUst.c target is that automake will then
assume the file is generated and thus be found in the builddir.
Using the BUILT_SOURCES target will ensure the
org_lttng_ust_LTTngUst.h and org_lttng_ust_jul_LTTngUst.h headers
are generated before LTTngUst.c is compiled.
Examples binaries in doc/examples require libs in liblttng-ust/.libs/,
and those libs require other libs in the same directory. For instance
liblttng-ust.so uses liblttng-ust-tracepoint.so symbols, so the latter
is needed to properly link example programs. Currently, ld is given
the lib dir only via -rpath (used for linking executables dependencies)
but not via -rpath-link (used for linking libs dependencies).
When compiling regularly, this does not seem to be a problem (maybe ld
is smart enough). However when cross-compiling with the
arm-linux-gnueabihf toolchain, -rpath-link is needed otherwise linking
fails with:
arm-linux-gnueabihf/bin/ld: warning: liblttng-ust-tracepoint.so.0, needed by ../../../liblttng-ust/.libs//liblttng-ust.so, not found (try using -rpath or -rpath-link)
This patch adds -rpath-link to doc/examples/Makefile.am.
Fix: disable liblttng-ust-dl if dlinfo is not available in C library
According to uClibc commit [1], dlinfo is not available in recent uClibc
(>0.9.33), but available in older version of uClibc (<=0.9.33) whith a
different prototype than glibc.
dlinfo used by liblttng-ust-dl require RTLD_DI_LINKMAP that is not
available in uClibc.
To be able to use LTTng UST with uClibc, we need to disable the Dynamic
Linker Tracing functionality [2] if RTLD_DI_LINKMAP is not detected.
lib_ring_buffer_write_commit_counter()'s 'buf_offset' argument should
contain offset of beginning of area used by the record being comitted.
However, lib_ring_buffer_commit() passes ctx->buf_offset, that gets
advanced by lib_ring_buffer_write() and thus points to just-after-
end-of-record at lib_ring_buffer_commit() time. This causes
lib_ring_buffer_write_commit_counter() to return without changing
commit_hot[idx].seq, due to
if (unlikely(subbuf_offset(offset - commit_count, chan)))
return;
Since after-crash data extraction tool checks 'seq' field to find out
how much data is in buffer, this results into inavailability of
data from partially-filled subbuffer for after-crash analysis.
This patch modifies lib_ring_buffer_write_commit_counter() and all its
callers to pass and expect the end of the area. So code works as it
should, and complete information becomes visible in crash dump.
[ Fix ported from lttng-modules. Changelog inspired from Nikita
Yushchenko's original patch. ]
Fix: .split() the CC environment variable in lttng-gen-tp
The CC variable might contain command line arguments which lttng-gen-tp
currently does not handle. This fix ensures that the arguments are
properly passed on to the compiler binary as if a normal Makefile
was used.
Fix: disable liblttng-ust-dl if dlinfo is not available in C library
According to uClibc commit [1], dlinfo is not available.
To be able to use LTTng UST with uClibc, we need to disable
the Dynamic Linker Tracing functionality [2].
Jesper Derehag [Thu, 27 Mar 2014 07:57:06 +0000 (08:57 +0100)]
Fix: python invocation through env
This modification calls the interpreter through env instead of
directly, which should be more portable for those distros which do not install
python into /usr/bin/.
Building LTTng-UST with --disable-shared compiles fine, but the
resulting installed static libraries silently fails to trace, because
tracepoint fails to dlopen liblttng-ust-tracepoint.so. So don't allow
this unsupported configure option.
Fix: liblttng-ust-libc-wrapper recursive use of calloc
When built with URCU with --disable-compiler-tls, URCU tls-compat.h uses
calloc in its TLS compat layer. This is used by RCU read-side, which is
used by tracepoint. So using a tracepoint in calloc triggers an infinite
recursion, since its first use of the TLS (with TLS compat) calls
calloc.
Fix this issue by keeping a per-thread nesting counter, and as soon as
we are allocating memory from a context nested within the memory
allocator, don't trace.
Don't trace from the static allocator: keep it as simple as possible,
since this is our fallback.
Lars Persson [Wed, 12 Mar 2014 09:28:49 +0000 (10:28 +0100)]
Use autoconf AM_MAINTAINER_MODE
Give distribution maintainers the option to skip rebuilding
autoconf and automake generated files. The default behaviour
is still to have the rebuild rules enabled.
Signed-off-by: Lars Persson <larper@axis.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Paul Woegerer [Fri, 28 Feb 2014 09:02:00 +0000 (10:02 +0100)]
Turn base address dump into experimental feature
Instead of having base address state dump enabled by default make it
necessary to explicitly enable it with environment variable
LTTNG_UST_WITH_EXPERIMENTAL_BADDR_STATEDUMP.
Documentation is updated to reflect this change and explain about the
experimental nature of this feature.
Signed-off-by: Paul Woegerer <paul_woegerer@mentor.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
David Goulet [Thu, 27 Feb 2014 16:53:22 +0000 (11:53 -0500)]
Fix: dynamically fetch the session daemon port
When connecting to the session daemon, fetch the port at each try. This
moves the read port file functions to the TCP session daemon client
where the connect happens.
If the port file is not found, simply retry 3 seconds later. Also, this
adds a timeout to the acquire semaphore of 3 seconds.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: David Goulet <dgoulet@efficios.com>
David Goulet [Thu, 27 Feb 2014 15:06:52 +0000 (10:06 -0500)]
Fix: JUL agent connect to user and root sessiond
In order to achieve such a thing, two tracepoints are used where one is
for the regular user and the other one for root. This is to avoid
duplicating the payload in the user and root traces.
Furthermore, if a root port is found, a new thread is started to handle
the second session daemon.
Paul Woegerer [Tue, 25 Feb 2014 09:47:00 +0000 (10:47 +0100)]
Fix: Dump executable base-address with readlink
The previous approach only worked if the traced executable was invoked
via its fully qualified path. Using readlink to determine the full
qualified path of the traced executable works reliably even when the
traced executable is started via PATH lookup.
Signed-off-by: Paul Woegerer <paul_woegerer@mentor.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Found by Coverity:
CID 1090602 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking "stream" suggests that it may be null,
but it has already been dereferenced on all paths leading to the check
CID 1021261 (#1 of 1): Argument cannot be negative (NEGATIVE_RETURNS)10.
negative_returns: "sysconf(30)" is passed to a parameter that cannot be
negative.
Fix: work-around glibc lying about dlsym()/dlerror() leafness
Especially in the LTTng-UST malloc instrumentation, we run into the
following situation:
1) Our calloc wrapper is called,
2) we setup the static allocator,
3) we call dlsym() to lookup the symbol of the real allocator,
4) dlsym() calls into calloc(), which is overridden by our own wrapper.
Our calloc does not see that the static allocator has been set,
because the stores setting up the static allocator have been optimized
away by gcc-4.8 (in O2), because the dlsym() prototype declares it
with the "leaf" attribute, and thus we end up doing an infinite
recursion, and eventually a segmentation fault.
Thanks to Alexander Monakov for pointing out the culprit of this glibc
bug.
projects / lttng-ust.git / log
