finish_consuming_dead_subbuffer: fix data_size read race, reread new consumed count
Make sure finish_consuming_dead_subbuffer always see a data_size that is non
0xffffffff only when the buffer data is entirely readable, else the code rely on
the commit_seq counter which is the proper solution.
The consumed count should be re-read in each test within the loop, otherwise the
mathematic formula to get the amount of data to read does not work wrt
buffer-size wrap-around.
fork: child should issue synchronize_rcu() for urcu-bp garbage collection
After a fork(), the child process should execute synchronize_rcu() before any
new thread can be created. Failure to do so could lead to a deadlock in the
unlikely scenario where a thread ID appearing in the parent is reused in the
child before GC is performed.
Add missing listener threads data vs fork() protection
The following races are problematic:
- fork() occurs concurrently with listener thread receiving commands.
- Mutexes and data structures can be left in incoherent state.
- fork() occurs concurrently with ust library destructor.
- listen_sock can be left in incoherent state in the child.
Protect these resources with their own specific mutex.
listener_thread_data_mutex protects all data/mutexes touched by the listener
thread. It is also held across fork to make sure the child see a coherent
version of these structures.
listen_sock_mutex protects the listen_sock teardown (pthread cancel done at
libust destructor). Is is also held across fork() to protect from concurrent
teardown of listen_sock. We add a check around listen_sock teardown to see if it
has already been deleted (which could happen if the destructor runs concurrently
with fork().
David Goulet [Thu, 3 Mar 2011 15:48:11 +0000 (10:48 -0500)]
Code base to fix the print errors in UST (v4)
Update:
v2: Use commit_seq instead of commit_count to fix a consumerd segfault when
accessing commit_count, since it is not mapped.
v3: Remove commented out code
v4: Remove unused variable
Jason Wessel [Thu, 3 Mar 2011 02:21:07 +0000 (21:21 -0500)]
ustctl: Fix memory allocation problem with compatibility args
The parenthesis were missing to make the malloc math have the correct
precedence. The addition needs to occur before the multiplication.
The result is the same but for clarity also change change char ** to
char * because we are allocating an array of char pointers.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jason Wessel [Thu, 24 Feb 2011 20:40:12 +0000 (21:40 +0100)]
Allow backward compatibility to ustctl <= 0.11 for some commands
The rewrite of the ustctl broke all the existing scripts
that make use of ustctl. This allows the original commands,
examples, and external scripts to continue working properly.
Nils Carlson [Fri, 4 Feb 2011 07:24:51 +0000 (08:24 +0100)]
libustctl: use direct socket communication
This patch changes libustctl to be socket instead of pid oriented.
The user is expected to connect to a pid using
ustctl_connect_pid(pid_t) which returns a socket file-descriptor and
then use the socket for the rest of the api. This reduces the amount
of open and closing systemcalls and also makes it possible for a
session daemon to detect process shutdown by the socket closing.
David, this ones for you. :-)
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com> Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Yannick Brosseau [Wed, 23 Feb 2011 17:46:22 +0000 (18:46 +0100)]
Fix libustctl_function_tests
After discussions, we concluded that the enable a non existing marker is a valid case, so we
move it to the working case section.
While being there, check that the re-enable a marker set the right errno
Signed-off-by: Yannick Brosseau <yannick.brosseau@gmail.com> Signed-off-by: Nils Carlson <nils.carlson@ericsson.com>
Yannick Brosseau [Mon, 21 Feb 2011 19:36:09 +0000 (20:36 +0100)]
TESTS: Add a delay at the start of the fork test for a more uniform testing.
Add a sleep to leave time for the ustconsumer thread to initialize correctly
before the fork.
Most of the time the consumer was not yet started at the time of the fork so
a bunch of initializations were not done and the fork code path was not
tested properly.
Signed-off-by: Yannick Brosseau <yannick.brosseau@gmail.com> Acked-by: Nils Carlson <nils.carlson@ericsson.com>
Jason Wessel [Tue, 22 Feb 2011 22:37:25 +0000 (17:37 -0500)]
usttrace: use short signal names for busybox compatibility
The kill command in coreutils will accept the short signal name, but
various versions of busybox will not accept the long signal name. For
compatibility with busybox use the short signal name.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jason Wessel [Tue, 22 Feb 2011 22:36:03 +0000 (17:36 -0500)]
usttrace: Use /bin/sh instead of /bin/bash for busybox compatibility
The busybox posix like shell does not understand the "function"
directive nor does it understand the syntax for redirecting a file via
a shell expanded variable with $(<$pidfilepath). Busybox also does
not typically provide a link to /bin/bash since busybox does not
provide bash.
It is possible to work around all these limitations in order to allow
user space tracing to work properly in a busybox based environment
with several syntax changes to the usttrace script.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Fix marker/tracepoint/trace_event lib list: expected to be sorted
Sort library lists.
List operations expect the library lists to be sorted by pointer addresses (this
was needed for iteration on kernel modules without having to hold the mutex
across read system calls). It's usefulness in userspace is debatable, but there
is clearly a bug here, since the code that iterates on the lists still expects
them to be sorted.
Yannick Brosseau [Sun, 20 Feb 2011 15:19:36 +0000 (10:19 -0500)]
tests: fix tap.c use of uninitialized pipe_r_file
[ Edit: updated patch header ]
The _tap_comment_stdout thread can start using pipe_r_file when it
is still uninitialized. Fix it by moving the initialization before the pthread
creation.
Yannick Brosseau [Fri, 18 Feb 2011 21:55:25 +0000 (16:55 -0500)]
Try harder to find a usable lttv in the tests scripts v2
Add many more attempts to auto-detect the path to the lttv executable or
the runlttv script in trace comparison tests.
Also support setting the LTTV env variable to directly set a path the
the lttv executable in addition to the RUNLTTV variable
Fixes odd alignment issues between linker and compiler, caused by compiler
using larger alignment values than expected for structures -- and the linker
adding unexpected padding within the sections. Use the same technique recently
introduced for Linux kernel tracepoints.
UST markers: fix structure alignment for recent gcc
GCC 4.5 (and possibly some late 4.4) choose to align structures on large
multiples, which breaks the __marker section (adding extra padding between
object sections). Increase the structure alignment to fix this issue.
Note that kernel tracepoints have recently been changed to use an array of
pointers (which are pointing to the actual tracepoint structures), thus removing
the requirement for these odd structure alignments.
But given that markers are in "maintainance mode", let's do the quick fix and
just increase the minimum alignment size.
Yannick Brosseau [Thu, 10 Feb 2011 15:51:41 +0000 (10:51 -0500)]
Remove PowerPC specific time reading function
The PPC version of the trace_clock_read64 was using the TB register
which is not constant accross implementations. The currently
measured time base on PPC was not accurate.
So, for now, we rely on the CLOCK_MONOTONIC.
This patch remove a bunch of #ifdef for x86 and PPC, since we now rely on
the same base clock. It also fix the build on PPC that was currently broken
David Goulet [Fri, 14 Jan 2011 17:14:31 +0000 (18:14 +0100)]
Fix variable declaration of the clock source v2
The ust_clock_source was static in clock.h header file.
Thus, tracectl.c was not able to modify that variable for
the trace clock read function. Bad timestamp was always
returned if CLOCK_TRACE was not available.
v2:
Comment by Mathieu Desnoyers, set clock source variable
to extern in clock.h
Signed-off-by: David Goulet <david.goulet@polymtl.ca> Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Acked-by: Nils Carlson <nils.carlson@ericsson.com>
Julien Desfossez [Wed, 12 Jan 2011 08:36:37 +0000 (09:36 +0100)]
At runtime, UST checks if the LTTng clock source is available
If not, CLOCK_MONOTONIC is the default fallback.
This makes UST use the LTTng kernel clock source. With this
clock support, LTTng and UST traces are perfectly synchronize.
Traces from these two tracer can be merged for analysis.
LTTng 0.240 or higher is needed for this new clock source.
Signed-off-by: Julien Desfossez <julien.desfossez@polymtl.ca> Signed-off-by: David Goulet <david.goulet@polymtl.ca> Acked-by: Nils Carlson <nils.carlson@ericsson.com>
Nils Carlson [Wed, 5 Jan 2011 12:53:43 +0000 (13:53 +0100)]
Rename libustd to libustconsumer and ustd to ust-consumerd
This is a fargoing but necessary renaming of some ust components.
The point of the renaming is to allow for a new daemon, ustd, which
will be able to keep track of tracing sessions and connect applications
with consumers. Also, the current names were non-sensical.
This patch is a step on the way towards creating a session daemon
that can connect trace producers and consumers in a nice way.
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com>
Nils Carlson [Tue, 30 Nov 2010 13:36:30 +0000 (14:36 +0100)]
Fix freeing of channels and buffers in buffers.c v2
Changes since v1:
Handle new naming conventions
buffers.c appeared to be very convoluted full of krefs and in
general very bad shape. I have tried to make the freeing and
unmapping of shared memory symmetrical in the sense that every
open has a corresponding close and every map has a corresponding
unmap.
I have removed all use of krefs. Assuming no concurrency this
should be fine. The kref usage was anyway broken.
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com>
Nils Carlson [Thu, 4 Nov 2010 16:45:06 +0000 (17:45 +0100)]
Add trace name handling throughout tracectl, ustcomm and ustcmd
This patch is the first real step in allowing multi-session
tracing. It pushes trace name all the way out to libustcmd
(though not yet to ustctl) and trace session support all
the way into tracectl in libust (though not yet down to markers).
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com> Acked-by: David Goulet <david.goulet@polymtl.ca>
Nils Carlson [Mon, 25 Oct 2010 10:57:30 +0000 (12:57 +0200)]
Make tracectl destructor aware of fork status
This little patch checks to see whether we have used a fork
wrapper to fork. If not we won't have a working UST in the process
and can skip the destructor stage.
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com> Acked-by: David Goulet <david.goulet@polymtl.ca>
Nils Carlson [Mon, 25 Oct 2010 10:06:31 +0000 (12:06 +0200)]
Function tests for libustcmd and a C tap library
This is a bunch of function tests for libustcmd, checking return
values and other things and emitting output directly in TAP format.
I wanted to get better feedback on what is working and what isn't
and also check that functions return intelligent values. The function
tests currently run without ustd. Eventually I want to link ustd into
the testcase as well to check that functions return errors if no ustd
instance was reachable, but right now it's enough that we get some
general feedback.
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com> Acked-by: David Goulet <david.goulet@polymtl.ca>
Nils Carlson [Mon, 11 Oct 2010 14:10:30 +0000 (16:10 +0200)]
Introduce a new communication protocol for UST v5
Changes since v4:
More scanning fixes in ustctl
Changes since v3:
Fix minor bugs in ustctl
Changes since v2:
Fix packing of the socket path and ustcmd handling
of socket path
Changes since v1:
Update after comments from David Goulet,
fix some style issues, a bug in GET_SOCK_PATH and
add a fixme for posterity.
This is once again a bit of a code-dump. The principal of this
patch is to get rid of most string parsing in UST and most
dynamic heap memory allocation by libust. A secondary goal
has been to introduce a command-response model so all commands
receive responses with a result code. This has been achived
through the following steps.
1. Create standardised message containers as structs ending in
a char array. The char array is used to pack strings into
while pointers in the struct point positions relative the data
segment in the struct. Unpacking the struct upon reception is
a simple matter of adding to the pointers the position in memory
of the char array.
2. Keeping a char array permanently allocated in libust that is
used to receive incoming data, avoiding allocation. This array
is large enough to receive any of the pre-defined message
structs.
3. Replacing all string matching for commands with enum based switch
statements. This will scale better over time.
4. All commands now receive responses containing a result as a negative
errno. Libustcmd now returns -1 and sets errno according to these.
Eventually these will need to be documented in manpages with
what each errno implies. Ustctl needs to check these in turn and
give meaningfull feedback.
5. Cleaning up tracectl.c quite a bit separating out control functions
and message handling.
6. Move channel marker scanning from tracectl out to ustctl.
Fifth is a pain
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com> Acked-by: David Goulet <david.goulet@polymtl.ca>
David Goulet [Thu, 30 Sep 2010 18:52:55 +0000 (20:52 +0200)]
Fix missing data pointer
This applies to tracepoint using a marker (trace_mark_tp)
At commit 9dec086e052cf7f583a3afaa3aab48a6de8d38ac,
the tracepoint_probe_register/unregister_noupdate function has been
changed with the addition of a void data pointer. In set_marker(),
the call to that function was not updated with this new parameter.
The effect was that on a second register_trace with an already
registered probe, the data ptr was corrupted with false data and
thus the probe checking failed to validate duplicates. So, the same
data was reproduced for a single trace call. Same behavior for
unregistering the marker, it was not found.
NOTE : For now, this pointer is NULL because we don't have at this
point tracing session.
Signed-off-by: David Goulet <david.goulet@polymtl.ca> Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
David Goulet [Tue, 28 Sep 2010 13:05:49 +0000 (15:05 +0200)]
New naming convention for internal macros
This is the first patch introducing the new naming convention
for the UST tracer. To prevent namespace pollution, _ust_ prefix
is added to internal macros which are visible in the global header.
For this patch, only the container_of macro is renamed and the
redundant definition in kcompat/simple.h is removed.
Signed-off-by: David Goulet <david.goulet@polymtl.ca> Acked-by: Nils Carlson <nils.carlson@ericsson.com>
Nils Carlson [Mon, 27 Sep 2010 08:54:48 +0000 (10:54 +0200)]
Re-write ustcomm parts of UST v2
Changes since v1:
Updated after comments from David Goulet and resulting insights.
* Add a continue after a failed accept
* Fix some malloc issues
* Fix some coding style in the patch
* make del_named_sock free the memory, even if all else fails.
Notice: valgrind test-case currently broken. Needs an exception.
Description:
This is a very big patch, and so it requires a bit of explaining.
This patch is a step on the way of accomplishing serveral goals I have in this
area:
1. Use enums for commands and eliminate text-based commands. This does not mean
that we will stop processing strings for trace/channel and marker names;
just that the long series of if statements with token and string matching
will be replaced with a switch statement. To this end I have created a
ustcomm_header struct that contains the length of the data-field and some
other fields. This allows us to first receive the header, allocate memory
for the data and then receive the data; eliminating all scanning of messages.
2. Reduce the complexity of the implementation. To put it simply, I don't like
callbacks. They reduce transparency and make it difficult to follow the
flow of the code; so I have eliminated multipoll replacing it with a normal
epoll. I have also replaced almost all the different server, connection and
source structs with one, called ustcomm_sock.
3. Make ustd scale better. Currently ustd scales terribly. We allocate one
thread per-cpu per-channel per-process, five applications each with three
channels on a four cpu machine leads to 5*3*4=60 threads. Part of the reason
for this multitude of threads was that we used a ustcomm_request call
(consisting of a send and a receive) to wait for a subbuffer to be written.
The sequence for a subbuffer to be written was as follows:
Ustd calls send with a 'get_subbuffer' command, and then recv in one of
the threads and hangs on the recv on the socket.
Upon filling the subbuffer the traced app writes '1' to a pipe.
The ust_thread inside the app which was listening to the other end of the
pipe wakes up when the '1' is written. The callback from multipoll calls
a send which sends a reply to the ustd thread over the socket.
The ustd thread wakes up and reads the message, continuing along in its
execution.
I replace this with a bit of a different mechanism, which should allow us
to eventually reduce the number of threads to one per cpu:
Ustd requests a buffer_fd which causes the ustd_thread inside the app
to send the file-descriptor for the read en of the pipe to ustd.
The ustd thread now does a read on the pipe, halting its execution until
the app fills the subbuffer and writes '1' to the pipe, waking up the ustd
thread.
Ustd now makes the 'get_subbuffer' call which the ust_thread inside the
app responds to with information about the subbuffer. Writes it and then
goes back to the read call, hanging on the pipe.
So we are still stuck on the multitude of threads, but we are in much better
position to move forward. Replacing the read with an epoll statement and then
pointing the epoll event data at the buffer struct containing the current
buffer to whitch the pipe belongs should be relatively easy. We can then
instead of spawning a new thread for each buffer just allocate the
buffer_info struct and assign it to one of the per-cpu threads in ustd to
poll on.
4. Replace poll with epoll which scales better, especially for
events << (nr of fds). This is complete.
5. Allow UST to handle arbitrarily long unix socket names. This is done by
carefull allocation of the socketaddr_un struct with a dynamic length.
Truncating is ugly and dangerous.
There is a lot of work still left to be done. This is only the first of a
number of patches that I expect in this area. If someone feels like tackling
ustd head on to reduce the number of threads that would be great.
I have kept Pierre-Marc's form of error handling for the I/O wrapping functions
because I want to propagate return codes up to the apps that are using them
so they can close file-descriptors and free associated resources. If somebody
knows of a better approach please make yourself heard.
Signed-off-by: Nils Carlson <nils.carlson@ericsson.com> Acked-by: David Goulet <david.goulet@polymtl.ca>
projects / ust.git / log
