Move completed trace archive chunks to an "archives" sub-folder

Users have expressed the desire to read all completed trace archive
chunks at once. Initialy, the requirement called for users to wait,
using the notification API, for chunks to be made available following
the completion of a rotation. Upon the reception of a rotation
completion notification, which contains the chunk's location, it would
be possible to safely consume the resulting trace archive chunk.

Given that using the notification API was deemed too complex for
certain users, it was decided that the creation of a folder of the
form "<start-time>-<end-time>-<idx>" must guarantee the readability
of its contents. This guarantee is currently not honored for the first
trace archive chunk. This is a known bug which will be addressed
before the release.

This requirement has evolved and it must now be possible for users to
point a reader to a session output directory, after an arbitrary
number of rotations, to consume all completed trace archive chunks
while tracing is ongoing.

To make this possible (and reliable), a reader must have a way to
infer which trace archive chunks are still being produced, and which
have been completed so as to not attempt to consume a trace that
is still being produced.

First, a quick refresher on the hierachy of a session output path.

Before a rotation occurs, a session output path has the following
hierarchy:

my_trace
  |__ust
       |__[...]
  |__kernel
       |__[...]

And, after four completed rotations:

my_trace
  |__<start-time>-<end-time>-0   <--- completed trace archive chunk
  |__<start-time>-<end-time>-1
  |__<start-time>-<end-time>-2
  |__<start-time>-<end-time>-3
  |__<start-time>-4              <--- trace archive being produced

As a consequence of this behaviour, it is not possible to safely point
a CTF reader to a trace output directory without a special
configuration option that would indicate that the reader should ignore
the usual lttng hierarchy ('ust' and 'kernel'). Indeed, it is not
possible to distinguish a completed trace from a trace being produced
before the completion of the first rotation of a session.

Moreover, relying on the format of the name of trace archive chunks to
infer their completeness is awfully restrictive in terms of our
ability to alter the trace archive chunk directory format in the
future.

This format is also not part of any CTF specification document meaning
that implementing this logic in a reference CTF
implementation (babeltrace) is ill-advised. It would be unexpected for
a reader to fail to read a trace simply because it is stored in a
directory of the form <ISO8601-timestamp>-<integer>, which is a
valid trace output directory.

Hence, this commit changes the hierarchy of the trace output directory
so that completed (and safe to read) trace archive chunks are stored
in an "archives" sub-folder under the tracing session output
directory. This results in the following trace output directory
hierarchy:

my_trace
  |__archives
  |   |__<start-time>-<end-time>-0 <--- completed trace archive chunk
  |   |__<start-time>-<end-time>-1
  |   |__<start-time>-<end-time>-2
  |   |__<start-time>-<end-time>-3
  |__<start-time>-4                <--- trace archive being produced

Pointing to the "archives" directory requires no LTTng-specific logic
to be implemented in the reader(s) to achieve the users' requirement.

It does require that users wait for the presence of an "archives"
directory in the trace output path.

Reference shell code is provided to implement such a check:
"""
if [ -d ${trace_output_path}/archives ]; then
  # invoke reader
fi
"""

Given that a user would have to wait for a first completed chunk to
appear or ensure (in whichever way) that a rotation has occured before
consuming trace archive chunks, this alternative does not seem more
complex.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

diff --git a/src/bin/lttng-sessiond/rotate.c b/src/bin/lttng-sessiond/rotate.c
index 7abfaed644b1e372a95ed623de3ade1eb7ba9fa3..c3413a2cab7b2e0d89ae2853a9773447de87572c 100644 (file)
--- a/src/bin/lttng-sessiond/rotate.c
+++ b/src/bin/lttng-sessiond/rotate.c
@@ -218,7 +218,7 @@ int rename_completed_chunk(struct ltt_session *session, time_t ts)
                 * session_root_path, so we need to create the chunk folder
                 * and move the domain-specific folders inside it.
                 */
-               ret = snprintf(new_path, sizeof(new_path), "%s/%s-%s-%" PRIu64,
+               ret = snprintf(new_path, sizeof(new_path), "%s/archives/%s-%s-%" PRIu64,
                                session->rotation_chunk.current_rotate_path,
                                start_time,
                                datetime, session->current_archive_id);
@@ -275,7 +275,7 @@ int rename_completed_chunk(struct ltt_session *session, time_t ts)
                        ret = -1;
                        goto end;
                }
-               ret = snprintf(new_path, sizeof(new_path), "%s/%s-%s-%" PRIu64,
+               ret = snprintf(new_path, sizeof(new_path), "%s/archives/%s-%s-%" PRIu64,
                                session_get_base_path(session),
                                start_datetime,
                                datetime, session->current_archive_id);

diff --git a/tests/regression/tools/rotation/rotate_utils.sh b/tests/regression/tools/rotation/rotate_utils.sh
index 3d622decaf7d4b43d494973a9d96092b4400c45d..3cd9fab6f5ea278437be9db2980c3e4cf72c1964 100644 (file)
--- a/tests/regression/tools/rotation/rotate_utils.sh
+++ b/tests/regression/tools/rotation/rotate_utils.sh
@@ -116,6 +116,11 @@ function rotate_timer_test ()
        nr_iter=0
        expected_chunks=3
 
+       # Wait for the "archives" folder to appear after the first rotation
+       until [ -d $local_path ]; do
+           sleep 1
+       done
+
        # Wait for $expected_chunks to be generated, timeout after
        # 3 * $expected_chunks * 0.5s.
        # On a laptop with an empty session, a local rotation takes about 200ms,

diff --git a/tests/regression/tools/rotation/test_kernel b/tests/regression/tools/rotation/test_kernel
index d7f04bed0275293d6b9b8ba5fff84652a4256d61..9a634b70b2376b49795a9b377015559f83db5f10 100755 (executable)
--- a/tests/regression/tools/rotation/test_kernel
+++ b/tests/regression/tools/rotation/test_kernel
@@ -55,7 +55,7 @@ function test_kernel_streaming ()
 {
        diag "Test kernel streaming with session rotation"
        create_lttng_session_uri $SESSION_NAME net://localhost
-       rotate_kernel_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*"
+       rotate_kernel_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*/archives"
 }
 
 function test_kernel_local ()
@@ -63,7 +63,7 @@ function test_kernel_local ()
        diag "Test kernel local with session rotation"
        create_lttng_session_ok $SESSION_NAME $TRACE_PATH
 
-       rotate_kernel_test "${TRACE_PATH}"
+       rotate_kernel_test "${TRACE_PATH}/archives"
 }
 
 function test_kernel_local_timer ()
@@ -74,7 +74,7 @@ function test_kernel_local_timer ()
        lttng_enable_rotation_timer_ok $SESSION_NAME 500ms
        start_lttng_tracing_ok $SESSION_NAME
 
-       rotate_timer_test "${TRACE_PATH}" 0
+       rotate_timer_test "${TRACE_PATH}/archives" 0
 }
 
 function test_kernel_streaming_timer ()
@@ -85,7 +85,7 @@ function test_kernel_streaming_timer ()
        lttng_enable_rotation_timer_ok $SESSION_NAME 500ms
        start_lttng_tracing_ok $SESSION_NAME
 
-       rotate_timer_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*" 0
+       rotate_timer_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*/archives" 0
 }
 
 plan_tests $NUM_TESTS

diff --git a/tests/regression/tools/rotation/test_ust b/tests/regression/tools/rotation/test_ust
index 69bc18e1c5864a29729c3fe5a8d4b973e832571a..a4988dada45cd9efa37aca1f23cc66715b706032 100755 (executable)
--- a/tests/regression/tools/rotation/test_ust
+++ b/tests/regression/tools/rotation/test_ust
@@ -75,7 +75,7 @@ function test_ust_streaming_uid ()
        create_lttng_session_uri $SESSION_NAME net://localhost
        enable_ust_lttng_event_ok $SESSION_NAME $EVENT_NAME
 
-       rotate_ust_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*" "ust/uid/*/*/" 0
+       rotate_ust_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*/archives" "ust/uid/*/*/" 0
 }
 
 function test_ust_local_uid ()
@@ -84,7 +84,7 @@ function test_ust_local_uid ()
        create_lttng_session_ok $SESSION_NAME $TRACE_PATH
        enable_ust_lttng_event_ok $SESSION_NAME $EVENT_NAME
 
-       rotate_ust_test "${TRACE_PATH}" "ust/uid/*/*/" 0
+       rotate_ust_test "${TRACE_PATH}/archives" "ust/uid/*/*/" 0
 }
 
 function test_ust_streaming_pid ()
@@ -94,7 +94,7 @@ function test_ust_streaming_pid ()
        enable_channel_per_pid $SESSION_NAME "channel0"
        enable_ust_lttng_event_ok $SESSION_NAME $EVENT_NAME "channel0"
 
-       rotate_ust_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*" "ust/pid/*/" 1
+       rotate_ust_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*/archives" "ust/pid/*/" 1
 }
 
 function test_ust_local_pid ()
@@ -104,7 +104,7 @@ function test_ust_local_pid ()
        enable_channel_per_pid $SESSION_NAME "channel0"
        enable_ust_lttng_event_ok $SESSION_NAME $EVENT_NAME "channel0"
 
-       rotate_ust_test "${TRACE_PATH}" "ust/pid/*/" 1
+       rotate_ust_test "${TRACE_PATH}/archives" "ust/pid/*/" 1
 }
 
 function test_ust_local_timer_uid ()
@@ -117,7 +117,7 @@ function test_ust_local_timer_uid ()
        # We just want the app to register, no event generated
        $TESTAPP_BIN 0 0 /dev/null 2>&1
 
-       rotate_timer_test "${TRACE_PATH}" 0
+       rotate_timer_test "${TRACE_PATH}/archives" 0
 }
 
 function test_ust_streaming_timer_uid ()
@@ -130,7 +130,7 @@ function test_ust_streaming_timer_uid ()
        # We just want the app to register, no event generated
        $TESTAPP_BIN 0 0 /dev/null 2>&1
 
-       rotate_timer_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*" 0
+       rotate_timer_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*/archives" 0
 }
 
 function test_ust_local_timer_pid ()
@@ -144,7 +144,7 @@ function test_ust_local_timer_pid ()
        # We just want the app to register, no event generated
        $TESTAPP_BIN 0 0 /dev/null 2>&1
 
-       rotate_timer_test "${TRACE_PATH}" 1
+       rotate_timer_test "${TRACE_PATH}/archives" 1
 }
 
 function test_ust_streaming_timer_pid ()
@@ -158,7 +158,7 @@ function test_ust_streaming_timer_pid ()
        # We just want the app to register, no event generated
        $TESTAPP_BIN 0 0 /dev/null 2>&1
 
-       rotate_timer_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*" 1
+       rotate_timer_test "${TRACE_PATH}/${HOSTNAME}/${SESSION_NAME}*/archives" 1
 }
 
 function test_incompatible_sessions ()
@@ -207,6 +207,8 @@ tests=( test_ust_streaming_uid test_ust_local_uid \
        test_ust_local_timer_pid test_ust_streaming_timer_pid \
        test_incompatible_sessions )
 
+#tests=( test_ust_local_timer_uid )
+
 for fct_test in ${tests[@]};
 do
        SESSION_NAME=$(randstring 16 0)