From 10c9e48bf29b5d7736b5c8635afb50f7c422e1a4 Mon Sep 17 00:00:00 2001 From: =?utf8?q?J=C3=A9r=C3=A9mie=20Galarneau?= Date: Wed, 26 Jun 2024 20:09:57 +0000 Subject: [PATCH] Fix: consumerd: consumed size miscomputed during statistics sampling MIME-Version: 1.0 Content-Type: text/plain; charset=utf8 Content-Transfer-Encoding: 8bit Issue observed ============== When specifying a size-based automatic rotation, sessions sometimes continuously rotate after the first size-based rotation is triggered. Steps to reproduce: $ ./lttng create the_test $ ./lttng enable-channel -u the_channel $ ./lttng enable-rotation --size=32M $ ./lttng enable-event -ua --channel the_channel $ ./lttng start # Produce more than 32MiB of tracing data and stop the application(s), then # observe a rotation occuring on every expiration of the channel's monitor timer. Cause ===== The consumer daemon samples statistics of the various streams it monitors on every expiration of their channel's "monitor" timer. One of the statistic it maintains is the channel's total consumed size. Once the stats are collected, the consumer sends a channel statistics sample message to the session daemon. One of the fields of this message is the amount of data that was consumed for the channel since the last statistics sample was sent. The session daemon relies on that information to aggregate the consumed size a session based on the consumed sizes of its various channels. The consumer maintains a per-channel 'last_consumed_size_sample_sent' which is used to compute the amount of data consumed since the last statistics sample was sent to the session daemon. The computation is erroneous since the count sent to the session daemon is 'the channel's total consumed data amount' minus 'the data consumed since the last statistics sample'. Solution ======== The counter maintained on a per-channel basis is now consumed_size_as_of_last_sample_sent: the total amount of data consumed for that channel as of the last statistics sample. On every expiration of the monitor timer, the total amount of data consumed for the channel is determined and the difference from that counter is sent to the sessiond daemon. The counter is then updated with the latest computed value. Known drawbacks =============== None. Change-Id: I327fde946826ab1da0b6511a13132f3afba1ac16 Signed-off-by: Jérémie Galarneau --- src/common/consumer/consumer-timer.cpp | 5 +++-- src/common/consumer/consumer.hpp | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/common/consumer/consumer-timer.cpp b/src/common/consumer/consumer-timer.cpp index 2ae7f0f73..70094ef4f 100644 --- a/src/common/consumer/consumer-timer.cpp +++ b/src/common/consumer/consumer-timer.cpp @@ -687,7 +687,8 @@ void sample_and_send_channel_buffer_stats(struct lttng_consumer_channel *channel msg.highest = highest; msg.lowest = lowest; - msg.consumed_since_last_sample = total_consumed - channel->last_consumed_size_sample_sent; + msg.consumed_since_last_sample = + total_consumed - channel->consumed_size_as_of_last_sample_sent; /* * Writes performed here are assumed to be atomic which is only @@ -712,7 +713,7 @@ void sample_and_send_channel_buffer_stats(struct lttng_consumer_channel *channel channel->key, msg.highest, msg.lowest); - channel->last_consumed_size_sample_sent = msg.consumed_since_last_sample; + channel->consumed_size_as_of_last_sample_sent = total_consumed; } } diff --git a/src/common/consumer/consumer.hpp b/src/common/consumer/consumer.hpp index 851b3e2b8..cad2e2c56 100644 --- a/src/common/consumer/consumer.hpp +++ b/src/common/consumer/consumer.hpp @@ -258,7 +258,7 @@ struct lttng_consumer_channel { uint64_t lost_packets = 0; bool streams_sent_to_relayd = false; - uint64_t last_consumed_size_sample_sent = false; + uint64_t consumed_size_as_of_last_sample_sent = 0; }; struct stream_subbuffer { -- 2.34.1