Fix: Crash when unregistering UST apps during shutdown
authorKienan Stewart <kstewart@efficios.com>
Wed, 10 Jul 2024 18:14:14 +0000 (14:14 -0400)
committerJérémie Galarneau <jeremie.galarneau@efficios.com>
Thu, 22 Aug 2024 21:05:42 +0000 (21:05 +0000)
Observed issue
==============

The following crash has been observed in v2.12.2:

```
function=0x55ac7c4c9600 <_ PRETTY FUNCTION .12873> "lttng_ustconsumer_close_metadata") at assert.c:92
function=0x55ac7c4c9600 <_ PRETTY FUNCTION .12873> "lttng_ustconsumer_close_metadata") at assert.c:101
```

The underlying cause is applicable in the current master branch as
well.

Cause
=====

There is a potential race between the threads the consumerd control
thread which handles commands coming from the sessiond and the main
thread when shutting down a consumerd.

Is it possible that the following happens:

1. `destroy_metadata_stream_ht` has the locks on `consumer_data`,
`channel`, `stream`
2. `lttng_ustconsumer_close_all_metadata` looks up the channel and starts to try and acquire a channel lock (`stream->chan->lock`)
3. `destroy_metadata_stream_ht` sets `stream->chan` to `null`
4. `destroy_metadata_stream_ht` releases the `stream`, `channel`, and `consumer_data` locks
5. `lttng_ustconsumer_close_all_metadata` now has the channel lock, and looks up `stream->chan` again to call `destroy_metadata_stream_ht`, and that member is now null

Solution
========

Acquire the stream lock after acquiring the channel lock.

part 2 follows: don't set stream->chan to null.

Known drawbacks
===============

None.

Change-Id: I1d27ea6ac08f3e7ed4624a8921cffb675be649d2
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
src/common/ust-consumer/ust-consumer.cpp

index de9838aab14284626a8a281a02afc59f60b95246..bb3e7346a7871b9641b15b04bdedb90eb66d9b95 100644 (file)
@@ -3191,7 +3191,9 @@ void lttng_ustconsumer_close_all_metadata(struct lttng_ht *metadata_ht)
                health_code_update();
 
                pthread_mutex_lock(&stream->chan->lock);
+               pthread_mutex_lock(&stream->lock);
                lttng_ustconsumer_close_metadata(stream->chan);
+               pthread_mutex_unlock(&stream->lock);
                pthread_mutex_unlock(&stream->chan->lock);
        }
 }
This page took 0.027193 seconds and 4 git commands to generate.