Fix: Crash when unregistering UST apps during shutdown
authorKienan Stewart <kstewart@efficios.com>
Tue, 16 Jul 2024 13:20:14 +0000 (09:20 -0400)
committerJérémie Galarneau <jeremie.galarneau@efficios.com>
Thu, 22 Aug 2024 21:11:19 +0000 (21:11 +0000)
Observed issue
==============

The following crash has been observed in v2.12.2:

```
function=0x55ac7c4c9600 <_ PRETTY FUNCTION .12873> "lttng_ustconsumer_close_metadata") at assert.c:92
function=0x55ac7c4c9600 <_ PRETTY FUNCTION .12873> "lttng_ustconsumer_close_metadata") at assert.c:101
```

The underlying cause is applicable in the current master branch as
well.

Cause
=====

There is a potential race between the threads the consumerd control
thread which handles commands coming from the sessiond and the main
thread when shutting down a consumerd.

Is it possible that the following happens:

1. `destroy_metadata_stream_ht` has the locks on `consumer_data`,
`channel`, `stream`
2. `lttng_ustconsumer_close_all_metadata` looks up the channel and starts to try and acquire a channel lock (`stream->chan->lock`)
3. `destroy_metadata_stream_ht` sets `stream->chan` to `null`
4. `destroy_metadata_stream_ht` releases the `stream`, `channel`, and `consumer_data` locks
5. `lttng_ustconsumer_close_all_metadata` now has the channel lock, and looks up `stream->chan` again to call `destroy_metadata_stream_ht`, and that member is now null

Solution
========

Acquire the stream lock after acquiring the channel lock.

part 2 follows: don't set stream->chan to null.

Known drawbacks
===============

None.

Change-Id: I1d27ea6ac08f3e7ed4624a8921cffb675be649d2
Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
src/common/ust-consumer/ust-consumer.c

index b43ae58ffd84b4be5c836130341d6c8b200b672c..af605052be6a067febbe3547e208f09f90995a4a 100644 (file)
@@ -3283,7 +3283,9 @@ void lttng_ustconsumer_close_all_metadata(struct lttng_ht *metadata_ht)
                health_code_update();
 
                pthread_mutex_lock(&stream->chan->lock);
+               pthread_mutex_lock(&stream->lock);
                lttng_ustconsumer_close_metadata(stream->chan);
+               pthread_mutex_unlock(&stream->lock);
                pthread_mutex_unlock(&stream->chan->lock);
 
        }
This page took 0.027523 seconds and 4 git commands to generate.