summary |
shortlog |
log |
commit | commitdiff |
tree
raw |
patch |
inline | side by side (from parent 1:
e15df1c)
The documentation of the RCU-based synchronization technique in lfstack
is too strict. It currently states that the cds_lfs_node structure
cannot be overwritten before a grace period has passed. However, lfstack
pop only use the next pointer as the replacement value when doing the
cmpxchg on the head. After the node has been pop'd from the stack,
concurrent cmpxchg trying to pop that same node will necessarily fail as
long as there is a grace period before pop/pop_all and re-adding the
node into the stack.
It is therefore sufficient to wait for a grace period between:
1) pop/pop_all and
2) freeing the node (to ensure existence for concurrent pop trying to
read node->next) or re-adding the node into the stack.
This node re-use constraint relaxation is only possible because we don't
care about node->next content read by concurrent pop: it will be simply
discarded by the cmpxchg on head. Be careful not to apply this relaxed
constraint to other data structures which care about the content of the
node's next pointer (e.g. wfstack).
This relaxed constraint allows implementing efficient free-lists (memory
allocation) with a lock-free allocation/free based on lfstack: it allows
re-using the memory backing the free-list node immediately after
allocation. The only requirement with respect to this use-case is to
wait for a grace period before putting the node back into the free-list.
Also update the test_urcu_lfs to poison the next pointer immediately
after pop/pop_all to make sure we test this relaxed constraint.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
CC: Lai Jiangshan <jiangshanlai@gmail.com>
CC: lttng-dev@lists.lttng.org
CC: rp@svcs.cs.pdx.edu
#include <urcu.h>
#include <urcu/cds.h>
#include <urcu.h>
#include <urcu/cds.h>
+#define POISON_PTR ((void *) 0x42UL)
+
/*
* External synchronization used.
*/
/*
* External synchronization used.
*/
if (snode) {
struct test *node;
if (snode) {
struct test *node;
+ snode->next = POISON_PTR;
node = caa_container_of(snode,
struct test, list);
if (sync == TEST_SYNC_RCU)
node = caa_container_of(snode,
struct test, list);
if (sync == TEST_SYNC_RCU)
cds_lfs_for_each_safe(head, snode, n) {
struct test *node;
cds_lfs_for_each_safe(head, snode, n) {
struct test *node;
+ snode->next = POISON_PTR;
node = caa_container_of(snode, struct test, list);
if (sync == TEST_SYNC_RCU)
call_rcu(&node->rcu, free_node_cb);
node = caa_container_of(snode, struct test, list);
if (sync == TEST_SYNC_RCU)
call_rcu(&node->rcu, free_node_cb);
* __cds_lfs_pop needs to be synchronized using one of the following
* techniques:
*
* __cds_lfs_pop needs to be synchronized using one of the following
* techniques:
*
- * 1) Calling __cds_lfs_pop under rcu read lock critical section. The
- * caller must wait for a grace period to pass before freeing the
- * returned node or modifying the cds_lfs_node structure.
+ * 1) Calling __cds_lfs_pop under rcu read lock critical section.
+ * Both __cds_lfs_pop and __cds_lfs_pop_all callers must wait for a
+ * grace period to pass before freeing the returned node or pushing
+ * the node back into the stack. It is valid to overwrite the content
+ * of cds_lfs_node immediately after __cds_lfs_pop and
+ * __cds_lfs_pop_all.
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop
* and __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop
* and __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* matching the technique used to synchronize __cds_lfs_pop:
*
* 1) If __cds_lfs_pop is called under rcu read lock critical section,
* matching the technique used to synchronize __cds_lfs_pop:
*
* 1) If __cds_lfs_pop is called under rcu read lock critical section,
- * both __cds_lfs_pop and cds_lfs_pop_all callers must wait for a
- * grace period to pass before freeing the returned node or modifying
- * the cds_lfs_node structure. However, no RCU read-side critical
- * section is needed around __cds_lfs_pop_all.
+ * both __cds_lfs_pop and __cds_lfs_pop_all callers must wait for a
+ * grace period to pass before freeing the returned node or pushing
+ * the node back into the stack. It is valid to overwrite the content
+ * of cds_lfs_node immediately after __cds_lfs_pop and
+ * __cds_lfs_pop_all. No RCU read-side critical section is needed
+ * around __cds_lfs_pop_all.
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop and
* __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop and
* __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* __cds_lfs_pop needs to be synchronized using one of the following
* techniques:
*
* __cds_lfs_pop needs to be synchronized using one of the following
* techniques:
*
- * 1) Calling __cds_lfs_pop under rcu read lock critical section. The
- * caller must wait for a grace period to pass before freeing the
- * returned node or modifying the cds_lfs_node structure.
+ * 1) Calling __cds_lfs_pop under rcu read lock critical section.
+ * Both __cds_lfs_pop and __cds_lfs_pop_all callers must wait for a
+ * grace period to pass before freeing the returned node or pushing
+ * the node back into the stack. It is valid to overwrite the content
+ * of cds_lfs_node immediately after __cds_lfs_pop and
+ * __cds_lfs_pop_all. No RCU read-side critical section is needed
+ * around __cds_lfs_pop_all.
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop
* and __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop
* and __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* matching the technique used to synchronize __cds_lfs_pop:
*
* 1) If __cds_lfs_pop is called under rcu read lock critical section,
* matching the technique used to synchronize __cds_lfs_pop:
*
* 1) If __cds_lfs_pop is called under rcu read lock critical section,
- * both __cds_lfs_pop and cds_lfs_pop_all callers must wait for a
- * grace period to pass before freeing the returned node or modifying
- * the cds_lfs_node structure. However, no RCU read-side critical
- * section is needed around __cds_lfs_pop_all.
+ * both __cds_lfs_pop and __cds_lfs_pop_all callers must wait for a
+ * grace period to pass before freeing the returned node or pushing
+ * the node back into the stack. It is valid to overwrite the content
+ * of cds_lfs_node immediately after __cds_lfs_pop and
+ * __cds_lfs_pop_all. No RCU read-side critical section is needed
+ * around __cds_lfs_pop_all.
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop and
* __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and
* 2) Using mutual exclusion (e.g. mutexes) to protect __cds_lfs_pop and
* __cds_lfs_pop_all callers.
* 3) Ensuring that only ONE thread can call __cds_lfs_pop() and