Add missing fall through annotations

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: symbol aliases with TLS compat

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Port: no symbols aliases on MacOS

There is no equivalent to symbols aliases on MacOS, this will
unfortunatly break the ABI for SONAME(6) and will require a rebuild of
client applications.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add -Wextra to CFLAGS

Edited by Mathieu Desnoyers:

Use /* fall through */ rather than __attribute__((fallthrough)) to
stay compatible with clang and gcc < 7. The fallthrough attribute
was introduced in gcc 7.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add silent mode to examples Makefiles

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

doc: update examples to API changes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

test multiflavor single compile unit

Test multiple liburcu flavors within a compile unit.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update README following API changes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Use new header locations for includes from urcu code

This also moves urcu/static/urcu-pointer.h to urcu/static/pointer.h.
Considering that it is not meant to be included directly by library
users, it should not cause any problem.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update call-rcu.h and defer.h comments and include guards

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rculfqueue.h: do not include urcu-call-rcu.h

urcu-call-rcu.h is included by rculfhqueue.h only for struct rcu_head
forward declaration, but as a result the urcu flavor needs to be chosen
beforehand, and therefore prevents using rculfqueue.h with multiple
urcu flavors in a given compile unit.

Remove that include and do a forward declaration of struct rcu_head
instead.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rculfhash: support use with multiple flavors per compile unit

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rculfhash: do not include urcu-call-rcu.h from public API

This include is not needed, and because of it rculfhash needs to be
included after the RCU flavor is chosen.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Refactor liburcu to support many flavors per compile unit

This refactoring keeps the prior use of liburcu "map" APIs unchanged.
However, it introduces the following new APIs:

Each urcu flavor is now available as its own header:

include/urcu/urcu-memb.h
include/urcu/urcu-mb.h
include/urcu/urcu-signal.h
include/urcu/urcu-bp.h
include/urcu/urcu-qsbr.h

The installed urcu headers that were not under the urcu/ subdirectory
are moved there:

include/urcu-call-rcu.h -> include/urcu/call-rcu.h
include/urcu-defer.h -> include/urcu/defer.h
include/urcu-flavor.h -> include/urcu/flavor.h
include/urcu-pointer.h -> include/urcu/pointer.h
include/urcu-bp.h -> include/urcu/urcu-bp.h
include/urcu.h -> include/urcu/urcu.h
include/urcu-qsbr.h -> include/urcu/urcu-qsbr.h

The liburcu "map" API is now only available for use when URCU_API_MAP is
defined before including the liburcu flavor headers.

The old headers are now placeholders defining URCU_API_MAP and including
the new headers for backward compatibility:

include/urcu-bp.h
include/urcu-call-rcu.h
include/urcu-defer.h
include/urcu-flavor.h
include/urcu-pointer.h
include/urcu-qsbr.h
include/urcu.h

The header include/urcu/urcu.h now includes the right header between the
memb, signal, or mb flavors based on the compiler defines.

The symbol names of liburcu flavors are cleaned up, favoring the
following hierarchy:

urcu_<flavor name>_...

This is an ABI-breaking change, however the previous symbols name were
kept as aliases to maintain backward compatibility. They will be removed
when the next SONAME bump occurs.

The new liburcu-memb.so shared object is introduced, properly
namespacing this flavor. It is a duplicate of the previous liburcu.so,
which is kept around for backward compatibility.

The new URCU_API_MAP macro is introduced, controlling whether the
urcu API "mapping" should stay defined after inclusion of the flavor
headers. Users wishing to use the prior urcu API should either
explicitly define URCU_API_MAP before including the urcu/urcu*.h flavor
headers, or include the flavor header files from the include toplevel
directory, which are placeholders for backward compatibility. Use of
many urcu flavors within the same _LGPL_SOURCE compile unit should not
use the "map" APIs.

Internally, the "map" header files are split into one header per
flavor. The include guards are removed, so their effect can be
applied more than once. A new include/urcu/map/clear.h header is
introduced, which undefines the mappings at the end of the flavor
header if URCU_API_MAP is not set.

The new APIs namespaced for each urcu flavor is the recommended way to
use liburcu. We can expect the prior APIs to eventually become
deprecated over time.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: only wait if work queue is empty in real-time mode

Unconditionally waiting for 10 ms after the completion of every batch
of jobs of the work queue in real-time mode appears to be a behaviour
inherited from the call-rcu thread.

While this is a fair trade-off in the context of call-rcu, it is less
evident that it is desirable in the context of a general-purpose
work queue. Waiting when work is available artificially degrades the
latency characteristics of the work queue.

If a workqueue user even need the explicit delay for batching (e.g. if
a call-rcu implementation would ever use the workqueue worker thread),
it can add it within the worker_before_wait_fct callback received as
argument from workqueue creation.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: don't wait after completion of a work queue job batch

As indicated in the previous patch of this series, waiting on
completion of a job batch from the work queue artificially increases
the latency of the work queue.

The previous patch removed the wait that is performed when the
work queue is observed to be empty and was observed as the cause of a
performance problem.

It is likely that waiting when the queue is observed to be non-empty
is similarly unintended. Note that I have not observed such a problem
myself.

If a workqueue user even need the explicit delay for batching (e.g. if
a call-rcu implementation would ever use the workqueue worker thread),
it can add it within the worker_before_wait_fct callback received as
argument from workqueue creation.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: don't wait after completion of job batch if work queue is empty

On completion of a batch of jobs from the work queue, a wait of 10
ms (using poll()) is performed if there is no work present in the
work queue before waiting on its futex.

The work queue thread's structure is inspired by the call-rcu thread.
In the context of the call-rcu thread, my understanding is that the
intention is to ensure that the thread does not continuously wake-up
to process a single queued item. This is fine as an application should
not wait for a call-rcu job to be executed (or at least I don't see a
use-case for that).

In the context of the work queue, waiting for more work to be available
artificially slows down the execution of work on which an application
may wait.

I have observed a case where LTTng's session daemon's shutdown is
takes around 4 seconds as a large number of cds_lfht objects are
destroyed. Removing the wait reduces the duration of this phase of the
shut-down to almost ~10ms.

If a workqueue user even need the explicit delay for batching (e.g. if
a call-rcu implementation would ever use the workqueue worker thread),
it can add it within the worker_before_wait_fct callback received as
argument from workqueue creation.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: workqueue: struct urcu_work vs rcu_head mixup

The workqueue code was derived from call-rcu, and its API
expects a struct urcu_work for work items, but it internally iterates
over struct rcu_head.

This is not an issue at runtime because both structures have the
exact same layout and content, but it is a type mixup nevertheless.

Use the right type in the implementation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: workqueue: update comments referring to call-rcu

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: mixup between URCU_WORKQUEUE_RT and URCU_CALL_RCU_RT

The work queue implementation is derived from the call-rcu thread. A
number of references seem to have been left in place when adapting the
code for its new purpose.

The URCU_CALL_RCU_RT flag is used by wake_worker_thread() while the
rest of the workqueue.c code uses URCU_WORKQUEUE_RT to determine if
the work queue was configured in real-time mode. Both flags are defined
to the same value (0x1) and the current internal user of the
work queue (lfht) never specifies any flags.

In practice, this does not cause any problem, but this mixup should
be fixed nevertheless.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

test_rwlock: Add per-thread count to verbose output

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add *.exe to gitignore for Cygwin

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: pthread_rwlock initialization on Cygwin

On Cygwin the PTHREAD_RWLOCK_INITIALIZER macro is not
sufficient to get a properly initialized pthread_rwlock_t
struct. Use the pthread_rwlock_init function instead which
should work on all platforms.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: compat_futex_noasync on Cygwin

The futex_noasync compat code uses a weak symbol to share state across
different shared object which is not possible on Windows with the
Portable Executable format. Use the async compat code for both cases.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

wfcqueue: allow defining CDS_WFCQ_WAIT_SLEEP to override `poll'

Users may want to use alternative sleeping behavior instead of
`poll'.  Make CDS_WFCQ_WAIT_SLEEP a macro which may be defined
before including wfcqueue.h.

This alternative behavior could include logging, performing
low-priority cleanup work, sleeping a shorter/longer interval
or any combination of that.

This will also make integration into glibc easier, as `poll'
linkage causes conformance test failures even when relegated
to an impossible code path:
https://public-inbox.org/libc-alpha/20180801092626.jrwyrojfye4avcis@whir/

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update documentation for call_rcu before/after fork

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add support for the RISC-V architecture

Tested in QEMU 2.12.0-rc0, requires --disable-compiler-tls to go
through the benchmarks reliably.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Tests: Add tap-driver.sh for automake < 1.12

Add the tap-driver.sh script in the aux directory to enable
systems with an automake version prior to 1.12 to bootstrap
the source tree from git.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Tests: Replace prove by autotools tap runner

This patch removes the dependency on the prove perl script
to run the TAP test suite. It replaces it with the autotools
shell TAP driver that only requires a shell and awk.

Custom arguments can be passed to the test runner with
env variables as follow:

  env LOG_DRIVER_FLAGS='--comments --ignore-exit' \
      TESTS='foo.test baz.test' make -e check

This tap driver also creates a log file for each test that
can then be used by another system to build a test report.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

liburcu-bp: Use membarrier private expedited when available

For the liburcu-bp flavor, use the membarrier private expedited command
when available. It is faster than the shared expedited command, but has
only been introduced in 4.14 Linux kernels.

When configured with --disable-sys-membarrier-fallback, liburcu-bp
will abort if running on a kernel that do not provide the membarrier
private expedited command (e.g. CONFIG_MEMBARRIER=n or kernel version
below 4.14).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

liburcu: Use membarrier private expedited when available

For the liburcu flavor, use the membarrier private expedited
command when available. It is faster than the shared expedited
command, but has only been introduced in 4.14 Linux kernels.

When configured with --disable-sys-membarrier-fallback, liburcu
will abort if running on a kernel that provide neither the shared
nor the private expedited membarrier commands. This is the case
if running on a CONFIG_MEMBARRIER=n kernel, or a kernel version
below 4.3.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rculfhash: improve error handling of mmap backend

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: don't use overlapping mmap mappings on Cygwin

The allocation scheme used by the mmap based RCU hash table is to make a
large unaccessible mapping to reserve memory without allocating it.
Then smaller chunks are allocated by overlapping read/write mappings which
do allocate memory. Deallocation is done by an overlapping unaccessible
mapping.

This scheme was tested on Linux, macOS and Solaris. However, on Cygwin the
mmap wrapper is based on the Windows NtMapViewOfSection API which doesn't
support overlapping mappings.

An alternative to the overlapping mappings is to use mprotect to change the
protection on chunks of the large mapping, read/write to allocate and none
to deallocate. This works perfecty on Cygwin and Solaris but on Linux a
call to madvise is also required to deallocate and it just doesn't work on
macOS.

For this reason, we keep to original scheme on all platforms except Cygwin.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Tests fix: errors in shell scripts

Fix all shellcheck errors in the test scripts, switch to posix
compatible syntax. Remove duplicated code already included in common.sh.
Call the tap.sh cleanup code from our exit trap instead of overriding
it.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Revert "Use initial-exec tls model"

This reverts commit 6fd172f599e8d798e68974a786dd930d876f182e.

The initial-exec model seems to behave differently than global-dynamic
with respect to lazy initialization, causing locks to be taken then
first time each thread touch the TLS. This introduces deadlocks
with library constructors waiting on other threads.

This will require further investigation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Use initial-exec tls model

The initial-exec tls model removes requirement on performing memory
allocation the first time a tls variable is touched by any given thread.
This is needed to ensure usage of the TLS from a signal handler works
fine.

Given that the link-editor figures out the right model to use at
runtime, we can change the tls model without changing the soname major
version.

This also brings interesting speedups over the GD model. This does not
affects TLS accesses performed by executables, but does affect TLS
accesses performed by libraries.

* Executable (no change)

./test_urcu 1 0 10
SUMMARY /media/truecrypt1/compudj/doc/userspace-rcu/tests/benchmark/.libs/test_urcu testdur   10 nr_readers   1 rdur      0 wdur      0 nr_writers   0 wdelay      0 nr_reads   4420328692 nr_writes            0 nr_ops   4420328692

(with initial-exec)
./test_urcu 1 0 10
SUMMARY /media/truecrypt1/compudj/doc/userspace-rcu/tests/benchmark/.libs/test_urcu testdur   10 nr_readers   1 rdur      0 wdur      0 nr_writers   0 wdelay      0 nr_reads   4424925864 nr_writes            0 nr_ops   4424925864

* Library

(with global-dynamic)
./test_urcu_dynamic_link 1 0 10
SUMMARY /media/truecrypt1/compudj/doc/userspace-rcu/tests/benchmark/.libs/test_urcu_dynamic_link testdur   10 nr_readers   1 rdur      0 wdur      0 nr_writers   0 wdelay      0 nr_reads    573209491 nr_writes            0 nr_ops    573209491

(with initial-exec)
./test_urcu_dynamic_link 1 0 10
SUMMARY /media/truecrypt1/compudj/doc/userspace-rcu/tests/benchmark/.libs/test_urcu_dynamic_link testdur   10 nr_readers   1 rdur      0 wdur      0 nr_writers   0 wdelay      0 nr_reads   1088836185 nr_writes            0 nr_ops   1088836185

Link: https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter8-20.html
Link: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#Common-Variable-Attributes
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: don't use membarrier SHARED syscall command in liburcu-bp

One main user of liburcu-bp (lttng-ust) invokes synchronize_rcu()
repeatedly, without batching (does not use call_rcu).

Those delays introduced by sys_membarrier SHARED command significantly
impacts application startup time. Therefore, revert to not using the
membarrier SHARED command.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Tests fix: add missing Cygwin thread id

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: assignment from incompatible pointer type warnings

On some platforms, mmap returns a caddr_t pointer which generates
compiler warnings, cast to the proper pointer type to eliminate them.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Tests fix: unused variable warnings

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: add missing m68k headers to dist

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Bump version to 0.11-pre

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Version 0.10.0

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Bump library soname due to urcu flavor structure change

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: use mutex_lock() wrapper in rculfhash

Fixes this coverity issue:

*** CID 1375952:  Error handling issues  (CHECKED_RETURN)
/src/rculfhash.c: 1950 in cds_lfht_resize()
1944     }
1945
1946     void cds_lfht_resize(struct cds_lfht *ht, unsigned long new_size)
1947     {
1948             resize_target_update_count(ht, new_size);
1949             CMM_STORE_SHARED(ht->resize_initiated, 1);
>>>     CID 1375952:  Error handling issues  (CHECKED_RETURN)
>>>     Calling "pthread_mutex_lock" without checking return value (as is done elsewhere 44 out of 50 times).
1950             pthread_mutex_lock(&ht->resize_mutex);
1951             _do_cds_lfht_resize(ht);
1952             pthread_mutex_unlock(&ht->resize_mutex);
1953     }
1954
1955     static

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Use workqueue in rculfhash

The RCU lock-free hash table currently requires that the destroy
function should not be called from within RCU read-side critical
sections. This is caused by the lazy resize, which uses the call_rcu
worker thread, even though all it really needs is a workqueue/worker
thread scheme.

Use the new internal workqueue API instead of call_rcu in rculfhash to
overcome this limitation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Implement urcu workqueues internal API

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add support for m68k architecture

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Tested-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Set -Wall globally in AM_CFLAGS

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: remove double use of PTHREAD_CFLAGS

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Re-add PTHREAD_CFLAGS to global CFLAGS

Fixes a build failure on Solaris and maybe other esoteric architectures.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Don't override user variables within the build system

Instead use the appropriatly prefixed AM_* variables as to not interfere
when a user variable is passed to a make command. The proper use of flag
variables is documented at :

https://www.gnu.org/software/automake/manual/automake.html#Flag-Variables-Ordering

Fixes #1095

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add report at the end of configure

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

uatomic-api docs: use the third-person singular

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add --enable-rcu-debug to configure

When used CONFIG_RCU_DEBUG is defined in urcu/config.h, thus the
debugging self-test are used at all time. This enables a permanent
built-in debugging behaviour.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

ARM32: use dmb ish (inner shareable domain) for smp barriers

Based on https://gcc.gnu.org/ml/gcc-patches/2014-11/msg01272.html:

From: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com>:

"Hans Boehm pointed out that we were using dmb sy instead of dmb ish.
Given that the ARM-ARM says that the inner shareability domain is really
the one that contains all PE's controlled by a single hypervisor or
operating system, it would be safe to replace all dmb sy's with dmb
ish's. "

Keep full system barriers for cmm_mb()/cmm_rmb()/cmm_wmb().

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: remove cmm_wmb() from rcu_xchg_pointer and rcu_cmpxchg_pointer

Both rcu_xchg_pointer() and rcu_cmpxchg_pointer() imply both release and
acquire barriers. Therefore, the extra cmm_wmb() is redundant and can be
removed.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: uatomic arm32: add missing release barrier before uatomic_xchg

__sync_lock_test_and_set() only imply a release barrier, but
uatomic_xchg() guarantees both acquire and release barrier semantics.
Therefore, add the missing release barrier.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Tests: Add verbose support to test script

Add support for the standard "V=1" to make the test runner script
verbose.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: add missing CONFIG_RCU_FORCE_SYS_MEMBARRIER to urcu/config.h.in

Expose this configuration define in the installed header.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Allow forcing the use of sys membarrier

When using the default (liburcu.so) and bulletproof (liburcu-bp.so)
flavours of Userspace RCU, kernel support for sys-membarrier is detected
dynamically and stored in the rcu_has_sys_membarrier_memb and
urcu_bp_has_sys_membarrier global variables.

Checking the value of these variables adds a small but measurable overhead
to smp_mb_slave. On systems which support sys-membarrier, it would be
nice to have a way of avoiding that overhead.

Here is the proposed approach: if CONFIG_RCU_FORCE_SYS_MEMBARRIER is
defined then rcu_has_sys_membarrier_memb/urcu_bp_has_sys_membarrier are
replaced with the constant 1, eliminating the overhead in smp_mb_slave.
As a sanity check, support for sys-membarrier is still detected at
startup. A program using liburcu or liburcu-bp compiled with this option
aborts in the library constructor if the membarrier system call is not
supported by the operating system.

Suggested-by: Duncan Sands <duncan.sands@deepbluecap.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: rcutorture: work-around signal issue on mac os x

Our MacOS X test machine with the following config:

15.6.0 Darwin Kernel Version 15.6.0
root:xnu-3248.60.10~1/RELEASE_X86_64

appears to have issues with liburcu-signal signal being delivered on top
of pthread_cond_wait. It seems to make the thread continue, and
therefore corrupt the rcu_head. Work around this issue by unregistering
the RCU read-side thread immediately after call_rcu (call_rcu needs us
to be registered RCU readers).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: rcutorture should register thread using call_rcu

From rcu-api.txt:

`call_rcu` should be called from registered RCU read-side threads.
For the QSBR flavor, the caller should be online.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: add missing backslash in Makefile.am

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Do not use wildcards in include/Makefile.am

Wildcards are not officially supported by autotools
in Makefiles since it needs to know the exact list
of files it has to work with.

Using an absolute path was a hack that worked as long
as the path to the header files from the top source dir
was the same as the install path of those files, which
is not the case anymore.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Bump version to 0.10-pre

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: check for rand_r() in compat-rand.h

The rand_r() function exists in newer Android versions,
detect it instead of relying on the __ANDROID__ define.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Move rand-compat to private src dir

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: remove AC_FUNC_MALLOC from configure.ac

AC_FUNC_MALLOC fails cross-compile builds and is
only used to detect non-standard glibc behavior where
malloc(0) does not return a null pointer.

We don't depend on that behavior since we would have
to ship a compat implementation of malloc() for this
macro to be of any use.

Keep it commented because autoscan will report it
as missing and it might get re-introduced.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: Re-organise source dir

Re-organise the sources, add a top level "src" and "include" dir and
move relevant files.

Disable autotools automated includes and define them manually. This
fixes problems with collision of header names with system headers.

Include the autoconf config.h in the default includes and remove it
where it's explicitely included. Remove _GNU_SOURCE defines since
it's detected at configure for platforms that requires it and added
to the config.h.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Cleanup: remove leftover manual pthread detection

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: update ax_pthread macro to handle newer clang

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update library current version due to adding destroy API

Both current and age need to be updated when adding an API.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Use pthread_self to get threadid on OSX

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: examples: use destroy API for queues/stacks

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Update library age due to new stack/queue destroy API

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: tests: invoke destroy APIs for queues/stacks

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: add missing destroy functions to queues/stack APIs

Queues and stack APIs that invoke pthread_mutex_init() should have a
"destroy" counterpart which calls pthread_mutex_destroy(), ortherwise
this causes small memory leaks on platforms where pthread_mutex_init
performs memory allocation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: add missing __cds_wfcq_init for LGPL API

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: memory leak on hash table destroy

There is a missing call to pthread_mutex_destroy(3) to match
pthread_mutex_init(3) in the hash table creation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Add failover for platforms without nproc

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: use clock_get_time for caa_get_cycles fallback on MacOSX

Use clock_get_time as fallback to read time for caa_get_cycles on
MacOSX. It should not matter much in practice, since x86 uses the cycle
counter.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: syscall-compat.h MacOSX support

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: Add solaris-build.md to dist

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

rculfhash: Documentation: clarify need for grace period before "re-using"

Grace period must be waited for in case a node removed from the hash
table is re-used, similarly to the reclaim use-case.

Reported-by: Luke K D'Alessandro <ldalessa@indiana.edu>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Port: build shared libraries in Cygwin

Link: https://lists.lttng.org/pipermail/lttng-dev/2016-May/026081.html
Signed-off-by: Sebastien Boisvert <sboisvert@gydle.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Port: fix compatibility header for Cygwin

Signed-off-by: Sebastien Boisvert <sboisvert@gydle.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add GNU Hurd support to syscall-compat.h

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Add support for aarch64_be

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: urcu-bp: re-initialize list head on library exit

In case an application would try to create threads after the urcu-bp
library destructor has run, make sure the arena chunk list is
re-initialized after the memory mappings are unmapped.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

bootstrap: Standardize on autoreconf -vi

Don't overwrite already generated files.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Harmonize bootstrap script across projects

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: examples make distcheck failure

"make distcheck" marks each source file on the srcdir in the extracted
dist tarball read-only. The examples copy from the srcdir into the
builddir before running the "make" examples, but this keeps the
read-only flag on the builddir directories, which fails the build
because the resulting objects cannot be created.

Fix this by ensuring the copied target directory for each example is
user-writeable.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

wfcqueue: add C++ compatibility API

Introduce __cds_wfcq_head_cast and cds_wfcq_head_cast for compability
of wfcqueue with c++. Those are effect-less in C, where transparent
unions are supported. However, in C++, those transform struct
cds_wfcq_head and struct __cds_wfcq_head pointers to
cds_wfcq_head_ptr_t.

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: CDS_WFCQ_WOULDBLOCK typing for c++

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: configure.ac: check for possibly required libs for clock_gettime

On some platforms, you have to link with rt to get clock_gettime. This
solves errors like:

 undefined reference to `clock_gettime'

Fixes: #996
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Support for NIOS2 architecture

Add support for the Altera NIOS2 CPU archirecture. The atomic operations
are handled by the GCC. The memory barriers on this systems are entirely
trivial too, since the CPU does not support SMP at all.

Signed-off-by: Marek Vasut <marex@denx.de>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

urcu_ref_get_safe: introduce new API

Add a urcu_ref_get_safe API, which returns a boolean. It takes the value
"false" if a LONG_MAX overflow would occur (get is not performed in this
case), or true otherwise.

It warns the user (at compile-time) if the return value is unchecked.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: handle reference count overflow

The urcu refcounting API features a look and feel similar to the Linux
kernel reference counting API, which has been the subject of
CVE-2016-0728 (use-after-free). Therefore, improve the urcu refcounting
API by dealing with reference counting overflow.

For urcu_ref_get(), handle this by comparing the prior value with
LONG_MAX before updating it with a cmpxchg. When an overflow would
occur, trigger a abort() rather than allowing the overflow (which is a
use-after-free security concern).

For urcu_ref_get_unless_zero(), in addition to compare the prior value
to 0, also compare it to LONG_MAX, and return failure (false) in both
cases.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Fix: compat_futex should work-around futex signal-restart kernel bug

When testing liburcu on a 3.18 Linux kernel, 2-core MIPS (cpu model :
Ingenic JZRISC V4.15  FPU V0.0), we notice that a blocked sys_futex
FUTEX_WAIT returns -1, errno=ENOSYS when interrupted by a SA_RESTART
signal handler. This spurious ENOSYS behavior causes hangs in liburcu
0.9.x. Running a MIPS 3.18 kernel under a QEMU emulator exhibits the
same behavior. This might affect earlier kernels.

This issue appears to be fixed in 3.19 since commit e967ef022 "MIPS: Fix
restart of indirect syscalls", but nevertheless, we should try to handle
this kernel bug more gracefully than a user-space hang due to unexpected
spurious ENOSYS return value.

Therefore, fallback on the "async-safe" version of compat_futex in those
situations where FUTEX_WAIT returns ENOSYS. This async-safe fallback has
the nice property of being OK to use concurrently with other FUTEX_WAKE
and FUTEX_WAIT futex() calls, because it's simply a busy-wait scheme.

The 4.2 kernel on parisc, and likely newer kernels too, are also
affected by a similar issue.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: Michael Jeanson <mjeanson@efficios.com>
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
CC: Ralf Baechle <ralf@linux-mips.org>
CC: linux-mips@linux-mips.org
CC: linux-kernel@vger.kernel.org
CC: "James E.J. Bottomley" <jejb@parisc-linux.org>
CC: Helge Deller <deller@gmx.de>
CC: linux-parisc@vger.kernel.org