rpc: On UNIX wait on condition variable instead of FD if header is for a different thread.

If rpc_socket_read() receives a header for a different thread, it tries to yield by
releasing the read mutex and waiting on the socket's read FD. On Linux systems, this has
been observed to cause a performance problem in cases where multiple threads are being
used. Threads expecting a different header can rapidly unlock and relock the read mutex,
as they resume when sock->read_code hasn't changed. This can result in contention on the
read mutex, which delays the thread that is expecting to consume the header.

This fix updates rpc_socket_read() on UNIX to wait on a condition variable instead of the
socket's read FD. The condition variable is signalled when sock->read_code changes. This
allows waiting threads to only resume once the header and payload have been consumed by
their target thread. This fix only targets UNIX platforms, as the Windows version that
p11-kit targets by default (Windows 2000) does not provide support for condition
variables.

Signed-off-by: Simon Haggett <simon.haggett@gmail.com>

1 files changed, 13 insertions, 0 deletions

diff --git a/common/compat.h b/common/compat.h
index 0062fae..bae0647 100644
--- a/common/compat.h
+++ b/common/compat.h
@@ -205,6 +205,19 @@ void        p11_recursive_mutex_init          (p11_mutex_t *mutex);
 #define p11_mutex_uninit(m) \
 	(pthread_mutex_destroy(m))
 
+typedef pthread_cond_t p11_cond_t;
+
+#define p11_cond_init(c) \
+	(pthread_cond_init (c, NULL))
+#define p11_cond_wait(c, m) \
+        (pthread_cond_wait (c, m))
+#define p11_cond_signal(c) \
+        (pthread_cond_signal (c))
+#define p11_cond_broadcast(c) \
+        (pthread_cond_broadcast (c))
+#define p11_cond_uninit(c) \
+        (pthread_cond_destroy (c))
+
 typedef pthread_t p11_thread_t;
 
 typedef pthread_t p11_thread_id_t;