xTask Thread Pool — Benchmark Report

Micro-benchmark comparison of xTaskSubmit / xTaskWait throughput before and after the optimizations introduced in commit 8eaf7a0:

1. xNote — Replace per-task pthread_mutex_t + pthread_cond_t (88 bytes) with a 4-byte one-shot notification using atomic + futex/ulock. Fast path is a single atomic load.
2. TLS Freelist — Per-thread task struct freelist eliminates malloc/free in the common submit-then-wait-on-same-thread path.
3. xMpsc Done-Queue — Replace mutex-protected done list with a lock-free MPSC queue so workers push completed tasks without contending on qlock.

Historical note. The "TLS Freelist" referenced below was the first iteration of the allocation optimisation. It has since been replaced by the shared multi-threaded slab allocator (xSlabMt, see slab.md), which removes the per-thread warm-up cost and handles cross-thread frees without falling back to malloc. Updated numbers under the current implementation are in the Post-Slab Update section at the end of this document.

Test Environment

Results

BM_Task_SubmitWait — Single-task round-trip

Submit one noop task and immediately wait. Measures the full overhead of allocation → enqueue → dispatch → completion → deallocation.

Modest improvement — the single-task path is dominated by thread wake-up latency (qcond signal → worker dequeue), which is unchanged. The xNote fast path doesn't help here because the waiter arrives before the worker finishes.

BM_Task_FanOut — Batch submit + GroupWait

Submit N tasks, then xTaskGroupWait(). Measures batch throughput with barrier synchronization.

Strong improvement across all fan-out widths. The lock-free xMpsc done-queue eliminates contention when workers push completed tasks concurrently. The xNote signal (atomic store + ulock wake) is cheaper than pthread_cond_broadcast + mutex lock/unlock.

BM_Task_SubmitWaitBatch — Submit N, then wait each

Submit N tasks, then xTaskWait() each individually. Exercises the TLS freelist (submit and wait on the same thread).

The TLS freelist shines at batch=1000: zero malloc/free overhead when the same thread submits and waits. At smaller batches, the improvement is more modest because the freelist is already warm after the first iteration.

BM_Task_ConcurrentSubmit — Multi-producer contention

N producer threads each submit 1,000 tasks concurrently, then GroupWait.

Mixed results. At low producer counts (1–2), the lock-free done-queue reduces contention and improves wall time by ~21%. At higher producer counts (4–8), the wall time increases — this is because the xMpsc push uses a CAS loop that can spin under heavy contention from 8 producers, while the old mutex-based approach serializes cleanly. The task queue submission itself still uses qlock, so the bottleneck shifts.

BM_Task_WorkerScaling — Throughput vs worker count

10,000 tasks with varying worker thread count.

At 4+ workers, the optimized version is significantly faster. The lock-free done-queue eliminates the bottleneck where all workers contend on qlock to append to the done list. At 8 workers, throughput improves by 49% and wall time drops by 30%. The 1-worker regression is noise — single-worker throughput is dominated by the serial dequeue path.

Summary

Key Takeaways

1. xMpsc done-queue is the biggest win. Replacing the mutex-protected done list with a lock-free MPSC queue eliminates the main contention point when multiple workers complete tasks simultaneously. This shows up most dramatically in WorkerScaling/8 (+49%) and FanOut/100 (+37%).

2. TLS freelist eliminates allocation overhead. When the same thread submits and waits (the event-loop offload pattern), task structs are recycled from a per-thread freelist with zero locks. This is most visible in SubmitWaitBatch/1000 (+36%).

3. xNote is a structural improvement. While the raw latency improvement is modest for single-task round-trips, xNote reduces struct xTask_ from ~136 bytes to ~48 bytes (−65%), eliminates pthread_mutex_init/pthread_cond_init/destroy calls, and makes the fast path (task already done) a single atomic load.

4. High-contention concurrent submit shows regression at 8 producers. The CAS-based xMpsc push can spin under extreme contention. This is a known trade-off — the lock-free path is faster for the common case (2–4 producers) but can degrade under pathological contention. Future work: consider work-stealing queues to eliminate the shared submission queue entirely.

libuv Baseline Comparison

Comparison against libuv 1.52.1's uv_queue_work API. libuv uses a global thread pool (default 4 workers) with pthread_cond_signal for precise wake-up. The libuv benchmarks use uv_run(UV_RUN_ONCE) to drive the event loop and collect completions.

Note on fairness: libuv's uv_queue_work is tightly integrated with its event loop — the after_work_cb fires on the loop thread during uv_run(), which avoids cross-thread synchronization for completion notification. xTask's xTaskWait() blocks the calling thread with a futex/ulock, which is a different (and more general) synchronization model. The comparison measures end-to-end throughput of "submit work → collect result" regardless of the underlying mechanism.

SubmitWait — Single-task round-trip (xTask vs libuv)

Essentially tied. Both are dominated by the same bottleneck: waking a sleeping worker thread via kernel syscall (ulock_wake / pthread_cond_signal).

FanOut — Batch submit + barrier (xTask vs libuv)

libuv is significantly faster at high fan-out. Key differences:

1. Completion path: libuv workers post completions to an async handle (pipe/eventfd write), and the loop thread drains them in a single uv__work_done() call — no per-task synchronization. xTask workers push to an xMpsc queue and signal xNote per task.
2. No per-task allocation: libuv's uv_work_t is caller-allocated (stack or embedding struct), while xTask mallocs a struct xTask_ per submit (mitigated by TLS freelist, but still present on first use).
3. Batch drain: libuv's uv__work_done() drains all completed work in one loop iteration, amortizing the event-loop overhead. xTask's xTaskGroupWait() spins on pending with a condvar.

SubmitWaitBatch — Submit N + wait each (xTask vs libuv)

Same pattern as FanOut. libuv's batch drain and zero-alloc model give it an edge at scale.

libuv Comparison Summary

Opportunities for Improvement

1. Batch drain in GroupWait: Instead of spinning on pending + condvar, drain the xMpsc done-queue in a batch (like libuv's uv__work_done()). This would amortize the per-task overhead of xNote signal + atomic decrement.

2. Caller-allocated tasks: Allow an xTaskSubmitInline(group, work_t*, fn) path where the caller provides the task struct (e.g. embedded in a larger request object), eliminating malloc entirely — matching libuv's uv_work_t model.

3. Coalesced wake: When multiple tasks complete in rapid succession, coalesce the xNote signals into a single kernel wake (batch futex_wake / ulock_wake). Currently each worker signals independently.

Post-Slab Update (2026-05)

The original measurements above were taken when task struct allocation went through a per-thread TLS freelist layered on top of malloc. That freelist has since been replaced by the new shared xSlabMt allocator (see slab.md), which removes the "first use pays malloc" cost on every thread and makes cross-thread free paths allocator-aware.

Test Environment (Post-Slab)

SubmitWait — Single-task round-trip (Post-Slab)

Down from ~5,700 ns wall / 3,400 ns CPU — the xSlabMt alloc is materially cheaper than the prior freelist-on-malloc path, even for the single-task case where allocation is already warm. Throughput rises to ~494 K ops/s.

FanOut — Batch submit + GroupWait (Post-Slab)

The large-batch throughput more than doubles versus the earlier measurement (3.76 M → 7.23 M ops/s at 10,000). xSlabMt lets both the submitting thread and the completing worker recycle task structs without ever touching malloc/free, removing the last per-task allocation from the batch path.

SubmitWaitBatch — Submit N + wait each (Post-Slab)

Comparable to the post-optimisation figures above; the submit-then-wait-on-same-thread path was already near-optimal with the TLS freelist, so the gain from xSlabMt is modest but positive.

ConcurrentSubmit — Multi-producer contention (Post-Slab)

The 8-producer regression that existed with the TLS freelist is still visible — the bottleneck is no longer allocation but the shared task submission queue and the xSlabMt spinlock under eight contending threads (see the slab doc's multi-threaded benchmark for the raw contention curve). Work-stealing and caller-inline task structs remain the right follow-ups here.

WorkerScaling — Throughput vs worker count (Post-Slab)

Single-worker throughput improves meaningfully (25 M → 66 M ops/s) — with only one worker there is no xMpsc contention and the allocation fast-path cost is what dominates, so the slab win shows through directly. At 4+ workers the done-queue CAS remains the bottleneck and the curve shape is unchanged from the prior run.

Key Takeaways (Post-Slab)

1. Shared slab > per-thread freelist for cross-thread recycle. The old TLS freelist was great when the same thread submitted and waited, but any task freed by a worker on a different thread had to bounce back to free(). xSlabMt removes that case entirely.
2. Single-task and single-worker paths are where the slab win shows clearest. In those scenarios there is no queue contention left, so allocator cost is front-and-centre.
3. Under heavy contention, allocation is no longer the bottleneck. 8-producer / 8-worker workloads are limited by the shared queues, not by task struct acquisition. The next round of work should target those queues, not the allocator.