Achieving Performance Parity Between the oneTBB and OpenMP oneMKL Backends

Choosing between OpenMP and oneTBB when linking oneMKL is an architectural choice between coordinated work-sharing and dynamic work-stealing. OpenMP routinely wins on large, regular dense kernels like DGEMM due to coordinated cache and memory reuse, but it breaks application composability when nested in larger parallel frameworks. oneTBB provides composable worker pools across complex software stacks, but on multi-socket or multi-NUMA systems it leaves 20–35% of peak performance on the table. Here, I investigate the computational mechanisms behind that gap across three common workloads, and show how to recover OpenMP-level throughput inside oneTBB without dropping into a secondary threading runtime.

Unless stated otherwise, all benchmarks were run on a Google Cloud h3-standard-88 VM featuring 88 physical Intel Xeon Platinum 8481C cores across four 22-core NUMA nodes, private 2 MiB L2 caches, and two 105 MiB shared last-level caches. SMT was disabled in the guest environment. Executables were built against oneMKL 2026.1 using either mkl_intel_thread with Intel libiomp5, or mkl_tbb_thread with oneTBB 2023.1. Matrices were touched in parallel prior to timed regions, and reported values reflect medians across warm runs.

Heterogeneous Tasks: Finding the Crossover

Here, work-sharing and work-stealing refer to the execution models used by the two oneMKL backends:

• OpenMP-oneMKL: Executes each kernel through a coordinated, bulk-synchronous team. OpenMP may schedule calls dynamically, but each call retains its own predetermined inner team.
• oneTBB-oneMKL: Exposes application tasks and nested kernel tasks to the same worker pool, allowing capacity to move between calls.

I built a benchmark imitating the task structure of a supernodal factorization to find where this trade-off shifts in practice. It consists of eight levels. Each task performs a dense Cholesky factorization (DPOTRF) followed by a level-3 Schur complement update (DGEMM). The task count halves at each level while matrix dimensions scale upward, with matrix orders spread deterministically by ±30% within each level to introduce realistic load imbalance:

OpenMP executed outer tasks using schedule(dynamic, 1) with thread-local oneMKL allocations:

#pragma omp parallel for schedule(dynamic, 1)
for (int i = 0; i < tasks.size(); ++i) {
    mkl_set_num_threads_local(inner_threads);
    factor_and_update(tasks[i]);
}

The oneTBB implementation submitted outer tasks to a single tbb::task_group, allowing nested mkl_tbb_thread sub-tasks to run on the global worker pool without a hard outer/inner boundary.

OpenMP’s dynamic schedule balances the outer calls, but it cannot redistribute an inner oneMKL team among those calls. TBB exposes both levels to one scheduler, so idle capacity can move across the entire nested workload.

The crossover is distinct. Across Levels 0–3, oneTBB wins by up to 18.7% because dynamic work-stealing smooths out iteration imbalance that fixed OpenMP inner-team reservations cannot absorb. Once the hierarchy narrows to Levels 4–7, inner-kernel compute efficiency begins to dominate outer imbalance.

At Level 7 (the single dense root), there is no outer imbalance left to absorb. OpenMP’s coordinated kernel outperforms oneTBB by 52.2%. That single root node accounts for nearly the entire 10% deficit across the full tree traversal.

Large xGEMM: Root Cause and Packing Recovery

On a single square DGEMM at n = 15,840, OpenMP oneMKL finished in 1.455 s while oneTBB oneMKL required 1.945 s—a 33.7% penalty.

I first suspected scheduler overhead or task over-decomposition. I profiled on-CPU stack samples with perf record -e cpu-clock:u -F 99 -g:

Sampling found little visible scheduler time, while task instrumentation found exactly 88 TBB tasks. The decisive difference was kernel selection: OpenMP executed mkl_blas_avx512_dgemm_kernel_0 alongside packing helpers (dgemm_dcopy_*), while oneTBB spent 99.86% of its execution time in mkl_blas_avx512_dgemm_kernel_nocopy_nn_b0/b1.

The oneTBB backend was not using packing.

This choice fits the runtime architecture. Bounded shared operand packing is fundamentally a work-sharing pattern: workers cooperatively fill a shared panel buffer, synchronize at a barrier, compute against that buffer, and synchronize again before overwriting it:

parallel_region(workers, [&](int worker) {
    for (KPanel k : k_panels) {
        pack_assigned_part(worker, k, shared_panel);
        barrier();

        compute_assigned_tiles(worker, shared_panel);
        barrier();
    }
});

Global barriers constrain a work-stealing scheduler because waiting participants cannot service unrelated work. They buy coordinated reuse at the cost of composability. The no-copy kernel preserves task independence, but gives up the packed microkernel’s performance on this large matrix.

Prototype 1: Public oneMKL Packing APIs

My first attempt imposed work-sharing inside TBB using oneMKL’s public packing interface. I divided matrix B into column panels, packed those panels across parallel TBB tasks, and dispatched long-lived row workers against the packed representations:

struct PackedPanel {
    int column, width;
    aligned_buffer bytes; // cblas_dgemm_pack_get_size(...)
};

// Epoch 1: pack B panels in parallel
tbb::parallel_for(0, panel_count, [&](int p) {
    cblas_dgemm_pack(CblasRowMajor, CblasBMatrix, CblasNoTrans,
                     rows_per_owner, panels[p].width, K, 1.0,
                     &B[panels[p].column], ldb, panels[p].bytes.data());
});

// Epoch 2: row owners compute against each packed panel
tbb::parallel_for(0, machine_threads, [&](int owner) {
    auto [r0, r1] = equal_row_slab(owner, machine_threads, M);
    for (const PackedPanel& panel : panels)
        cblas_dgemm_compute(/* A[r0:r1, :] * packed(panel) -> C slab */);
});

Panel widths between 720 and 1,440 columns recovered OpenMP-level wall time (1.494 s). However, because oneMKL’s public packing interface (cblas_dgemm_pack) is intended for persistent reuse across multiple GEMM calls rather than transient panel streaming, retaining these full panels required 2.154 GiB of explicit buffer memory.

Prototype 2: OpenBLAS Microkernels + NUMA Arenas

That memory cost led me to implement transient panel cycling using OpenBLAS’s internal packing routines and microkernels. I distributed 88 long-lived row-owner tasks across four NUMA-constrained tbb::task_arena instances, 22 per node.

For each K panel, the row owners packed their private A slabs, cooperatively packed a single shared B panel, synchronized through a quickly written CyclicBarrier, and executed the microkernel:

tbb::parallel_for(row_owners, [&](Owner owner) {
    for (KPanel k : k_panels) {
        pack_private_A(owner.rows, k, owner.packed_A);
        pack_assigned_B_columns(owner, k, shared_packed_B);
        barrier.arrive_and_wait();

        openblas_microkernel(owner.rows, all_columns, k.width,
                             owner.packed_A, shared_packed_B,
                             C[owner.rows]);
        barrier.arrive_and_wait();
    }
});

The smaller buffers reduced explicit packed storage from 2.154 GiB to 247.5 MiB. With NUMA task arenas, execution time reached 1.453 s, within 0.7% of OpenMP oneMKL.

xGEMV: Locality Fencing and Intra-Node Stealing

While GEMM possesses sufficient arithmetic intensity to partially mask NUMA traversal penalties, GEMV is strictly memory-bandwidth bound. Evaluating a 131,072 × 16,384 double-precision matrix (17.18 GB) across the four NUMA nodes highlights how placement dictates performance:

Distributed page placement is mandatory for full-system bandwidth: it yields a 3.81× speedup under OpenMP.

I reproduced this placement in oneTBB with four tbb::task_arena instances. I gave each arena an explicit NUMA constraint, first-touched its row block there, and enqueued local GEMV operations:

for (int node = 0; node < numa_nodes; ++node) {
    tbb::task_arena::constraints c;
    c.set_numa_id(node).set_max_concurrency(cores_on(node));
    arenas[node] = std::make_unique<tbb::task_arena>(c);

    arenas[node]->execute([&] {
        first_touch_rows(A, y, node_row_range(node));
    });
}

for (int node = 0; node < numa_nodes; ++node) {
    for (int part = 0; part < calls_per_node; ++part) {
        arenas[node]->enqueue([&, node, part] {
            auto rows = split(node_row_range(node), calls_per_node, part);
            cblas_dgemv(/* A[rows, :] * x -> y[rows] */);
        });
    }
}
wait_for_all_arena_calls();

A tbb::task_scheduler_observer recorded every scheduler entry during nested oneMKL work and found zero entries on the wrong NUMA node.

Because assigning exactly one call per arena resulted in slight tail latencies across NUMA node completions, introducing moderate intra-node over-decomposition provided local stealing slack:

Across these runs, four tasks per NUMA node reached 398–401 GB/s, within 1.4–2.1% of OpenMP.

The Operational Rule

OpenMP-oneMKL composes dynamic outer scheduling with internally work-shared kernels, whereas oneTBB-oneMKL exposes both layers to one scheduler. Coordinated work-sharing simplifies ownership and reuse. A unified scheduler makes it easier to redistribute resources across nested workloads.

A library can support three operational tiers within oneTBB:

1. Composable Mode: Default stealable tasks across a global worker pool for heterogeneous trees and multi-component pipelines.
2. Locality-Aware Mode: NUMA-constrained task_arena instances with intra-node over-decomposition for bandwidth-bound operations.
3. Coordinated Mode: Synchronized panel-packed work-sharing executed across long-lived TBB workers when a large dense kernel is granted dedicated ownership of the machine.

Across both experiments, the useful pattern was to share across expensive locality boundaries and steal within them.