[ptweezy]

What and why

ZSTD_cpuSupportsBmi2() runs CPUID on every CCtx and DCtx init, and each one-shot ZSTD_compress() and ZSTD_decompress() call does an init. In a VM, CPUID traps to the hypervisor and costs about 2 µs instead of about 100 ns (#4746); on the WSL2 machine below it costs about 420 ns. Callers that create a context per message pay this on every message: librdkafka calls ZSTD_decompress() and creates a CStream per batch, and systemd creates a DCtx per journal field it decompresses.

This change caches the result in a function-local static int that starts at -1. Loads and stores use relaxed __atomic builtins, so concurrent first calls are race-free; every thread stores the same value. Without __atomic, the function falls back to the uncached probe. That only matters if DYNAMIC_BMI2 is forced on for another compiler; by default it requires GCC ≥ 4.8 or Clang, which both have __atomic. The static is per translation unit, so a normal build keeps two caches per process, one in zstd_compress.c and one in zstd_decompress.c.

-mbmi2, kernel, MSVC, and non-x86 builds never call this function.

Related: #4746 and #4767 removed the probe for -mbmi2 and kernel builds, respectively. #4888 (open) caches its new APX probe the same way and does not touch the BMI2 probe. The two PRs edit adjacent lines of zstd_internal.h, so whichever merges second needs a trivial rebase.

Performance

• x86-64: WSL2 (Hyper-V) on an AMD Ryzen 7 9800X3D, Ubuntu 24.04, GCC 13.3.0. One CPUID costs 415 to 421 ns in this VM.
• arm64 control: Apple M4, macOS, Apple clang 21. DYNAMIC_BMI2 is 0, so the probe never runs.
• Library: lib/libzstd.a from dev and from this branch, built with default flags (make -C lib libzstd.a). Bench: built with -O2, source below.
• Data: deterministic synthetic JSON log records of 256 B, 1 KiB, 4 KiB, and 64 KiB.
• Runs: per case, one warm-up, then 9 rounds of 20,000 calls (2,000 at 64 KiB), taking the median round. Two interleaved passes per build; the table shows the lower of the two medians.

x86-64, ns per call:

case                                            size   base ns  patched ns  speed-up
create+free DCtx                                   -       870          13    66.92x
create+free CCtx                                   -       914          44    20.77x
ZSTD_decompress() one-shot                       256      1768         816     2.17x
ZSTD_decompress() one-shot                      1024      2119        1182     1.79x
ZSTD_decompress() one-shot                      4096      3251        2191     1.48x
ZSTD_decompress() one-shot                     65536     16590       16312     1.02x
ZSTD_compress() one-shot, level 1                256      3107        2010     1.55x
ZSTD_compress() one-shot, level 1               1024      3650        2552     1.43x
ZSTD_compress() one-shot, level 1               4096      5955        5117     1.16x
ZSTD_compress() one-shot, level 1              65536     45072       43322     1.04x
ZSTD_compress() one-shot, level 3                256      3460        2066     1.67x
ZSTD_compress() one-shot, level 3               1024      3812        2711     1.41x
ZSTD_compress() one-shot, level 3               4096      6171        5036     1.23x
ZSTD_compress() one-shot, level 3              65536     46373       45441     1.02x
new DCtx + decompressStream (systemd)            256      1843         857     2.15x
new DCtx + decompressStream (systemd)           1024      2147        1265     1.70x
new DCtx + decompressStream (systemd)           4096      3528        2248     1.57x
new DCtx + decompressStream (systemd)          65536     16523       15608     1.06x
reference: reused DCtx (no context made)         256       808         790     1.02x
reference: reused DCtx (no context made)        1024      1154        1146     1.01x
reference: reused DCtx (no context made)        4096      2145        2186     0.98x
reference: reused DCtx (no context made)       65536     15260       15657     0.97x
reference: reused CCtx (no context made)         256      1927        1906     1.01x
reference: reused CCtx (no context made)        1024      2482        2410     1.03x
reference: reused CCtx (no context made)        4096      4819        4804     1.00x
reference: reused CCtx (no context made)       65536     43802       43600     1.00x

Context create+free saves about 860 ns, the cost of two CPUIDs. One-shot and new-DCtx calls up to 4 KiB save 0.8 to 1.4 µs. Calls on a reused context do not change. On arm64, every row is within 1% of dev.

Not measured: bare-metal x86-64, Intel CPUs, and cloud VMs.

Bench source (ctx_ab.c)

/* Times public API call patterns that create a context: one-shot
 * ZSTD_compress() and ZSTD_decompress(), a new DCtx per frame, and
 * create+free. The "reference" rows reuse a context and should not change.
 * Build once against each libzstd.a and run the binaries interleaved.
 *
 *   cc -O2 -I<tree>/lib ctx_ab.c <tree>/lib/libzstd.a -o ctx_ab
 *   ./ctx_ab [label] [rounds]
 */
#define _POSIX_C_SOURCE 199309L
#define ZSTD_STATIC_LINKING_ONLY
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "zstd.h"

static double now(void)
{
    struct timespec t;
    clock_gettime(CLOCK_MONOTONIC, &t);
    return (double)t.tv_sec + (double)t.tv_nsec / 1e9;
}

/* Deterministic JSON-like log records. */
static void fill(char* buf, size_t n, unsigned seed)
{
    static const char* words[] = {"\"level\":\"info\"", "\"msg\":\"request done\"", "\"status\":200",
        "\"path\":\"/api/v1/items\"", "\"user\":\"u-1842\"", "\"ms\":12", "\"region\":\"us-east-1\"",
        "\"trace\":\"4bf92f3577b34da6\"", "\"bytes\":5531", "\"method\":\"GET\""};
    size_t i = 0;
    unsigned x = seed * 2654435761u + 1;
    while (i < n) {
        const char* w;
        size_t l;
        x = x * 1103515245u + 12345u;
        w = words[(x >> 16) % 10];
        l = strlen(w);
        if (i + l + 1 > n) l = n - i;
        memcpy(buf + i, w, l);
        i += l;
        if (i < n) buf[i++] = (x >> 8) % 7 == 0 ? '\n' : ',';
        if (i < n && (x >> 20) % 5 == 0) buf[i++] = (char)('0' + (x >> 24) % 10);
    }
}

#define MAXN (64 * 1024)
static char src[MAXN], comp[MAXN + 1024], out[MAXN];
static volatile size_t sink;

typedef double (*case_fn)(size_t n, int iters);
static size_t clen;

static double c_create_dctx(size_t n, int it) { int i; double t; (void)n; t = now(); for (i = 0; i < it; i++) { ZSTD_DCtx* d = ZSTD_createDCtx(); sink += (size_t)d; ZSTD_freeDCtx(d); } return (now() - t) / it; }
static double c_create_cctx(size_t n, int it) { int i; double t; (void)n; t = now(); for (i = 0; i < it; i++) { ZSTD_CCtx* c = ZSTD_createCCtx(); sink += (size_t)c; ZSTD_freeCCtx(c); } return (now() - t) / it; }
static double c_decompress(size_t n, int it) { int i; double t = now(); for (i = 0; i < it; i++) sink += ZSTD_decompress(out, n, comp, clen); return (now() - t) / it; }
static double c_compress1(size_t n, int it) { int i; double t = now(); for (i = 0; i < it; i++) sink += ZSTD_compress(comp, sizeof comp, src, n, 1); return (now() - t) / it; }
static double c_compress3(size_t n, int it) { int i; double t = now(); for (i = 0; i < it; i++) sink += ZSTD_compress(comp, sizeof comp, src, n, 3); return (now() - t) / it; }
static double c_fresh_dstream(size_t n, int it)
{
    int i; double t = now();
    for (i = 0; i < it; i++) {
        ZSTD_DCtx* d = ZSTD_createDCtx();
        ZSTD_inBuffer in; ZSTD_outBuffer o;
        in.src = comp; in.size = clen; in.pos = 0;
        o.dst = out; o.size = n; o.pos = 0;
        sink += ZSTD_decompressStream(d, &o, &in);
        ZSTD_freeDCtx(d);
    }
    return (now() - t) / it;
}
static ZSTD_DCtx* rd; static ZSTD_CCtx* rc;
static double c_reused_d(size_t n, int it) { int i; double t = now(); for (i = 0; i < it; i++) sink += ZSTD_decompressDCtx(rd, out, n, comp, clen); return (now() - t) / it; }
static double c_reused_c(size_t n, int it) { int i; double t = now(); for (i = 0; i < it; i++) sink += ZSTD_compressCCtx(rc, comp, sizeof comp, src, n, 1); return (now() - t) / it; }

static int cmpd(const void* a, const void* b) { double x = *(const double*)a, y = *(const double*)b; return x < y ? -1 : x > y; }

int main(int argc, char** argv)
{
    struct { const char* name; case_fn f; int needs_frame; } cases[] = {
        {"create+free DCtx", c_create_dctx, 0},
        {"create+free CCtx", c_create_cctx, 0},
        {"ZSTD_decompress() one-shot", c_decompress, 1},
        {"ZSTD_compress() one-shot, level 1", c_compress1, 0},
        {"ZSTD_compress() one-shot, level 3", c_compress3, 0},
        {"new DCtx + decompressStream (systemd)", c_fresh_dstream, 1},
        {"reference: reused DCtx (no context made)", c_reused_d, 1},
        {"reference: reused CCtx (no context made)", c_reused_c, 0},
    };
    size_t sizes[] = {256, 1024, 4096, 65536};
    const char* label = argc > 1 ? argv[1] : "run";
    int rounds = argc > 2 ? atoi(argv[2]) : 7;
    size_t ci, si;
    rd = ZSTD_createDCtx(); rc = ZSTD_createCCtx();
    printf("# %s  zstd %s\n", label, ZSTD_versionString());
    for (ci = 0; ci < sizeof cases / sizeof cases[0]; ci++) {
        for (si = 0; si < 4; si++) {
            size_t n = sizes[si];
            double r[31], med;
            int k, it;
            if (ci < 2 && si > 0) break; /* size doesn't apply */
            fill(src, n, (unsigned)n);
            clen = ZSTD_compress(comp, sizeof comp, src, n, 3);
            if (ZSTD_isError(clen)) { fprintf(stderr, "compress failed\n"); return 1; }
            it = n >= 65536 ? 2000 : 20000;
            cases[ci].f(n, it / 10); /* warm up */
            if (rounds > 31) rounds = 31;
            for (k = 0; k < rounds; k++) r[k] = cases[ci].f(n, it);
            qsort(r, (size_t)rounds, sizeof r[0], cmpd);
            med = r[rounds / 2];
            if (ci < 2) printf("%-44s %8s  %9.0f ns\n", cases[ci].name, "-", med * 1e9);
            else printf("%-44s %6zu B  %9.0f ns\n", cases[ci].name, n, med * 1e9);
        }
    }
    ZSTD_freeDCtx(rd); ZSTD_freeCCtx(rc);
    return 0;
}

Testing

• make check and make -C tests test-fuzzer test-zstream (90 s each): pass on x86-64 (WSL2, GCC 13.3.0) and arm64 (macOS, Apple clang 21).
• make tsan-test-zstream: passes on x86-64. It ran under setarch x86_64 -R because GCC 13's libtsan fails at startup with this WSL kernel's ASLR settings.
• Compressed output is byte-identical between dev and this branch for 544 frames: sizes 0 B to 1 MiB, levels -5 to 19, one-shot and streaming APIs. The digest is the same on x86-64 and arm64.
• With the library and test built with clang 18 -fsanitize=thread, 8 threads racing on the first context creation produce no reports in 11 runs.
• On macOS with -arch x86_64, lib/ compiles with -Werror, with the c89build flags, and as C++ (the last two with asm off).

The x86-64 runs used the same change on 49cf517. Since then, dev has changed only CI files and programs/zstdcli.c. On arm64, make check also passes on this branch as submitted.

Checklist

• I searched open pull requests and issues: this problem is not already being addressed.
• Bug fix: it comes with a reproducer or a test that fails without the fix. (Not a bug fix.)
• Performance change: numbers from repeated runs are included, with CPU, compiler, flags and data.
  A gain in one configuration is not a general gain: see the Performance section of CONTRIBUTING.md.
• I stated below whether AI or other automated tools were used.

Tools used

Claude was used to help write the change, the benchmark, and the correctness checks.

[meta-cla]

Hi @ptweezy!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!