Skip to content

Keep one object per subject in the feature flag exposure cache (FFE optimization) - #13533

Draft
dougqh wants to merge 2 commits into
dougqh/ffe-toolbox-shadingfrom
dougqh/ffe-exposure-cache
Draft

dougqh wants to merge 2 commits into
dougqh/ffe-toolbox-shadingfrom
dougqh/ffe-exposure-cache

Conversation

@dougqh

@dougqh dougqh commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Moves ExposureDeduplicationCache from an access-ordered LinkedHashMap onto the toolbox's Hashtable.D2, keyed by flag and subject:

  • One object per subject. The table entry (Exposure) holds both keys, the last allocation, variant and serial id, and its links in the least-recently-used order. Before, each subject cost a LinkedHashMap entry, a Key and a Sent.
  • Lookups build no objects. contains and recordIfChanged look up by the two strings without a key object, and recordIfChanged updates the entry in place when the exposure changes. Before, each call built a Key, a Sent and the varargs array of Objects.hash.
  • Same eviction. Exact LRU, the same capacity (65,536 subjects) and the same lock. A cache with a capacity of 0 still remembers nothing.

Motivation

The cache is long-lived: once a service has seen 65,536 subjects, it stays full. Every object it holds is old-generation state that each marking cycle has to walk. Per subject, with compressed oops:

Before After
Objects per subject 3: LinkedHashMap entry (40 B), Key (24 B), Sent (24 B) 1: Exposure (56 B)
Full cache, without the bucket array 5.8 MB, 196,608 objects 3.7 MB, 65,536 objects
Built per lookup, in the source Key, Sent, an Object[2] nothing

The "after" sizes come from a class histogram at the end of a benchmark run: 65,536 Exposure instances, 3,670,016 B. The "before" sizes are computed from the field layouts. The bucket array has 131,072 slots either way.

Measured on its own by Bits: this PR's head before the review follow-ups (af61b74853) against #13532's head (8dd30e7901). Both sides get the same benchmark-only overlay (#13529's ClientEvaluationBenchmark and a consent param). Unique subjects, JDK 21.0.12, -prof gc; base, #13533, #13534 and #13535 forks rotate fork by fork, so those PRs share the base runs. "Noise" means the fork ranges overlap.

Retained size of a full cache (65,536 entries, JOL):

Base PR
Total 9,969,344 B / 328,007 objects 7,872,200 B / 196,936 objects (−21% B, −40% objects)
Without subject Strings 6,291,552 B / 196,611 objects 4,194,408 B / 65,540 objects (−33% B, −67% objects)
Per entry 96 B, 3 objects 64 B, 1 object

256 MB heap, average time (5 forks at 1 thread, 3 at 4 threads):

Case Threads Time per evaluation Bytes per evaluation GC ms per million evaluations
scale/flags 1 887 → 881 ns (noise) 1,102 → 1,095 (noise) 25.6 → 24.4 (noise)
scale/flags 4 2.35 → 2.35 µs (noise) 1,105 → 1,097 (noise) 25.6 → 23.5 (noise)
scale/splits 1 23.15 → 22.35 µs (noise) 40,193 → 40,046 (noise) 1,266 → 1,057 (−17%)
scale/splits 4 47.24 → 45.86 µs (noise) 39,752 → 39,592 (noise) 853 → 753 (noise)

128 MB heap, G1, 4 threads, throughput (3 forks; per million evaluations, whole fork from the GC logs):

Case Throughput Pause ms Promoted MB Full GCs Humongous-triggered GCs after setup
scale/flags 1,341k → 1,360k (noise) 64.1 → 63.9 (noise) 7.84 → 7.44 (noise) 5 → 3 16 → 13
scale/splits 63k → 65k (+3.3%) 2,058 → 1,857 (−10%) 157 → 163 (noise) 4 → 3 14 → 11
  • Evaluation time and bytes per evaluation don't change. The win is the 2 MB and 131k objects a full cache no longer holds.
  • The only GC-time drops outside noise are on scale/splits: −17% GC ms per million evaluations at 1 thread, and −10% pause on the small heap.
  • Full and humongous GC counts barely move. Those come from the payloads at flush time (see Build feature flag event payloads in chunks instead of large arrays (FFE optimization) #13535), not from the cache.
  • Measured before the cache's table was made lazy (see the review follow-ups), so a client that records no exposures also skips the 512 KiB bucket array.

Additional Notes

  • Depends on the toolbox. Stacked on Shade the perf toolbox into dd-openfeature (FFE optimization) #13532, which shades the toolbox into the SDK. The published jar gains 19 relocated Hashtable classes.
  • ExposureDeduplicationCacheTest passes unchanged, including the LRU tests. The coverage exclusions for Key and Sent are removed with those classes.
  • Checked locally on this branch (stacked on Shade the perf toolbox into dd-openfeature (FFE optimization) #13532): all 1,601 dd-openfeature tests pass, and Spotless, SpotBugs (spotbugsMain, spotbugsTest, spotbugsJmh) and checkToolboxRelocation pass. The sandbox run on the earlier base had 1,609 tests: Benchmark FFE evaluation on the scale suites #13485 adds the scale-suite tests.
  • Possible follow-up: drop the exact LRU for second-chance eviction on ConcurrentHashtable, so contains reads without the lock. With 4 threads, JFR showed 61 contended enters (86 ms) on this cache's lock in 11 s, and every hit rewrites up to 6 link fields.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Review follow-ups

  • The Hashtable.D2 table is created on the first recorded exposure. At full capacity its bucket array is 512 KiB, a humongous allocation on a small G1 heap, which a client that records no exposures no longer pays for.

🤖 Generated with Claude Code

@dougqh dougqh added type: feature Enhancements and improvements tag: performance Performance related changes tag: ai generated Largely based on code generated by an AI or LLM comp: openfeature OpenFeature labels Nov 18, 2026
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 60.81% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8064973 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.02 s 14.06 s [-0.9%; +0.4%] (no difference)
startup:insecure-bank:tracing:Agent 12.93 s 13.02 s [-1.4%; +0.1%] (no difference)
startup:petclinic:appsec:Agent 17.16 s 16.98 s [+0.1%; +2.0%] (maybe worse)
startup:petclinic:iast:Agent 16.81 s 17.06 s [-2.3%; -0.6%] (maybe better)
startup:petclinic:profiling:Agent 16.67 s 16.65 s [-1.2%; +1.5%] (no difference)
startup:petclinic:sca:Agent 17.10 s 16.89 s [+0.4%; +2.1%] (maybe worse)
startup:petclinic:tracing:Agent 15.72 s 16.25 s [-7.5%; +1.0%] (no difference)

Commit: 80649732 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh dougqh changed the title Keep one object per subject in the feature flag exposure cache Keep one object per subject in the feature flag exposure cache (FFE optimization) Nov 19, 2026
@dougqh
dougqh force-pushed the dougqh/ffe-toolbox-shading branch from 3d4204f to 8dd30e7 Compare November 19, 2026 21:24
@dougqh
dougqh force-pushed the dougqh/ffe-exposure-cache branch from dc0a80f to af61b74 Compare November 19, 2026 21:24
@dougqh
dougqh force-pushed the dougqh/ffe-toolbox-shading branch from 8dd30e7 to cdda826 Compare November 19, 2026 22:38
@dougqh
dougqh force-pushed the dougqh/ffe-exposure-cache branch from af61b74 to 741cd7d Compare November 19, 2026 22:41
@dougqh
dougqh force-pushed the dougqh/ffe-exposure-cache branch from 741cd7d to 53bec3c Compare November 20, 2026 13:36
@dougqh
dougqh force-pushed the dougqh/ffe-toolbox-shading branch from 89895cd to 8c81722 Compare November 20, 2026 13:39
@dougqh
dougqh force-pushed the dougqh/ffe-exposure-cache branch from 53bec3c to 17e1eaa Compare November 20, 2026 13:39
@pr-commenter

pr-commenter Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

LLM Validation

LLM Validation Gate — java-agent-instructions

✅ PASS

  • No blocking-case regressions; the quality change is within noise (baseline/candidate confidence intervals overlap).

6 case(s) · quality 81.1 → 81.4 (+0.3) · win-rate 50% [47%–53%]

Details — analysis and per-case results

Analysis

Changed instruction file(s): AGENTS.md.

No safety or blocking-case regressions across 6 case(s). Overall pairwise win-rate 50% [47%–53%], quality +0.3 — see the verdict above for whether that clears the noise band.

Results

  • Pairwise win-rate: 50% [47%–53%] — candidate's share of blind comparisons (90% CI; spanning 50% = no clear difference)
  • Overall quality: 81.1 → 81.4 (/100, +0.3)
  • Bad signals introduced (advisory): 0
  • Candidate criteria coverage (advisory): 23/27 (85%) — expected_criteria the candidate met; does not affect the gate
  • Blocking-case regressions: 0

Cases

Case Mode Quality Δ Win-rate (90% CI) Safety
java-core-validation-001 warn -4.2 42% [28%–55%] ok
java-unit-test-conventions-001 warn +1.9 50% [50%–50%] ok
java-premain-safety-001 warn +1.7 50% [50%–50%] ok
java-ci-download-guidance-001 warn -0.8 50% [50%–50%] ok
java-documentation-only-validation-001 warn +3.3 58% [45%–72%] ok
java-existing-spock-maintenance-001 warn 0.0 50% [50%–50%] ok

Per-dimension scores, token usage, latency, and estimated cost are in the CI job logs.

datadog-bits and others added 2 commits November 20, 2026 22:53
One entry per subject, keyed by flag and subject without a key object,
updated in place and linked in LRU order: a lookup allocates nothing,
and a full cache holds one object per subject instead of three.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: dougqh <dougqh@gmail.com>
At full capacity its bucket array is 512 KiB, a humongous allocation on a
small G1 heap, which a client that records no exposures should not pay for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh force-pushed the dougqh/ffe-toolbox-shading branch from 0e0d85e to 676974d Compare November 21, 2026 03:53
@dougqh
dougqh force-pushed the dougqh/ffe-exposure-cache branch from a9c8f34 to 8064973 Compare November 21, 2026 03:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: openfeature OpenFeature tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants