Skip to content

Add CompactList, with a cursor, to the toolbox (FFE optimization) - #13601

Draft
dougqh wants to merge 2 commits into
dougqh/toolbox-componentfrom
dougqh/toolbox-compact-list
Draft

dougqh wants to merge 2 commits into
dougqh/toolbox-componentfrom
dougqh/toolbox-compact-list

Conversation

@dougqh

@dougqh dougqh commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

  • Adds datadog.trace.util.CompactList to components/toolbox: an immutable list stored at its exact size. It's one shared empty list, a single object for one element, and the object plus one exact-size array beyond. Elements may be null.
    • copyOf(Collection) returns a CompactList (a CompactList argument is returned as is), and empty() returns the shared empty list.
    • It has its own iterator(), forEach and isEmpty, plus a cursor() for walks that stop early or carry state in locals. The cursor requires a list without null elements, since null marks the end.
  • Adds ImmutableListBenchmark (internal-api JMH), in the style of ImmutableMapBenchmark. It compares how small immutable lists iterate, and whether they allocate.
  • Adds BenchmarkUtils.warmUpListDispatch, which broadens the profile of the JDK's shared list internals before a trial.

Motivation

Long-lived parsed state, such as a configuration, holds many small lists, mostly with zero or one element. As exact-size ArrayLists, each costs a list object plus an array. Mixing list classes to save that (emptyList, singletonList, ArrayList) makes every loop over them megamorphic, and escape analysis then keeps their iterators. CompactList is one class at its exact size, and walking it allocates nothing:

  • Own iterator, forEach and isEmpty. AbstractList's are shared by every subclass, so in an application their size()/get() calls go megamorphic and the iterator escapes.
  • A new iterator or cursor on every path, the empty list included. A shared one on any path would make the result a merged allocation, which escape analysis can't remove. The iterator and the cursor share one index walk rather than wrapping each other.

ImmutableListBenchmark (Bits, JDK 21, 3 forks, 4 threads, steal median 0.14%). Five lists of sizes 0, 1, 1, 2 and 8, walked per op. Median fork in ns/op, then B/op. compact here is the same design, measured before it moved into the toolbox. polluted first makes the JDK's shared list internals see many list classes, as in an application:

Profile, operation arrayList jdkImmutable (List.of) jdkMixed compactInherited compact
clean, enhanced for 37.3, 0 66.1, 160 165, 112 37.8, 0 32.4, 0
clean, forEach 35.3, 0 28.1, 0 77.9, 80 38.9, 0 25.0, 0
clean, indexed 32.3, 0 45.0, 0 113, 0 34.8, 0 34.0, 0
polluted, enhanced for 37.6, 0 67.6, 160 159, 112 194, 160 32.4, 0
polluted, forEach 34.8, 0 28.9, 0 85.4, 80 161, 160 22.6, 0
polluted, indexed 32.4, 0 49.5, 0 119, 0 34.7, 0 34.7, 0
  • compact allocates nothing in any case. Against ArrayList it's −13% on enhanced-for and −29% to −35% on forEach, but +5–7% on the indexed loop (small, though the forks don't overlap). It beats List.of copies everywhere.
  • compactInherited, the same list inheriting AbstractList's iterator, is fine in a quiet JVM. Polluted, it allocates 160 B and runs about 5× slower. PrintInlining shows AbstractList$Itr's size() and get() as virtual calls; with its own iterator, everything inlines from a monomorphic profile.
  • jdkMixed allocates even unpolluted: three classes make the caller's own call sites megamorphic.
  • List.of copies allocate 160 B per enhanced-for walk in both profiles, because the List12 and ListN iterators escape.

In dd-openfeature's configuration (#13556), the inherited iterator cost up to +174% time and +66 KB per evaluation with a polluted profile. This design matched master's bytes, and variations came out 20% faster.

Additional Notes

Contributor Checklist

🤖 Generated with Claude Code

@dougqh dougqh added type: feature Enhancements and improvements comp: core Tracer core tag: performance Performance related changes tag: ai generated Largely based on code generated by an AI or LLM labels Nov 21, 2026
@pr-commenter

pr-commenter Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

LLM Validation

LLM Validation Gate — java-agent-instructions

✅ PASS

  • No blocking-case regressions; the quality change is within noise (baseline/candidate confidence intervals overlap).

6 case(s) · quality 80.3 → 80.6 (+0.2) · win-rate 50% [50%–50%]

Details — analysis and per-case results

Analysis

Changed instruction file(s): AGENTS.md.

No safety or blocking-case regressions across 6 case(s). Overall pairwise win-rate 50% [50%–50%], quality +0.2 — see the verdict above for whether that clears the noise band.

Results

  • Pairwise win-rate: 50% [50%–50%] — candidate's share of blind comparisons (90% CI; spanning 50% = no clear difference)
  • Overall quality: 80.3 → 80.6 (/100, +0.2)
  • Bad signals introduced (advisory): 0
  • Candidate criteria coverage (advisory): 23/27 (85%) — expected_criteria the candidate met; does not affect the gate
  • Blocking-case regressions: 0

Cases

Case Mode Quality Δ Win-rate (90% CI) Safety
java-core-validation-001 warn +1.4 50% [50%–50%] ok
java-unit-test-conventions-001 warn +1.1 50% [50%–50%] ok
java-premain-safety-001 warn 0.0 50% [50%–50%] ok
java-ci-download-guidance-001 warn -1.9 50% [50%–50%] ok
java-documentation-only-validation-001 warn +0.6 50% [50%–50%] ok
java-existing-spock-maintenance-001 warn +0.3 50% [50%–50%] ok

Per-dimension scores, token usage, latency, and estimated cost are in the CI job logs.

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 60.81% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 89db448 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.04 s 14.07 s [-0.9%; +0.5%] (no difference)
startup:insecure-bank:tracing:Agent 12.99 s 13.10 s [-1.9%; +0.1%] (no difference)
startup:petclinic:appsec:Agent 17.74 s 17.48 s [+0.6%; +2.3%] (maybe worse)
startup:petclinic:iast:Agent 17.53 s 17.60 s [-1.4%; +0.5%] (no difference)
startup:petclinic:profiling:Agent 17.41 s 17.42 s [-1.1%; +1.1%] (no difference)
startup:petclinic:sca:Agent 17.82 s 17.82 s [-0.8%; +0.8%] (no difference)
startup:petclinic:tracing:Agent 16.77 s 16.28 s [-1.3%; +7.4%] (no difference)

Commit: 89db448d · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

dougqh and others added 2 commits November 20, 2026 22:53
ImmutableListBenchmark compares how small lists held in long-lived parsed
state iterate (enhanced for, forEach, indexed) across ArrayList, the JDK's
immutable lists, a mix of emptyList/singletonList/ArrayList, and a compact
list with and without its own iterator, each structure holding lists of
sizes 0, 1, 1, 2 and 8.

BenchmarkUtils.warmUpListDispatch broadens the profiles of the JDK's
shared list internals (AbstractList's iterator, AbstractCollection.isEmpty,
Iterable.forEach) first when profile=polluted, since a quiet benchmark JVM
lets escape analysis remove iterators it wouldn't in an application.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An immutable list at its exact size for small lists held in long-lived
state: one shared empty list, one object for a single element, and one
exact-size array beyond. Walking it allocates nothing: its own iterator,
forEach and isEmpty keep their calls monomorphic where AbstractList's
shared ones go megamorphic, and iterator() and cursor() return a new
object on every path, the empty list included, so escape analysis can
remove them. The iterator and the cursor share one index walk.

ImmutableListBenchmark's compact arm now uses it, with a cursor arm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh force-pushed the dougqh/toolbox-compact-list branch from 1b82c60 to 89db448 Compare November 21, 2026 03:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant