Repository navigation
Benchmark FFE evaluation on the scale suites - #13485
gh-worker-dd-mergequeue-cf854d[bot] merged 16 commits into
Conversation
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Add ScaleSuiteBenchmark, which evaluates each case of the opt-in scale suite of the shared FFE fixtures. The setup checks every expected value and reason first. Pin ffe-system-test-data to the scale suite PR head.
The scale benchmark sat on the JSON value change and checked JSON integers by type, so it could not set a baseline on master. It now compares JSON numbers by value, covers every case file of the scale and scale-json suites, can isolate one case with the index parameter, and prints each case's index. A load benchmark measures parsing each suite's configuration and prints the heap it retains, so changes that move work to load time show their cost. The submodule points to the scale-suite fixture branch, which adds the N x M dimensions and JSON values up to 10 MB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5a15d5c to
c18784a
Compare
The scale benchmarks failed the forbidden API check on System.out, and nothing outside a manual JMH run checked the scale cases, so a sibling change that resolves a large configuration wrongly would go unnoticed. - Add ScaleSuiteTest: evaluate every case of the scale and scale-json suites twice, and fail when the benchmark misses a case file. The cases, their check and the suite reader move to the test sources and the benchmarks reuse them; the reader uses JsonReading. - Allow System.out in the benchmark setups (@SuppressForbidden). - Check every case again after the measurement, fail on an error code, and reject expectations the check does not handle. - Rerun the jmh task when the fixtures change. - Measure retained heap after a first parse, as the smallest of three signed samples. - Describe a row as the unweighted mean over the cases of a file, say that hooks are not measured and when to run three forks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Reminder: before merge, repin the fixture submodule to the commit that DataDog/ffe-system-test-data#78 lands as on
Check after the repin: |
The fixture PR now checks that the committed scale files match the generator and fixes its docs. The case and configuration files are unchanged, so the benchmark numbers do not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in ScaleSuiteTest, the benchmark hardening, and the fixtures pin f425d578. ScaleSuiteTest now also checks the four case files of the regex suite, so the cross-SDK regex contract runs in CI. ScaleCase accepts an expected errorCode, so the PARSE_ERROR cases of unsupported-syntax.json are compared, not skipped. semantics and unsupported-syntax are checked but not benchmarked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixtures add a case file with ONE_OF lists of the integers 1 to 1000 and of the powers of two up to 2^52, which an index keyed on exact doubles must still match. Pin them and benchmark the file; its 8 cases pass on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the scale/one-of-small-numbers case file and the fixtures pin 08e86993. The benchmark and ScaleSuiteTest keep both the new case file and the regex suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SDK is the main reader of the ffe-system-test-data submodule and the only reader of the vendored experimentation opt-out cases, so both move from the smoke-test resources to dd-openfeature/src/test/fixtures. They stay outside any source set: the tests and benchmarks read them from disk, and the smoke tests copy ufc-config.json and evaluation-cases into their resources. The jmh task declares the scale suites as a file tree, which is empty rather than missing when the submodule is not initialized, so a jmh run on such a checkout no longer fails before it starts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The suites read the system properties and environment, so setting DD_EXPERIMENTAL_FLAGGING_PROVIDER_SPAN_ENRICHMENT_ENABLED turned span enrichment on: the benchmarks measured the enriched metadata path and the expected selection metadata no longer described it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each case file parsed its suite again, 14 times for the 7.8 MB scale configuration. One provider per suite, shut down after all cases, parses each configuration once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mcculls
left a comment
There was a problem hiding this comment.
The 2 benchmarks look like a reasonable base to build on
Keep the fixtures submodule at dd-openfeature/src/test/fixtures, pinned to 508a342. Master bumped it at the old smoke-test path to 29ba6bf, which is 4 commits behind that pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
4075f1f
into
master
Take master's version of every file from #13485, which merged as a squash commit, so this branch only carries the shard hashing change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master has the scale benchmarks of #13485 as one squashed commit, with the fixtures moved to dd-openfeature/src/test/fixtures. Take master's benchmarks, fixture path and fixture pin, and drop the old fixture path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#13485 merged as a squash commit, so the conflicts take master's version of its files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Takes master's version of every #13485 file; only this branch's SemVer changes differ from master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Benchmark the FFE scale evaluation suite Add ScaleSuiteBenchmark, which evaluates each case of the opt-in scale suite of the shared FFE fixtures. The setup checks every expected value and reason first. Pin ffe-system-test-data to the scale suite PR head. Benchmark every scale dimension on master, and the configuration load The scale benchmark sat on the JSON value change and checked JSON integers by type, so it could not set a baseline on master. It now compares JSON numbers by value, covers every case file of the scale and scale-json suites, can isolate one case with the index parameter, and prints each case's index. A load benchmark measures parsing each suite's configuration and prints the heap it retains, so changes that move work to load time show their cost. The submodule points to the scale-suite fixture branch, which adds the N x M dimensions and JSON values up to 10 MB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Index ONE_OF and NOT_ONE_OF lists when the configuration loads Each ONE_OF or NOT_ONE_OF evaluation scanned its list and compared the attribute with each element. With a number attribute, the scan also parsed each element as a double. A 10,000-element list cost about 25 us for a string attribute and 90 us for a number attribute. The parser now builds a MembershipIndex for each list of 2 or more elements: a HashMap for string comparisons and primitive long-keyed tables for numeric comparisons, each mapping a key to the element's first position. A lookup costs about 2-4 ns at every list size. The index gives the same result as the scan, including the numeric leniency (42 matches "042") and the errors that the scan raises at an unconvertible element. When a result depends on such an error, or for an attribute type that the index does not handle, the evaluator runs the scan as before. Spread ONE_OF number keys over the table, and test the parser builds the index The long table hashed with the low bits of a Fibonacci product. A whole number below 2^21 has a key whose low 32 bits are zero, so small integers and powers of two all started at slot 0, and a lookup in a list such as 1..1000 walked about 500 slots (792 ns instead of 5 ns). The slot now takes the high bits of the product, which depend on every bit of the key. - MembershipIndexTest: start slots of small integers, powers of two, prices in cent steps and large IDs; it fails on the old hash. - UniversalFlagConfigParserTest: the parser builds the index for lists of two or more elements, and not for one element. - FlagEvaluatorMembershipTest: attribute types that the index leaves to the scan (Instant, list, map), and nested list and object elements. - A list without numeric strings shares one table for both number lookups. - Comments match the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Check the scale suites in the unit tests and harden the benchmarks The scale benchmarks failed the forbidden API check on System.out, and nothing outside a manual JMH run checked the scale cases, so a sibling change that resolves a large configuration wrongly would go unnoticed. - Add ScaleSuiteTest: evaluate every case of the scale and scale-json suites twice, and fail when the benchmark misses a case file. The cases, their check and the suite reader move to the test sources and the benchmarks reuse them; the reader uses JsonReading. - Allow System.out in the benchmark setups (@SuppressForbidden). - Check every case again after the measurement, fail on an error code, and reject expectations the check does not handle. - Rerun the jmh task when the fixtures change. - Measure retained heap after a first parse, as the smallest of three signed samples. - Describe a row as the unweighted mean over the cases of a file, say that hooks are not measured and when to run three forks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge the #13485 base (a887657) into the ONE_OF index branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Pin the fixtures to the latest head of the scale suite PR The fixture PR now checks that the committed scale files match the generator and fixes its docs. The case and configuration files are unchanged, so the benchmark numbers do not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge the #13485 base (09289ff) into the ONE_OF index branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Benchmark one-of lists of small numbers The fixtures add a case file with ONE_OF lists of the integers 1 to 1000 and of the powers of two up to 2^52, which an index keyed on exact doubles must still match. Pin them and benchmark the file; its 8 cases pass on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge the #13485 base (30df999) into the ONE_OF index branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Benchmark the scale cases with the client's layered context The OpenFeature client passes the provider a new LayeredEvaluationContext on each call, while the benchmark reused one MutableContext. A rule on `id` when the context has no `id` reads the layered context's key set, which it builds again on each call, so the benchmark did not show what applications pay for it. evaluateLayered wraps each case's context the way the client does, as ExperimentationOptOutBenchmark.evaluateFreshLayeredContext does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge the #13485 base (8d0f39c) into the ONE_OF index branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge master (4075f1f) into the ONE_OF index branch #13485 merged as a squash commit, so the conflicts take master's version of its files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Test the probe lengths of the number index The test now replays the inserts and bounds the slots that hits and misses read, over integers, negative integers, powers of two, cent and .99 prices, large IDs, and integers mixed with powers of two in one list. The earlier hash fails six of the seven cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Index every attribute type, and do not scan when the scan throws Any number attribute is compared through doubleValue, and any other attribute as its string, so Instant, List and Map attributes use the index too. Only an attribute that is neither a string nor a boolean, against a list with a nested element, still scans: an equal map can print its keys in another order. When the scan throws, the index returns the element it throws at, and the evaluator compares the attribute with that element only, which throws the same exception. The differential test now also compares the error messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Use static imports and Javadoc in the membership changes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Keep the ONE_OF string index in flat arrays The HashMap of strings to first positions held a node and often a boxed Integer per value. On the scale configuration it added 4.4 MB in about 130,000 long-lived objects that every concurrent mark traces. A flat open-addressed table, modeled on StringIndex, keeps the strings, their cached hashes and, only for lists with a number element, their first positions. Lists without strings or without numbers share one empty table. Co-authored-by: Douglas Q Hawkins <douglas.hawkins@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: leo.romanovsky <leo.romanovsky@datadoghq.com>
Benchmark the FFE scale evaluation suite Add ScaleSuiteBenchmark, which evaluates each case of the opt-in scale suite of the shared FFE fixtures. The setup checks every expected value and reason first. Pin ffe-system-test-data to the scale suite PR head. Benchmark every scale dimension on master, and the configuration load The scale benchmark sat on the JSON value change and checked JSON integers by type, so it could not set a baseline on master. It now compares JSON numbers by value, covers every case file of the scale and scale-json suites, can isolate one case with the index parameter, and prints each case's index. A load benchmark measures parsing each suite's configuration and prints the heap it retains, so changes that move work to load time show their cost. The submodule points to the scale-suite fixture branch, which adds the N x M dimensions and JSON values up to 10 MB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Parse each FFE SemVer attribute once per evaluation A flag with many SemVer conditions on the same attribute parsed the attribute value once per condition: the scale-semver-ladder fixture parses it up to 100 times per evaluation. The parser now marks flags with SemVer conditions, and their evaluations keep a small cache of attribute parses, invalid versions included. Other flags allocate nothing for it. ParsedSemver.parse no longer copies the core components, the build metadata or the remainder, and prerelease comparison compares identifiers in place instead of copying them. Numeric conditions convert operands to double without boxing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Keep numeric conversion errors identical and place the SemVer tests Numeric comparisons convert numbers without boxing and send every other operand through mapValue, so an unconvertible or null operand still fails with the same exception and message, which reaches getErrorMessage() when full evaluation data is observed. Tests compare those messages with mapValue's for a few bad inputs. Flag.hasSemverConditions documents that only the parser sets it, before the configuration is published. The parse cache derives its count from its entries. The ladder tests move into FlagEvaluatorTest and the parser flag test into UniversalFlagConfigParserTest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Check the scale suites in the unit tests and harden the benchmarks The scale benchmarks failed the forbidden API check on System.out, and nothing outside a manual JMH run checked the scale cases, so a sibling change that resolves a large configuration wrongly would go unnoticed. - Add ScaleSuiteTest: evaluate every case of the scale and scale-json suites twice, and fail when the benchmark misses a case file. The cases, their check and the suite reader move to the test sources and the benchmarks reuse them; the reader uses JsonReading. - Allow System.out in the benchmark setups (@SuppressForbidden). - Check every case again after the measurement, fail on an error code, and reject expectations the check does not handle. - Rerun the jmh task when the fixtures change. - Measure retained heap after a first parse, as the smallest of three signed samples. - Describe a row as the unweighted mean over the cases of a file, say that hooks are not measured and when to run three forks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Pin the fixtures to the latest head of the scale suite PR The fixture PR now checks that the committed scale files match the generator and fixes its docs. The case and configuration files are unchanged, so the benchmark numbers do not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Test that an evaluation parses each SemVer value once A test now counts the SemVer parses of one evaluation's rules, so bypassing the cache fails. New precedence rows cover a numeric prerelease identifier that is not the last one. Core components use Long.parseUnsignedLong on the version's range instead of a hand-written parser. The null-element message test is skipped on JDKs without helpful NullPointerException messages instead of comparing null with null, and the toDouble Javadoc states what it converts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge the #13485 base (09289ff) into the SemVer parse-once branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Benchmark one-of lists of small numbers The fixtures add a case file with ONE_OF lists of the integers 1 to 1000 and of the powers of two up to 2^52, which an index keyed on exact doubles must still match. Pin them and benchmark the file; its 8 cases pass on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge the #13485 base (30df999) into the SemVer parse-once branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Merge master (#13485 squash 4075f1f) into the SemVer parse-once branch Takes master's version of every #13485 file; only this branch's SemVer changes differ from master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Drop the identity check SpotBugs flags in SemverParseCache SpotBugs reports ES_COMPARING_PARAMETER_STRING_WITH_EQ for the `cached == value || cached.equals(value)` fast path. String.equals already returns early on identity, so the explicit == adds nothing; scale/semver time and allocation are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Compare SemVer attributes in place instead of caching their parse An evaluation now validates and compares each SemVer attribute against the pre-parsed comparand in one pass over the string, without building a ParsedSemver. This removes the per-evaluation SemverParseCache and the hasSemverConditions marker on Flag, which only existed to feed that cache. ParsedSemver.compare(String, ParsedSemver) returns exactly compare(parse(version), comparand), or INVALID_ORDERING when parse would reject the version. Tests check that over every pair of the valid, invalid, spec-ordering and edge-case versions, and over seeded random strings. Suggested by Doug Hawkins in the review of #13488. Co-authored-by: Douglas Q Hawkins <douglas.hawkins@datadoghq.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Share SemVer suffix validation and test the in-place compare further parse and compare(String, ParsedSemver) now validate the prerelease and build metadata through one helper, so a grammar change lands in one place. The helper allocates nothing and returns the end of the prerelease. New tests reject non-ASCII digits in both paths, fuzz the suffixes of equal cores, check SemVer precedence through the String overload, cover SEMVER operator edge cases in the evaluator, and catch a truncating numeric conversion. The javadocs now say the attribute is compared without allocating rather than in one pass, and describe how null fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
What Does This Do
Adds JMH benchmarks that measure FFE flag evaluation on the opt-in
scaleandscale-jsonsuites of the shared fixtures, and sets a baseline formaster:ScaleSuiteBenchmarkevaluates the cases of each case file through the provider and reports time and allocation for each evaluation.ScaleSuiteLoadBenchmarkmeasures how long each suite's configuration takes to parse, and prints the heap that the parsed configuration retains.ScaleSuiteTestruns the same cases in the unit tests, so a change that resolves a large configuration wrongly fails CI instead of only a manual benchmark run.This is the base of a stack. Three sibling PRs are stacked on it, and each one reports its before/after numbers against the baseline below:
ONE_OFandNOT_ONE_OFlists when the configuration loadsThis PR has no change to shipped code. The submodule bump also brings in fixture
maincommits that add 8 core cases (one time-bounded holdout flag);CanonicalFixtureTestpasses on all of them.Motivation
We need flag evaluation in Java to win a performance-sensitive customer, and this PR is the foundation: a repeatable measurement for every change that follows. The canonical fixtures are small, so they do not show where an evaluation's cost grows with configuration size: list length, allocation count, rule depth, shard count, regex input length, or JSON value size. The suites in DataDog/ffe-system-test-data#78 cover these dimensions at N and N × M, up to JSON values of 4 MB. This PR runs our evaluator over them, so every performance change shows its effect on the same inputs, on the same machine.
Additional Notes
What the benchmark covers.
@Param casesholds all 17 case files:scale/flags,numeric,one-of,one-of-many,one-of-small-numbers,regex,regex-many,rules,rules-sweep,semver,splits,time-windows,variations,wide-context, andscale-json/json-shapes,json-sizes,json-values. The fixtureevaluation-suites/scale/AGENTS.mddescribes each one.index=all, the operations go through the cases of a file in turn, so a row is the unweighted mean over the 2 to 16 cases of the file, and its slowest case dominates it: thejson-sizesrow is mostly its 4 MB case. A change that targets one case should also report that case with-p index=<n>; the setup prints the index of each case.serialId,doLog, or split features, an error code, or on an expectation the check does not handle, so a run that resolves a case wrongly fails instead of reporting a number. All 122 cases pass onmaster, including the sharded ones.ScaleSuiteTestevaluates every case twice in the unit tests, parsing each suite once. It reads the benchmark's@Paramlist and fails unless that list names every case file of the fixtures.TestSettings.of(), so JVM properties and environment variables, such as the span enrichment flag, cannot change the metadata path being measured.evaluation-suites/regexsuite (dialect and backtracking cases, notscale/regexabove) is not here, because the JDK engine does not finish its backtracking cases. Evaluate feature flag regex conditions in linear time #13487 adds it.Decisions.
masterreturns JSON integers asDouble. Return read-only JSON flag values with exact integers #13484 makes them exact, and the core fixture Add JSON number precision fixture ffe-system-test-data#77 checks the type. Comparing by value lets one benchmark run on every PR in the stack. This check guards the measurement;CanonicalFixtureTestis the conformance check of the types.508a342on the fixtures'main, the merge commit of Add opt-in scale, scale-json, and regex evaluation suites ffe-system-test-data#78. Its suites are the same as the PR head the rows below were measured on. The same commit also brings the core layer cases from Add mutually exclusive layer test fixtures ffe-system-test-data#66, which pass in the unit and smoke tests.main.dd-openfeature. The SDK reads most of the shared fixtures, so theffe-system-test-datasubmodule and the vendoredexperimentation-opt-outcases now live indd-openfeature/src/test/fixtures/. The smoke tests copyufc-config.jsonandevaluation-casesfrom there. The fixtures are a plain directory, not atestFixturessource set: every reader opens them from disk, and a source set would package the 26 MB submodule into jars, including a published variant ofdd-openfeature. Branches stacked on this one need the same path change.jmhtask declares the suites as an input, so a repin reruns the benchmarks instead of reporting the task up to date. The input is a file tree, so a checkout without the submodule initialized can still run every other benchmark.jackson-core, which the module already uses.Baseline on
master. AWS workspace, Intel Xeon Platinum 8175M, 16 vCPUs, JDK 21.0.10. JMH AverageTime, 1 fork, 3 warm-up and 5 measurement iterations of 2 s,-prof gc. Each row is the mean over the cases of the file. The time and allocation figures predate the latest commit, which leaves the measured evaluation unchanged; the retained heap uses the revised method. Theone-of-small-numbersrow and thescaleload row were measured later, after the fixtures added that file and its 2 flags; the earlierscaleparse time was 95.8 ms, and 2 more flags out of 10,030 do not explain the difference, so treat the parse time as within this machine's run-to-run variation. Thejson-sizesrow and thescale-jsonload row were measured again after the fixtures replaced the 10 MB value with a 4 MB one, with three forks while the machine was heavily loaded (load average about 70 on 16 vCPUs): their allocation and retained heap are exact, but compare their times only with runs made back to back on the same machine. Theone-of,one-of-many,rules,rules-sweep,splits,variations, andwide-contextrows were measured again at911924f, after the fixtures added first-match cases to those files (load average about 15): a row's mean covers the new cases, so compare those rows only with runs on the same pin. One fork suits allocation and large time differences; for a time difference under about 15%, run three forks (-Pjmh.forks=3, or-f 3with the JMH jar).scale/flags(10,000 flags)scale/wide-contextscale/time-windowsscale/numericscale/rulesscale/one-of-small-numbersscale/semverscale/regex-manyscale/splitsscale/rules-sweepscale/variationsscale/one-ofscale/one-of-manyscale/regexscale-json/json-valuesscale-json/json-shapesscale-json/json-sizesThrough the client's context. The rows above pass the provider one reused context (
evaluate). The OpenFeature client passes a newLayeredEvaluationContexton each call instead, andevaluateLayeredmeasures that. It adds 48 B per evaluation for most cases. A rule onidcosts much more when the context has noid, because the layered context builds its key set again on each call:scale/wide-contextcaseevaluateevaluateLayeredscale-id-wide-context, match (index 3)scale-id-wide-context, miss (index 4)scalescale-jsonThe stacked PRs address
json-*,one-of*, andregex*. The baseline also shows the next candidates.splitsallocates 34 KB per call (a newMessageDigestand hex string for each shard).semverallocates 16 KB (the attribute is parsed again for each condition).variationsscans 1,000 allocations.Run all case files, with both evaluation methods:
Only the rows of the main table (
evaluate), in about half the time:Run one case:
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]