Per-component / tag cardinality limits in client-side stats by dougqh · Pull Request #11387 · DataDog/dd-trace-java

dougqh · 2026-05-15T22:55:00Z

What Does This Do

Implements per-component / per-tag cadinality limits to improve user experience under high load.

Motivation

Previously, there was a single global cap and per-component / tag caches that helped curtail allocation and unbound live objects, but this approach had a couple problems.

While the aggregate table was sized capped, failure to insert into the aggregate table would lead to silent data loss. There weren't any obvious indications to the customer when metrics were lost.

And under extreme loads, the caches would degenerate to constantly missing and allocating which would result in long GC cycles.

The per-element limiting allows us to substitute a sentinel value to indicate what was dropped and why to the trace agent / backend. Additionally, this change includes logging and metrics to indicate to the user what is happening locally.

Additional Notes

The cardinality handlers introduced in this change serve dual roles. They both track cardinality and act as caches for UTF8 encodings.

By cardinality limiting first, constant allocation from string concatenation and UTF8 encoding is avoided. And given that a cache and cardinality limiter are basically both sets of recently used values, it seemed most efficient to combine them.

The one difference between the cardinality limiter and the cache is that the cardinality limiter is regularly fully reset -- which hurts the use of the limiter as a cache. To make up for that, the cardinality limiter also holds onto the values used in the previous cycle for reuse in the new cycle.

Claude's Summary

Stack: master → #11382 → #11478 → this PR. Bounds client-stats label cardinality, reworks peer-tag handling, renames the aggregator, and adds a design doc. The lazy-errorLatencies memory win that originally lived in this PR's downstream (#11389) was extracted ahead of #11387 into #11478 during a stack resequence, so the per-entry footprint reduction lands independently of the cardinality machinery.

Cardinality control: replaces the per-field DDCaches with PropertyCardinalityHandler / TagCardinalityHandler. Each has a per-field budget; once exhausted, the sentinel-substitution behavior is gated by the new trace.stats.cardinality.limits.enabled flag (default false). With the flag on, overflow values canonicalize to a blocked_by_tracer sentinel UTF8BytesString and collapse to one bucket. With the flag off (the default), the cache size is still capped at the same budget but over-cap values get freshly-allocated UTF8BytesStrings and flow to distinct buckets — so the wire format is identical to Update client-side stats to use light weight Hashtable #11382. Handlers reset every reporting cycle in either mode.
Canonicalize before hashing: AggregateTable.findOrInsert runs every label through its handler before computing the lookup hash, so cardinality-blocked values collapse into one bucket instead of fragmenting into N entries.
Peer tags reworked: producer captures peer-tag values only (parallel String[] to a PeerTagSchema.names array). tag:value interning happens on the aggregator thread via TagCardinalityHandler. The schema is synced once per trace via PeerTagSchema.currentSyncedTo(Set) with an identity-check fast path.
Rename: ConflatingMetricsAggregator → ClientStatsAggregator.
Producer-side wins identified via JFR: use the cached span.kind byte ordinal through a new CoreSpan.getSpanKindString() (skips a tag-map lookup per metrics-eligible span); hoist schema.names out of capturePeerTagValues; avoid toString() allocation in isSynthetic.
Cleanups: fix TracerHealthMetrics.previousCounts size bug that would have silently dropped the new statsInboxFull counter; drop dead clearAggregates().
Docs: new docs/client_metrics_design.md covering the pipeline shape, the canonical-key trick, thread-safety contract, reporting cadence, failure modes, and benchmark numbers.

Benchmark

`ClientStatsAggregatorDDSpanBenchmark` — producer publish() latency

(64 client-kind DDSpans per op, real CoreTracer)

Stage	µs/op
master baseline	6.428
stack tip before this PR	2.454
+ peer-tag schema hoist	2.410
+ cached span-kind ordinal	1.995

~3.2× over master end to end on the producer side.

Aggregator bench suite vs v1.62.0 + master + #11382

Re-measured 2026-05-27 with three benches in matrix form: full adversarial (all four label dimensions vary) and two cardinality-isolation companions (only resource varies; only peer.hostname varies). Same machine state, same JMH config (8 producer threads, 2×15s warmup + 5×15s, 1 fork, throughput mode). The HighCardinality* and Adversarial benches were backported onto the v1.62.0 tag using its ConflatingMetricsAggregator constructor and HealthMetrics.NO_OP (v1.62.0 predates the inbox split so per-iteration drop counters are not directly comparable).

	v1.62.0 release	master (post-#11381)	#11382	#11387 limits OFF (default)	#11387 limits ON
`AdversarialMetricsBenchmark` (ops/s)	444,290 ± 1,616,937	14,276,351 ± 1,091,138	32,556,300 ± 4,321,490	23,480,978 ± 2,221,623	8,068,173 ± 1,754,400
vs v1.62.0	1.00×	32.1×	73.3×	52.9×	18.2×
`HighCardinalityResource` (ops/s)	4,854,335 ± 1,214,233	8,168,005 ± 3,493,716	35,739,452 ± 2,556,684	28,866,978 ± 1,251,950	25,095,814 ± 1,934,690
vs v1.62.0	1.00×	1.68×	7.36×	5.94×	5.17×
`HighCardinalityPeer` (ops/s)	6,902,209 ± 368,641	10,110,142 ± 3,380,594	37,638,634 ± 6,673,337	29,635,631 ± 5,710,512	27,408,255 ± 1,722,131
vs v1.62.0	1.00×	1.46×	5.45×	4.29×	3.97×
Adversarial `onStatsAggregateDropped`	n/a (`HealthMetrics.NO_OP`)	155,251,623	16,568,738	12,336,616	0
Resource `onStatsAggregateDropped`	n/a	188,023,595	16,557,066	9,773,903	0
Peer `onStatsAggregateDropped`	n/a	223,260,962	14,904,938	17,983,372	0

Customer headline: vs the shipping v1.62.0 release, this branch at the default flag setting (limits OFF) delivers ~50× throughput on adversarial cardinality and ~5–6× on single-axis cardinality. With the flag ON (sentinel-substitution active), ~18× / 4–5× plus zero onStatsAggregateDropped — i.e. the cardinality cap actually saves the bench from data loss. v1.62.0's Adversarial per-iteration progression shows the classic degradation curve (1.08M warmup → 277K → 199K) where the LRU cache thrashes catastrophically; this PR holds steady-state across iterations in either flag mode.

Reading the trade-off:

onStatsAggregateDropped = 0 only with limits ON. That's the safety guarantee the feature pays for. Every other config drops 10–225 M aggregate updates under adversarial cardinality because over-cap values fragment into distinct buckets and saturate tracerMetricsMaxAggregates.
Adversarial-bench limits-on cost is real. All four label dimensions exhaust their cap simultaneously, so every snapshot pays the full sentinel-substitution + blockedCounts++ + warnedCardinality bookkeeping on all four fields. Single-axis benches (HighCardinality*) show a much smaller limits-on penalty (~10%) because only one dimension is over-cap. Workloads with one runaway dimension and the rest bounded sit much closer to the limits-off throughput.
Variance collapses dramatically with limits on. ±1.72 M / ±1.75 M / ±1.93 M on limits-on vs ±5–6.67 M without. Bounded cardinality means no eviction sweeps, stable table size, no per-cycle GC churn — predictable throughput. For workloads paged on p99 latency spikes during reporting cycles, this is often more valuable than peak throughput.
Benches are adversarial. Designed to saturate every capacity bound at once; realistic workloads with smaller working sets see proportionally smaller throughput gaps between configs. The 19–66% limits-on penalty vs Update client-side stats to use light weight Hashtable #11382 is an upper bound, not a steady-state cost.

Architecture note on the limits-off cost. Limits-off matches #11382's wire format exactly, but still costs ~19% on HighCardinality* and ~28% on Adversarial. The gap comes from AggregateTable.findOrInsert canonicalizing every snapshot before lookup — required for the sentinel collapse in limits-on, but pure overhead in limits-off where the hash is content-stable across raw vs canonicalized forms. A two-path findOrInsert (hash-raw on limits-off, canonicalize-first on limits-on) would likely close most of the gap; deferred as a follow-up optimization if the default-off cost matters in practice.

Test plan

:dd-trace-core:test — metrics tests pass (existing + new AggregateTableTest cases for cardinality collapse)
JMH benchmark numbers reproduce locally
No behavior change to client-stats wire payload for traces within the cardinality budget

🤖 Generated with Claude Code

Replaces the per-field DDCache layer inside AggregateEntry with the two new cardinality handlers. Each per-field handler holds a small HashMap working set; when its budget is exhausted, subsequent values collapse to a stable "blocked_by_tracer" sentinel UTF8BytesString rather than growing without bound. The handlers are reset on the aggregator thread at the end of each report() cycle (10s default), so the cardinality budget refreshes per reporting interval. Caches replaced (limits preserved from the prior DDCache sizes): RESOURCE_HANDLER 32 SERVICE_HANDLER 32 OPERATION_HANDLER 64 SERVICE_SOURCE_HANDLER 16 TYPE_HANDLER 8 SPAN_KIND_HANDLER 16 HTTP_METHOD_HANDLER 8 HTTP_ENDPOINT_HANDLER 32 GRPC_STATUS_CODE_HANDLER 32 PEER_TAG_HANDLERS per-tag-name TagCardinalityHandler, each 512 Two production-only changes to the handlers as the user wrote them: - Fixed import: datadog.collections.tagmap6lazy.TagMap doesn't exist; TagCardinalityHandler now imports datadog.trace.api.TagMap which has the Entry API the handler uses. - Added TagCardinalityHandler.register(String) overload so AggregateEntry's peer-tag canonicalization doesn't have to allocate a TagMap.Entry per call -- the snapshot already carries peer-tag values as a flattened String[] {name, value, ...}. AggregateEntry split into two construction paths: - forSnapshot(snapshot, agg): the hot path; runs each field through the appropriate handler. - of(...): test-only factory; bypasses the handlers and creates UTF8 instances directly, so tests don't pollute static handler state. Content- equality on the resulting entry still matches the production-built one. Thread-safety: handlers are HashMap-backed and not safe for concurrent access. Both forSnapshot and resetCardinalityHandlers must be called from the aggregator thread. After the prior commits that moved MetricKey construction to the aggregator thread, this is the only thread that canonicalizes; the test factory path runs on test threads but doesn't touch the handlers. Reset semantics: clearing the handler's working set drops the {value -> UTF8BytesString} mapping but doesn't invalidate existing AggregateEntry fields -- those keep their UTF8BytesString references alive on their own. Subsequent snapshots with the same content still resolve to the existing entries via content-equality matches(). New values after reset get freshly allocated UTF8BytesStrings via the handler. Known limitation (not fixed here): hashOf(SpanSnapshot) hashes from the raw snapshot fields, not from the post-handler canonical form. So when cardinality is exceeded, multiple distinct raw values that collapse to the "blocked_by_tracer" sentinel still produce distinct hashes and land in different AggregateEntry buckets -- the wire payload will carry multiple rows that all label as blocked. This is the same behavior the prior DDCache-based design would have had at capacity. Collapsing those into a single sentinel entry would require canonicalizing before hashing and is a follow-up. Tests: new CardinalityHandlerTest covers PropertyCardinalityHandler and TagCardinalityHandler in isolation (hit/miss, over-limit blocking, reset behavior, sentinel stability). Existing ConflatingMetricAggregatorTest / SerializingMetricWriterTest / AggregateTableTest all pass unchanged because the test factory bypasses handlers. Benchmarks (2 forks x 5 iter x 15s) -- producer side unchanged because the handlers live on the consumer thread: SimpleSpan bench: 3.114 +- 0.045 us/op (prior: 3.123 +- 0.018) DDSpan bench: 2.364 +- 0.113 us/op (prior: 2.412 +- 0.022) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The prior commit ran every snapshot through the cardinality handlers but still hashed the raw snapshot fields. When a field exceeded its cardinality budget the handlers collapsed many distinct values to a single "blocked_by_tracer" sentinel, but the raw hashes were still all different -- so the blocked entries fragmented across the AggregateTable. This commit makes hash + match work off the canonical (post-handler) UTF8BytesString fields, so blocked values land in the same bucket and merge into one entry. How the lookup path changes --------------------------- A new package-private AggregateEntry.Canonical scratch buffer: - holds the 10 canonical UTF8BytesString refs, primitives, peerTags list, and the precomputed keyHash; - exposes populate(SpanSnapshot) which runs each field through the appropriate handler and computes the long hash from the canonical refs; - exposes matches(AggregateEntry) for content-equality lookup; - exposes toEntry(AggregateMetric) which copies its refs into a fresh AggregateEntry on miss. AggregateTable holds one Canonical instance and reuses it per findOrInsert. On a hit nothing is allocated -- the buffer's refs feed the bucket walk and matches() directly. On a miss the refs are copied into the new entry and the buffer is overwritten on the next call. Hash function ------------- hashOf now takes UTF8BytesString fields (plus primitives + peerTags list) instead of raw CharSequence/String from the snapshot. UTF8BytesString.hashCode returns the underlying String's hash, so: - content-equal entries built via AggregateEntry.of(...) (test factory, bypasses handlers) produce the same hash as entries built via Canonical.toEntry(...) (production, via handlers); - all values that collapsed to "blocked_by_tracer" share that sentinel instance and therefore that hashCode -- they land in the same bucket and merge into one entry. Matches ------- The SpanSnapshot-keyed matches() on AggregateEntry is gone. Lookup goes through Canonical.matches(entry) which compares the buffer's UTF8 fields against the entry's UTF8 fields via Objects.equals (content equality on UTF8BytesString). This is needed because across handler resets the UTF8BytesString instance referenced by an existing entry differs from the freshly-issued instance for the same content -- content-equality lets the existing entry survive resets. The peerTagPairsRaw field on AggregateEntry was previously kept for matching against snapshot.peerTagPairs (the flat String[]). Canonical.matches uses List.equals on the encoded UTF8 peerTags directly, so peerTagPairsRaw is dropped. New test in AggregateTableTest -- cardinalityBlockedValuesCollapseIntoOneEntry inserts 50 distinct services into a table whose SERVICE_HANDLER has a cardinality limit of 32, and asserts the final size is 33 (the 32 in-budget services plus a single collapsed "blocked_by_tracer" entry, not 50 separate entries). Benchmarks (2 forks x 5 iter x 15s) -- producer side unchanged: SimpleSpan bench: 3.117 +- 0.026 us/op (prior: 3.114 +- 0.045) DDSpan bench: 2.344 +- 0.114 us/op (prior: 2.364 +- 0.113) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…chema-indexed handlers Replaces the producer's early {@code (name, value)}-pair encoding with a schema-based design: peer-tag values are captured into a parallel String array, and the consumer applies the matching {@link TagCardinalityHandler} by index using a {@link PeerTagSchema}'s parallel name/handler arrays. This removes the {@code Map<String, TagCardinalityHandler>} the prior commit left in {@code AggregateEntry} -- handler lookup is now a single array dereference instead of a hashmap probe. PeerTagSchema ------------- New package-private class that holds: - {@code String[] names} -- peer-tag names in stable order - {@code TagCardinalityHandler[] handlers} -- parallel to names Two schemas exist: a static singleton {@code INTERNAL} for the internal-kind {@code base.service} case, and a {@code CURRENT} schema for the peer- aggregation kinds (client/producer/consumer) that lazily refreshes when {@code features.peerTags()} returns a different set of names. Each {@link SpanSnapshot} captures the schema reference it was built against so producer and consumer agree on the indexing even if {@code CURRENT} changes between capture and consumption. A fast-path identity check (cached last input Set instance) keeps the {@code currentSyncedTo} call cheap: when the producer hands in the same Set instance as last time -- the steady-state case -- {@code currentSyncedTo} returns immediately without iterating names. The {@code matches()} loop only runs when the Set instance changes, which in production is rare (only on remote-config reconfiguration). Snapshot shape -------------- {@code SpanSnapshot.peerTagPairs} (a flat {@code [name0, value0, name1, value1, ...]} array) is replaced by: - {@code PeerTagSchema peerTagSchema} -- nullable; schema for the values - {@code String[] peerTagValues} -- parallel to schema.names The producer captures only values; the consumer constructs the encoded {@code "name:value"} UTF8 forms via {@code schema.handler(i).register(value)} on its own thread. Consumer-side cleanups bundled in --------------------------------- While here, also addresses the perf review items raised against the prior commit: - {@code hashOf}'s peer-tag loop is now indexed iteration; no more iterator allocation per snapshot. - {@code Canonical} now owns a reusable {@code peerTagsBuffer} ArrayList that's cleared+refilled per {@code populate} call -- zero allocation on the hit path. The buffer is copied into an immutable list only on miss when the entry needs to own it long-term. - {@code Canonical.matches} uses indexed list comparison; no iterator alloc in {@code List.equals}. - The {@code HashMap<String, TagCardinalityHandler> PEER_TAG_HANDLERS} on {@code AggregateEntry} is gone, replaced by the {@link PeerTagSchema}'s parallel array layout. Benchmark (2 forks x 5 iter x 15s) ---------------------------------- SimpleSpan bench: 3.165 +- 0.032 us/op (prior: 3.117 +- 0.026) DDSpan bench: 2.727 +- 0.018 us/op (prior: 2.344 +- 0.114) Some producer-side regression from the per-snapshot schema sync (volatile read + identity check). The fast-path identity comparison keeps it small; hoisting the sync out of the per-snapshot loop is possible but would change behavior in the edge case where {@code features.peerTags()} returns different Sets within a single trace (covered by an existing test). Choosing correctness over the marginal speedup. Tests ----- AggregateTableTest's snapshot builder is updated to construct a schema + values via {@code PeerTagSchema.currentSyncedTo}, exercising the same code path as production. Existing peer-tag test in {@code ConflatingMetricAggregatorTest} still passes unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The "Conflating" in the name dates from the prior design that used a Batch pool + pending map to conflate up to 64 hits per inbox slot. That mechanism is gone -- the producer now publishes one SpanSnapshot per span and the consumer's AggregateTable is the conflation point. The new name matches the existing protocol/metric terminology (HealthMetrics.onClientStat*, stats.flush_payloads, etc.). File renames: ConflatingMetricsAggregator.java -> ClientStatsAggregator.java ConflatingMetricAggregatorTest.groovy -> ClientStatsAggregatorTest.groovy ConflatingMetricsAggregatorBenchmark -> ClientStatsAggregatorBenchmark ConflatingMetricsAggregatorDDSpan* -> ClientStatsAggregatorDDSpan* Plus all symbol references in MetricsAggregatorFactory and the test fixtures that referenced the old class name. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Three small follow-ups carried over from a /techdebt pass: - TracerHealthMetrics: previousCounts array was sized 51, but the prior commits added a 52nd reporter (statsInboxFull). Without this fix the new counter's report() call would throw ArrayIndexOutOfBoundsException; the Flush task swallows that exception, so the failure would be silent (statsInboxFull would just never make it to statsd). - Aggregator: removes the now-dead public clearAggregates() method. The ClearSignal route from ClientStatsAggregator.disable() supplanted it several commits ago; the method had no remaining callers. - TagCardinalityHandler: removes the unused register(TagMap.Entry) overload and its isValidType helper. The String-keyed overload covers all current callers (AggregateEntry's peer-tag canonicalization). - PeerTagSchema: spotless-driven javadoc reflow only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

ClientStatsAggregator.publish was calling features.peerTags() + PeerTagSchema.currentSyncedTo for every span. Peer-tag configuration is stable for the duration of a single trace publish in production -- DDAgentFeaturesDiscovery returns the same Set instance until remote-config reconfiguration -- so the per-snapshot sync is wasted work. Move the sync to once per publish(trace) and pass the resolved schema to the inner publish(span, isTopLevel, peerAggSchema). INTERNAL-kind spans still use the static PeerTagSchema.INTERNAL regardless. Behavior boundary ----------------- Schema changes from features.peerTags() now take effect at the next publish(trace) call rather than mid-trace. Production-equivalent (a trace takes microseconds to milliseconds; remote-config refreshes are seconds apart), but a Spock test that used `>>> [...]` to mock different peerTags() returns on successive calls within one trace no longer makes sense in the new model. That test is rewritten to assert the production-relevant case: peer-tag NAMES are stable, peer-tag VALUES vary per span, distinct value combinations produce distinct aggregate buckets. Benchmark (2 forks x 5 iter x 15s) ---------------------------------- SimpleSpan bench: 3.133 +- 0.057 us/op (prior: 3.165 +- 0.032) DDSpan bench: 2.454 +- 0.082 us/op (prior: 2.727 +- 0.018) Recovers ~270 ns/op on the DDSpan bench -- most of the regression introduced by the per-snapshot lookup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

JFR profiling showed ~21% of producer CPU time spent in tag-map lookups during ClientStatsAggregator.publish. One of those lookups -- span.kind -- is redundant because DDSpanContext already caches the kind as a byte ordinal that resolves to a String via a small array. - Add CoreSpan.getSpanKindString() with a default that falls back to the tag map for non-DDSpan impls; DDSpan overrides to delegate to the context's cached resolution. - Hoist schema.names array out of the capturePeerTagValues loop. - Avoid an unnecessary toString() in isSynthetic by declaring SYNTHETICS_ORIGIN as String and using contentEquals. Benchmark (ClientStatsAggregatorDDSpanBenchmark): before: 2.410 us/op after: 1.995 us/op (~17% improvement) vs. master baseline (6.428 us/op): now ~3.2x faster. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Captures the producer/consumer split, the canonical-key trick that makes cardinality-blocking actually save space, the once-per-trace peer-tag schema sync, the role of each file in datadog.trace.common.metrics, and the rationale behind the redesign from ConflatingMetricsAggregator. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Monotonically increases each time the discovered peerTags Set differs from the previous one. Lets callers detect peer-tag config changes with a long compare instead of a Set.equals (or leaning on Set-identity, which was an implementation accident, not part of the public contract). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ggregator PeerTagSchema previously held its current schema + last-synced-Set in static volatile fields with a synchronized rebuild. The "is it stale?" signal was an identity check on the Set instance returned by features.peerTags() -- a correct but indirect reading of a DDAgentFeaturesDiscovery invariant. Replace that with: - ClientStatsAggregator keeps its own (volatile PeerTagSchema, volatile long cachedPeerTagsRevision) cache pair, rebuilt under synchronized when the revision returned by features.peerTagsRevision() doesn't match. - PeerTagSchema becomes a pure data holder: static factory PeerTagSchema.of, the INTERNAL singleton, and an instance resetCardinalityHandlers(). The static CURRENT, LAST_SYNCED_INPUT, and the synchronized rebuild block are gone. - Aggregator gains a Runnable onResetCardinality hook fired right after AggregateEntry.resetCardinalityHandlers(). ClientStatsAggregator wires it to reset its cached schema's handlers each report cycle. - AggregateEntry.resetCardinalityHandlers() resets PeerTagSchema.INTERNAL directly instead of the removed PeerTagSchema.resetAll(). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

datadog-prod-us1-3 · 2026-05-19T20:03:45Z

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 9 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-java | agent_integration_tests

🔧 Fix in code (Fix with Cursor).
4 failed tests with IllegalAccessError at MetricsIntegrationTest.groovy:44.

DataDog/apm-reliability/dd-trace-java | spotless

🔧 Fix in code (Fix with Cursor).
Spotless format violations in src/main/java/datadog/trace/api/Config.java. Please run './gradlew spotlessApply' to fix.

DataDog/apm-reliability/dd-trace-java | check_smoke 1/4

🔄 Retry job. This looks flaky and may succeed on retry.
Could not read workspace metadata from /go/src/github.com/DataDog/apm-reliability/dd-trace-java/.gradle/caches/8.14.5/groovy-dsl/00559432caf37d0d481f5cc567dc28bc/metadata.bin due to unexpected EOF.

View all 9 failed jobs.

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 690b796 | Docs | Datadog PR Page | Give us feedback!}

Pulls in the util-hashtable design pass (MAX_RATIO, insertHeadEntry keyHash overload, MutatingTableIterator, Entry.next privatization, getOrCreate, MAX_BUCKETS rename, doc tightening, etc.) and the AggregateTable simplifications that came with it. Reconciliation notes: - AggregateEntry / AggregateMetric: this branch keeps the split design (AggregateEntry holds labels + an AggregateMetric counter ref). The optimize-metric-key branch had collapsed them into a single AggregateEntry. Resolution: keep the split (HEAD's design) -- it's more recent and supports the cardinality-canonicalization layer. - AggregateEntryTest.java (new in optimize-metric-key, exercises the collapsed design) deleted; AggregateMetricTest.groovy stays as the counter-side coverage for the split design. - AggregateTable: apply the optimize-metric-key cleanups on top of the Canonical-pattern findOrInsert -- Support.create(n, MAX_RATIO), Support.bucket for the chain head, Support.insertHeadEntry(keyHash), Support.mutatingTableIterator for evictOneStale and expungeStaleAggregates, Support.forEach for forEach. Also add the context-passing forEach overload to match the BiConsumer the Aggregator already uses. - Aggregator.report: keep the BiConsumer + context lambda (non-capturing); body adapted to entry.aggregate.clear() for the split design. - Aggregator.Drainer: keep AggregateMetric as the findOrInsert return type (matches the table's actual signature). - SerializingMetricWriter, SerializingMetricWriterTest, ClientStatsAggregatorTest, AggregateTableTest, SpanSnapshot, MetricWriter: restore HEAD's versions where the auto-merge had taken the optimize-metric-key shape (counters via entry.* vs entry.aggregate.*) -- HEAD's versions match this branch's design. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Adopts the optimize-metric-key design choice: one entry type that holds both the canonical label fields and the counter / histogram state. The prior split (AggregateMetric for counters, AggregateEntry for labels) required every counter read to hop through entry.aggregate -- ~30 sites across SerializingMetricWriter, the Aggregator, and the test suites. - AggregateEntry now owns ERROR_TAG, TOP_LEVEL_TAG, the okLatencies and errorLatencies histograms, hitCount/errorCount/topLevelCount/duration counters, and the recordOneDuration / recordDurations / clear methods that used to live on AggregateMetric. - AggregateMetric.java and AggregateMetricTest.groovy deleted. - AggregateTable.findOrInsert now returns AggregateEntry (not the inner AggregateMetric); Canonical.toEntry no longer takes an AggregateMetric arg. - Aggregator.Drainer reverts to AggregateEntry; the report lambda calls entry.clear() directly. - SerializingMetricWriter, ClientStatsAggregator imports, and all three test files updated to read counters from entry.* (not entry.aggregate.*). - AggregateEntryTest.java added with the recordOneDuration / recordDurations / clear coverage that AggregateMetricTest.groovy used to provide. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- Replace `// nullable` comments on AggregateEntry's 4 nullable label fields (entry + Canonical scratch buffer) with `@Nullable` annotations. Also annotate the matching getters and of(...) factory parameters. - Move the cache revision into PeerTagSchema as a final field (peerTagsRevision), built via PeerTagSchema.of(names, revision). One field on the schema carries the cache key, so the hot path is a single volatile read + long compare against schema.peerTagsRevision -- no separate cachedPeerTagsRevision field on ClientStatsAggregator. When peer tags are unconfigured the cache stores an empty schema (size 0) carrying the revision rather than null, so subsequent publishes still short-circuit on the fast path. peerTagSchemaFor treats `schema.size() == 0` as "skip peer-agg processing" for client/producer/consumer kinds. INTERNAL is built with a -1L sentinel revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Reset was split between two owners: Aggregator.report called AggregateEntry.resetCardinalityHandlers (static handlers + INTERNAL) then ran a separate onResetCardinality callback that ClientStats wired up to reset its cached non-INTERNAL peer-agg schema. Anyone adding a new handler had to know which side to put it on. Make the callback the only entry point. ClientStatsAggregator. resetCardinalityHandlers (renamed from resetCachedPeerAggSchema) now calls AggregateEntry.resetCardinalityHandlers() itself plus the cached peer-agg schema reset. Aggregator.report just runs the callback -- it no longer knows about AggregateEntry's static state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Each handler is now typed to its SpanSnapshot field type, so the HashMap's key class has well-defined equals/hashCode rather than the abstract CharSequence interface. For String-typed fields (service, spanKind, httpMethod, httpEndpoint, grpcStatusCode) the cache hits reliably. For CharSequence-typed fields (resource, operationName, serviceSource, type) consistency still depends on the producer returning a single concrete class per field -- a pre-existing runtime contract -- but the type system now prevents call sites from accidentally passing a different shape. registerOrEmpty is now generic so it threads T through. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Previously, reset() cleared the only cache, so every reporting cycle re-allocated UTF8BytesString instances for every property value seen again. Sustained allocations on the aggregator thread proportional to the sum of per-field cardinality limits, ~bytes/sec, on every reset. Split the state in two: - seenThisCycle (HashSet<T>): consumed-budget tracking, cleared on reset(). - utf8Cache (LinkedHashMap in access-order, 2x cardinalityLimit): long-lived; survives reset; LRU eviction once full. Workloads with stable value sets pay zero UTF8 allocations after the first cycle. The reused instances also short-circuit downstream equals to identity comparisons. Drops the TODO at the prior allocation site. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The 9 property limits and the peer-tag value limit were sprinkled inline. Pull them into a single class with per-field javadoc so the sizing rationale lives in one place. Six values change from the DDCache-inherited defaults based on workload analysis: - RESOURCE 32 -> 128 (highest-cardinality field; tight today) - HTTP_ENDPOINT 32 -> 64 (same shape as RESOURCE for HTTP-heavy) - TYPE 8 -> 16 (DDSpanTypes catalogue is ~30) - HTTP_METHOD 8 -> 16 (WebDAV/custom verbs push past 8) - SPAN_KIND 16 -> 8 (OTel defines exactly 5 standard kinds) - GRPC_STATUS 32 -> 24 (gRPC spec has exactly 17 codes) SERVICE, OPERATION, SERVICE_SOURCE, and PEER_TAG_VALUE keep their current values. Net worst-case memory delta: roughly +90 KB driven by the RESOURCE and HTTP_ENDPOINT bumps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The doc described an old design where the producer thread per-trace read a peerTagsRevision() and rebuilt the cached PeerTagSchema under a monitor. The actual implementation (cascaded from #11381) runs reconcile once per report cycle on the aggregator thread via the onReportCycle hook, keyed on getLastTimeDiscovered(). Producers do nothing more than a volatile read of the cached schema. Updates: - Producer-side flow: drop the per-trace sync description; document the volatile-read steady state and the one-time synchronized bootstrap on first publish. - New "Aggregator-side reconcile" section under "Reporting cadence and cardinality reset" describing the timestamp fast path, the same-tags slow path that preserves warm handlers, and the read-order race fix (timestamp before names). - Memory and lifetime: replace peerTagsRevision pairing with the on-schema lastTimeDiscovered + per-aggregator-instance lifecycle. - "Why the redesign" point 6: rewritten to describe the aggregator- thread reconcile rather than the producer-side revision check. Resolves dougqh's open review thread about peerTagsRevision vs lastTimeDiscovered. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Both PropertyCardinalityHandler and TagCardinalityHandler linear-probe on (value.hashCode() & capacityMask). Without a spreader, inputs that share a low-bit pattern (e.g. URL templates with a common prefix, or String.hashCode values clustered around 0 for short strings) collapse onto the same probe chain. With the load factor capped at 0.5 the chain length is bounded but can still grow under pathological inputs. Mixing the input hash with its upper half (h ^ (h >>> 16)) before masking spreads the high bits down, same trick HashMap.hash uses. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Pure formatting -- google-java-format reflows of Javadoc paragraph breaks and parameter wrapping. No behavior change. Picked up from a prior session's spotlessApply that wasn't bundled into the relevant commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… dougqh/control-tag-cardinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java # dd-trace-core/src/main/java/datadog/trace/common/metrics/ClientStatsAggregator.java # dd-trace-core/src/main/java/datadog/trace/common/metrics/PeerTagSchema.java # dd-trace-core/src/test/groovy/datadog/trace/common/metrics/ClientStatsAggregatorTest.groovy # dd-trace-core/src/test/java/datadog/trace/common/metrics/AggregateEntryTest.java # dd-trace-core/src/test/java/datadog/trace/common/metrics/PeerTagSchemaTest.java

Spotless tidy missed in commit b382df5. No semantic change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… dougqh/control-tag-cardinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java # dd-trace-core/src/test/groovy/datadog/trace/common/metrics/ClientStatsAggregatorTest.groovy

… dougqh/control-tag-cardinality

dd-octo-sts · 2026-05-26T18:31:28Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Startup Time

Scenario	This PR	master	Change
insecure-bank / iast	14,157 ms	14,022 ms	+1.0%
insecure-bank / tracing	12,950 ms	12,986 ms	-0.3%
petclinic / appsec	16,508 ms	15,465 ms	+6.7%
petclinic / iast	15,513 ms	16,515 ms	-6.1%
petclinic / profiling	16,380 ms	16,459 ms	-0.5%
petclinic / tracing	15,804 ms	15,783 ms	+0.1%

Commit: 690b7963 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

… dougqh/control-tag-cardinality

… dougqh/control-tag-cardinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java # dd-trace-core/src/test/groovy/datadog/trace/common/metrics/ClientStatsAggregatorTest.groovy # dd-trace-core/src/test/java/datadog/trace/common/metrics/AggregateEntryTest.java

Mirrors the #11382 cleanup. Production AggregateEntry never invokes equals/hashCode -- AggregateTable bucketing goes through keyHash + Canonical.matches directly. The contract existed only to support Spock mock argument matchers. - Delete equals/hashCode from production AggregateEntry; class stays final. - Add src/test AggregateEntryTestUtils.equals/hashCode that implements the same field-wise contract (peerTags compared as an encoded list, consistent with hashOf on this branch). - Update Spock argument matchers from `writer.add(AggregateEntry.of(...))` to `writer.add({ AggregateEntryTestUtils.equals(it, AggregateEntry.of(...)) })`. - For loop-driven expectations, hoist the fixture into a per-iteration `def expected = ...` local so it's captured by value rather than by reference to the loop variable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… dougqh/control-tag-cardinality # Conflicts: # dd-trace-core/src/test/groovy/datadog/trace/common/metrics/ClientStatsAggregatorTest.groovy # dd-trace-core/src/test/java/datadog/trace/common/metrics/AggregateEntryFixtures.java # dd-trace-core/src/test/java/datadog/trace/common/metrics/AggregateEntryTestUtils.java

… dougqh/control-tag-cardinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java

- Drop unused PeerTagSchema.hashCode/equals + cachedHashCode field; the schema is never compared via Object.equals or used as a Set/Map key. hasSameTagsAs(Set<String>) is the only schema-equivalence primitive in use, and it's a name-content check that doesn't go through hashCode. - Canonical.populatePeerTags now skips null values directly instead of round-tripping them through handler.register only to filter EMPTY back out. Cheap win on sparse-null peer-tag arrays from capturePeerTagValues. - Canonical.matches reordered to compare highest-cardinality fields (resource, service, operationName) first for faster short-circuit on bucket-chain collisions; UTF8 fields use direct .equals (the EMPTY-sentinel invariant guarantees non-null on both sides) instead of Objects.equals + dead null branches. - PropertyCardinalityHandler/TagCardinalityHandler.register compute the mixed hash once and reuse the start index for both the current-cycle and prior-cycle probe paths. The probe helper is inlined since it's no longer shared. - ClientStatsAggregator.reconcilePeerTagSchema now flushes the outgoing schema's accumulated blockedCounts via resetCardinalityHandlers() before discarding it on a tag-set change, otherwise partial-cycle block telemetry would silently disappear. - Document the "one ClientStatsAggregator per JVM" invariant implied by AggregateEntry's static cardinality handlers and PeerTagSchema.INTERNAL, plus the load-bearing size guard in TagCardinalityHandler.isBlockedResult that prevents accidental sentinel materialization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…dinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateTable.java

dougqh · 2026-05-27T12:55:54Z


  private synchronized void discoverIfOutdated(final long maxElapsedMs) {
    final long now = System.currentTimeMillis();
-    final long elapsed = now - discoveryState.lastTimeDiscovered;


Claude, this change is unnecessary

…matches Optional fields on AggregateEntry (serviceSource, httpMethod, httpEndpoint, grpcStatusCode) carry UTF8BytesString.EMPTY when the snapshot had no value. SerializingMetricWriter (and its test) previously read this contract via reference-eq against EMPTY, leaking the storage choice into callers. Add hasServiceSource / hasHttpMethod / hasHttpEndpoint / hasGrpcStatusCode predicates on AggregateEntry; switch the four serializer sites and the mirroring four test sites to call them. The EMPTY-sentinel is now an internal implementation detail of AggregateEntry. Reorder AggregateEntry.hashOf so its field order mirrors Canonical.matches (UTF8 fields first, then peer-tag list, then primitives). The hash value itself is order-stable across all callers; this is purely so future readers can reason about lookup and equality in lockstep. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

When peer-tag discovery returns a different tag set than the cached schema carries, ClientStatsAggregator.reconcilePeerTagSchema replaces the schema. Previously every per-tag TagCardinalityHandler was rebuilt from scratch, losing the prior-cycle UTF8 cache for tags that survived the rebuild -- so a persistent tag like peer.hostname re-allocated UTF8BytesStrings for every value on the cycle following the rebuild. Split PeerTagSchema.resetCardinalityHandlers into two operations: - resetCardinalityHandlers: full rotate (called at end-of-cycle on the cached schema). - flushBlockedCounts: telemetry-only flush (used by reconcile on the outgoing schema before discard, so partial-cycle counts still reach HealthMetrics without disturbing the handlers). Add a donor overload PeerTagSchema.of(names, state, healthMetrics, previous) that transfers TagCardinalityHandler instances by name for any tag present in both the previous and replacement schemas. Names absent from the previous schema get fresh handlers; names absent from the replacement schema are dropped along with the outgoing schema. reconcilePeerTagSchema now calls flushBlockedCounts then constructs the replacement via the donor overload. The end-of-cycle reset that runs immediately after reconcile rotates the (now-transferred) handlers in the normal way, so cycle N+1 sees cycle N's UTF8 values as priorValues for persisting tags. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ebuilds" This reverts commit 703c9e1.

Cardinality limiting alters the wire format under high cardinality -- overflow values get the "blocked_by_tracer" sentinel and collapse into one aggregate bucket. Customers with dashboards or alerts keyed on specific tag values would see the sentinel value appear unexpectedly if their workload exceeds the per-field budgets. Put the substitution behavior behind a config flag so the rollout is opt-in. With the flag off (the new default), the per-field handlers still act as bounded UTF8 reuse caches sized by their cardinality budget, but over-cap values get a freshly-allocated UTF8BytesString instead of the sentinel and flow to distinct aggregate buckets. The wire format is unchanged from master for any workload. With the flag on, current behavior (sentinel substitution + bucket collapse). Handler refactor: - PropertyCardinalityHandler / TagCardinalityHandler take a useBlockedSentinel constructor arg. On cap-exhaust the cache no longer claims a slot (since it's full), but prior-cycle reuse still runs so repeat over-cap values pay only the probe, not the allocation. - TagCardinalityHandler.isBlockedResult now reads cacheBlocked directly rather than calling blockedByTracer(), so query-time never forces the sentinel to materialize when limits are disabled. - Test-convenience single-arg constructors default useBlockedSentinel to true so existing tests of the limits-enabled mode don't churn; new tests cover the disabled mode. Wiring: - AggregateEntry exposes static final LIMITS_ENABLED read from Config.get() at class init, threaded through all PropertyCardinality Handlers and the PeerTagSchema-built TagCardinalityHandlers. Config: - GeneralConfig.TRACE_STATS_CARDINALITY_LIMITS_ENABLED (default false) - Config.isTraceStatsCardinalityLimitsEnabled() - Registered in metadata/supported-configurations.json Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Three small doc additions calling out behavior that's correct in code but easy to miss on a cold read: - AggregateTable.evictOneStale: explain that this is no longer just a pathological-case backstop. With LIMITS_ENABLED=false (the new default), over-cap values flow to distinct buckets and maxAggregates becomes the load-bearing cardinality enforcement -- the cursor- resumed scan was added for this regime. - AggregateEntry.LIMITS_ENABLED: document the over-cap repeat tradeoff in disabled mode (over-cap values can't promote into the current cache so repeats re-allocate) and the class-init caveat (static final read of Config, frozen for the JVM at first class load -- tests needing to exercise the limits-on path through the static handlers must construct handlers directly with explicit useBlockedSentinel args). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…dinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateTable.java # dd-trace-core/src/main/java/datadog/trace/common/metrics/PeerTagSchema.java # dd-trace-core/src/test/java/datadog/trace/common/metrics/AggregateEntryTest.java # dd-trace-core/src/test/java/datadog/trace/common/metrics/AggregateEntryTestUtils.java # dd-trace-core/src/test/java/datadog/trace/common/metrics/PeerTagSchemaTest.java

…dinality

…rdinality

dougqh added type: enhancement Enhancements and improvements tag: performance Performance related changes tag: no release notes Changes to exclude from release notes comp: metrics Metrics tag: ai generated Largely based on code generated by an AI or LLM labels May 15, 2026

dougqh commented May 15, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/common/metrics/PeerTagSchema.java Outdated

dougqh mentioned this pull request May 16, 2026

Memory-efficiency pass on ClientStatsAggregator + adversarial benchmark #11389

Draft

5 tasks

dougqh force-pushed the dougqh/optimize-metric-key branch from 050a998 to 3738c85 Compare May 18, 2026 19:20

dougqh and others added 8 commits May 18, 2026 15:24

dougqh force-pushed the dougqh/control-tag-cardinality branch from 8020ec4 to 1221b2b Compare May 18, 2026 19:25

dougqh and others added 2 commits May 19, 2026 15:43

dougqh commented May 19, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java Outdated

dougqh and others added 3 commits May 19, 2026 16:44

Remove accidentally-staged .claude/worktrees entries

ece78c9

dougqh commented May 19, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/common/metrics/ClientStatsAggregator.java Outdated

dougqh and others added 5 commits May 19, 2026 20:02

dougqh and others added 3 commits May 21, 2026 17:37

dougqh mentioned this pull request May 22, 2026

Defer MetricKey construction to the aggregator thread #11381

Merged

6 tasks

dougqh and others added 2 commits May 22, 2026 14:13

Reflow reconcilePeerTagSchema Javadoc after merge

e4ed029

Spotless tidy missed in commit b382df5. No semantic change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

dougqh mentioned this pull request May 26, 2026

Add span-derived primary tags (CSS v1.3.0) #11402

Draft

6 tasks

dougqh added 2 commits May 26, 2026 12:06

Merge remote-tracking branch 'origin/dougqh/optimize-metric-key' into…

a6b7854

… dougqh/control-tag-cardinality

dougqh and others added 6 commits May 26, 2026 16:02

Merge remote-tracking branch 'origin/dougqh/optimize-metric-key' into…

3d8cccc

… dougqh/control-tag-cardinality

Merge remote-tracking branch 'origin/dougqh/optimize-metric-key' into…

ad01d7d

… dougqh/control-tag-cardinality

Merge remote-tracking branch 'origin/dougqh/optimize-metric-key' into…

49124e5

… dougqh/control-tag-cardinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java

dougqh mentioned this pull request May 26, 2026

Update client-side stats to use light weight Hashtable #11382

Open

5 tasks

dougqh and others added 2 commits May 27, 2026 08:20

Merge branch 'dougqh/optimize-metric-key' into dougqh/control-tag-car…

553bfbd

…dinality # Conflicts: # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateEntry.java # dd-trace-core/src/main/java/datadog/trace/common/metrics/AggregateTable.java

dougqh commented May 27, 2026

View reviewed changes

dougqh and others added 7 commits May 27, 2026 09:06

Revert "Preserve warm TagCardinalityHandlers across peer-tag schema r…

92a9af3

…ebuilds" This reverts commit 703c9e1.

Merge branch 'dougqh/optimize-metric-key' into dougqh/control-tag-car…

c48399d

…dinality

dougqh mentioned this pull request May 27, 2026

Lazy-allocate error latency histogram on AggregateEntry #11478

Open

2 tasks

Merge branch 'dougqh/lazy-error-latencies' into dougqh/control-tag-ca…

690b796

…rdinality

dougqh changed the base branch from dougqh/optimize-metric-key to dougqh/lazy-error-latencies May 27, 2026 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Per-component / tag cardinality limits in client-side stats#11387

Per-component / tag cardinality limits in client-side stats#11387
dougqh wants to merge 59 commits into
dougqh/lazy-error-latenciesfrom
dougqh/control-tag-cardinality

dougqh commented May 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

datadog-prod-us1-3 Bot commented May 19, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts Bot commented May 26, 2026 •

edited

Loading

Uh oh!

dougqh May 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

dougqh commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Claude's Summary

Benchmark

ClientStatsAggregatorDDSpanBenchmark — producer publish() latency

Aggregator bench suite vs v1.62.0 + master + #11382

Test plan

Uh oh!

Uh oh!

datadog-prod-us1-3 Bot commented May 19, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

dougqh May 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

dougqh commented May 15, 2026 •

edited

Loading

`ClientStatsAggregatorDDSpanBenchmark` — producer publish() latency

datadog-prod-us1-3 Bot commented May 19, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

dd-octo-sts Bot commented May 26, 2026 •

edited

Loading