Skip to content

profiling(ddprof): migrate context bridge to the all-native API (Phase 2)#11899

Draft
rkennke wants to merge 2 commits into
masterfrom
rkennke/profiler-all-native-context-phase2
Draft

profiling(ddprof): migrate context bridge to the all-native API (Phase 2)#11899
rkennke wants to merge 2 commits into
masterfrom
rkennke/profiler-all-native-context-phase2

Conversation

@rkennke

@rkennke rkennke commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What

Migrates the profiler context bridge off the deprecated DirectByteBuffer (DBB) context API onto java-profiler's all-native API (setTraceContext / clearTraceContext / setContextValue / clearContextValue).

This is Phase 2 of the all-native context work (Phase 1 = DataDog/java-profiler#631).

Why

  • Fixes the virtual-thread use-after-free: the DBB path cached a DirectByteBuffer over the native OTEP record; on a carrier that a mounted virtual thread later migrated off, the record was freed while the cached buffer kept being written. The all-native API resolves the current carrier's record inside each JNI call — no cached buffer to dangle.
  • Per-activation perf: activation collapses from setContext + 2×setContextValue (3 JNI calls) into one setTraceContext (~28% faster on the combined cycle per the design-note benchmark).

Changes

  • DatadogProfilingIntegration.activate → one setTraceContext(...) carrying trace/span context + operation & resource attributes; close/clearContextclearTraceContext().
  • DatadogProfiler: setContextValue/clearContextValue/reapplyAppContext/syncNativeAppContext are now all-native. reapplyAppContext uses a per-slot native setContextValue loop — native setContextValue publishes the record (valid=1) even with no active span, so app context stays visible between spans (preserves Restore app context attributes wiped by ddprof setContext on span activation #11646). A native batch reapply is deferred to a measured follow-up (java-profiler PROF-15361).
  • AppContextSnapshot simplified to strings-only (the native path resolves each value's encoding via the process-wide cache — no cached id/utf8/snapshotTags DBB read).
  • snapshot() uses the new native copyContextTags read (no ThreadContext/DBB, so it observes native writes without resetting the record).
  • ContextSetter kept only for offsetOf + size (pure Java); no DBB usage remains.

Behavior notes

  • Per-slot reapply is not atomic across app attributes. reapplyAppContext publishes each app-managed attribute with its own native setContextValue (detach→attach), so a profiler sample landing mid-reapply can observe a partial app-attribute set on activation — a window the old batch fast-path (setContextValuesByIdAndBytes, single attach) did not have. This is deliberate for the expected small app-attribute cardinality (0–2); the single-call batch reapply is the measured follow-up (PROF-15361). Trace/span + operation/resource stay atomic (one setTraceContext).
  • Defensive zero-span guard. The native setTraceContext rejects spanId==0 (it is the activation path; clearing is clearTraceContext). Span ids are non-zero by construction (IdGenerationStrategy never yields 0, DDSpanId.ZERO means "no span", and DDSpanContext is the only ProfilerContext), so the bridge routes a stray zero span to clearTraceContext() rather than letting the IllegalArgumentException be swallowed over stale context.

Dependency / how to build

Requires java-profiler with the all-native API — DataDog/java-profiler#631 (ddprof ≥ the Phase-1 release). Until that ships, build/test against a local snapshot:

# in java-profiler (PR #631 branch):
./gradlew publishToMavenLocal          # com.datadoghq:ddprof:1.47.0-SNAPSHOT
# here:
./gradlew :dd-java-agent:agent-profiling:profiling-ddprof:test -PddprofUseSnapshot=true

Status / verification

  • Draft — gated on the java-profiler Phase-1 release (Start implementation of new tracer API #631) so the ddprof pin can move to a published 1.47.0. CI here stays red until then because it compiles against the pinned 1.46.0, which predates the all-native API (cannot find symbol on the new calls) — not a code defect.
  • Verified GREEN on Linux against local 1.47.0-SNAPSHOT:
    • ddprof unit suites DatadogProfilerTest / DatadogProfilerConfigTest / DatadogProfilerRecordingTest51 tests, skipped=0, 0 failures.
    • dd-smoke-tests:profiling-integration-tests (CodeHotspotsTest, JFRBasedProfilingIntegrationTest) — 20 tests, 0 failures (end-to-end agent + profiler + JFR, exercising span/op/resource context).

🤖 Generated with Claude Code

rkennke and others added 2 commits July 9, 2026 16:40
Switch the profiler context bridge off the deprecated DirectByteBuffer (DBB)
context API onto java-profiler's all-native API (setTraceContext /
clearTraceContext / setContextValue / clearContextValue). This eliminates the
virtual-thread use-after-free (the DBB cached buffer that dangled on carrier
migration) and folds the per-activation sequence (setContext + two
setContextValue) into a single native call.

- DatadogProfilingIntegration.activate: one setTraceContext(...) carrying
  trace/span context + operation and resource attributes (3 JNI calls -> 1);
  close/clearContext: clearTraceContext() (wipes op/resource slots too).
- DatadogProfiler: setContextValue/clearContextValue/reapplyAppContext/
  syncNativeAppContext are now all-native. reapplyAppContext uses a per-slot
  native setContextValue loop (native setContextValue publishes valid=1, so app
  context stays visible without an active span — preserves PR #11646). A native
  batch reapply is deferred to a measured follow-up (java-profiler PROF-15361).
- AppContextSnapshot simplified to strings-only (the native path resolves each
  value's encoding via the process-wide cache; no cached id/utf8/snapshotTags).
- snapshot() uses the new native copyContextTags read (no ThreadContext/DBB, so
  it observes native writes without resetting the record).
- ContextSetter kept only for offsetOf + size (pure Java); no DBB usage remains.

Requires java-profiler with the all-native API (ddprof >= the phase-1 release;
DataDog/java-profiler#631). Build/test with -PddprofUseSnapshot=true against a
local publishToMavenLocal 1.47.0-SNAPSHOT until that ships. ddprof suite is
Linux-gated (assumeTrue(isLinux)) — verify on Linux/CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ContextValue

The native setTraceContext rejects spanId==0 with IllegalArgumentException (it is
the activation path; clearing is clearTraceContext). The bridge's catch(Throwable)
would swallow that throw and leave the previous span's context stale on the thread.
Span ids are non-zero by construction (IdGenerationStrategy never yields 0,
DDSpanId.ZERO means "no span", and DDSpanContext is the only ProfilerContext), so
this is defensive: route a zero span to a clean clearTraceContext instead of a
silently-swallowed throw over stale state.

Also document clearContextValue(int)'s return contract (@param/@return) and add a
testContextRegistration scenario asserting a zero-span activation does not throw and
still reapplies app context.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant