Skip to content

feat(agent): complete ClassFile API probe parity for Java 26+ class files#843

Open
jbachorik wants to merge 37 commits into
developfrom
muse/classfileapi-return-duration
Open

feat(agent): complete ClassFile API probe parity for Java 26+ class files#843
jbachorik wants to merge 37 commits into
developfrom
muse/classfileapi-return-duration

Conversation

@jbachorik

@jbachorik jbachorik commented May 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements full probe parity in ClassFileApiBackend — the instrumentation backend for class files with major version > 69 (Java 26+), where ASM cannot parse the class file format. Uses the JDK ClassFile API (java.lang.classfile.*, available since JDK 24).

Probe kinds implemented (17 total)

Kind Notes
ENTRY Method entry; @Self, @ProbeClassName, @ProbeMethodName
RETURN Normal return; @Return (all primitives + reference, with boxing), @Duration
ERROR Uncaught-exception exit; @TargetInstance (throwable), @Duration
CALL Before/after method call; @TargetInstance, @TargetMethodOrField, @Return, @Duration, ordinary args
LINE Before/after source line; @ProbeClassName, @ProbeMethodName
FIELD_GET Before/after getfield/getstatic; @TargetInstance, @TargetMethodOrField, @Return
FIELD_SET Before putfield/putstatic; @TargetInstance, @TargetMethodOrField, ordinary value arg
ARRAY_GET Before/after array load; @TargetInstance, @Index, @Return
ARRAY_SET Before array store; @TargetInstance, @Index, ordinary value arg
CHECKCAST Before/after checkcast; @TargetInstance, ordinary type-name arg
INSTANCEOF Before/after instanceof; @TargetInstance, ordinary type-name arg
THROW Before explicit athrow; @TargetInstance (thrown throwable)
CATCH At exception handler entry; @TargetInstance (caught throwable)
NEWARRAY Before/after primitive, reference, and multi-dimensional array allocation; ordinary element-type and dimension args, @Return (after only)
NEW Before new; after first <init> completes; ordinary type-name arg, @Return (after only)
SYNC_ENTRY After monitorenter; @TargetInstance (lock object)
SYNC_EXIT Before/after monitorexit; @TargetInstance (lock object)

Guard semantics

  • Level guards: enforced at the MethodHandle layer (HandlerRepositoryImpl). The INVOKEDYNAMIC instruction is always emitted; the level check fires at runtime with no bytecode overhead.
  • Const sampling (@Sampled): MethodTracker.hit() is called independently at each ENTRY, RETURN, and ERROR probe site.
  • Adaptive sampling (@Sampled(kind=Adaptive)): MethodTracker.hitAdaptive() is called once at method entry and the result is stored in a shared local slot; RETURN and ERROR Adaptive probes load that slot rather than re-calling, ensuring paired entry/exit measurement. Each exit site also calls MethodTracker.updateEndTs() to close the sampling window — matching the behavior of the ASM MethodTrackingContext.

Key implementation details

  • Single-pass CodeTransform over the CodeModel element stream; synthesizes exception handler in atEnd() when needed for ERROR/@Duration probes.
  • pendingNewStack (LIFO) tracks uninitialized new references across the new Foo / dup / <init> sequence.
  • catchHandlerTypes pre-scanned from ExceptionCatch pseudo-instructions before transform so handler labels are known before their LabelTarget is encountered.
  • Method pattern matching uses om.isMethodRegexMatcher() flag (slashes are stripped by setMethod()), handles empty pattern as match-all, and # as same-name self-reference — consistent with the ASM Instrumentor.
  • When both a NEW AFTER probe and a CALL probe target the same <init>, both fire: constructor args are backed up via backupCallStack before the invoke, and CALL AFTER probes are fired in the NEW AFTER block after the constructor completes.

Bug fixes (found via adversarial multi-agent review)

  • filterForMethod regex patterns were silently dead: setMethod() strips /…/ delimiters before storing, so the old slash-detection check in filterForMethod was always false — all regex-pattern ENTRY/RETURN/CALL probes never matched. Also fixed: empty pattern now matches all methods (was silently dropped), and # self-reference is now resolved via om.getTargetName().
  • LINE AFTER probe wrong line number: lineNumber.line() - 1 only equals the previous line when source lines are consecutive; fixed to use lastLine.
  • Multi-catch handler type overwritten: Map<Label, String> silently discarded all but the last exception type for catch (A | B e) blocks; changed to Map<Label, List<String>> with computeIfAbsentcanEmitCatchProbe now checks against all caught types.
  • NEW AFTER + CALL probe conflict: in a single-pass transform, when an <init> call matched a pending NEW AFTER probe the CALL block never ran. Fixed by collecting CALL AFTER handlers before emitting the constructor invoke, saving constructor args, and firing both probe types in the NEW AFTER block.
  • Adaptive sampling not paired: hitAdaptive() was called independently at every RETURN/ERROR site, breaking the entry/exit pairing that lets MethodTracker adjust the sampling window. Fixed by calling hitAdaptive() once at method entry, storing the decision in a shared local slot, and gating RETURN/ERROR Adaptive probes on that slot with a single updateEndTs() call per exit.

Test plan

  • ./gradlew :btrace-agent:test --tests "*ClassFileApiBackend*" with JAVA_HOME=~/.sdkman/candidates/java/26-tem — 105 tests, all pass
  • ./gradlew :btrace-dist:build && ./gradlew :integration-tests:test -Pintegration for full end-to-end validation (requires JDK 26 target)

🤖 Generated with Claude Code


This change is Reviewable

…ackend

ClassFileApiBackend (used for Java 26+ class files) previously skipped
handlers with @return or @duration parameters. This change adds full support:

- @return: dup/dup2 based on TypeKind.slotSize(), store to an allocated
  local slot, load before INVOKEDYNAMIC. Void methods are silently skipped.
- @duration: System.nanoTime() captured at method entry; delta computed at
  each RETURN point and in a finally-block exception handler (matching the
  existing ASM instrumentation behavior).

Bug fixes applied after chorus review:
- emitProbeCall: pre-validate all handler arguments before pushing any onto
  the stack; an early return mid-loop would leave orphaned stack values and
  cause a VerifyError on class loading.
- emitProbeCall: load the return-value slot using the actual store TypeKind
  from the ReturnInstruction, not TypeKind.fromDescriptor on the handler
  descriptor (which would yield REFERENCE for AnyType→Object handlers,
  mismatching the ISTORE/LSTORE used during capture).
- atStart/accept: bind startLabel in accept() alongside entryTsSlot
  allocation so they are always in sync; an unpairable label in atEnd
  could leave an unclosed try region.
- atEnd: register exceptionCatchAll after binding all three labels; replace
  stream pipeline with a plain for-loop.

Tests cover @return (int, long, void-skip), @duration (normal exit, exception
exit), and combined @return+@duration slot-collision check. Tests that require
class file version 70 parsing (Java 26+) skip automatically on JDK < 26
via Assumptions.assumeTrue.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbachorik jbachorik changed the title feat(agent): implement @Return and @Duration in ClassFileApiBackend feat(agent): complete ClassFile API method probe support Jun 5, 2026
jbachorik and others added 17 commits June 5, 2026 23:28
Add Kind.CATCH and Kind.ERROR probe support to ClassFileApiBackend.

CATCH probes fire at typed exception handler entry points (throwable is
TOS); the handler labels are pre-scanned from ExceptionCatch elements
before the CodeTransform runs to handle stream ordering. ERROR probes
fire via a synthetic catch-all handler that wraps the entire method body,
sharing the entryTs slot with RETURN @duration probes when both are
present. needsExceptionHandler separates exception-handler creation from
hasDuration so ERROR probes work even without @duration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add Kind.NEWARRAY probe support to ClassFileApiBackend for newarray
(primitive), anewarray (reference), and multianewarray instructions.

Before probes fire with ordinary args (element type String, dimension
count int) without touching the allocation operand stack. After probes
store the freshly allocated array to a local, fire with @return carrying
the array reference, then restore TOS for the original caller.

Location.clazz() matches the element type Java name (e.g. "int",
"java.lang.String"); an empty clazz matches all array allocations.
@duration, @TargetInstance, and @TargetMethodOrField are rejected by
canEmitNewArrayProbe as they have no meaningful semantics here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements Kind.NEW probes in ClassFileApiBackend: BEFORE fires before
the 'new' instruction with the allocated type name as a String argument;
AFTER fires after the matching INVOKESPECIAL <init> call with the fully
initialized reference available via @return. Nested constructor calls
are handled correctly via a per-method pending-new stack.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement SYNC_ENTRY and SYNC_EXIT probe kinds in ClassFileApiBackend.
MonitorInstruction is detected via the ClassFile API; MONITORENTER fires
SYNC_ENTRY handlers and MONITOREXIT fires SYNC_EXIT handlers. When
@TargetInstance is requested, the lock object is DUP'd and stored to a
local before the monitor instruction executes so both BEFORE and AFTER
probes can reference it. Unsupported parameter kinds (@duration, @return,
etc.) cause the probe to be silently rejected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add level guard and sampling support to ClassFileApiBackend:
- Level guards (via @Level) continue to work at the MethodHandle layer
  (HandlerRepositoryImpl.applyLevelGuard); INVOKEDYNAMIC is always
  emitted and the guard fires at runtime without bytecode-level checks.
- Const and Adaptive sampling for ENTRY, RETURN, and ERROR probes is
  now implemented: MethodTracker.registerCounter is called at
  instrumentation time; MethodTracker.hit / hitAdaptive is emitted as
  bytecode before each probe call so non-sampled invocations skip the
  INVOKEDYNAMIC entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbachorik jbachorik changed the title feat(agent): complete ClassFile API method probe support feat(agent): complete ClassFile API probe parity for Java 26+ class files Jun 15, 2026
…pers

- Remove dead HIT_ADAPTIVE_DESC constant (identical to HIT_DESC)
- Extract registerSampledCounters() to collapse three identical registration loops
- Delete emitErrorProbeWithSampling(); consolidate into emitWithSamplingGuard()
  by changing its parameter from Runnable to BooleanSupplier (all probe kinds
  now share a single sampling-guard implementation)
- Merge filterForNew() into filterForNewArray() (identical method bodies)
- All filter*() methods now use lazy ArrayList allocation, returning
  Collections.emptyList() instead of a fresh ArrayList when nothing matches

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jbachorik and others added 12 commits June 16, 2026 09:59
…tion

- filterForMethod: use om.isMethodRegexMatcher() flag instead of slash-detection
  (setMethod() strips delimiters before storing, so the slash check was always false,
  silently dead-dropping all regex method patterns); handle empty pattern as match-all
  and '#' as same-name self-reference, matching the ASM Instrumentor behavior
- LINE AFTER probe: fire with lastLine instead of lineNumber.line()-1; the subtraction
  only produced the correct answer when source lines were consecutive
- CATCH probe: store all exception types per handler label in a List instead of a single
  String so that multi-catch blocks (catch (A | B e)) are matched correctly; a probe
  typed to the overwritten type was silently never emitted in the multi-catch case

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two correctness bugs fixed in ClassFileApiBackend:

1. NEW AFTER + CALL probe conflict: in a single-pass transform, when an
   <init> call matches a pending NEW AFTER probe the CALL block never ran.
   Fix collects CALL AFTER handlers before emitting the constructor invoke,
   backs up constructor args via backupCallStack/restoreCallStack, and fires
   both NEW AFTER and CALL AFTER probes in the NEW AFTER block.

2. Adaptive sampling pairing: hitAdaptive() was called independently at
   every RETURN/ERROR site, breaking the entry/exit pairing that lets
   MethodTracker adjust the sampling window. Fix calls hitAdaptive() once
   at method entry and stores the result in a shared local slot (sHitSlot).
   RETURN and ERROR Adaptive probes load that slot, and each exit site
   calls updateEndTs() once to close the window.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…re-entry on JDK 26+

Without this fix, TraceAll probes on classes reachable from
MethodHandleNatives.linkCallSite() (e.g. java.util.Collections,
Collections$CopiesList) caused infinite recursion → StackOverflowError
and a JPLIS assertion failure on JDK 26/27 in premain startup mode.

Root cause (two-part):

1. ASM cannot parse class-file major > 69 (JDK 26+), so
   LinkerInstrumentor silently skipped patching
   MethodHandleNatives.linkCallSite() with the LinkingFlag guard.
   Fixed by ClassFileApiLinkerGuard, which uses the ClassFile API
   to apply the same guard for JDK 26+ class bytes.

2. ClassFileApiBackend emitted invokedynamic probe dispatch calls
   without the LinkingFlag.get() != 0 check that Instrumentor wraps
   around every invokeBTraceAction().  When linkCallSite() internally
   calls java.util.Collections.nCopies(), the not-yet-linked probe
   dispatch in nCopies() re-entered linkCallSite() for its own
   bootstrap — circular dependency → SOE.  Fixed by adding the
   LinkingFlag guard (check before pushing any args) to all three
   invokedynamic emission sites in ClassFileApiBackend:
   emitProbeCall, emitNewObjectProbe, emitSyncProbe.

Also guards ClassFilter from probing java/util/concurrent/ and
java/lang/classfile/ classes, and ensures BTraceTransformer swallows
Throwable rather than rethrowing through JVMTI/JPLIS where a
pending JNI exception triggers a native assertion on JDK 27+.

Verified: testTraceAll() passes on JDK 24, 26, and 27.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…abstract/native in ClassFileApiBackend

btrace.verify.transformed triggered a spurious warn on every ClassFile API
instrumented class because ASM ClassReader throws on major > 69 (JDK 26+).
Guard the verification block with a major-version check so it only runs
when ASM can actually parse the output.

ClassFileApiBackend was silently passing abstract and native methods into
the handler-collection and method-transform paths, only dropping them later
when no CodeModel was found. Mirrors Instrumentor's explicit ACC_ABSTRACT
check by testing AccessFlag.ABSTRACT | NATIVE before any handler work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t timeouts

The prior commit eagerly loaded ClassFileApiBackend (71 KB) and
ClassFileApiLinkerGuard (3.6 KB) at static-init time of BackendSelector
and LinkerInstrumentor respectively. On JDK 8/11/21 both loads fail with
UnsupportedClassVersionError — but only after MaskedClassLoader reads the
.classdata entry from btrace.jar and defineClass() rejects the bytes.

On slow CI disks (shared NFS, ephemeral storage, AV scanning) this
startup-path JAR read plus defineClass failure added 1-2 s to the first
class transformation. Because BackendSelector's class-init lock serialises
all concurrent transform threads while the load is in flight, the delay
was felt by every thread in the retransformation batch — intermittently
pushing the 10-second integration-test timeout, causing testOnMethod,
testOnMethodSubclass, and testOnMethodTrackRetransform to fail on JDK 8,
11, and 21.

Fix: make both loads strictly lazy — they are only attempted when
select()/addGuardClassFileApi() is first called with a major version
above MAX_ASM_MAJOR_VERSION (69). On JDK 8-25 no such version ever
appears, so the disk reads simply never happen on those JVMs.

Also make the 4 MB command-processor stack conditional on JDK 26+
(class-file major >= 70), where ClassFile API's StackMapGenerator can
exhaust the default stack depth on complex methods. On older JVMs the
default stack size (0 = JVM-chosen) is used, avoiding any potential
OS-level stack-allocation overhead in constrained CI environments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…K versions

The 10-second default was reliably hit on slow CI runners (JDK 11, 17)
for tests that require class retransformation (testOnMethod,
testOnMethodSubclass, testOnMethodTrackRetransform, testReflection,
Test HDR Histogram Metrics Integration).  The JDK 25+ special-case
bump to 20 s already proved adequate there, so promote that value as
the universal default and drop the now-dead version-probe code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three JUnit 5 tests in ClassFileApiTests verify ENTRY, RETURN+@return,
and RETURN+@duration probes against java.lang.Math methods via the
resources.MainJdkApi target app.

On JDK 26+ the JDK class files have class-file major version ≥ 70 and
instrumentation is routed through the ClassFile API backend.  On older
JDKs the same tests exercise the ASM backend, so both code paths are
covered by CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The complex OnMethodTest.java probe (subtype matching + regex field
patterns) requires BTrace to scan all loaded classes during attach.  On
slower JDK 8/11 CI runners this can take 20-25 s, so 20 s was still
insufficient.  30 s covers observed worst-case attach durations for the
heaviest probes in the matrix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-test conflict

Test classes run concurrently under --parallel; ClassFileApiTests was
competing with BTraceFunctionalTests for port 2020, causing
"Port 2020 unavailable" failures on every JDK in CI.  Use the same
ephemeral-port pattern as ExtensionLifecycleIntegrationTest and
ExternalTypeAdapterIntegrationTest.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
testStartup() was using unbounded CountDownLatch.await() calls — if the
JVM startup + agent initialisation stalled (observed on JDK 27 EA when
TraceAll retransforms all classes through the ClassFile API backend),
the test would hang forever and exhaust the CI job time budget.

Fix: use a 4× multiplier of the per-test timeout for startup-mode latches
so a slow run fails fast rather than blocking the whole CI job.

Also increase the CI job timeout-minutes from 10 to 15 to give EA builds
headroom now that the hang is bounded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…probes

Two bugs in ClassFileApiBackend caused VerifyError on JDK 27 when retransforming
JDK class files (major version > 69) with RETURN-location probes:

1. GENERATE_STACK_MAPS: The default STACK_MAPS_WHEN_REQUIRED can preserve original
   StackMapTable frames that pre-date the insertion of new local variables (retValSlot,
   durationSlot).  Preserved frames omit the new TOP slots, producing a frame
   inconsistency that the JVM verifier rejects.  Switching to GENERATE_STACK_MAPS
   forces a complete data-flow recomputation of all frames after transformation.

2. startLabel ordering: The try-region start label was previously bound before the
   entry-timestamp store (invokestatic nanoTime; lstore entryTsSlot).  If nanoTime
   threw, the exception handler would see entryTsSlot as TOP and then fail at
   lload entryTsSlot.  Binding startLabel after lstore guarantees entryTsSlot is
   always LONG at every throw-point inside [startLabel, endLabel).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e with probe parameter

When a RETURN probe declares `@Return int result` and the target class has
multiple method overloads (e.g. Math.abs(int/long/float/double)), the probe
matched ALL overloads due to the absent type filter in @OnMethod.  For the
non-int overloads (LONG/FLOAT/DOUBLE), emitProbeCall() would load the actual
return value (e.g. LONG, 2 slots) but the invokedynamic descriptor was fixed
as (I)V (1 int slot), creating a stack-size mismatch during GENERATE_STACK_MAPS
computation.  This caused ClassFile.transformClass() to throw
IllegalArgumentException, which instrument() caught and returned null — leaving
the class uninstrumented and the probe silently silent.

Fix: in emitProbeCall()'s pre-validation, require that the probe's @return
parameter type is either a reference type (Object/AnyType, handled via boxing)
or exactly matches the actual method return TypeKind.  If not, the argument is
marked unsatisfiable, emitProbeCall() returns false without emitting any
bytecode, and the probe is skipped for that overload while the compatible
overloads are still correctly instrumented.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant