Skip to content

docs(bqaa): document ADK Java preview-readiness fixes (pause/resume tracing, drop stats, redaction)#1975

Open
caohy1988 wants to merge 4 commits into
google:mainfrom
caohy1988:bqaa-java-preview-docs
Open

docs(bqaa): document ADK Java preview-readiness fixes (pause/resume tracing, drop stats, redaction)#1975
caohy1988 wants to merge 4 commits into
google:mainfrom
caohy1988:bqaa-java-preview-docs

Conversation

@caohy1988

@caohy1988 caohy1988 commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Documents the ADK Java BQAA "preview-readiness fixes" — shipped in adk-java v1.7.0 (released 2026-07-20) — in the BigQuery Agent Analytics page:

All behavioral claims were verified against the plugin source (BigQueryAgentAnalyticsPlugin, BigQueryLoggerConfig, BigQueryUtils, JsonFormatter, PluginState, BatchProcessor, TraceManager); the reviewed commit ba23601c has no agentanalytics diffs against the v1.7.0 tag.

What changed

  • ADK 2.0 section: Java v1.7.0 language-support chip added (per the Enhance BigQuery Agent Analytics with new options #1942 convention). Java emits TOOL_PAUSED and the pause/resume pairing keys, but stores pause_kind / function_call_id at the top level of attributes (no attributes.adk envelope, which is Python-only), and does not emit AGENT_TRANSFER / AGENT_STATE_CHECKPOINT / EVENT_COMPACTION. Includes the Java variant of the base-table pairing query, and notes that HITL_*_REQUEST_COMPLETED rows carry the same top-level pair keys (no dedicated HITL-completion view).
  • Views footnote: Java (v1.7.0+) creates v_tool_paused and the pause columns on v_tool_completed; the three workflow views are Python-only.
  • Reliability fixes intro + Dropped-event observability: Java getDropStats() with all eight drop reasons (queue_full, append_error, serialization_error, after_close, shutdown_timeout, writer_permit_exhausted, writer_create_error, late_after_finalize); append_error defined per the BatchProcessor contract (any batch-prep/append failure other than AppendSerializationError); tabbed code samples; closing guidance treats any non-zero counter as actual loss.
  • Built-in redaction: Java (v1.7.0+) redacts the six sensitive keys plus temp:-prefixed keys — but not secret: (Python-only). Java contentFormatter contract documented (thread-safe, fast/non-blocking, return a new object) plus fail-closed behavior on exceptions.
  • Tracing and observability: Java v1.7.0+ uses the same internal ID-only span tracking (no OTel span export). trace_id is a 32-char OTel ID when inherited from an ambient span; Python generates that format as its fallback, while Java falls back to the ADK invocation ID.
  • Public methods (Java tab): getDropStats() and the JVM shutdown hook that drains pending events at exit when close() was never called.

Review feedback addressed

Publish checklist

  • adk-java release containing the fixes is tagged (v1.7.0, 2026-07-20)
  • Preview admonition and TODO(publish) comment deleted; gated sections converted to the section-level language-support-tag convention (Java v1.7.0 chip on the ADK 2.0 section); all inline "builds newer than v1.6.0" prose replaced with v1.7.0 references
  • getDropStats() reason keys and top-level pause_kind / function_call_id attribute location re-verified against the tag (no agentanalytics changes between ba23601c and v1.7.0)
  • Fix the stale three-reason Javadoc on BigQueryAgentAnalyticsPlugin.getDropStats() upstream in adk-java (tracked separately, per review)

🤖 Generated with Claude Code

@netlify

netlify Bot commented Jul 18, 2026

Copy link
Copy Markdown

Deploy Preview for adk-docs-preview ready!

Name Link
🔨 Latest commit 8e4b8e5
🔍 Latest deploy log https://app.netlify.com/projects/adk-docs-preview/deploys/6a62916bd7e9520008707ea2
😎 Deploy Preview https://deploy-preview-1975--adk-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@google-cla

google-cla Bot commented Jul 18, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@caohy1988

Copy link
Copy Markdown
Contributor Author

Review against current google/adk-java/main (ba23601c09927) and the current live adk-docs page:

The Java-specific direction is good, and most behavioral claims check out. I found two blockers and three smaller documentation gaps before this is ready to publish.

Blocking

  1. Rebuild/rebase this on current adk-docs/main. The branch carries the stale pre-Enhance BigQuery Agent Analytics with new options #1942 version of this page: it reintroduces the obsolete Python “not in v2.2.0” callout, while current main documents ADK 2.0 support with a Python v2.3.0 language-support tag. It also carries the old top-level Java v1.4.0 support tag; the live page now says v1.5.0. This explains the current dirty/conflicting merge state and the 9-commit, +434/−38 PR diff. Please replay only the Java-specific tip commit onto current main and preserve the current section-level version-tag style.

  2. getDropStats() exposes eight Java reasons, not three. PluginState.getDropStats() currently returns:

    • queue_full
    • append_error
    • serialization_error
    • after_close
    • shutdown_timeout
    • writer_permit_exhausted
    • writer_create_error
    • late_after_finalize

    Source: PluginState.getDropStats() and BatchProcessor.getDropStats().

    Please update the overview, Java reason table, sample output, and publish checklist. Suggested definitions for the missing reasons:

    Reason Meaning
    after_close A row reached an already-closed per-invocation processor.
    shutdown_timeout Queued rows remained when the bounded final drain expired.
    writer_permit_exhausted The live-writer safety cap was exhausted, normally during a Storage outage or delayed cleanup.
    writer_create_error StreamWriter construction or processor startup failed.
    late_after_finalize Async work completed after the invocation was finalized or while the plugin was closing.

    The Java method's own Javadoc still lists only the original three reasons, but the returned map and regression tests cover all eight; that source comment should be corrected separately.

Additional gaps

  1. Fix the absolute “32-character hex Trace ID” wording. Java correctly falls back to the ADK invocation ID, which is not guaranteed to be 32-hex. Suggested opening: “Trace identifier. It is a 32-character OpenTelemetry trace ID when inherited from an ambient span; Python also generates that format as its fallback, while Java falls back to the ADK invocation ID.” See TraceManager.getTraceId().

  2. Mention Java HITL completion pair keys. Java places top-level pause_kind and function_call_id on HITL_*_REQUEST_COMPLETED rows too, allowing a base-table join from the TOOL_PAUSED HITL row to its completion. The current text says to look for the completion event but does not explain that it is directly pairable. There is still no dedicated HITL-completion view. See the resume callback.

  3. Document the contentFormatter contract. The Java callback must be thread-safe, fast/non-blocking, and return a new object instead of mutating its input. See BigQueryLoggerConfig.contentFormatter().

Verified as correct

  • v1.6.0 is still the latest published Java release; the July 9/16 fixes remain unreleased.
  • Java emits TOOL_PAUSED with top-level pause_kind / function_call_id and no attributes.adk envelope.
  • Java does not emit AGENT_TRANSFER, AGENT_STATE_CHECKPOINT, or EVENT_COMPACTION.
  • v_tool_paused and the pair columns on v_tool_completed are implemented.
  • Java redacts the six sensitive key names plus temp: (not secret:), and formatter failure clears the row content.
  • ID-only tracing, invocation-ID fallback, cross-region routing, and the bounded JVM shutdown-hook drain are accurately described.

Once the branch is rebuilt on current main and the eight-reason contract is fixed, the release-gating approach looks good for the next Java tag.

Covers google/adk-java@c685ece46b (redaction, table bootstrap, drop
stats) and google/adk-java@2027a4b53d (tracing, lifecycle, redaction,
HITL/TOOL_PAUSED pairing), rebuilt on current main and gated as
post-v1.6.0 until the next adk-java release is tagged:

- Top note gating the new Java behavior, with a TODO(publish) marker.
- ADK 2.0 section: Java emits TOOL_PAUSED and the pause/resume pairing
  keys at the top level of attributes (no attributes.adk envelope),
  without AGENT_TRANSFER / AGENT_STATE_CHECKPOINT / EVENT_COMPACTION;
  Java variant of the base-table pairing query; HITL completions carry
  the same top-level pair keys for direct base-table joins.
- Views footnote: Java creates v_tool_paused and the pause columns on
  v_tool_completed only; the three workflow views are Python-only.
- Dropped-event observability: Java getDropStats() with all eight drop
  reasons (queue_full, append_error, serialization_error, after_close,
  shutdown_timeout, writer_permit_exhausted, writer_create_error,
  late_after_finalize).
- Built-in redaction: Java redacts the six sensitive keys plus temp:
  (not secret:); contentFormatter contract (thread-safe, non-blocking,
  no mutation) and fail-closed behavior.
- Tracing: Java ID-only span tracking; trace_id wording no longer
  claims 32-hex absolutely — Java falls back to the ADK invocation ID.
- Public methods (Java): getDropStats() and the JVM shutdown-hook drain.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@caohy1988
caohy1988 force-pushed the bqaa-java-preview-docs branch from b1f10df to 478c32a Compare July 18, 2026 11:25
@caohy1988

Copy link
Copy Markdown
Contributor Author

All five points addressed in 478c32a (branch rebuilt, force-pushed):

  1. Rebuilt on current main — replayed only the Java-specific changes onto 3e6ee758; the PR is now one commit, +122/−23. No stale "not in v2.2.0" callout or old top-level Java v1.4.0 tag; the current section-level version-tag style is untouched (the unreleased Java behavior is gated in prose + a TODO(publish) HTML comment, since there is no Java version to put in a chip yet).
  2. Eight drop reasons — the Java table in Dropped-event observability now lists all eight keys (queue_full, append_error, serialization_error, after_close, shutdown_timeout, writer_permit_exhausted, writer_create_error, late_after_finalize) with your suggested definitions; the sample output and publish checklist were updated, and the intro bullet now defers to that section instead of listing keys inline. Verified against PluginState.getDropStats() at ba23601c. The stale three-reason Javadoc upstream is noted in the checklist as a separate fix.
  3. trace_id wording — schema-table row and the tracing-section fallback bullet now use the suggested framing: 32-char OTel ID when inherited from an ambient span; Python generates that format as its fallback; Java falls back to the ADK invocation ID.
  4. HITL completion pair keys — the "Java attribute location" note now states that HITL_*_REQUEST_COMPLETED rows carry the same top-level pause_kind / function_call_id, so a HITL TOOL_PAUSED row joins directly to its completion on function_call_id in the base table, and that there is no dedicated HITL-completion view.
  5. contentFormatter contract — the "Built-in redaction in Java" note documents the thread-safe / fast, non-blocking / return-a-new-object-do-not-mutate contract plus the fail-closed behavior on exceptions.

Still draft pending the next adk-java tag; all gating phrasings are greppable via v1.6.0.

🤖 Generated with Claude Code

@caohy1988

Copy link
Copy Markdown
Contributor Author

Fresh review of 478c32a7dba5ae10b325e434c5259e1a49d6c695: the five original review points are resolved. I found three remaining issues before this should leave draft.

Findings

  1. P1 — Non-zero counters represent actual loss, not merely “risk”

    The closing guidance says only sustained queue_full, retry_exhausted, or Java append_error indicates BQAA is “at risk of data loss.” In Java, every non-zero counter represents rows that never reached BigQuery, including shutdown_timeout, writer_create_error, and the other lifecycle counters. Current wording, Java source contract.

    Suggested replacement:

    Any non-zero count means analytics rows were dropped before reaching BigQuery. Alert on every reason; sustained queue_full or write-error counts generally indicate throughput or Storage Write health problems.

  2. P2 — Java append_error is defined too narrowly

    The table describes it exclusively as a Storage Write API failure after retries or a non-retryable response. The implementation uses append_error for every exception covering Arrow batch preparation and append processing except a wrapped AppendSerializationError. That includes conversion failures, timeouts, interruptions, and other unexpected failures. Current table, implementation.

    Suggested definition:

    Batch preparation or append failed for a cause other than AppendSerializationError, including timeouts, exhausted or non-retryable writes, and unexpected conversion failures.

  3. P2 — The publish checklist would preserve ad-hoc version callouts

    The draft gating is reasonable while no Java version exists. Once the release ships, however, merely replacing the three v1.6.0 phrasings would leave the top version admonition and inline “builds newer than…” callouts. PR Enhance BigQuery Agent Analytics with new options #1942 explicitly established section-level language-support-tag blocks as the convention and removed this style of callout. Current gating, PR #1942.

    Please update the publish checklist to remove the preview admonition and convert the applicable sections to Java version chips once the release number is known.

Verified in this pass

  • Single commit correctly rebuilt on current main.
  • All eight Java drop keys match ba23601c.
  • Trace fallback, HITL pair keys, redaction, contentFormatter, views, and shutdown behavior match current source/tests.
  • Netlify preview is clean on desktop and 375 px mobile; tabs work, with no overflow, console errors, or failed requests.
  • GitHub Actions, header check, and Netlify are green. CLA remains the only external failing check.

- Closing guidance: any non-zero drop counter is actual loss, not risk;
  alert on every reason, with queue_full / write-error counts called out
  as throughput or Storage Write health signals.
- Java append_error: broaden the definition to match BatchProcessor —
  any batch-preparation or append failure other than
  AppendSerializationError (timeouts, exhausted or non-retryable writes,
  unexpected conversion failures).
- TODO(publish) marker: on release, delete the preview admonition and
  convert gated sections to section-level language-support-tag Java
  chips per PR google#1942, instead of keeping prose version callouts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@caohy1988

Copy link
Copy Markdown
Contributor Author

All three findings addressed in fe5adff:

  1. P1 — loss vs. risk: the closing guidance now reads: "Any non-zero count means analytics rows were dropped before reaching BigQuery. Alert on every reason; sustained queue_full or write-error counts (retry_exhausted / non_retryable in Python, append_error in Java) generally indicate throughput or Storage Write health problems."
  2. P2 — append_error definition: broadened per the BatchProcessor flush path — "Batch preparation or append failed for a cause other than AppendSerializationError — including timeouts, exhausted or non-retryable writes, and unexpected conversion failures." (Verified: the entire Arrow-populate → append → response-error path increments droppedAppendError for any exception whose cause isn't AppendSerializationError.)
  3. P2 — publish plan: both the in-file TODO(publish) comment and the PR checklist now say to delete the preview admonition entirely at release time and convert the gated sections to section-level language-support-tag Java chips per the Enhance BigQuery Agent Analytics with new options #1942 convention (e.g. a Java vX.Y.0 chip next to the Python chip on the ADK 2.0 section), removing all inline "builds newer than v1.6.0" prose (grep: v1.6.0).

🤖 Generated with Claude Code

adk-java v1.7.0 (2026-07-20) ships the July preview-readiness fixes;
verified no agentanalytics changes between the reviewed commit
ba23601c and the v1.7.0 tag. Per the publish checklist:

- Delete the preview 'ADK Java version requirement' admonition and its
  TODO(publish) marker.
- Add the 'Java v1.7.0' language-support chip to the ADK 2.0 workflow
  section (PR google#1942 convention).
- Convert the inline 'builds newer than v1.6.0' callouts to v1.7.0
  version references throughout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@caohy1988
caohy1988 marked this pull request as ready for review July 21, 2026 23:14
@caohy1988

Copy link
Copy Markdown
Contributor Author

adk-java v1.7.0 (2026-07-20) is out and ships both preview-readiness commits — verified there are no agentanalytics changes between the last-reviewed commit ba23601c and the v1.7.0 tag, so all reviewed claims hold for the release.

Publish checklist executed in ba107cf:

  • Deleted the preview "ADK Java version requirement" admonition and its TODO(publish) comment.
  • Added the Java v1.7.0 language-support-tag chip to the Agent workflow and pause/resume events (ADK 2.0) section, next to the Python chip (per the Enhance BigQuery Agent Analytics with new options #1942 convention).
  • Replaced every inline "builds newer than v1.6.0" / "first release after v1.6.0" callout with a v1.7.0 reference (grep v1.6.0 is clean).

Marked the PR ready for review. Remaining: the stale three-reason getDropStats() Javadoc fix upstream in adk-java (tracked separately), and the CLA check.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants