Skip to content

Add pagination metrics for known tests fetch#11925

Draft
calvinbayer wants to merge 1 commit into
masterfrom
add-known-tests-pagination-metrics
Draft

Add pagination metrics for known tests fetch#11925
calvinbayer wants to merge 1 commit into
masterfrom
add-known-tests-pagination-metrics

Conversation

@calvinbayer

Copy link
Copy Markdown

What does this PR do?

Adds three new distribution metrics to track known tests (Early Flake Detection) pagination behavior:

  • civisibility.known_tests.pages_fetched - Number of pages fetched during pagination
  • civisibility.known_tests.total_fetch_ms - Wall-clock time from first to last page request
  • civisibility.known_tests.total_request_ms - Sum of individual per-page request durations

Motivation

Currently, only per-request timing is tracked. These new metrics provide observability into:

  • How often pagination occurs (single vs multi-page responses)
  • Backend pagination performance (total wall-clock time)
  • Network overhead vs local processing time (total_fetch_ms vs total_request_ms)

Implementation Details

  • Metrics are emitted only on successful completion after all pages are fetched
  • If fetch fails midway (network error, invalid response), metrics are not emitted
  • Per-request metrics remain unchanged (backward compatible)
  • Added enum entries to CiVisibilityDistributionMetric
  • Updated ConfigurationApiImpl.getKnownTestsByModule() to track timing

Testing

  • Tests pass
  • Code formatted with spotless

Checklist

  • Tests passing
  • Code formatted (spotless)
  • Backward compatible (no breaking changes)
  • Follows existing telemetry patterns

@datadog-datadog-prod-us1

This comment has been minimized.

Add three new distribution metrics to track known tests pagination:
- civisibility.known_tests.pages_fetched - number of pages fetched
- civisibility.known_tests.total_fetch_ms - wall-clock time for entire fetch
- civisibility.known_tests.total_request_ms - sum of per-page request times

Metrics are emitted only on successful completion of pagination.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@calvinbayer calvinbayer force-pushed the add-known-tests-pagination-metrics branch from a8b5777 to e5964e9 Compare July 13, 2026 11:45
@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.94 s [-0.6%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent 12.90 s 13.02 s [-1.6%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent 16.80 s 16.75 s [-0.6%; +1.2%] (no difference)
startup:petclinic:iast:Agent 16.92 s 16.42 s [-1.5%; +7.6%] (no difference)
startup:petclinic:profiling:Agent 16.71 s 16.80 s [-1.9%; +0.8%] (no difference)
startup:petclinic:sca:Agent 16.92 s 16.67 s [+0.7%; +2.3%] (maybe worse)
startup:petclinic:tracing:Agent 16.05 s 16.00 s [-0.7%; +1.5%] (no difference)

Commit: e5964e9d · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant