Skip to content

feat(storage): add run_benchmark_tests.sh for automated time-based GC… - #18138

Draft
shradhakatyal wants to merge 14 commits into
mainfrom
add-gcs-read-benchmark-runner
Draft

feat(storage): add run_benchmark_tests.sh for automated time-based GC…#18138
shradhakatyal wants to merge 14 commits into
mainfrom
add-gcs-read-benchmark-runner

Conversation

@shradhakatyal

Copy link
Copy Markdown
  • Adds run_benchmark_tests.sh under packages/google-cloud-storage/cloudbuild/ to automate high-bandwidth GCS read microbenchmarks for large objects (10 GiB file sizes).
  • Dynamically updates config.yaml to test multi-process worker pools (PROCESSES=48) across standard REST and BidiReadObject DirectPath gRPC (zonal buckets).
  • Automatically verifies and pre-seeds required GCS test objects prior to benchmark execution to ensure consistent benchmark conditions.
  • Emits structured JSON metrics via --benchmark-json for automated ingestion into Cloud Build CI

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an automated Google Cloud Storage read microbenchmark runner script (run_benchmark_tests.sh) designed for GitHub CI/CD and high-bandwidth GCE VMs. The review feedback identifies two critical areas for improvement: first, a race condition and a NameError in the Python pre-seeding block where concurrent processes attempt to write to the same payload file and reference an undefined variable; second, a style guide violation where the configuration loaded from config.yaml is not type-validated before indexing.

Comment thread packages/google-cloud-storage/cloudbuild/run_benchmark_tests.sh
Comment thread packages/google-cloud-storage/cloudbuild/run_benchmark_tests.sh
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

⚡ GCS DirectPath Read Performance Benchmark

Status: PASSED | Commit: adb2e006 | Target VM: gcs-benchmark-runner-us-west4-a (c4-standard-192)

Workload Pattern Measured Throughput (MiB/s) Network Bandwidth CPU Usage Status
read_zonal_bidi_grpc Execution Completed See Logs - Passed
📊 Detailed Telemetry & System Information
  • Storage Target: gs://gcs-read-bench-zb-us-west4-a (Zonal Rapid Storage)
  • Transport: BidiReadObject gRPC DirectPath (ALTS)
  • DirectPath gRPC streaming metrics verified.
  • Build Logs: View Cloud Build Execution Logs

@shradhakatyal

Copy link
Copy Markdown
Author

/gcbrun adb2e00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant