Skip to content

Add offline cluster data exporter tool - #704

Merged
milindsrivastava1997 merged 3 commits into
mainfrom
codex/cluster-data-exporter
Sep 13, 2026
Merged

Add offline cluster data exporter tool#704
milindsrivastava1997 merged 3 commits into
mainfrom
codex/cluster-data-exporter

Conversation

@milindsrivastava1997

Copy link
Copy Markdown
Contributor

Summary

This adds the offline Google/Alibaba cluster trace exporter as an independent evaluation tool under tools/cluster-data-exporter/.

Before this PR, ASAPQuery-backend had ingest receivers but no CSV replay source for these traces. After this PR, users can run the standalone exporter, scrape it with Prometheus, and feed Prometheus Remote Write into the backend's existing profile.

Included

  • Google 2011 task-usage replay
  • Alibaba Node and MSResource replay for 2021 and 2022
  • Alibaba sort/recompress preprocessing script
  • Exporter + Prometheus Compose example
  • Synthetic fixtures and live /metrics smoke coverage
  • Optional --exit-after-eof-ms grace period, defaulting to legacy immediate shutdown
  • Documentation and standalone Cargo packaging

Verification

  • Standalone build/tests passed
  • cargo clippy --all-targets -- -D warnings passed
  • Python, shell, YAML, formatting, and whitespace checks passed
  • Live smoke test passed for Google and Alibaba Node/MSResource across both years
  • Full backend workspace test run passed under loopback-enabled execution; the sandboxed pre-commit rerun blocked only existing socket-dependent tests with PermissionDenied

The Docker lint hook was skipped during commit because the sandbox Docker daemon is inaccessible.

@milindsrivastava1997
milindsrivastava1997 merged commit b1a58ca into main Sep 13, 2026
1 check failed
@milindsrivastava1997
milindsrivastava1997 deleted the codex/cluster-data-exporter branch September 13, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant