Skip to content

Bound SDK event logs while preserving screenshot evidence - #2899

Open
miguelg719 wants to merge 12 commits into
evals/consolidation-08-native-routingfrom
evals/consolidation-09-event-retention
Open

Bound SDK event logs while preserving screenshot evidence#2899
miguelg719 wants to merge 12 commits into
evals/consolidation-08-native-routingfrom
evals/consolidation-09-event-retention

Conversation

@miguelg719

@miguelg719 miguelg719 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Bounds retained SDK event logs while keeping actionable diagnostics and trajectory evidence. Pi screenshot compaction and its Buffer-aware trajectory consumer land together, preventing intermediate screenshot loss.

Validation: 118 SDK/event tests, five Pi adapter/pipeline tests, 12 build tasks, and SDK/core/eval typechecks passed. The pipeline test imports the actual built Pi compactor and checks retained image bytes for compacted Buffer and historical base64 events.

Stack position: 9/17. Base: evals/consolidation-08-native-routing. Each PR contains the change relative to its immediate predecessor.

Reviewer entry points:

Readiness: local validation is described above; GitHub CI and automated review feedback are pending. This PR is open as non-draft so automated reviewers can run.


Summary by cubic

SDK event logs previously kept stream and lifecycle noise at visible level and retained repeated payloads; they now drop non-content fragments, keep completed events at debug level, and keep failures at level 1 across Claude, Codex, DeepAgents, Eve, Fx, Mastra, and Pi. This bounds diagnostics without changing model-facing results or trajectory evidence.

  • Exports shared harnessEventLogLevel classification with SDK-specific handling for nested tool failures, command failures, provider errors, and failed results.
  • Retains Pi screenshots once as Buffer values, removes duplicate base64 payloads from logs and non-assistant messages, and emits omission markers above the 8 MiB per-image or 64 MiB per-run limits.
  • Compacts Mastra step-finish and finish events to remove repeated request bodies while preserving usage and trajectory data; log details are clipped after credential redaction.

Written for commit 15fe01c. Summary will update on new commits.

Review in cubic

@changeset-bot

changeset-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 15fe01c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 21 files

Architecture diagram
sequenceDiagram
    participant SDK as SDK Agent
    participant Session as Session Runner
    participant Classifier as harnessEventLogLevel
    participant Logger as Harness Logger
    participant Pi as Pi Session Compactor
    participant Adapter as Pi Adapter
    
    Note over SDK,Adapter: Event ingestion and logging classification
    SDK->>Session: Emit raw event
    Session->>Classifier: Classify event type
    alt Stream delta or bare lifecycle
        Classifier-->>Session: undefined (drop)
        Session->>Logger: Skip logging
    else Error event
        Classifier-->>Session: level 1
        Session->>Logger: Log with level 1
    else Content event
        Classifier-->>Session: level 2
        Session->>Logger: Log with level 2
    end
    
    Note over Session,Adapter: Pi screenshot evidence pipeline
    SDK->>Session: tool_execution_end with base64 image
    Session->>Pi: compactPiEvent(event)
    Pi->>Pi: Decode image block to Buffer
    Pi-->>Session: Compacted event with bytes
    
    Session->>Adapter: Pass compacted trajectory events
    alt Buffer image block
        Adapter->>Adapter: Use bytes directly
        Adapter-->>Session: Image modality with Buffer
    else Legacy base64 block
        Adapter->>Adapter: Decode base64 to Buffer
        Adapter-->>Session: Image modality with Buffer
    end
    
    Note over Session,Logger: Log sanitization
    Session->>Session: withoutImageData(event)
    Session->>Logger: Log with [image N bytes] placeholder
    Logger-->>Session: Sanitized log entry
    
    Note over Session,Adapter: Mastra event compaction
    Session->>Session: compactMastraEvent(event)
    alt Retained event type
        Session-->>Session: Keep full event
    else step-finish/finish
        Session->>Session: Strip request/response bodies
        Session-->>Session: Keep only stepResult and usage
    end
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/integrations/pi-sdk/src/session.ts Outdated
Comment thread packages/integrations/claude-agent-sdk/src/session.ts Outdated
Comment thread packages/integrations/eve-sdk/src/session.ts Outdated
Comment thread packages/integrations/codex-sdk/src/session.ts Outdated
Comment thread packages/integrations/mastra-sdk/src/session.ts
Comment thread packages/integrations/fx-sdk/src/session.ts Outdated
Comment thread packages/integrations/pi-sdk/src/session.ts Outdated
Comment thread packages/integrations/pi-sdk/src/session.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants