Skip to content

[Bug]: Codex token counts saturate at 2,147,483,647 in Usage & Spend #469

Description

@Popidge

Existing issues

  • I searched existing issues and did not find a duplicate.

What happened?

Observed behavior

Settings → Usage & Spend → Detailed shows:

  • Input: 2,147,483,647
  • Cached: 2,147,483,647
  • Output: 36,417,423
  • Both 7-day and 30-day totals: 2,183,901,070
Image

Affected area

  • Tray panel
  • Settings UI
  • Config file / settings persistence
  • CLI
  • Provider-specific behavior
  • Installer / release packaging
  • Startup / background behavior
  • Other

Steps to reproduce

An isolated Rust harness using the repository's unchanged merge_codex_record_into_packed function merged two records, each with input 1,500,000,000, cached 1,400,000,000, output 100, and reasoning None.

  • Expected packed totals: [3000000000, 2800000000, 200]
  • Actual: [2147483647, 2147483647, 200]

This confirms the saturation in source. The installed binary and private session logs were not inspected, so the exact stage responsible for this screenshot is not established. Full application tests were not run for this investigation.

Logs, screenshots, or recordings


App version

v0.56.8 beta

Windows version

Windows 11 25H2

Additional context

Cause found in source

The scanner uses signed 32-bit token storage and saturating addition. Counts therefore silently stop at i32::MAX rather than continuing above it.

The UI total calculation then adds input and output in u64: 2,147,483,647 + 36,417,423 = 2,183,901,070, exactly matching the screenshot. Cache-read tokens are already included in input and are not added again.

Expected behavior / suggested fix

Preserve token counts above 2³¹−1 through parsing, cumulative totals/deltas, per-file and day/model caches, retained reports, and summaries. Audit the parser's i32 fields and narrowing as i32 casts too; widening only the UI or final summary will not fix this.

Rebuild or invalidate affected persisted token caches and retained reports: widening already-saturated numbers cannot restore them. Preserve the existing bounded scan and partial-history behavior.

Add regression coverage for individual records and cumulative totals above the boundary, same-bucket and cross-file aggregation, cross-day retained reports, cache round trips, and recovery from old caches. Keep cached-input and reasoning accounting semantics unchanged.

Suggested agent prompt for fix, from GPT-6 Astra:

Fix Codex token counts silently saturating at 2,147,483,647 in nesszer/Win-CodexBar. Follow AGENTS.md and keep the patch focused.

The bottleneck is the scanner's i32 token pipeline: rust/src/core/jsonl_scanner.rs (token structs, Vec day buckets, merge_codex_record_into_packed, cached_cost_report_from_days), rust/src/core/jsonl_scanner/codex/{helpers,parser}.rs, and rust/src/cost_scanner/codex/cache_days.rs. rust/src/cost_scanner/codex.rs converts the already-capped cached report to u64 too late.

Use a consistent 64-bit representation through parsing, cumulative totals/deltas, aggregation, cache persistence, and reports. Remove token narrowing casts and preserve handling of negative/invalid input, cumulative resets, fork deduplication, cached input, and optional reasoning. Audit shared consumers so the type change does not break other providers. Do not double-count cache-read tokens in totals.

Ensure old caches and previous_report values that may already contain saturation are invalidated/rebuilt from source logs using the existing bounded scan workflow. Simply deserializing old capped values into wider fields is insufficient.

Add focused regression tests: two records with input 1,500,000,000 and cached 1,400,000,000 each must yield input 3,000,000,000 and cached 2,800,000,000; also test individual/cumulative values above i32::MAX, cross-file/day aggregation, retained reports, persistence, and old-cache recovery. Run the relevant Rust tests, formatting, and required repository checks; report any checks you could not run. Return a concise change summary and validation evidence.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions