Skip to content

fix: address review findings on billing, markup, and prompt fidelity - #15

Open
getdownandcode wants to merge 1 commit into
featfrom
fix/review-bugfixes
Open

getdownandcode wants to merge 1 commit into
featfrom
fix/review-bugfixes

Conversation

@getdownandcode

Copy link
Copy Markdown
Owner

Summary

Fixes all 8 findings from the code review (bugs 1\u20138), each with regression tests where meaningful.

High

1. Free-tier users billed at paid rates \u2014 commit_flow now passes free_tier=cfg.free_tier to both telemetry calls (initial generation and regenerate), so rows are stored with the correct flag and get_invocations no longer reprices them at paid rates. Free-tier usage can no longer trip budget_cap_usd.

2. Rich markup ate text in square brackets \u2014 commit preview panel, git stdout/stderr, and the a/e/r/c menus now render as literal text (rich.text.Text / rich.markup.escape / \[…\]). fix(cli): handle list[str] input [skip ci] displays verbatim; git's [main 1a2b3c4] … line keeps the branch and hash; invalid input containing [/x] no longer crashes with MarkupError.

Medium

3. Truncated files lost their filename \u2014 patches replaced by [patch omitted for size: …] are now prefixed with File <path> (<status>): in the prompt (new _prompt_diff_text, OMITTED_NOTE_PREFIX).

4. Binary changes never reached the model \u2014 binary files are now included in the prompt as Binary file <path> (<status>).

5. pr-summary / changelog ignored ignore_globs \u2014 both now pass extra_ignores=cfg.ignore_globs to DiffExtractor, matching commit and debug-diff.

6. Cached chunk summaries were billed \u2014 CachedProvider exposes last_hit; summarize_omitted_diffs excludes cache hits from billed tokens, and the orchestrator uses last_hit (instead of probing cache internals) to classify the main call. A fully cached rerun now costs $0.

Low

7. Budget warning printed twice \u2014 new check_budget flag (default True) on generate_commit_message/run_generation_pipeline; callers that already ran check_budget_cap pass check_budget=False.

8. Misleading fallback message \u2014 token-counting failures now print "Token budgeting failed, using template fallback" instead of "API unavailable".

Testing

  • 304 tests pass (uv run pytest -q) \u2014 includes new regressions: free-tier $0 spend, literal brackets end-to-end, prompt labels for truncated/binary files, cached-chunk $0 billing, single budget check, ignore_globs in pr-summary
  • ruff check . and ruff format . clean
  • mypy (strict) clean
  • One stale test that pinned the old bug-6 behaviour was updated (test_generate_commit_message_fully_cached_rerun_costs_nothing)
@@ tests/test_orchestrator.py
-    # Final call cost was 0 (cache hit), but chunk_cost MUST still be preserved!
-    assert res2.estimated_cost_usd == chunk_cost
+    # No API tokens were billed on the rerun: cached chunk summaries are excluded.
+    assert res2.estimated_cost_usd == 0.0

Note: this PR is based on feat (not main) so it contains only the fixes \u2014 they'll flow into main via the open #14.

- free-tier: pass free_tier to commit telemetry so spend is not repriced at paid rates (bug 1)
- ui: render preview, git output, and menus as literal text so [brackets] survive (bug 2)
- prompt: label size-truncated and binary files by path in the prompt (bugs 3, 4)
- config: honour ignore_globs in pr-summary and changelog extractors (bug 5)
- cache: don't bill chunk summaries served from cache; track last_hit on CachedProvider (bug 6)
- budget: check the monthly cap once per command via check_budget flag (bug 7)
- fallback: say 'Token budgeting failed' instead of 'API unavailable' (bug 8)
@getdownandcode

Copy link
Copy Markdown
Owner Author

🤖 Automated PR Review: #15 - fix: address review findings on billing, markup, and prompt fidelity

1. Summary of Changes

  • Scope: Addresses 8 defect findings from previous code review across billing metrics, terminal UI rendering, prompt fidelity, and chunk cache accounting.
  • Key Files Modified: src/gitmate/committer.py, src/gitmate/orchestrator.py, src/gitmate/chunking.py, src/gitmate/providers/cache.py, src/gitmate/changelog.py, src/gitmate/pr_summary.py, src/gitmate/fallback.py, src/gitmate/token_budget.py, tests/test_committer.py, tests/test_orchestrator.py, tests/test_pr_summary.py

2. Analysis & Key Highlights

  • Strengths:
    • Accurate Telemetry & Spend Isolation: Properly threads free_tier=active_cfg.free_tier into commit_flow telemetry calls, preventing free-tier users from having usage repriced at commercial API rates.
    • Crash-Resistant Rich UI: Wraps user message text, git command stdout/stderr, and prompt options with Text(...) and escape(...), neutralizing MarkupError exceptions when commit messages contain unescaped brackets like [skip ci] or [feat].
    • O(1) Cache State Tracking: Introduces last_hit attribute on CachedProvider, eliminating intrusive cache key probing and ensuring cached chunk summaries are not double-billed on subsequent runs.
    • Prompt Fidelity: Restores file path and change status context for omitted/truncated patches (File <path> (<status>): ...) and binary files (Binary file <path> (<status>)).
  • Observations:

3. Actionable Suggestions & Concerns

  • [Severity: Medium] [tests/test_committer.py:270, 295, 470]: Ambient GIT_EDITOR shadows EDITOR in interactive edit unit tests. In test_interactive_edit_compound_editor_and_comment_stripping, test_interactive_edit_aborts_on_empty_commit_message, and test_interactive_edit_trailing_whitespace_stripped, monkeypatch.setenv("EDITOR", ...) is called without clearing higher-precedence editor variables. When GIT_EDITOR is present in the ambient environment (common in CI/subshells, e.g. GIT_EDITOR=false), _resolve_editor() selects GIT_EDITOR, causing editor execution to fail with exit code 1.
    # Ensure clean isolation across all interactive edit tests:
    monkeypatch.delenv("GIT_EDITOR", raising=False)
    monkeypatch.delenv("VISUAL", raising=False)
    monkeypatch.setenv("EDITOR", f"{sys.executable} -c \"{editor_script}\"")
  • [Severity: Low] [src/gitmate/orchestrator.py:266]: Inconsistent provider duck-typing. chunking.py:68 checks getattr(provider, "last_hit", False), whereas orchestrator.py:266 uses isinstance(active_provider, CachedProvider) and active_provider.last_hit.
    # Suggestion: Use consistent duck-typing in orchestrator.py:
    cache_hit = bool(getattr(active_provider, "last_hit", False))
  • [Severity: Low] [tests/test_orchestrator.py:446-447, 461-462]: Unpack typing suppression. The tests use # type: ignore[arg-type] due to dictionary unpacking (**kwargs / **common). Calling generate_commit_message(diffs=_sample_diff(), cfg=cfg, provider=provider, counter=FakeCounter()) with explicit keyword arguments preserves type checking and avoids suppression comments.

4. Merge Status

Verdict: READY TO MERGE
Reasoning: Resolves all 8 target defects with rigorous regression tests, clean type hints, and zero breaking changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant