Skip to content

fix: derive default llm-rate from resolved --workers#2

Merged
rasros merged 6 commits into
mainfrom
fix/llm-rate-from-workers
Jul 10, 2026
Merged

fix: derive default llm-rate from resolved --workers#2
rasros merged 6 commits into
mainfrom
fix/llm-rate-from-workers

Conversation

@rasros

@rasros rasros commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Problem

DEFAULT_LLM_RATE = min(2 / DEFAULT_WORKERS, 0.2) was computed once at import time from os.cpu_count() and baked in as the argparse default, so it ignored the --workers the user actually chose. The design intent is "keep ~2 LLM calls in flight", but on a 20-core box with --workers 4 the rate stayed pinned at 0.10 (~0.4 concurrent calls, a much slower/cheaper run than intended); the reverse case over-shot.

Fix

  • Extract _default_llm_rate(workers) = min(2/workers, 0.2).
  • --llm-rate now defaults to None; after parsing, when the user did not pass it explicitly, it's derived from the resolved ns.workers. An explicit --llm-rate still wins.
  • Help text updated to describe the derivation.
  • DEFAULT_LLM_RATE retained (from DEFAULT_WORKERS) for --help display and back-compat.

Tests

  • test_default_llm_rate_tracks_workers (0.2 cap at --workers 4; 2/40 at --workers 40).
  • test_explicit_llm_rate_overrides_workers_derivation.

@rasros rasros merged commit 363adc2 into main Jul 10, 2026
1 check passed
@rasros rasros deleted the fix/llm-rate-from-workers branch July 10, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant