Skip to content

fix(test): diagnose skipped duplicate-name checks in debug mode - #350

Open
yuhaouno wants to merge 3 commits into
TestSprite:mainfrom
yuhaouno:fix/duplicate-name-debug-diagnostics
Open

yuhaouno wants to merge 3 commits into
TestSprite:mainfrom
yuhaouno:fix/duplicate-name-debug-diagnostics

Conversation

@yuhaouno

@yuhaouno yuhaouno commented Sep 25, 2026 •

Copy link
Copy Markdown

When the best-effort duplicate-name lookup fails, test create currently continues without explaining why the advisory was skipped, even under --debug. This adds a debug-only stderr diagnostic to the shared lookup used by both --code-file and --plan-from.

Refs #186

The change preserves creation and exit behavior, keeps non-debug output byte-identical, and isolates diagnostic-write failures so they cannot prevent creation. It covers only the duplicate-name lookup; the broader issue remains open, and the skill-nudge work completed in #317 is untouched.

Validation:

  • 10 new regression cases cover both creation paths, JSON/text output, default and verbose-only silence, successful lookups, and a throwing diagnostic sink. Against the original source, the four diagnostic assertions fail; they pass with this change.
  • Build and CLI version smoke check pass.
  • ESLint passes with 0 errors (85 existing unused-disable warnings); Prettier and TypeScript checks pass.
  • Local end-to-end suite: 101 passed, 1 skipped.
  • Full unit/coverage suite (env -u NO_COLOR npm run test:coverage): 3,706 passed, 2 skipped; lines 89.98%, branches 87.98%, functions 92.39% (all above the 80% gate). The host sets NO_COLOR=1, which makes seven existing ANSI ticker tests fail; unsetting it makes all 30 ticker tests and the complete suite pass. No ticker code was changed.

Locally verified on macOS / Node 24.16.0. Linux Node 20/22 and Windows remain for repository CI.

Summary by CodeRabbit

  • Documentation
    • Clarified that test creation performs a best-effort lookup for an existing test with the same name for both code-file and plan-based creation. The lookup is skipped during dry runs.
  • Bug Fixes
    • Lookup failures no longer prevent test creation. With --debug, skipped lookups and their reasons are reported on stderr; otherwise, they remain silent. Failures while reporting diagnostics also do not interrupt creation.

@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: TestSprite/testsprite-cli/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 445d63fd-3f6f-4325-91ea-f4c5293c4321

📥 Commits

Reviewing files that changed from the base of the PR and between 1eba879 and f826fff.

📒 Files selected for processing (1)
  • src/commands/test.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/commands/test.test.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 5 remain after this review.


Walkthrough

The duplicate-name advisory reports lookup failures to stderr only when --debug is enabled. Lookup failures and diagnostic-write errors do not block creation. Both code-file and plan-based creation paths pass the debug setting.

Changes

Duplicate-name advisory

Layer / File(s) Summary
Debug diagnostics across create paths
src/commands/test.ts, src/commands/test.test.ts, DOCUMENTATION.md
The advisory reports lookup failures to stderr in debug mode and swallows errors from diagnostic writes. Both create paths pass the debug setting. Tests cover lookup success and failure, output modes, and diagnostic-write errors. The documentation describes the advisory, including its behavior under --dry-run.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to f826f

Lookup failures remain non-fatal in both creation paths, with diagnostics limited to debug mode. The change is ready to merge after normal checks.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding debug-mode diagnostics for skipped duplicate-name checks in test creation.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant