Develop#161
Merged
Merged
Conversation
… and git-workflow command RCA: Users and AI agents lacked structured onboarding, troubleshooting, and tool selection guidance when connecting a Provar project via ProvarDX MCP, causing long friction loops and repeated tool selection mistakes that baked-in agent workflows would prevent. Fix: Added three MCP prompts (provar.guide.onboarding, troubleshoot, orchestration), one resource (provar://docs/tool-guide), and a tracked /git-workflow slash command covering Jira ticket creation, branch naming, worktree setup with yarn install, and the full dev/PR lifecycle. Smoke test updated to 54 entries. Gitignore tightened to track .claude/commands/.
RCA: CI workflow pinned QualityOrchestrator at v1.0.0, requiring manual edits to pick up every subsequent patch or minor release, causing the action to drift behind the latest available version. Fix: Created floating v1 tag on mrdailey99/QualityOrchestrator (currently at v1.0.2) and updated CI_Execution.yml to reference @v1, so the workflow automatically uses the latest v1.x release without any further changes needed.
RCA: Copilot flagged 14 issues — wrong tool schemas in guide prompts and docs, missing build copy step, hardcoded cloudId in a public repo file, broken gitignored file reference. Fix: Add PROVAR_TOOL_GUIDE.md to package.json build copy; fix all wrong tool params in guide docs and prompts (properties_generate output_path, --plan-name, testrun_rca project_path, testcase_step_edit test_case_path, testplan add-instance hyphen, defect run_id); remove hardcoded cloudId; remove broken agents ref. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…r-version PDX-0: chore(ci): update QualityOrchestrator to floating v1 tag
…ides PDX-0: feat(mcp): add user-facing guide prompts, tool-guide resource, and git-workflow command
… repo at release time RCA: NitroX component packages were statically bundled in the repo and not updated automatically; the source of truth is the ProvarTesting/factPackages GitHub repo (main branch), so packages would silently drift stale between releases. Fix: Added scripts/fetch-nitrox-packages.cjs to the prepack hook; it downloads all component package files from factPackages@main, regenerates NITROX_COMPONENT_CATALOG.md, and writes NITROX_CATALOG_SOURCE.json with the commit SHA. On failure (no token, network error) it logs a warning and falls back to the committed catalog — the release is never blocked. A new provar://nitrox/catalog-source MCP resource exposes the bundled version so consumers can verify which factPackages commit is in use.
…ings RCA: The factPackages repo stores component files under fact-*/src/components/ not fact-*/components/, so the path-matching regexes and catalog builder needed updating; additionally nine pre-existing unicorn/numeric-separators-style lint warnings in updateChecker.ts and its test file were left unaddressed. Fix: Updated PKG_JSON_RE and COMPONENT_FILE_RE in fetch-nitrox-packages.cjs to match the fact-*/src/ layout and adjusted buildCatalogFromDir to navigate the src/ subdirectory; ran eslint --fix on updateChecker.ts and updateChecker.test.ts to resolve all numeric-separator warnings, leaving the project at 0 lint errors and 0 warnings.
RCA: downloadRaw() used the branch name (main) in the raw URL rather than the resolved commit SHA, so files could be fetched from a different commit than the one the tree listing described; additionally both httpsGet and httpsGetBuffer had no timeout, meaning a stalled network connection would block prepack indefinitely. Fix: Added REQUEST_TIMEOUT_MS (15s) to both http helpers via req.setTimeout/req.destroy so hangs fail fast and fall through to the graceful fallback; updated downloadRaw to accept and use the commitSha parameter so all downloads are pinned to the same commit as the tree.
…ponent-package-retrieval PDX-463: feat(mcp): fetch NitroX component packages from factPackages repo at release time
…er startup (#158) RCA: No bin entry in package.json forced users through a two-step sf CLI plugin install before connecting Claude Desktop, creating unnecessary onboarding friction. Fix: Added provardx bin entry pointing to bin/mcp-start.js; lightweight ESM entrypoint parses mcp start flags, validates --allowed-paths as required, then delegates to the same server bootstrap used by the sf plugin path.
* PDX-464: feat(mcp): fetch NitroX schemas from internal source at build time RCA: FactComponent.schema and FactPackage.schema were bundled statically and never refreshed from the canonical internal source, risking stale schema validation in released packages. Fix: Extended fetch-nitrox-packages.cjs to download both schemas from the same commit SHA as the component catalog, write to src/mcp/rules/ and root-level copies, and record schemasUpdated in NITROX_CATALOG_SOURCE.json. Falls back to bundled schemas with a warning on any failure. * PDX-464: fix(mcp): address Copilot review — schema consumers, repo field, schemasUpdated normalisation RCA: Four review issues: docs incorrectly named runtime tools as schema consumers; repo field exposed internal URL in MCP resource; readCatalogSource did not normalise missing schemasUpdated from older build artifacts; fallback object also contained the internal URL. Fix: Corrected docs to describe IDE/SchemaStore as schema consumers; removed repo field from emitted JSON and fallback; normalised schemasUpdated to null in readCatalogSource try-path when field is absent; updated tests to cover the new normalisation and assert no repo field in fallback.
…alidate (#159) * PDX-466: feat(mcp): add AJV JSON schema validation alongside hardcoded NX rules RCA: provar_nitrox_validate only ran hardcoded NX001–NX010 semantic rules; structural errors (wrong types, extra properties, enum violations) encoded in FactComponent.schema.json were never caught at validation time. Fix: Added Ajv2020 as a runtime dependency; schema is lazily loaded from lib/mcp/rules/FactComponent.schema.json on first call and validated in parallel with existing rules. Violations are returned as NX_SCHEMA_<KEYWORD> issues (ERROR for type/required, WARNING for additionalProperties/pattern/enum). Falls back to hardcoded-rules-only when schema is unavailable. * PDX-466: fix(test): replace no-explicit-any with typed ValidateFunction import in NX_SCHEMA tests RCA: ESLint no-explicit-any rule rejected the any parameter type used for the schemaOverride parameter in AJV schema override tests; the eslint-disable comment was positioned on the wrong line. Fix: Added import type { ValidateFunction } from ajv/dist/2020.js and replaced all any usages with properly typed ValidateFunction and a narrow IssueShape type alias for the return value. * PDX-466: fix(mcp): address Copilot review comments on nitrox-ajv-schema-validation RCA: Copilot flagged incorrect 'in parallel' wording (validation is synchronous/sequential), an overly broad ERROR severity mapping in ajvErrorToIssue (MIN_ITEMS/MINIMUM/MAXIMUM should be WARNING), and broken markdown rendering of NX_SCHEMA_* in docs (underscores parsed as italic markers). Fix: Reworded tool description and docs to 'sequential' passes; narrowed ERROR set to REQUIRED and TYPE only; fixed NX_SCHEMA_* heading and inline text with backtick quoting; updated docs table to show MIN_ITEMS as WARNING.
…nstall tag (#160) * PDX-467: chore(docs): bump version to 1.5.0 stable and remove @beta install tag RCA: The 1.5.0 release is ready for stable promotion; all docs and install commands still referenced the @beta dist-tag and pre-release version string. Fix: Updated package.json and server.json to 1.5.0, removed @beta from all install commands in README.md, docs/mcp.md, and docs/mcp-pilot-guide.md, and updated the stale-cache unit test to use the latest channel. * PDX-467: test(mcp): make stale-cache test release-agnostic by deriving channel at runtime RCA: Hardcoded version and channel values in the stale-cache test will fail once the branch version cycles back to a prerelease (beta/rc) after the 1.5.0 stable release. Fix: Derive currentVersion and channel from the running version at test time, mirroring the pattern used in the fresh-cache test, so the test remains valid across any semver channel.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR prepares the ProvarDX CLI for a stable 1.5.0 release while expanding MCP usability via zero-install startup, new guided prompts/resources, and NitroX schema-backed validation + build-time catalog/schema refresh.
Changes:
- Promote release from
1.5.0-beta.*to1.5.0across manifests/docs and add aprovardxzero-installbin/mcp-start.jsentrypoint. - Add three new MCP “guide” prompts and a new MCP resource for NitroX catalog source metadata, plus a bundled tool-selection guide doc.
- Add AJV-based JSON schema validation for NitroX
.po.json(emitsNX_SCHEMA_*issues) and a release-time script to refresh NitroX catalog + schemas.
Reviewed changes
Copilot reviewed 19 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| test/unit/mcp/updateChecker.test.ts | Adjusts cache staleness tests to derive channel/version dynamically. |
| test/unit/mcp/server.test.ts | Adds unit tests for readCatalogSource() behavior and normalization. |
| test/unit/mcp/nitroXTools.test.ts | Adds tests for AJV-backed NX_SCHEMA_* validation behavior. |
| test/unit/bin/mcpStart.test.ts | Adds argument-validation tests for the new bin/mcp-start.js entrypoint. |
| src/mcp/update/updateChecker.ts | Minor constant formatting tweaks (no functional update flow change). |
| src/mcp/tools/nitroXTools.ts | Adds AJV schema loading/compilation and additive schema validation pass. |
| src/mcp/server.ts | Registers new MCP resources (NitroX catalog source + tool guide) and adds readCatalogSource(). |
| src/mcp/prompts/index.ts | Registers new guide prompts. |
| src/mcp/prompts/guidePrompts.ts | New “onboarding”, “troubleshoot”, and “orchestration” prompt definitions. |
| server.json | Updates published MCP server version metadata to 1.5.0. |
| scripts/mcp-smoke.cjs | Expands smoke coverage to include new prompts and updates expected counts. |
| scripts/fetch-nitrox-packages.cjs | New release utility to fetch NitroX packages/catalog + schema files and write source metadata. |
| README.md | Updates install/quickstart guidance, adds “zero-install” npx-based setup path. |
| package.json | Bumps to 1.5.0, adds AJV dep, adds provardx bin entry, extends prepack/compile artifacts. |
| docs/PROVAR_TOOL_GUIDE.md | New bundled tool selection guide doc. |
| docs/NITROX_CATALOG_SOURCE.json | Adds a committed default source metadata JSON file. |
| docs/mcp.md | Updates install guidance and documents schema sourcing + new NitroX resource. |
| docs/mcp-pilot-guide.md | Updates install guidance away from @beta. |
| bin/mcp-start.js | New lightweight ESM entrypoint that validates args, validates license, checks updates, starts MCP over stdio. |
| .gitignore | Refines .claude ignore rules and clarifies tracked command docs. |
| .github/workflows/CI_Execution.yml | Changes QualityOrchestrator action version reference. |
| .claude/commands/git-workflow.md | Adds a documented git workflow command for Claude tooling. |
Comments suppressed due to low confidence (1)
README.md:104
- This link text says "7 MCP prompts", but this PR registers 3 new
provar.guide.*prompts (total is now higher). Update the prompt count so the README stays accurate.
📖 docs/mcp.md — full setup, all 35+ tools, 7 MCP prompts, troubleshooting.
</details>
---
💡 <a href="/ProvarTesting/provardx-cli/new/main?filename=.github/instructions/*.instructions.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add Copilot custom instructions</a> for smarter, more guided reviews. <a href="https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn how to get started</a>.
Comment on lines
+66
to
+78
| function ajvErrorToIssue(err: ErrorObject): NitroXIssue { | ||
| const keyword = err.keyword.replace(/([a-z])([A-Z])/g, '$1_$2').toUpperCase(); | ||
| const instancePath = err.instancePath; | ||
| const appliesTo = instancePath ? instancePath.replace(/^\//, '').replace(/\//g, '.') : 'root'; | ||
| const pathParts = instancePath.split('/').filter(Boolean); | ||
| const severity: 'ERROR' | 'WARNING' = ['REQUIRED', 'TYPE'].includes(keyword) ? 'ERROR' : 'WARNING'; | ||
| const issue: NitroXIssue = { | ||
| rule_id: `NX_SCHEMA_${keyword}`, | ||
| severity, | ||
| message: `Schema: ${instancePath || 'root'} — ${err.message ?? 'validation failed'}`, | ||
| applies_to: appliesTo, | ||
| }; | ||
| if (pathParts.length > 0) issue.field = pathParts[pathParts.length - 1]; |
Comment on lines
+2
to
+5
| "repo": "https://github.com/ProvarTesting/factPackages", | ||
| "branch": "main", | ||
| "commitSha": null, | ||
| "fetchedAt": null |
| "node": ">=18.0.0 <25.0.0" | ||
| }, | ||
| "bin": { | ||
| "provardx": "./bin/mcp-start.js" |
Comment on lines
+53
to
+57
| "mcpServers": { | ||
| "provar": { | ||
| "command": "npx", | ||
| "args": ["-y", "@provartesting/provardx-cli", "mcp", "start", "--allowed-paths", "/path/to/your/provar/project"] | ||
| } |
4 tasks
mrdailey99
added a commit
that referenced
this pull request
May 12, 2026
RCA: Four issues identified in post-merge Copilot review: AJV error mapping for additionalProperties/required used instancePath (parent object) instead of err.params, NITROX_CATALOG_SOURCE.json had an exposed repo field and missing schemasUpdated key, the provardx-cli bin alias was absent making npx resolution ambiguous, and prompt counts in README/docs were stale at 7 (now 11). Fix: ajvErrorToIssue now special-cases additionalProperties and required keywords to pull field/applies_to from err.params; committed JSON normalised to stable shape; package.json adds provardx-cli bin alias; README.md and docs/mcp.md updated to 11 MCP prompts.
mrdailey99
added a commit
that referenced
this pull request
May 12, 2026
…talog source shape, bin alias, prompt count + test coverage (#164) * PDX-476: fix(mcp): address Copilot review follow-ups from PR #161 (#162) RCA: Four issues identified in post-merge Copilot review: AJV error mapping for additionalProperties/required used instancePath (parent object) instead of err.params, NITROX_CATALOG_SOURCE.json had an exposed repo field and missing schemasUpdated key, the provardx-cli bin alias was absent making npx resolution ambiguous, and prompt counts in README/docs were stale at 7 (now 11). Fix: ajvErrorToIssue now special-cases additionalProperties and required keywords to pull field/applies_to from err.params; committed JSON normalised to stable shape; package.json adds provardx-cli bin alias; README.md and docs/mcp.md updated to 11 MCP prompts. * PDX-476: test(mcp): assert field/applies_to for AJV additionalProperties and required errors RCA: Copilot review noted that the new ajvErrorToIssue params-based mapping for additionalProperties and required keywords had no test coverage for the field and applies_to values, leaving the mapping free to silently regress. Fix: Expanded NX_SCHEMA_ADDITIONAL_PROPERTIES test to assert field and applies_to equal the extra property name; added NX_SCHEMA_REQUIRED case asserting field and applies_to equal the missing property name; added applies_to/field assertions to the NX_SCHEMA_TYPE test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.