PDX-466: feat(mcp): add AJV JSON schema validation to provar_nitrox_validate#159
Merged
mrdailey99 merged 3 commits intoMay 12, 2026
Merged
Conversation
…d NX rules RCA: provar_nitrox_validate only ran hardcoded NX001–NX010 semantic rules; structural errors (wrong types, extra properties, enum violations) encoded in FactComponent.schema.json were never caught at validation time. Fix: Added Ajv2020 as a runtime dependency; schema is lazily loaded from lib/mcp/rules/FactComponent.schema.json on first call and validated in parallel with existing rules. Violations are returned as NX_SCHEMA_<KEYWORD> issues (ERROR for type/required, WARNING for additionalProperties/pattern/enum). Falls back to hardcoded-rules-only when schema is unavailable.
…on import in NX_SCHEMA tests
RCA: ESLint no-explicit-any rule rejected the any parameter type used for the schemaOverride parameter in AJV schema override tests; the eslint-disable comment was positioned on the wrong line.
Fix: Added import type { ValidateFunction } from ajv/dist/2020.js and replaced all any usages with properly typed ValidateFunction and a narrow IssueShape type alias for the return value.
Quality Orchestrator🟢 LOW · 🧪 Tests to Run · Running 1 of 43 tests
▶ Run commandnpx vitest run \
unit/mcp/nitroXTools.test.ts⚡ quality-orchestrator · |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds AJV-based JSON Schema validation to the provar_nitrox_validate MCP tool so NitroX .po.json files are checked both for existing semantic NX rules and for structural schema conformance, with schema issues surfaced as NX_SCHEMA_* rule IDs and reflected in scoring.
Changes:
- Introduces a lazy, cached AJV 2020-12 schema validator (with graceful fallback when the schema file isn’t available).
- Extends
validateNitroXContent()to optionally run a schema-validation pass and translate AJV errors intoNX_SCHEMA_*issues. - Adds unit tests for
NX_SCHEMA_*behaviors and updates MCP docs + package version/dependency metadata.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/mcp/tools/nitroXTools.ts |
Adds AJV schema loading/caching, error-to-issue mapping, and integrates schema validation into NitroX content validation + tool description updates. |
test/unit/mcp/nitroXTools.test.ts |
Adds tests covering NX_SCHEMA_* issue generation and severity behavior using schema overrides. |
docs/mcp.md |
Documents NX_SCHEMA_* rule IDs and updates provar_nitrox_validate docs to describe the added schema pass. |
package.json |
Adds ajv runtime dependency and bumps version to 1.5.0-beta.19. |
server.json |
Bumps server/package version to 1.5.0-beta.19. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 'Validate a NitroX .po.json (Hybrid Model component page object) against schema rules.', | ||
| 'Works for any NitroX-mapped component type: LWC, Screen Flow, Industry Components, Experience Cloud, HTML5.', | ||
| 'Returns a quality score (0–100) and a list of issues with rule IDs (NX001–NX010), severity, and suggestions.', | ||
| 'Runs two passes in parallel: hardcoded semantic rules (NX001–NX010) and JSON schema validation (NX_SCHEMA_* rule IDs).', |
Comment on lines
+71
to
+73
| const severity: 'ERROR' | 'WARNING' = ['REQUIRED', 'TYPE', 'MIN_ITEMS', 'MINIMUM', 'MAXIMUM'].includes(keyword) | ||
| ? 'ERROR' | ||
| : 'WARNING'; |
Comment on lines
+1646
to
+1649
| Validate a NitroX `.po.json` (Hybrid Model component page object) against the FACT schema rules. Returns a quality score (0–100) and a combined list of issues from two validation passes that run in parallel: | ||
|
|
||
| 1. **Hardcoded semantic rules (NX001–NX010)** — always run | ||
| 2. **JSON schema validation (NX*SCHEMA*\*)** — runs when the bundled `FactComponent.schema.json` is available; falls back to hardcoded-rules-only if the schema cannot be loaded |
| | NX009 | INFO | Interaction `name` contains characters outside `[A-Za-z0-9 ]` | | ||
| | NX010 | INFO | `bodyTagName` contains whitespace | | ||
|
|
||
| **JSON schema rules (NX*SCHEMA*\*):** |
…ma-validation RCA: Copilot flagged incorrect 'in parallel' wording (validation is synchronous/sequential), an overly broad ERROR severity mapping in ajvErrorToIssue (MIN_ITEMS/MINIMUM/MAXIMUM should be WARNING), and broken markdown rendering of NX_SCHEMA_* in docs (underscores parsed as italic markers). Fix: Reworded tool description and docs to 'sequential' passes; narrowed ERROR set to REQUIRED and TYPE only; fixed NX_SCHEMA_* heading and inline text with backtick quoting; updated docs table to show MIN_ITEMS as WARNING.
Collaborator
Author
|
All four review comments addressed in commit 8b60e3d:
|
mrdailey99
added a commit
that referenced
this pull request
May 12, 2026
* PDX-0: feat(mcp): add user-facing guide prompts, tool-guide resource, and git-workflow command RCA: Users and AI agents lacked structured onboarding, troubleshooting, and tool selection guidance when connecting a Provar project via ProvarDX MCP, causing long friction loops and repeated tool selection mistakes that baked-in agent workflows would prevent. Fix: Added three MCP prompts (provar.guide.onboarding, troubleshoot, orchestration), one resource (provar://docs/tool-guide), and a tracked /git-workflow slash command covering Jira ticket creation, branch naming, worktree setup with yarn install, and the full dev/PR lifecycle. Smoke test updated to 54 entries. Gitignore tightened to track .claude/commands/. * PDX-0: chore(ci): update QualityOrchestrator to floating v1 tag RCA: CI workflow pinned QualityOrchestrator at v1.0.0, requiring manual edits to pick up every subsequent patch or minor release, causing the action to drift behind the latest available version. Fix: Created floating v1 tag on mrdailey99/QualityOrchestrator (currently at v1.0.2) and updated CI_Execution.yml to reference @v1, so the workflow automatically uses the latest v1.x release without any further changes needed. * PDX-0: fix(mcp): address Copilot review comments on PR #153 RCA: Copilot flagged 14 issues — wrong tool schemas in guide prompts and docs, missing build copy step, hardcoded cloudId in a public repo file, broken gitignored file reference. Fix: Add PROVAR_TOOL_GUIDE.md to package.json build copy; fix all wrong tool params in guide docs and prompts (properties_generate output_path, --plan-name, testrun_rca project_path, testcase_step_edit test_case_path, testplan add-instance hyphen, defect run_id); remove hardcoded cloudId; remove broken agents ref. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump package json version to 1.5.0-beta.18 * Bump version to 1.5.0-beta.18 in server.json * PDX-463: feat(mcp): fetch NitroX component packages from factPackages repo at release time RCA: NitroX component packages were statically bundled in the repo and not updated automatically; the source of truth is the ProvarTesting/factPackages GitHub repo (main branch), so packages would silently drift stale between releases. Fix: Added scripts/fetch-nitrox-packages.cjs to the prepack hook; it downloads all component package files from factPackages@main, regenerates NITROX_COMPONENT_CATALOG.md, and writes NITROX_CATALOG_SOURCE.json with the commit SHA. On failure (no token, network error) it logs a warning and falls back to the committed catalog — the release is never blocked. A new provar://nitrox/catalog-source MCP resource exposes the bundled version so consumers can verify which factPackages commit is in use. * PDX-463: fix(mcp): correct factPackages path layout and fix lint warnings RCA: The factPackages repo stores component files under fact-*/src/components/ not fact-*/components/, so the path-matching regexes and catalog builder needed updating; additionally nine pre-existing unicorn/numeric-separators-style lint warnings in updateChecker.ts and its test file were left unaddressed. Fix: Updated PKG_JSON_RE and COMPONENT_FILE_RE in fetch-nitrox-packages.cjs to match the fact-*/src/ layout and adjusted buildCatalogFromDir to navigate the src/ subdirectory; ran eslint --fix on updateChecker.ts and updateChecker.test.ts to resolve all numeric-separator warnings, leaving the project at 0 lint errors and 0 warnings. * PDX-463: fix(mcp): address PR review comments on fetch-nitrox-packages RCA: downloadRaw() used the branch name (main) in the raw URL rather than the resolved commit SHA, so files could be fetched from a different commit than the one the tree listing described; additionally both httpsGet and httpsGetBuffer had no timeout, meaning a stalled network connection would block prepack indefinitely. Fix: Added REQUEST_TIMEOUT_MS (15s) to both http helpers via req.setTimeout/req.destroy so hangs fail fast and fall through to the graceful fallback; updated downloadRaw to accept and use the commitSha parameter so all downloads are pinned to the same commit as the tree. * PDX-465: feat(mcp): add bin entry to enable zero-install npx MCP server startup (#158) RCA: No bin entry in package.json forced users through a two-step sf CLI plugin install before connecting Claude Desktop, creating unnecessary onboarding friction. Fix: Added provardx bin entry pointing to bin/mcp-start.js; lightweight ESM entrypoint parses mcp start flags, validates --allowed-paths as required, then delegates to the same server bootstrap used by the sf plugin path. * PDX-464: fetch NitroX schemas from internal source at build time (#157) * PDX-464: feat(mcp): fetch NitroX schemas from internal source at build time RCA: FactComponent.schema and FactPackage.schema were bundled statically and never refreshed from the canonical internal source, risking stale schema validation in released packages. Fix: Extended fetch-nitrox-packages.cjs to download both schemas from the same commit SHA as the component catalog, write to src/mcp/rules/ and root-level copies, and record schemasUpdated in NITROX_CATALOG_SOURCE.json. Falls back to bundled schemas with a warning on any failure. * PDX-464: fix(mcp): address Copilot review — schema consumers, repo field, schemasUpdated normalisation RCA: Four review issues: docs incorrectly named runtime tools as schema consumers; repo field exposed internal URL in MCP resource; readCatalogSource did not normalise missing schemasUpdated from older build artifacts; fallback object also contained the internal URL. Fix: Corrected docs to describe IDE/SchemaStore as schema consumers; removed repo field from emitted JSON and fallback; normalised schemasUpdated to null in readCatalogSource try-path when field is absent; updated tests to cover the new normalisation and assert no repo field in fallback. * PDX-466: feat(mcp): add AJV JSON schema validation to provar_nitrox_validate (#159) * PDX-466: feat(mcp): add AJV JSON schema validation alongside hardcoded NX rules RCA: provar_nitrox_validate only ran hardcoded NX001–NX010 semantic rules; structural errors (wrong types, extra properties, enum violations) encoded in FactComponent.schema.json were never caught at validation time. Fix: Added Ajv2020 as a runtime dependency; schema is lazily loaded from lib/mcp/rules/FactComponent.schema.json on first call and validated in parallel with existing rules. Violations are returned as NX_SCHEMA_<KEYWORD> issues (ERROR for type/required, WARNING for additionalProperties/pattern/enum). Falls back to hardcoded-rules-only when schema is unavailable. * PDX-466: fix(test): replace no-explicit-any with typed ValidateFunction import in NX_SCHEMA tests RCA: ESLint no-explicit-any rule rejected the any parameter type used for the schemaOverride parameter in AJV schema override tests; the eslint-disable comment was positioned on the wrong line. Fix: Added import type { ValidateFunction } from ajv/dist/2020.js and replaced all any usages with properly typed ValidateFunction and a narrow IssueShape type alias for the return value. * PDX-466: fix(mcp): address Copilot review comments on nitrox-ajv-schema-validation RCA: Copilot flagged incorrect 'in parallel' wording (validation is synchronous/sequential), an overly broad ERROR severity mapping in ajvErrorToIssue (MIN_ITEMS/MINIMUM/MAXIMUM should be WARNING), and broken markdown rendering of NX_SCHEMA_* in docs (underscores parsed as italic markers). Fix: Reworded tool description and docs to 'sequential' passes; narrowed ERROR set to REQUIRED and TYPE only; fixed NX_SCHEMA_* heading and inline text with backtick quoting; updated docs table to show MIN_ITEMS as WARNING. * PDX-467: chore(docs): bump version to 1.5.0 stable and remove @beta install tag (#160) * PDX-467: chore(docs): bump version to 1.5.0 stable and remove @beta install tag RCA: The 1.5.0 release is ready for stable promotion; all docs and install commands still referenced the @beta dist-tag and pre-release version string. Fix: Updated package.json and server.json to 1.5.0, removed @beta from all install commands in README.md, docs/mcp.md, and docs/mcp-pilot-guide.md, and updated the stale-cache unit test to use the latest channel. * PDX-467: test(mcp): make stale-cache test release-agnostic by deriving channel at runtime RCA: Hardcoded version and channel values in the stale-cache test will fail once the branch version cycles back to a prerelease (beta/rc) after the 1.5.0 stable release. Fix: Derive currentVersion and channel from the running version at test time, mirroring the pattern used in the fresh-cache test, so the test remains valid across any semver channel.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ajv(v8, Draft 2020-12) as a runtime dependencyprovar_nitrox_validatenow runs two validation passes in parallel: existing hardcoded NX001–NX010 rules and structural JSON schema validation against the bundledFactComponent.schema.jsonNX_SCHEMA_<KEYWORD>issues (ERROR fortype/required, WARNING foradditionalProperties/pattern/enum)1.5.0-beta.19Jira
https://provartesting.atlassian.net/browse/PDX-466
Test plan
yarn compilepassesyarn test:onlypasses (954 tests, 4 new NX_SCHEMA tests)node scripts/mcp-smoke.cjspasses (54/54)yarn lintpassesChanges
src/mcp/tools/nitroXTools.ts: addedAjv2020import, lazy schema validator,ajvErrorToIssue(), updatedvalidateNitroXContent()to accept optional override and run AJV pass; updated tool descriptiontest/unit/mcp/nitroXTools.test.ts: addedNX_SCHEMA_ rules (AJV schema override)test suite (4 tests)docs/mcp.md: documentedNX_SCHEMA_*rule IDs and updatedprovar_nitrox_validatesectionpackage.json/server.json: addedajvdependency, bumped to1.5.0-beta.19