Add schema overlay support to the validation engine

[matboros]

Summary

Adds an additionalSchemas option to EngineConfig that lets callers merge extra CloudFormation resource-provider schemas on top of the bundled schemas before validation. This provides an extensibility point for validating templates against resource schemas that are not (yet) part of the engine's bundled set.

Motivation

The engine validates against the CloudFormation provider schemas compiled into the binary. There is currently no way for an embedder to extend or override those schemas at runtime. As a result, a template that uses a resource property the bundled schema does not know about produces false positives such as:

• F3002 — "Additional properties are not allowed ('…' was unexpected)"
• W3030 — a value "is not one of […]" for a property whose allowed set has since expanded

This change lets callers supply the corresponding resource-provider schema so those properties validate correctly. Overlays are additive and per-construction: when a caller stops supplying an overlay, validation reverts to the bundled schema with no residual state.

What changed

• EngineConfig.additionalSchemas (AdditionalSchemaSource { typeName, schema }): each entry is a complete CloudFormation resource-provider schema (registry JSON) supplied as a string. Overlays are merged onto the bundled schema for the matching typeName:

  • properties and definitions are deep-merged,
  • required is unioned,
  • enum values on an overlay property replace the bundled enum for that property path,
  • other defined fields on the overlay override the bundled value; nothing is removed.

  A typeName with no bundled schema is inserted verbatim.

• schema-validator: new SchemaValidator::with_additional_schemas(...) and CompiledSchemaStore::apply_overlay(...), plus a new overlay module that compiles each overlay and merges it into the store.

• Input validation lives in AdditionalSchemaSource::resolve() and fails fast on invalid JSON or a missing type name, keeping the validator constructor infallible.

• Refactor (separate commit): the raw-registry → compiled-schema transform and its types were moved out of the build-only (full-gated) codegen module into a new dependency-free data-source::compiled_schema module, now shared by the build-time codegen and the runtime overlay path. This is a pure move — the emitted compiled_schemas.json is byte-identical, and the module pulls in no dependencies beyond serde/serde_json.

• Bindings (WASM, JVM): thread additionalSchemas through the engine constructors and surface overlay errors to the caller (an invalid overlay throws/raises at engine construction).

TypeScript surface (auto-generated via tsify):

interface AdditionalSchemaSource { typeName: string; schema: string; }
interface EngineConfig {
  customRules?: ExternalRuleSource[];
  guardRules?: ExternalRuleSource[];
  additionalSchemas?: AdditionalSchemaSource[];   // new
}

Testing

• New unit and end-to-end tests in schema-validator (100% line coverage of the overlay module), covering: adding a new property, enum replacement, required union, deep/nested property and definition merges, $ref handling, schema-level metadata overrides, and hard-fail on invalid input.
• Verified through the built WASM package that an overlay schema adding a property to a resource type suppresses the corresponding F3002, and an overlay expanding a property's enum suppresses the corresponding W3030, while the bundled behavior is unchanged when no overlay is supplied.
• cargo build --workspace, cargo test, and cargo clippy pass; the generated compiled_schemas.json is unchanged by the refactor.

Notes

• The API is content-based (schema JSON strings) rather than a filesystem path, so it works uniformly across the WASM, JVM, and native bindings; discovering/reading schema files is left to the embedder.
• Backward compatible: the new field is optional/defaulted for the TypeScript and JVM bindings; only exhaustive Rust struct literals need ..Default::default() (updated in-repo).

[matboros]

CI status _ the two red checks are a pre-existing main failure, not from this change

The failing jobs are only the binding smoke-test jobs _ test-wasm and test-jvm (all OSes). Everything substantive is green: format, lint (clippy), audit, all Rust test jobs (ubuntu/windows/macos), and CodeQL.

Root cause. Both jobs fail on the same assertion, e.g. in bindings-wasm/tests/smoke.test.ts:

AssertionError: expected undefined to be 'Bucket'
expect(d.resourceId).toBe('Bucket');
expect(d.resourceType).toBe('AWS::S3::Bucket');

StandardDiagnostic no longer has top-level resourceId / resourceType _ #206 (new Entity struct / logical ids) and #216 moved them into a nested entity { logicalId, entityType, resourceType }. The current serialized type is:

interface StandardDiagnostic {
ruleId: string;
severity: Severity;
message: string;
source: RuleOrigin;
entity?: Entity; // <- resourceId/resourceType now live here
propertyPath?: string;
...
}

The smoke tests, however, haven't been updated since #176 and still assert the removed resourceId / resourceType fields, so they now read undefined. The JVM binding tests fail for the same reason.

Why this isn't from this PR. This change only adds additionalSchemas support; it doesn't touch the diagnostics crate, StandardDiagnostic, the entity model, or the WASM/JVM tests. Those files on this branch are identical to main, so
test-wasm / test-jvm fail on main independently of this PR.

Suggested resolution. These smoke tests should be updated to assert entity?.logicalId / entity?.resourceType (WASM) and the equivalent on the JVM side _ ideally in a separate fix owned by the diagnostics refactor. Happy to rebase
once that lands, or to open that fix separately if you'd like.

[satyakigh]

Thanks for the PR, this is an useful feature! (Ignore the failing jvm/wasm tests, as long as the rust tests pass its good)
Think there are a couple of issues:

1. oneOf/anyOf/allOf duplication causes false-positive. merge_into appends one_of/any_of/all_of/if_then_else instead of replacing them. Since a complete provider schema can be supplied, overlaying any type with a top-level oneOf duplicates the branches.
   • Potential fix: replace instead of extend one_of/any_of/all_of/if_then_else when the overlay supplies them, or dedupe by structural equality
2. Inline overlay on a $ref property silently drops the referenced definition's constraints
   • Potential fix: before clearing ref_name, resolve the definition and merge its fields into base, then apply the overlay on top, or reject the input
3. resolve() accepts non-object schemas. schema: "42" with an explicit typeName resolves Ok and compiles to an empty no-op overlay
   • Potential fix: reject any non-object JSON in resolve()

[matboros]

Thanks for the thorough review! All three addressed in 1bb1dd8:

1. oneOf/anyOf/allOf/if_then_else duplication. merge_into/merge_prop now replace these when the overlay supplies them, rather than appending _ so a complete provider schema no longer doubles the branches. If the overlay omits them,
   the bundled ones are kept.

• Tests: merge_replaces_composition_keywords_instead_of_appending, merge_keeps_composition_when_overlay_omits_it.

2. Inline overlay on a $ref property dropping the definition's constraints. Went with your first suggestion: when an inline overlay extends a $ref base, the referenced definition is now inlined first (preserving its
   properties/required/constraints), then the overlay is applied on top. To make the definition available at merge time, merge_into merges definitions first and passes the combined definition set down to merge_prop. If the ref can't be
   resolved, it falls back to dropping only the dangling ref so the overlay still takes effect.

• Test: merge_inline_overlay_on_ref_preserves_definition.

3. resolve() accepting non-object schemas. AdditionalSchemaSource::resolve() now rejects any non-object JSON (e.g. "42") with a clear error, so it can no longer compile to an empty no-op overlay.

• Test: additional_schema_resolve_rejects_non_object.

cargo build, cargo test, cargo clippy, and cargo fmt --all --check all pass. Thanks again _ the composition-append one in particular would have been a nasty false positive.

[satyakigh]

Reviewing still

Review withdrawn by reviewer; feedback will be shared privately.

[satyakigh]

.

Reviewing still