Skip to content

Json schema yaml validation#1054

Open
dennispaagman wants to merge 2 commits into
rubysec:masterfrom
dennispaagman:json-schema-yaml-validation
Open

Json schema yaml validation#1054
dennispaagman wants to merge 2 commits into
rubysec:masterfrom
dennispaagman:json-schema-yaml-validation

Conversation

@dennispaagman
Copy link
Copy Markdown

I mentioned using JSON Schema to validate / lint the yml files in the Bundler slack as a possible improvement to the workflows for this repo.

I've made a PoC below to see what's possible. It already flagged a few inconsistenties that I also fixed, so that show's there some possible value here.


Right now there are a couple of rules defined in various places:

This PR condenses those into two JSON schemas and adds a test to validate all files against the schemas by converting the yml into JSON-like ruby hashes. (Disclaimer: Claude Code was used, but I checked all output. And I hand wrote this PR description 😂)

JSON Schema is nice as it's more powerful in what you can define than kwalify does (like not allowing certain values). it's supported in most modern editors (you can get autocomplete on values, warnings, etc working when editing files). In a way it can also work as a single source of truth for the rules.


There are a couple of considerations that I can think of from the top of my head:

  • It compares values, not syntax of yml files, so yamllint is probably still needed for that.
  • Editor support does not come out of the box. You need a magic comment to each yml file or set up it yourself.
  • The transformation to Ruby objects is a bit clunky. There are JS based linters that can just run as is, but that would require introducing a JS toolchain into the repo.
  • Maybe it's too strict? I don't have the experience to judge that for this repo.
  • It needs to be added as a github action workflow step.
  • Can we remove kwalify? There's quite a bit of duplication with that and the advisory_* specs. I left them for now.
  • Maybe deduplicate the shared stuff in the schemas.

I've made two separate commits, one that adds the schemas and it's test and one that fixes the issues it flagged in existing files. I've fixed them as I think makes sense, but they can definitely use a second pair of eyes.

Curious to hear what you think, I just wanted to at least put some effort in to show what it could look like.

@dennispaagman dennispaagman force-pushed the json-schema-yaml-validation branch from d034ea4 to c9a9e75 Compare May 17, 2026 18:17
@jasnow
Copy link
Copy Markdown
Contributor

jasnow commented May 17, 2026

I would like to keep the old "rake" method and add this too.

@jasnow
Copy link
Copy Markdown
Contributor

jasnow commented May 17, 2026

Would the README file change any with this new validation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants