Skip to content

JWT claim + docs-vs-implementation detector (v0.3.0)#22

Merged
Shakargy merged 1 commit into
mainfrom
v0.3.0-jwt-claim-docs-contradiction
Jul 10, 2026
Merged

JWT claim + docs-vs-implementation detector (v0.3.0)#22
Shakargy merged 1 commit into
mainfrom
v0.3.0-jwt-claim-docs-contradiction

Conversation

@Shakargy

Copy link
Copy Markdown
Owner

Summary

Second verification slice, proving the claim registry generalizes.

  • New built-in claim jwt-authentication - "Authentication uses JWT access tokens", powered by the existing JWT purpose classifier (access vs invitation/verification tokens)
  • New contradiction detector (docs-vs-implementation): documentation claims JWT while the only JWT usage found is invitation-purpose -> CONTRADICTED with both sides and exact paths
  • Honesty rule preserved: docs with NO usage found is WEAK (missing evidence), never CONTRADICTED - absence is not positive conflicting evidence
  • Evaluator registry replaces single-claim dispatch: adding claims = adding definitions

Bug found and fixed

Signal metadata_json was hardcoded to '{}' at persistence since V0 - every extractor's metadata silently dropped. Concept detection never noticed (it runs in-process on live Signal objects); the verification engine is the first feature reading metadata back from the database, and it exposed the bug immediately. Fixed with a regression test asserting the JWT purpose survives the round-trip.

Verification

  • 126 tests passing (6 new: SUPPORTED/WEAK/CONTRADICTED/UNKNOWN for the JWT claim, both-sided contradiction content, metadata persistence regression, verify-all returns both claims)
  • demo-saas now verifies both claims: billing-webhook-signature SUPPORTED, jwt-authentication SUPPORTED (access usage + decision corroboration)
  • build + twine clean

No breaking changes. Version 0.3.0. No tag, no publish - awaiting approval.

…3.0)

Second verification slice:

- New built-in claim jwt-authentication ("Authentication uses JWT access
  tokens"), powered by the v0.0.6 JWT purpose classifier.
- New contradiction detector: documentation claims JWT while the only JWT
  usage found is invitation/verification tokens -> CONTRADICTED with both
  sides. Documentation with no usage at all stays WEAK, never contradicted:
  absence is not positive conflicting evidence.
- Evaluator registry replaces the single hardcoded dispatch, so adding claims
  is adding definitions.
- fix: signal metadata was hardcoded to '{}' at persistence, silently dropping
  extractor metadata (including JWT purpose) since V0. Concept detection never
  noticed (in-process); verification is the first DB reader of metadata and
  exposed it. Regression test added.

126 tests (6 new). No breaking changes. Version 0.3.0.
@Shakargy Shakargy merged commit 2cfc70b into main Jul 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant