Skip to content

feat(contract): add API schema drift detection#401

Open
sorlen008 wants to merge 1 commit intojackwener:mainfrom
sorlen008:feat/contract-testing
Open

feat(contract): add API schema drift detection#401
sorlen008 wants to merge 1 commit intojackwener:mainfrom
sorlen008:feat/contract-testing

Conversation

@sorlen008
Copy link
Contributor

Closes #50

What this adds

A contract subcommand that detects when upstream APIs change their response shape — the #1 cause of silent adapter breakage.

Commands

  • opencli contract snapshot <site> <command> [args...] — run a command and save its response schema as a baseline
  • opencli contract check <site> <command> [args...] — run the command again and diff against the baseline
  • opencli contract list — list all saved baselines

How it works

  1. Schema capture — recursively traverses JSON responses and extracts structure (keys, types, nested objects, array element types). Array schemas merge all elements into a union.
  2. Schema diff — compares baseline vs current: fields added (+), removed (-), type changed (~). Recurses into nested objects and array items. Paths use dot notation (data.items[].slug).
  3. Storage — baselines saved at ~/.opencli/contracts/<site>/<command>.json
  4. Exit codescheck exits with code 1 if drift is detected, making it CI-friendly.

Example

# Save baseline
opencli contract snapshot hackernews top

# Later, check for drift
opencli contract check hackernews top
# ✓ No schema drift detected

# If the API changed:
# ~ data[].score: number → string
# - data[].by
# + data[].author
# ✗ 3 schema differences detected

Files

  • src/contract.ts (new) — schema capture, diff engine, storage, formatting
  • src/contract.test.ts (new) — 25 unit tests covering all core functions
  • src/cli.ts — wired up contract subcommand group

Testing

  • npx tsc --noEmit — zero type errors
  • npm test — all 25 new tests pass. 3 pre-existing failures unrelated.

Add contract testing module that captures JSON response schemas,
compares them against saved baselines, and reports structural drift
(fields added, removed, or type-changed).

New CLI subcommands:
- `opencli contract snapshot <site> <command>` — save baseline schema
- `opencli contract check <site> <command>` — diff against baseline
- `opencli contract list` — list saved contracts

Schema storage at ~/.opencli/contracts/<site>/<command>.json.
Includes 25 unit tests covering schema capture, diffing, formatting,
and end-to-end drift detection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sorlen008
Copy link
Contributor Author

UAT Results (Windows 11)

Test Result
Unit tests (25/25) PASS
Type check (tsc --noEmit) PASS
Schema capture + diff (manual) PASS — correctly detects field removal, addition, type changes
Contract storage (save/load round-trip) PASS
CLI wiring (contract list, --help) PASS

All 5 tests pass. No bugs found — clean implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: daily contract check for API schema drift with test accounts

1 participant