Skip to content

Add Evidence Lab for claim-level RAG reliability - #1

Draft
YuCPbit wants to merge 2 commits into
masterfrom
portfolio/documentops
Draft

Add Evidence Lab for claim-level RAG reliability#1
YuCPbit wants to merge 2 commits into
masterfrom
portfolio/documentops

Conversation

@YuCPbit

@YuCPbit YuCPbit commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • add evidence-backed document extraction with exact source offsets and risk-aware review routing
  • add a claim-level reliability evaluator for RAG and AI-agent outputs
  • verify quote integrity and preserve SHA-256 source provenance
  • classify claims as direct, inferred, unsupported, or contradicted
  • detect material amount, date, percentage, and negation mismatches
  • add an interactive Evidence Lab and a reproducible 10-case gold-set benchmark

Validation

  • python -m unittest discover -s tests -v — 14 tests passed
  • bash -n scripts/demo-evidenceops.sh
  • python scripts/benchmark-evidence-evaluation.py — 10/10 gold-set cases classified correctly
  • git diff --check

Portfolio focus

This extension demonstrates a focused AI reliability workflow for contracts, policies, research records, and other source-grounded systems where citations, provenance, contradictions, and abstention decisions matter.

@YuCPbit YuCPbit changed the title Add evidence-backed document intelligence agent Add Evidence Lab for claim-level RAG reliability Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant