Skip to content

feat(models): sync Enkrypt safety scores daily - #5622

Open
emilieschario wants to merge 5 commits into
mainfrom
feat/enkrypt-model-scores
Open

feat(models): sync Enkrypt safety scores daily#5622
emilieschario wants to merge 5 commits into
mainfrom
feat/enkrypt-model-scores

Conversation

@emilieschario

@emilieschario emilieschario commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Add daily, change-aware Enkrypt ingestion as optional model metadata—not runtime safety enforcement or a UI change.

  • Match only reviewed model identities and write only changed scores in one batch.
  • Use one shared statistics cache, with eligibility and freshness checked at final publication.
  • Keep health checks read-only; an external monitor owns notifications and recovery.
  • Use one consolidated migration and keep ingestion/publication independently disabled by default.

Verification

  • 1,272 focused tests passed, including database integration and review-regression coverage.
  • Migration consistency checks, lint, and typechecks passed.
  • Authenticated local checks confirmed publication and repeated unchanged runs without score-row rewrites.
  • Re-review found no remaining actionable P1/P2 issues in the implementation.

The broader schema suite has a pre-existing missing-fixture failure; validation above refers to the targeted checks. No credentials, live score values, or raw provider responses are included.

Visual Changes

None.

Follow-up actions

  • Merge current main and regenerate the single Enkrypt migration as 0235_enkrypt_sync_state.
  • Review CI results for the merged revision.
  • Provision isolated staging configuration through the shared environment workflow; migrate and bootstrap the catalog.
  • Run authenticated staging syncs and verify unchanged-row preservation, freshness, failure behavior, and both disable controls.
  • Configure one external monitor for the authenticated read-only health endpoint; test alert delivery, recovery, and scheduler-wide failure detection.
  • Confirm redistribution, attribution, and retention requirements before enabling publication.
  • Deploy the approved revision with one cron owner; enable ingestion and publication separately after their gates are satisfied.

Ready for code review. Production deployment and enablement have not been performed.

@emilieschario emilieschario changed the title feat(models): add external safety benchmark metadata feat(models): sync Enkrypt safety scores daily Sep 1, 2026
@emilieschario
emilieschario marked this pull request as ready for review September 1, 2026 21:35
@emilieschario emilieschario self-assigned this Sep 1, 2026
@kilo-code-bot

kilo-code-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed Enkrypt daily ingestion, identity matching, publication gating, shared stats cache, health probe, cron/API routes, and schema/migration changes; no high-confidence defects on changed lines.

Files Reviewed (45 files)
  • ENVIRONMENT.md
  • apps/web/src/app/api/cron/check-enkrypt-health/route.test.ts
  • apps/web/src/app/api/cron/check-enkrypt-health/route.ts
  • apps/web/src/app/api/cron/sync-enkrypt/route.test.ts
  • apps/web/src/app/api/cron/sync-enkrypt/route.ts
  • apps/web/src/app/api/cron/sync-model-stats/route.test.ts
  • apps/web/src/app/api/cron/sync-model-stats/route.ts
  • apps/web/src/app/api/models/stats/[slug]/route.test.ts
  • apps/web/src/app/api/models/stats/[slug]/route.ts
  • apps/web/src/app/api/models/stats/route.test.ts
  • apps/web/src/app/api/models/stats/route.ts
  • apps/web/src/app/api/openrouter/models/route.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/index.ts
  • apps/web/src/lib/config.server.ts
  • apps/web/src/lib/model-stats/enkrypt-errors.ts
  • apps/web/src/lib/model-stats/enkrypt-fingerprint.test.ts
  • apps/web/src/lib/model-stats/enkrypt-fingerprint.ts
  • apps/web/src/lib/model-stats/enkrypt-identity.test.ts
  • apps/web/src/lib/model-stats/enkrypt-identity.ts
  • apps/web/src/lib/model-stats/enkrypt-publication.test.ts
  • apps/web/src/lib/model-stats/enkrypt-publication.ts
  • apps/web/src/lib/model-stats/enkrypt-status.test.ts
  • apps/web/src/lib/model-stats/enkrypt-status.ts
  • apps/web/src/lib/model-stats/enkrypt.test.ts
  • apps/web/src/lib/model-stats/enkrypt.ts
  • apps/web/src/lib/model-stats/model-stats-cache.test.ts
  • apps/web/src/lib/model-stats/model-stats-cache.ts
  • apps/web/src/lib/model-stats/sync-enkrypt.test.ts
  • apps/web/src/lib/model-stats/sync-enkrypt.ts
  • apps/web/src/lib/model-stats/sync-openrouter.test.ts
  • apps/web/src/lib/model-stats/sync-openrouter.ts
  • apps/web/src/lib/organizations/organization-models.test.ts
  • apps/web/src/lib/organizations/organization-models.ts
  • apps/web/src/lib/organizations/organization-types.ts
  • apps/web/src/routers/admin-router.ts
  • apps/web/src/scripts/enkrypt-coverage.ts
  • apps/web/src/tests/fixtures/enkrypt-scores.ts
  • apps/web/src/tests/openrouter-models.test.ts
  • apps/web/vercel.json
  • docs/enkrypt-sync-operations.md
  • packages/db/src/migrations/0221_enkrypt_sync_state.sql
  • packages/db/src/migrations/meta/0221_snapshot.json
  • packages/db/src/migrations/meta/_journal.json
  • packages/db/src/schema-types.ts
  • packages/db/src/schema.ts

Reviewed by grok-4.6 · Input: 309K · Output: 24.7K · Cached: 1.2M

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant