deterministic-testing

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

Deterministic runtime for agent evaluation

KayaDB distributed key-value storage engine

Regression memory for coding agents — seal how your repo behaves, then check over MCP (or in CI) that an edit didn't regress it: pass/drift/regressed/missing. With seal history, bisection, and staleness queries.

Reproduce race conditions in real PostgreSQL application code and keep the failing order as a regression test.

every failed agent trace is automatically turned into a regression test, so your agent never makes the same mistake twice

Deterministic, zero-dependency static SAP S/4HANA OData and ERP workflow simulator.

Local-first C# project for deterministic prompt versioning, A/B evaluation, and evidence-based promotion using structured scoring.

Reliability testing for MCP servers: deterministic faults, bounded recovery, and executable regressions.

Retired

Safe Rust metaprogramming and adversarial testing: typed code generation with receipts, proc macros, generated trials, mutation, replay, and reproducible evidence.

Deterministic net.Conn/net.Listener fault injection for Go — the testing/synctest philosophy applied to the network layer.

Autonomous quality-engineering system for Android mobile apps. Learns intent from stories and docs, builds a versioned app twin, explores native APKs on an emulator via Appium (UiAutomator2), and reports only findings reproduced twice without AI.

Record normalized DeepSeek Harness tool results and replay them offline with strict matching and zero tool-body execution.

The deterministic heap groomer for C/C++ memory debugging.

Deterministic UI testing framework for TS/JS — canvas-first. Virtual time, scene contracts, model-driven discovery.

Injectable time for Go: a Clock interface, time-package-compatible timers, tickers and contexts, and a deterministic fake clock for tests, zero dependencies

A Raft consensus library built from scratch in Rust, with a deterministic network simulator for testing leader election, log replication, and partition recovery.

Deterministic evals for AI agents: skill-gain A/B across 11 models (no LLM judge), plus two zero-API regression suites for skill libraries and content merges.

WordleBench — Deterministic AI Wordle benchmark. Compare 34+ LLMs (GPT-5, Claude 4.5, Gemini, Grok, Llama) head-to-head on accuracy, speed, and cost across 50 standardized words.