reliability-testing

Environmental-test lab platform: SOP execution, ISO 17025 report generation, fixture tracking, and AI-assisted scheduling for temp/humidity chambers. FastAPI · React · SQLite · CI-gated pytest/Vitest.

Reliability testing for MCP servers: deterministic faults, bounded recovery, and executable regressions.

A middleware for LangChain agents that intentionally injects failures (exceptions) into tool and model calls to test agent resilience.

Graph Simulator lets you build and test network configurations with node redundancy and failure scenarios, providing insights into stability and necessary improvements. It logs disconnections and allows for multi-instance simulations to find optimal reliability.

MIT-licensed, inactive n8n reliability samples with synthetic fixtures and no credentials

End-to-end reliability testing for multi-step AI agent workflows. Chaos testing, cascade failure detection, E2E benchmarks. What DeepEval/Langfuse/LangSmith can't test. Works with LangChain, CrewAI, AutoGen, OpenAI. npm install agent-reliability

Webhook delivery reliability testing platform - retry logic validation, idempotency verification, payload integrity, timeout handling, dead letter queue testing. TypeScript + Redis + GitHub Actions CI.

Crash-test your AI agent before real users do.

Conducted in Python, this analysis examines the internal consistency (using Kuder-Richardson Formula 20 [KR-20]), inter-rater reliability (using Intraclass Correlation Coefficient [ICC] and Cohen's Kappa), item difficulty and discrimination indices, and the relationship between multiple-choice and essay sections for mixed-format achievement tests.

Measure LLM output consistency from the command line.

Manual testing of a Productivity Suite application, validating core features for task management, collaboration, and reporting to ensure accuracy, usability, and reliability.

Production-style Databricks Lakehouse Medallion Pipeline using PySpark, Delta Lake, CDC, SCD2, Data Quality, Reliability Testing, Monitoring and Performance Tuning.

Synthetic customer generation and reliability testing platform for conversational voice agents.

SMS-MAN Reliability Report: virtual number delivery benchmarks

Python API Test Automation and Reliability Framework with pytest, YAML/JSON test definitions, REST API testing, PostgreSQL validation, failure classification, defect reports, Docker Compose, and GitHub Actions CI.

Portable, content-addressed reliability evidence for LLM systems. Capture how a model behaves under perturbation; preserve, verify, and diff the evidence across model changes.

Deterministic fault-injection and invariant-testing laboratory for durable SQLite job queues.

Reliability lab for idempotent game reward delivery: Outbox, ledger guard, retries, and invariant tests

Public research artifacts, evaluation frameworks, prototype workflows, and technical documentation for LLM reliability, structured analysis, and applied AI systems.