selective-classification

A list of papers that studies out-of-distribution (OOD) detection and misclassification detection (MisD)

Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

PyTorch implementation of our ECCV 2022 paper "Rethinking Confidence Calibration for Failure Prediction"

A decision-safety lab for loan approval: trains a baseline classifier, calibrates probabilities (ECE/Brier), sweeps confidence thresholds to build a coverage, quality frontier and outputs a defensible abstention policy (auto-decide vs review). Includes a Streamlit dashboard for report cards, triage UI, and data quality checks.

Longform article reframing abstention (reject option / selective prediction) as product design, not model weakness. Covers coverage as a KPI, calibration as a prerequisite, threshold selection under review capacity and risk, queue/UX design for human-in-the-loop workflows, and anti-patterns that break safety in production.

Decision-safe evaluation + Streamlit dashboard for AI vs Human vs Post-Edited AI text detection. Generates a reliability report card (Accuracy, Macro F1, ECE, Brier), calibration plots, confidence histograms, and a coverage-vs-performance abstention curve. Recommends an operating threshold for human-review routing.

Promises and Pitfalls of Threshold-based Auto-labeling (NeurIPS 2023, Spotlight)

Code for "Budgeted Classification with Rejection: An Evolutionary Method with Multiple Objectives", at IEEE CEC 2022

Robust Selective Classification of Skin Lesions with Asymmetric Costs

Reliable and Efficient Selective Machine Learning

Confidence-aware calibration framework for blood agglutination image classification (MobileNetV2/EfficientNetV2 + AutoGluon, isotonic calibration, class-conditional review policy). Externally validated on NIH Malaria dataset. Paper + reproducible pipeline.

More human oversight can make an AI agent less safe. Headroom is a human-in-the-loop firewall for coding agents that measures when to trust the human: oversight as resource allocation, not just classification.

‘‘I know that I don’t know... and I explain why’’ Robust abstention via counterfactual explanations

Event-level risk-controlled triage for conjunction data message streams

Recorded EEG decoding and reusable command audits with participant results, timing, instructed rest and reproducible evaluation

Applied NLP decision-support project for CFPB complaint routing, comparing TF-IDF + Linear SVM with DistilBERT through leakage-safe evaluation, selective routing, Human Review, and 2025 retrospective analysis.

Risk-aware banking support router for 77 intents using calibrated classification, selective prediction, human escalation policies, and FastAPI.

Few-shot text classifiers that know when they don't know: selective classification with certified risk (SGR) + episodic prototypical fine-tuning. 12 Portuguese corpora, 7 OOD baselines.

Official implementation of the ICLR paper "Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It"

8-Chapter Reproducible Training Capsule (Docker / Jupyter) on Deep Learning, Uncertainty Estimation (MC Dropout / Ensembles) & Selective Risk Control