Calibrate detector defaults using benchmark sweep data

[Breakthrough]

It might be worth revisiting detector defaults on a more systematic basis. A full parameter sweep of all detectors across BBC Planet Earth, AutoShot, and ClipShots is now complete (see benchmarks page):

Detector	v0.7 default	Swept best	Mean F1 (tol=1)	Proposal
HistogramDetector	threshold=0.05, bins=256	threshold=0.20, bins=128	54.3 -> 66.3	change
HashDetector	threshold=0.395, size=16	threshold=0.35, size=8	65.8 -> 69.8	change
ContentDetector	threshold=27	threshold=31	72.2 -> 73.4	keep
AdaptiveDetector	threshold=3.0, window=2	threshold=3.5, window=3	74.7 -> 76.3	keep

ThresholdDetector is out of scope for these datasets, because it detects fades, not shot changes. The ordering comes from benchmark/analyze_sweep.py, which requires more than just the best average: a candidate must sit on a plateau (not a spike), beat the current default even with any one dataset removed, not trade away precision (over-splitting is the most common complaint), and clear a minimum +2.0 F1 gain to justify changing user-visible output. Full output is collapsed below.

We also may want to come up with a strategy for dealing with changing the defaults if the addition of more datasets warrants it. At minimum this can help guide on what reasonable values might be.

Rollout

• Land in a minor release (target 0.7.2), not a patch release.
• Changelog callouts with the old values and how to restore them (e.g. detect-hist -t 0.05 -b 256).
• Regenerate the README / website benchmark tables afterwards.

Tasks

• HistogramDetector default threshold=0.20, bins=128
• HashDetector default size=8, threshold=0.35 (or 0.375)
• docstrs / CLI help updated
• Guidance for tuning content/adaptive
• scripts/benchmark_defaults.sh rerun; README + website tables regenerated
• Changelog entries with restore-old-behavior instructions

Default Recommendations from Benchmark Datasets

Generated by benchmark/analyze_sweep.py.

detect-content

Cell	Mean F1@1	BBC	AutoShot	ClipShots	Params
default	72.15	87.72	72.24	56.50	min_scene_len=0.6, threshold=27
best	73.39	85.95	71.51	62.73	min_scene_len=0.6, threshold=31

Candidates (mean F1@1 within 1 of best):

Mean	Micro	NbrDrop	Robust	WeightStable	PrecFloor	Material	Verdict	Params
73.39	72.27	0.41	yes	NO	yes	NO	-	min_scene_len=0.6, threshold=31
73.37	72.76	0.50	yes	NO	yes	NO	-	min_scene_len=0.6, threshold=33
73.32	72.10	1.46	yes	NO	yes	NO	-	min_scene_len=0.4, threshold=33
73.15	71.36	1.00	yes	NO	yes	NO	-	min_scene_len=0.6, threshold=29
73.09	71.23	1.85	yes	NO	yes	NO	-	min_scene_len=0.4, threshold=31
73.04	72.19	1.15	yes	NO	yes	NO	-	min_scene_len=0.4, threshold=35
72.99	72.46	0.22	yes	NO	yes	NO	-	min_scene_len=0.8, threshold=31
72.92	72.65	0.61	yes	NO	yes	NO	-	min_scene_len=0.6, threshold=35
72.87	72.81	0.56	yes	NO	yes	NO	-	min_scene_len=0.8, threshold=33
72.77	71.71	0.80	yes	NO	yes	NO	-	min_scene_len=0.8, threshold=29
72.72	70.06	2.44	NO	yes	yes	NO	-	min_scene_len=0.4, threshold=29

With min_scene_len fixed at the default-equivalent 0.6s:

Mean F1@1	BBC	AutoShot	ClipShots	Params
73.39	85.95	71.51	62.73	min_scene_len=0.6, threshold=31
73.37	84.27	70.95	64.89	min_scene_len=0.6, threshold=33
73.15	87.28	72.05	60.13	min_scene_len=0.6, threshold=29

Recommendation: KEEP current default (no candidate passes all criteria; see verdict column for which gate fails).

detect-adaptive

Cell	Mean F1@1	BBC	AutoShot	ClipShots	Params
default	74.71	92.52	75.48	56.14	adaptive_threshold=3, min_scene_len=0.6, window_width=2
best	76.34	90.41	76.27	62.32	adaptive_threshold=3.5, min_scene_len=0.6, window_width=3

Candidates (mean F1@1 within 1 of best):

Mean	Micro	NbrDrop	Robust	WeightStable	PrecFloor	Material	Verdict	Params
76.34	74.10	1.15	yes	NO	yes	NO	-	adaptive_threshold=3.5, min_scene_len=0.6, window_width=3
76.21	74.56	0.75	yes	NO	yes	NO	-	adaptive_threshold=4, min_scene_len=0.6, window_width=3
76.18	73.36	1.44	yes	NO	yes	NO	-	adaptive_threshold=3.5, min_scene_len=0.4, window_width=3
76.16	74.06	0.72	yes	NO	yes	NO	-	adaptive_threshold=4, min_scene_len=0.4, window_width=3
75.45	74.09	1.21	yes	NO	yes	NO	-	adaptive_threshold=4.5, min_scene_len=0.6, window_width=3
75.44	73.73	1.16	yes	NO	yes	NO	-	adaptive_threshold=4.5, min_scene_len=0.4, window_width=3

With min_scene_len fixed at the default-equivalent 0.6s:

Mean F1@1	BBC	AutoShot	ClipShots	Params
76.34	90.41	76.27	62.32	adaptive_threshold=3.5, min_scene_len=0.6, window_width=3
76.21	87.87	76.17	64.58	adaptive_threshold=4, min_scene_len=0.6, window_width=3
75.45	85.28	75.71	65.37	adaptive_threshold=4.5, min_scene_len=0.6, window_width=3

Recommendation: KEEP current default (no candidate passes all criteria; see verdict column for which gate fails).

detect-hash

Cell	Mean F1@1	BBC	AutoShot	ClipShots	Params
default	65.78	85.01	66.82	45.50	size=16, threshold=0.4
best	69.83	86.38	68.98	54.12	size=8, threshold=0.35

Candidates (mean F1@1 within 1 of best):

Mean	Micro	NbrDrop	Robust	WeightStable	PrecFloor	Material	Verdict	Params
69.83	67.60	1.64	yes	yes	yes	yes	PASS	size=8, threshold=0.35
69.63	68.78	2.62	NO	yes	yes	yes	-	size=8, threshold=0.4
69.63	68.77	-0.00	yes	yes	yes	yes	PASS	size=8, threshold=0.375

Recommendation: CHANGE to size=8, threshold=0.35 (+4.05 mean F1@1 vs default).

detect-hist

Cell	Mean F1@1	BBC	AutoShot	ClipShots	Params
default	54.31	81.21	61.29	20.43	bins=256, threshold=0.05
best	66.27	82.10	68.99	47.72	bins=128, threshold=0.2

Candidates (mean F1@1 within 1 of best):

Mean	Micro	NbrDrop	Robust	WeightStable	PrecFloor	Material	Verdict	Params
66.27	61.87	0.07	yes	yes	yes	yes	PASS	bins=128, threshold=0.2
66.23	62.08	0.04	yes	yes	yes	yes	PASS	bins=128, threshold=0.21
66.20	61.58	0.18	yes	yes	yes	yes	PASS	bins=128, threshold=0.19
66.19	62.25	0.05	yes	yes	yes	yes	PASS	bins=128, threshold=0.22
66.17	62.63	0.08	yes	yes	yes	NO	-	bins=128, threshold=0.24
66.14	62.42	-0.02	yes	yes	yes	NO	-	bins=128, threshold=0.23
66.09	62.70	0.07	yes	yes	yes	NO	-	bins=128, threshold=0.25
66.02	61.11	0.03	yes	yes	yes	yes	PASS	bins=128, threshold=0.18
66.01	62.73	0.13	yes	yes	yes	NO	-	bins=128, threshold=0.26
65.99	60.87	0.17	yes	yes	yes	yes	PASS	bins=128, threshold=0.17
65.88	62.77	0.23	yes	yes	yes	NO	-	bins=128, threshold=0.27
65.83	60.46	0.05	yes	yes	yes	yes	PASS	bins=128, threshold=0.16
65.77	59.99	0.13	yes	yes	yes	yes	PASS	bins=128, threshold=0.15
65.65	62.69	0.23	yes	yes	yes	NO	-	bins=128, threshold=0.28
65.64	59.37	0.25	yes	yes	yes	yes	PASS	bins=128, threshold=0.14
65.42	62.60	0.25	yes	yes	yes	NO	-	bins=128, threshold=0.29
65.39	58.63	0.42	yes	yes	yes	yes	PASS	bins=128, threshold=0.13
65.28	60.49	0.08	yes	yes	yes	yes	PASS	bins=256, threshold=0.21

Recommendation: CHANGE to bins=128, threshold=0.2 (+11.97 mean F1@1 vs default).

[MuhammadBilalKhan267]

It looks like the recommended HashDetector and HistogramDetector defaults from this issue are already present on main (0.35/8 and 0.20/128 respectively), and the changelog also includes #559 entries.

The issue checklist should to be updated to reflect what has already been completed and what work is still outstanding.