docs: ledger row 31 root cause for stalled bulk audits

docs/app_bugs.md

@@ -42,7 +42,7 @@ Format: append-only. When a bug is fixed in the app, mark `status: fixed` and th
 | 28 | evaluations list / sorting | Clicking a sortable column header (e.g. "Evaluation Name") on the evaluations table's first click does **nothing at all** — no visual reorder, and the header's `aria-sort` attribute stays `None`/absent. The second click (and every click after) does re-sort the rows correctly. Confirmed live 2026-08-13 with a scripted repro that clicked the "Evaluation Name" header 3 times in a row with a 3s settle between each: click 1 → row order and `aria-sort` unchanged from the pre-click state; click 2 → rows visibly reordered; click 3 → rows reordered again — but `aria-sort` never changes from `None` across all three clicks, so screen-reader users get no indication a sort is active even once engaged (a secondary a11y defect on top of the primary one-click-eaten defect). The `DataTable-module_Heading__*` header button is rendered by the shared `opub-ui` `DataTable` component (no local source in `ParakhAI-frontend` — it's a compiled dependency), so the root cause could not be pinned to an exact line; likely a debounce/state-init bug where the component's internal sort-state only starts reacting to clicks after the first one initializes it. | open | Log in as `TEST_EMAIL_1` → AI Maker → CivicDataLab → Evaluations. Click the "Evaluation Name" column header once. Row order is unchanged and `document.querySelector("th button:has-text('Evaluation Name')").getAttribute('aria-sort')` stays `null`. Click it again — rows now visibly reorder. | `tests/e2e/test_evaluations_list_controls.py::TestSortableHeaders::test_sort_by_name_toggles_order` | 2026-08-13 |
 
 | 30 | evaluations list / default sort | The default (unfiltered, unsorted) `audits` GraphQL query the evaluations list loads with does **not** order by recency — a newly created draft never appears on the list's landing page, no matter how long you wait or how many times you hard-reload. Root-caused 2026-08-17 via a live network capture of the actual `GetAudits` response (not a synthetic probe): four separate fresh-draft creations in the same org (audit ids 2042–2045, confirmed to exist individually via `audit(auditId)` and to rank at the top of a raw `audits(limit:20)` GraphQL call with no `sortOptions`) each produced an identical captured frontend response — `ids=['1916','1957','1953','1943','1944','1954','1961','1909','1964','1958']`, `total=141` — completely unrelated to the just-created id and byte-identical across all four runs, including after a full `page.reload()` + 45s app-ready wait. Response headers carried no `cache-control`/`age`/`x-cache` (200, no HTTP caching), so this isn't a CDN/browser cache — the frontend's actual `GetAudits` call (with its real field selection, distinct from a bare `id name status` probe) is deterministically returning the same fixed page of ~10 rows regardless of what's been created since, meaning its effective sort key is something other than recency (or a query-plan/index quirk that happens to be stable across requests). User impact: creating an evaluation and clicking "Back to List" gives no way to find your own draft without knowing to search or filter — among 141 rows it could be anywhere. Related to bug #13 (same `audits`/`GetAudits` query family, full-field-vs-light-field behavior) but a distinct symptom — not a hang, a wrong/stale-looking default ordering. | open | Log in as `TEST_EMAIL_1` → AI Maker → CivicDataLab → New Evaluation → Bulk → Start (creates a draft, capture its `auditId` from the URL) → Back to List. The draft's row (`a[href*='auditId=<id>']`) is absent from the table even after a full page reload and a 45s wait. Confirm via GraphQL that the audit exists (`audit(auditId: "<id>")`) and that a raw `audits(limit:20)` call (no `sortOptions`) ranks it near the top — yet the frontend's own list request returns a fixed, unrelated set of older ids. Reproduced 4/4 times in complete isolation (`-n 1 --reruns 0`, no concurrent suite load) 2026-08-17. | `tests/e2e/test_add_evaluation_bulk.py::TestBulkDraftPersistence::test_new_draft_appears_in_draft_tab_with_wizard_link` | 2026-08-17 |
-| 31 | bulk evaluation / library run stall | Two Bulk runs on prod using a prompt library on the General scope froze mid-run: audit 910 (Cultural, Bias, Gemma 4 26B) at 74% and audit 911 (Technical, all 3 modules, GPT-5 Mini) at 84%, both IN_PROGRESS with `totalTests` 0 for 30+ min. `updateAudit(status: CANCELLED)` left them IN_PROGRESS. 8 other bulk runs (library and custom, all scopes) completed in 1-3 min. Found by the pairwise matrix (`tests/e2e/test_evaluation_matrix.py`, opt-in via `EVAL_MATRIX`). Filed as ParakhAI-Backend#134 (2026-11-05); not fixed. Companion observation: audit 912 (Haiku) went FAILED in seconds with `No successful AuditTest objects found` — model inference failure, passes on re-run.  Follow-up 2026-11-06: re-ran the General/Technical/all-modules/library case once; audit 941 went FAILED in ~47s instead of stalling, so the stall is not deterministic (model chosen at random by the test; failure reason not read). |
+| 31 | bulk evaluation / library run stall | Two Bulk runs on prod using a prompt library on the General scope froze mid-run: audit 910 (Cultural, Bias, Gemma 4 26B) at 74% and audit 911 (Technical, all 3 modules, GPT-5 Mini) at 84%, both IN_PROGRESS with `totalTests` 0 for 30+ min. Cancel is refused by design for bulk audits in IN_PROGRESS (`Cannot cancel a bulk evaluation in IN_PROGRESS status`; only DRAFT/PENDING_REVIEW are cancellable), so nothing can clear them from the API. 8 other bulk runs (library and custom, all scopes) completed in 1-3 min. Found by the pairwise matrix (`tests/e2e/test_evaluation_matrix.py`, opt-in via `EVAL_MATRIX`). Filed as ParakhAI-Backend#134 (2026-11-05); not fixed. Companion observation: audit 912 (Haiku) went FAILED in seconds with `No successful AuditTest objects found` — model inference failure, passes on re-run.  Follow-up 2026-11-06: re-ran the General/Technical/all-modules/library case once; audit 941 went FAILED in ~47s instead of stalling, so the stall is not deterministic (model chosen at random by the test; failure reason not read).  Probable root cause from reading ParakhAPI `apps/model_audits/tasks.py` (not confirmed against prod logs): `run_audit_task` does `raise self.retry(exc=e)` inside `try/except MaxRetriesExceededError`, but Celery re-raises the original `exc` once retries are used up, so that handler never runs. The AuditTask stays RUNNING, the Redis `completed` counter is never incremented (progress freezes below 100%), and the failed chord header means `finalize_audit_callback` never runs (totalTests stays 0). Any one task failing 4 times strands the whole audit IN_PROGRESS. |
 | 32 | evaluations list / status filter + pagination | Applying a status filter while on page 2+ keeps the old page offset. The list shows `GraphQL Error: offset (10) is out of range (0–2)` with a Retry button instead of the filtered rows. Seen in CI 2026-11-05 (screenshot of shard 2). Filed as ParakhAI-frontend#477. | open | Evaluations list with 2+ pages → click Next → open Filter Status → pick a status → Apply. | `tests/e2e/test_evaluations.py::TestEvaluationsPagination::test_status_filter_resets_to_first_page` (xfail, non-strict) | 2026-11-05 |
 
 ## Conventions