Skip to content

Fix top review findings: coverage labeling, error handling, UX dead ends - #9

Open
daphnehanse11 wants to merge 2 commits into
mainfrom
fix/review-tier1
Open

Fix top review findings: coverage labeling, error handling, UX dead ends#9
daphnehanse11 wants to merge 2 commits into
mainfrom
fix/review-tier1

Conversation

@daphnehanse11

Copy link
Copy Markdown
Collaborator

Fixes the highest-priority findings from a full code review of the app (backend, frontend, and a live walkthrough).

What users see wrong today

  1. "No coverage" when the real answer is "full-price marketplace." Anyone with a $0 premium tax credit (income over the 400% FPL cliff, which is back in 2026) or anyone 65+ showed no coverage pill at all — reading as "you lose insurance."
  2. Good news framed as bad. An income drop that moves the whole family onto Medicaid ($300/mo → $0) was headlined "Your new income ends your ACA tax credit."
  3. Silent zeros. Any PolicyEngine calculation error was swallowed into $0, so bad input produced a confident all-zeros result instead of an error.
  4. Raw platform errors. Simulations regularly take 30–60s on a Modal cold start; without maxDuration/timeouts, users saw Unexpected token 'A'... when the proxy tried to JSON-parse an HTML error page.
  5. Seven questions, then a dead end. Households with employer coverage answered the whole wizard before being told the tool can't help them.

Changes

Backend

  • Coverage typing: Marketplace (full price) for $0-PTC people, Medicare for 65+ (dual-eligibles surface as Medicare); counterfactual remaps carry the new field through
  • _run_simulation raises on failed core variables (medicaid, chip, premium_tax_credit, slcsp); lcbp/chip_premium stay soft-default 0 since their absence is a data gap
  • Move events: after-side bronze estimate now uses the destination state's bronze/silver ratio (was ~$1,000/yr off on e.g. TX→CA)
  • lcbp/chip_premium added to metadata with a non-benefit category, so totalBenefits no longer counts the gross bronze premium as a benefit; also fixes the raw chip_premium label leaking to the API
  • 500 responses no longer include internal exception text (logged server-side instead)

Frontend

  • ESI heads-up moved to wizard step 5, with an explicit "Continue without employer coverage" path
  • Result headlines prioritize per-person coverage transitions (Medicaid gained/lost, Medicare) over tax-credit mechanics; lost-PTC copy now says "you'd pay full price"
  • Stale-response guard (requestSeq): a slow simulation can no longer save a scenario for a household you've already left or rewrite the URL; error card and change wizard are now mutually exclusive
  • API routes: maxDuration = 120, 110s backend fetch timeout with a friendly 504, content-type-checked error parsing, deduplicated into src/lib/backendProxy.ts
  • Share links: decodeScenario restores pregnantMember (pregnancy scenarios previously recomputed against a non-pregnant baseline)
  • ResultsView hooks moved above the early return (hooks-order crash); added app/error.tsx error boundary
  • Move-specific silver explainer sentence only shows for move scenarios
  • README live URL fixed (old link 404s)

Verification

  • pytest tests/ — 34 passed
  • npm run build + tsc + eslint clean (3 pre-existing dead-code warnings untouched)
  • Manual walkthrough on localhost against the real backend: above-cliff couple shows Marketplace (full price) pills; 66-year-old shows Medicare; family income drop headlines "qualifies you and your family for Medicaid"; ESI "Yes" blocks at step 5; invalid state returns a clean JSON error
  • Known upstream issue (pre-existing, not addressed here): for a childless CA couple, PolicyEngine reports both PTC=$0 and Medicaid=$0 in a ~$30k–32k band (~142–151% FPL) — the two eligibility checks disagree. Flagged for separate investigation.

🤖 Generated with Claude Code

Backend:
- Label unsubsidized marketplace ("Marketplace (full price)") and
  Medicare (65+) instead of showing "No coverage" when PTC is $0
- Raise on failed core-variable calculations instead of silently
  returning $0 (optional lcbp/chip_premium still default to 0)
- Use the destination state's bronze/silver ratio for the after side
  of a move's ACA premium estimate
- Categorize lcbp and chip_premium so totalBenefits no longer counts
  the gross bronze premium or CHIP fees as benefits
- Stop leaking internal exception text in 500 responses

Frontend:
- Block employer-coverage households at wizard step 5 with an inline
  heads-up (plus a "continue without employer coverage" path) instead
  of after all seven questions
- Lead result headlines with per-person coverage transitions
  ("qualifies you and your family for Medicaid") instead of tax-credit
  mechanics that framed good news as bad
- Add stale-response guard so slow simulations can't clobber a newer
  household or wedge the UI; hide the change wizard while the error
  card is shown
- Proxy routes: maxDuration=120, backend fetch timeout, safe handling
  of non-JSON error bodies, shared helper (was duplicated)
- Restore pregnantMember when decoding share links
- Fix hooks-order violation in ResultsView; add route error boundary
- Show the move-specific silver explainer only for move scenarios

README: point Live link at the working deployment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cc-clean Ready Ready Preview, Comment Jul 27, 2026 6:35pm
coverage-compass Ready Ready Preview, Comment Jul 27, 2026 6:35pm

Request Review

- SimulationComputeError with a user-safe message replaces the ValueError
  that embedded raw PolicyEngine exception text; full details go to the
  server log, and the Flask API returns it as 500 (engine fault), not 400
- modal_app.py gets the same sanitization as the Flask API (it previously
  still returned raw exception text) and logs tracebacks in-container
- Marriage/divorce counterfactual side-simulations run non-strict again:
  their aggregated results are discarded, so a failed variable there no
  longer aborts the whole comparison
- restoreScenario bumps the request sequence and clears loading, so an
  in-flight simulation can't override a restored saved scenario
- decodeScenario rejects unknown event types from crafted share links
  (previously rendered an "undefined …" headline)
- Backing out of the ESI heads-up fully resets the answer so the summary
  no longer shows "Job coverage: Yes"
- test_compare: fake _run_simulation accepts the new strict kwarg

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@daphnehanse11

Copy link
Copy Markdown
Collaborator Author

Second review round (adversarial pass over this diff) found and fixed in 3d2bef1:

  • Error sanitization was incomplete: the new raise-on-failure path embedded raw PolicyEngine exception text and returned it as a 400, and modal_app.py (the production backend) had no sanitization at all. Now a dedicated SimulationComputeError carries a user-safe message (details logged server-side), the Flask API returns it as 500, and Modal matches.
  • Marriage/divorce counterfactual side-simulations now run non-strict — their aggregated results are discarded, so a failed variable there no longer aborts a whole comparison that previously succeeded.
  • restoreScenario now invalidates in-flight simulations (clicking a saved scenario during a run no longer gets silently overridden).
  • decodeScenario rejects unknown event types from crafted share links (previously rendered "undefined …" headlines).
  • Backing out of the ESI heads-up fully resets the answer in the "So far" summary.

Full backend suite: 34 passed. One deploy-time check for the reviewer: maxDuration = 120 in the API routes requires Vercel Pro or Hobby with Fluid compute (limit 300s); legacy Hobby without Fluid caps at 60s.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant