Skip to content

Add randomized correlation-trap analysis (analyze_traps) with trap metrics and heuristics#344

Open
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/add-analyze_traps-method-to-weightwatcher-09lem1
Open

Add randomized correlation-trap analysis (analyze_traps) with trap metrics and heuristics#344
charlesmartin14 wants to merge 1 commit intomasterfrom
codex/add-analyze_traps-method-to-weightwatcher-09lem1

Conversation

@charlesmartin14
Copy link
Copy Markdown
Member

Motivation

  • Provide a reproducible, randomized workflow to detect and inspect correlation "traps" (isolated spike modes) in layer weight matrices.
  • Expose per-trap vector metrics and order-invariant statistics to characterize localization vs diffuseness of traps.
  • Add a compact heuristic to assess trap severity (diffuseness / risk) and tooling to visualize traps for debugging and model surgery.

Description

  • Introduces WeightWatcher.analyze_traps(...) and supporting methods including apply_analyze_traps, analyze_single_trap, identify_trap_mode_indices, compute_original_basis_for_traps, apply_trap_mp_fit, and _trap_result_columns to return one DataFrame row per detected trap.
  • Adds per-trap vector metrics (_trap_vector_metrics) and order-invariant statistics (_trap_vector_order_invariant_stats) plus helpers (_gini_abs, _topk_mass_fractions) and per-trap aggregation into the result rows.
  • Implements assess_trap_diffuseness(...) to compute trap_diffuseness_score, trap_risk_score, and trap_assessment, and adds plotting via plot_trap_analysis.
  • Makes permutation deterministic/reproducible by adding an optional rng parameter to permute_matrix(...) and threading rng through apply_permute_W and analyze_traps.
  • Adds unit tests: tests/test_analyze_traps.py (exercise API, metrics, reproducibility, columns) and tests/test_trap_diffuseness.py (heuristic behavior).
  • Updates documentation and packaging: bumps __version__ to 0.8.0, updates README.md with usage examples for analyze_traps, and records new changes in Changelog.txt.

Testing

  • Ran the new unit tests tests/test_analyze_traps.py and tests/test_trap_diffuseness.py via the repository test runner (python -m unittest / pytest); the tests executed and passed in the development environment.
  • Smoke-tested trap plotting/save behavior via analyze_traps(..., plot=True, savefig=...) to ensure no exceptions when saving figures.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant