bioreason: default GRPO LoRA to q_proj/v_proj and add a target lister

[balajiadarsha]

Description

Follow-up to #9, which makes training.lora_target_modules take effect. This branch is built on #9's latest commit (5c9e8c2), so merge #9 first. Until then the diff below also shows #9's changes. The only new commit is the last one. Independent of #10; they touch different files.

1. GRPO adapts only q_proj and v_proj by default. bioreason_grpo.yaml now ships lora_target_modules: [q_proj, v_proj]. On OLMo-1B that is 32 of 112 linear layers (0.164% of backbone params trainable instead of 0.933%).
2. SFT declares the key as null, keeping its behavior (every nn.Linear except lm_head). With the key declared in both YAMLs, training.lora_target_modules=[...] overrides it without Hydra's + prefix; before, no YAML declared it and the plain override failed in struct mode.
3. tools/list_lora_targets.py lists the leaf names that can be targeted for any model, with layer counts. It builds the backbone on the meta device (only config.json is read, no weights) and uses the trainer's own _select_lora_targets, so its list matches what training accepts:

   $ python tools/list_lora_targets.py model=prism_olmo_1b
   Backbone: allenai/OLMo-1B-0724-hf
   LoRA-targetable nn.Linear leaf names (excluded: ['lm_head']):
     down_proj                  16 layers
     gate_proj                  16 layers
     k_proj                     16 layers
     o_proj                     16 layers
     q_proj                     16 layers
     up_proj                    16 layers
     v_proj                     16 layers
     total                     112 layers
   

   --backbone-id <HF id or path> skips Hydra. Both YAMLs and the notebook (section 9b) point to it.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that changes existing behavior)
• Documentation update
• Infrastructure, CI, or tooling

Compatibility

• src.* import paths
• the prism CLI surface
• Hydra config keys
• checkpoint keys
• launcher flags
• None of the above

training.lora_target_modules is now declared in bioreason_grpo.yaml and bioreason_sft.yaml. BioReason GRPO results change: new GRPO runs adapt q/v only, so they are not directly comparable with earlier all-layer GRPO runs. Pass training.lora_target_modules=null to get the old behavior.

How Has This Been Tested?

Aurora login node, CPU only:

pytest tests/test_bioreason_lora_targets_config.py tests/test_lora_target_modules.py \
       tests/test_train_coerce.py tests/test_isoflop_plan.py      # 53 passed, 3 skipped (torchtune)
ruff check src tests tools scripts applications examples        # ruff 0.16.9: all checks passed
python tools/list_lora_targets.py model=prism_olmo_1b           # 112 layers, 7 names (above)
python tools/list_lora_targets.py                               # OLMo-7B-0724: 224 layers
python tools/list_lora_targets.py --backbone-id Qwen/Qwen3-0.6B # 196 layers

New tests (no torchtune needed, so they run in CI): the shipped defaults for both YAMLs, the plain override without + (list and null) giving a real list, and the tool's name counting.

End-to-end: GRPO with [q_proj, v_proj] was already run on Aurora when testing #9 (passed: 32 layers adapted, adapter saved and merged at eval). Aurora runs of this branch's defaults are in progress and will be reported here.

• Platform tested on: Aurora login node (CPU) and Aurora compute nodes.
• Tests skipped, and why: torchtune-dependent tests from lora: honor lora_target_modules and fix the sidecar save crash #9 skip in CI, as before.

New Dependencies, Data, or Assets

None.

Checklist

• I have performed a self-review of my own changes
• The code follows the project's style guidelines (ruff check, ruff 0.16.9 per requirements/ci.txt)
• I have commented my code where the intent is not obvious
• I have made corresponding changes to the documentation
• I have added tests that cover this change, or explained why none are needed
• New and existing tests pass locally
• No secrets, credentials, or absolute institutional paths are included

AI/LLM Assistance

• AI/LLM tools were primarily used to generate code or artifacts in this PR
• I have reviewed all such output line-by-line and understand it

Written with Claude Code (Claude Opus 5.5).

🤖 Generated with Claude Code