Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.aurora.example
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ PRISM_DATA_ROOT="/lus/flare/projects/<project>/<user>/prism_data"
# Interleaved tokenizers. Two are committed under tokenizers/ in the repo;
# there is no builder script. Referenced by the
# prism_olmo*_ts presets in src/conf/model/.
PRISM_TOKENIZERS="/lus/flare/projects/<project>/<user>/BaseMM_PRISM/tokenizers"
PRISM_TOKENIZERS="/lus/flare/projects/<project>/<user>/PRISM/tokenizers"

# Staged third-party generator assets, e.g. the pinned OmniGen2 checkout used
# by src/conf/image_generation/*.json (which append "/OmniGen2").
Expand Down
6 changes: 3 additions & 3 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ HF_HOME="/flare/<project>/<your_username>"

# PRISM Project Root
# Path to the base of the repository
PRISM_DIR="/flare/<project>/<your_username>/BaseMM_PRISM"
PRISM_DIR="/flare/<project>/<your_username>/PRISM"

# Shared Hugging Face Hub (Optional but strongly recommended on Aurora)
# A shared directory containing pre-downloaded models to avoid re-downloading.
Expand Down Expand Up @@ -73,7 +73,7 @@ PRISM_DATA_ROOT="/flare/<project>/<your_username>/data"

# Interleaved tokenizers (two are committed under tokenizers/), referenced by the
# prism_olmo*_ts presets in src/conf/model/.
PRISM_TOKENIZERS="/flare/<project>/<your_username>/BaseMM_PRISM/tokenizers"
PRISM_TOKENIZERS="/flare/<project>/<your_username>/PRISM/tokenizers"

# Staged third-party generator assets, e.g. the pinned OmniGen2 snapshot used
# by the image-generation configs in src/conf/image_generation/.
Expand All @@ -100,7 +100,7 @@ PRISM_OUTPUT_ROOT="/flare/<project>/<your_username>/outputs"
# Custom OLMo tokenizer with interleaving tokens added, used as llm_tokenizer_id
# by the prism-olmo-*-interleaved presets. Build with
# the committed tokenizers/ directory, or point at your own path.
# PRISM_OLMO1B_INTERLEAVED_TOKENIZER="/flare/<project>/<your_username>/BaseMM_PRISM/tokenizers/prism-olmo-1b-interleaved"
# PRISM_OLMO1B_INTERLEAVED_TOKENIZER="/flare/<project>/<your_username>/PRISM/tokenizers/prism-olmo-1b-interleaved"

# Shared HF hub snapshot of the walrus geometry encoder weights. When set and
# the file exists, src/encoders/geometry.py loads it directly instead of
Expand Down
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Code owners for BaseMM_PRISM.
# Code owners for PRISM.
#
# GitHub requests a review from the owning entry whenever a matching file
# changes. Branch protection on `main` will require one approving review at the
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Other solutions or features you considered, and why they fall short.
## Compatibility Impact

Would this change any of the following? See *Compatibility* in
[CONTRIBUTING.md](https://github.com/AI-ModCon/BaseMM_PRISM/blob/main/CONTRIBUTING.md).
[CONTRIBUTING.md](https://github.com/AI-ModCon/PRISM/blob/main/CONTRIBUTING.md).

- [ ] `src.*` import paths
- [ ] the `prism` CLI surface
Expand Down
4 changes: 2 additions & 2 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ its license and provenance. Write "None" if there are none.
- [ ] I have added tests that cover this change, or explained why none are needed
- [ ] New and existing tests pass locally
- [ ] No secrets, credentials, or absolute institutional paths are included
(see [SECURITY.md](https://github.com/AI-ModCon/BaseMM_PRISM/blob/main/SECURITY.md))
(see [SECURITY.md](https://github.com/AI-ModCon/PRISM/blob/main/SECURITY.md))

## AI/LLM Assistance

See *Guidelines for AI/LLM-Assisted Contributions* in
[CONTRIBUTING.md](https://github.com/AI-ModCon/BaseMM_PRISM/blob/main/CONTRIBUTING.md). All PRs require a human reviewer who is
[CONTRIBUTING.md](https://github.com/AI-ModCon/PRISM/blob/main/CONTRIBUTING.md). All PRs require a human reviewer who is
accountable for the review — a project norm, not yet enforced by branch
protection.

Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,4 @@ will make up v0.1.0.
pretrained causal-LM backbone, with validated training paths on Aurora,
Polaris, and Perlmutter. -->

[Unreleased]: https://github.com/AI-ModCon/BaseMM_PRISM/commits/main
[Unreleased]: https://github.com/AI-ModCon/PRISM/commits/main
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ abstract: >-
type: software
version: 0.1.0
license: Apache-2.0
repository-code: "https://github.com/AI-ModCon/BaseMM_PRISM"
url: "https://github.com/AI-ModCon/BaseMM_PRISM"
repository-code: "https://github.com/AI-ModCon/PRISM"
url: "https://github.com/AI-ModCon/PRISM"
keywords:
- multimodal
- machine learning
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,8 @@ Historical experiment IDs and their ordering are stable — see *Compatibility*.
1. Fork the repository, then clone your fork **with submodules**:

```bash
git clone --recursive https://github.com/<your-username>/BaseMM_PRISM.git
cd BaseMM_PRISM
git clone --recursive https://github.com/<your-username>/PRISM.git
cd PRISM
```

If you already cloned without `--recursive`:
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ No HPC allocation, DAOS container, or accelerator needed.
> *Installation* instead.

```bash
git clone --recursive https://github.com/AI-ModCon/BaseMM_PRISM.git
cd BaseMM_PRISM
git clone --recursive https://github.com/AI-ModCon/PRISM.git
cd PRISM

python -m venv .venv && source .venv/bin/activate
pip install -r requirements/base.txt
Expand Down Expand Up @@ -344,7 +344,7 @@ Full reference, including the export key-renaming and the serving flags:
## Project structure

```
BaseMM_PRISM/
PRISM/
├── src/
│ ├── train.py # Hydra entry point
│ ├── model.py # UnifiedTransformer
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ PRISM vendors the Walrus geometry encoder as a git submodule, so clone
recursively:

```bash
git clone --recursive https://github.com/AI-ModCon/BaseMM_PRISM.git
cd BaseMM_PRISM
git clone --recursive https://github.com/AI-ModCon/PRISM.git
cd PRISM
```

If you already cloned without `--recursive`:
Expand Down
2 changes: 1 addition & 1 deletion docs/platforms/aurora_operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ cat $PBS_NODEFILE
Open a new terminal on the UAN and run:

```bash
cd /lus/flare/projects/ModCon/ngetty/BaseMM_PRISM
cd /lus/flare/projects/<project>/<user>/PRISM

# Quick test (5 steps, 1B model):
python tools/launch_aurora_web.py \
Expand Down
8 changes: 4 additions & 4 deletions docs/platforms/running_on_polaris.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ counterpart to [Aurora operations](aurora_operations.md).
```bash
ssh polaris
cd /eagle/ModCon/ngetty/repos
git clone git@github.com:AI-ModCon/BaseMM_PRISM.git # or rsync from Aurora
cd BaseMM_PRISM
git clone git@github.com:AI-ModCon/PRISM.git # or rsync from Aurora
cd PRISM

# Build the venv on top of the conda base module (uses uv if available)
PRISM_VENV=/eagle/ModCon/ngetty/envs/.venv-polaris \
Expand Down Expand Up @@ -98,7 +98,7 @@ recommended layout:
datasets/<dataset>/shards/*.tar
models/<hf-org--hf-model>/ # HF snapshot dirs
envs/.venv-polaris/ # built by setup_polaris_env.sh
repos/BaseMM_PRISM/ # checkout
repos/PRISM/ # checkout
logs/<DESIGN>/ # PBS stdout/stderr
```

Expand Down Expand Up @@ -206,7 +206,7 @@ Training Zone A: 100%|██████| 30/30 [00:31<00:00, 1.02s/it, loss=8.9
`from src.X` imports resolve. Aurora launchers don't need this
because their per-rank shell isn't `-l`.
- **rsync `--exclude='data'` is too greedy.** It matches both the
intended repo-root `BaseMM_PRISM/data` AND `BaseMM_PRISM/src/data`.
intended repo-root `PRISM/data` AND `PRISM/src/data`.
Use anchored excludes (`--exclude='/data'`) when staging the repo
to Polaris.
- **Outbound network is via proxy.** Set
Expand Down
8 changes: 4 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -102,10 +102,10 @@ data = [
]

[project.urls]
Homepage = "https://github.com/AI-ModCon/BaseMM_PRISM"
Repository = "https://github.com/AI-ModCon/BaseMM_PRISM"
Issues = "https://github.com/AI-ModCon/BaseMM_PRISM/issues"
Documentation = "https://github.com/AI-ModCon/BaseMM_PRISM/tree/main/docs"
Homepage = "https://github.com/AI-ModCon/PRISM"
Repository = "https://github.com/AI-ModCon/PRISM"
Issues = "https://github.com/AI-ModCon/PRISM/issues"
Documentation = "https://github.com/AI-ModCon/PRISM/tree/main/docs"

[project.scripts]
prism = "src.cli:app"
Expand Down
2 changes: 1 addition & 1 deletion src/conf/model/prism_olmo1b_linear_interleaved_ts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ backbone_id: allenai/OLMo-1B-0724-hf
# Custom tokenizer with interleaving tokens added.
# Placeholder path — set to wherever you've built/copied this tokenizer
# (see PRISM_OLMO1B_INTERLEAVED_TOKENIZER default in src/config.py).
tokenizer_id: "/flare/<project>/<user>/BaseMM_PRISM/tokenizers/prism-olmo-1b-interleaved"
tokenizer_id: "/flare/<project>/<user>/PRISM/tokenizers/prism-olmo-1b-interleaved"

modalities: [text,time_series]

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
backbone_id: allenai/Olmo-3-7B-Instruct
# Custom tokenizer with interleaving tokens added.
# Placeholder path — set to wherever you've built/copied this tokenizer.
tokenizer_id: "/flare/<project>/<user>/BaseMM_PRISM/tokenizers/prism-olmo-3-7b-instruct-interleaved"
tokenizer_id: "/flare/<project>/<user>/PRISM/tokenizers/prism-olmo-3-7b-instruct-interleaved"

modalities: [text,time_series]

Expand Down
2 changes: 1 addition & 1 deletion src/conf/model/prism_olmo7b_linear_interleaved_ts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ backbone_id: allenai/OLMo-7B-0724-hf
# Custom tokenizer with interleaving tokens added.
# Placeholder path — set to wherever you've built/copied this tokenizer
# (see PRISM_OLMO1B_INTERLEAVED_TOKENIZER default in src/config.py).
tokenizer_id: "/flare/<project>/<user>/BaseMM_PRISM/tokenizers/prism-olmo-1b-interleaved"
tokenizer_id: "/flare/<project>/<user>/PRISM/tokenizers/prism-olmo-1b-interleaved"

modalities: [text,time_series]

Expand Down
6 changes: 3 additions & 3 deletions src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -543,7 +543,7 @@ def from_preset(cls, preset_name: str):
"PRISM_OLMO1B_INTERLEAVED_TOKENIZER",
# Custom tokenizer with interleaving tokens added; build it with
# the committed tokenizers/ directory, or point this at your own path.
"/flare/<project>/<user>/BaseMM_PRISM/tokenizers/prism-olmo-1b-interleaved",
"/flare/<project>/<user>/PRISM/tokenizers/prism-olmo-1b-interleaved",
),
"modalities": ["text", "time_series"],
"freeze_backbone": True,
Expand All @@ -566,7 +566,7 @@ def from_preset(cls, preset_name: str):
"PRISM_OLMO1B_INTERLEAVED_TOKENIZER",
# Custom tokenizer with interleaving tokens added; build it with
# the committed tokenizers/ directory, or point this at your own path.
"/flare/<project>/<user>/BaseMM_PRISM/tokenizers/prism-olmo-1b-interleaved",
"/flare/<project>/<user>/PRISM/tokenizers/prism-olmo-1b-interleaved",
),
"modalities": ["text", "time_series"],
"freeze_backbone": True,
Expand All @@ -589,7 +589,7 @@ def from_preset(cls, preset_name: str):
"PRISM_OLMO1B_INTERLEAVED_TOKENIZER",
# Custom tokenizer with interleaving tokens added; build it with
# the committed tokenizers/ directory, or point this at your own path.
"/flare/<project>/<user>/BaseMM_PRISM/tokenizers/prism-olmo-1b-interleaved",
"/flare/<project>/<user>/PRISM/tokenizers/prism-olmo-1b-interleaved",
),
"modalities": ["text", "time_series"],
"freeze_backbone": True,
Expand Down
2 changes: 1 addition & 1 deletion tests/test_scits_fix_static.py
Original file line number Diff line number Diff line change
Expand Up @@ -455,7 +455,7 @@ def test_no_user_specific_hub_paths_in_model_configs():

Tokenizer paths under a project dir are still allowed: the interleaved TS
configs legitimately point at custom tokenizers checked in under
/flare/ModCon/<user>/BaseMM_PRISM/tokenizers/.
/flare/<project>/<user>/PRISM/tokenizers/.
"""
offenders = []
for cfg in sorted((REPO_ROOT / "src" / "conf" / "model").glob("*.yaml")):
Expand Down
4 changes: 2 additions & 2 deletions tools/ci/site_path_allowlist.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"docs/modalities/image_decoder/docci_data.md": 3,
"docs/modalities/timeseries.md": 6,
"docs/models/auroragpt_vlm.md": 1,
"docs/platforms/aurora_operations.md": 4,
"docs/platforms/aurora_operations.md": 3,
"docs/platforms/daos_setup.md": 10,
"docs/platforms/running_on_polaris.md": 8,
"docs/results/patrick_tsqa_ab.md": 3,
Expand Down Expand Up @@ -50,7 +50,7 @@
"tools/isoflop_collect.py": 1,
"tools/isoflop_launch.py": 1,
"tools/isoflop_plan.py": 4,
"tools/launch_aurora.py": 3,
"tools/launch_aurora.py": 2,
"tools/launch_aurora_daos.py": 2,
"tools/launch_aurora_web.py": 1,
"tools/launch_baremetal.py": 2,
Expand Down
5 changes: 4 additions & 1 deletion tools/launch_aurora.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,10 @@ def main():
env_config = load_dotenv()

# Defaults from .env or hardcoded fallbacks
default_prism_dir = env_config.get("PRISM_DIR", "/flare/ModCon/ngetty/BaseMM_PRISM")
# Fall back to the checkout this launcher lives in, not a hardcoded user path.
default_prism_dir = env_config.get(
"PRISM_DIR", os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
default_hf_home = env_config.get("HF_HOME", "/flare/ModCon/sandeep")
env_config.get("PROJECT_ALLOCATION", "ModCon")
env_config.get("QUEUE_NAME", "debug")
Expand Down
2 changes: 1 addition & 1 deletion tools/launch_polaris.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ def main() -> int:
env_config = load_dotenv()

# When --prism-dir isn't given, prefer the launcher's own location over
# cwd. That way `ssh polaris "cd /eagle/.../BaseMM_PRISM && \
# cwd. That way `ssh polaris "cd /eagle/.../PRISM && \
# python tools/launch_polaris.py ..."` Just Works from a different host.
if args.prism_dir:
prism_dir = args.prism_dir
Expand Down
Loading
Loading