Skip to content

Add CancerVerse dataset support to the viewer backend#78

Merged
aperson30 merged 2 commits into
mainfrom
cancerverse-dataset-support
Jul 18, 2026
Merged

Add CancerVerse dataset support to the viewer backend#78
aperson30 merged 2 commits into
mainfrom
cancerverse-dataset-support

Conversation

@aperson30

Copy link
Copy Markdown
Collaborator

What

Adds a second dataset — CancerVerse (22.8k CT-only cases, CV_%08d ids, CSV metadata) — alongside PanTS. Fully additive and dataset-dispatched: every PanTS path stays byte-for-byte unchanged, and CancerVerse is inert unless CANCERVERSE_PATH is set.

CancerVerse has no masks yet (no combined_labels, segmentations, or organ intensities), so every mask-dependent endpoint degrades gracefully to {"masks_available": false} instead of 404/crashing.

Changes (flask-server/)

  • constants.pyCANCERVERSE_PATH, CANCERVERSE_LOWRES_PATH (default /home/visitor/cancerverse_lowres), DATASET_PREFIXES.
  • api/utils.pyget_cancerverse_id, get_dataset_from_case_id, get_folder_id, get_case_nifti_paths; loads the CancerVerse CSV through the same _norm_cols into DF_CV (stays None if the path/CSV is absent — never raises); select_dataset_df() switches search on ?dataset=.
  • api/api_blueprint.pyget-main-nifti serves CV CT + ?res=low from the CancerVerse paths; get-segmentations / get-specific-segmentations / get-report-data return {"masks_available": false} for CV; previews degrade gracefully; /api/search honours ?dataset=cancerverse|all.
  • services/npz_processor.py, preprocess_meshes.pyget_cancerverse_id parallels.
  • .env.example (new) + README — document the new env vars.

Notes / deviations from the spec

  • utils.py was edited per explicit maintainer sign-off (normally off-limits). Kept surgical: one added entry to case_cols, the new helpers, and the DF_CV/select_dataset_df block — no existing search logic changed.
  • Used a central dispatch helper (get_dataset_from_case_id + get_case_nifti_paths) rather than mechanically wrapping each scattered get_panTS_id call. PanTS branches are untouched; CV is a pure if. Same result, far lower regression risk.
  • The spec said add two lines to .env.env is gitignored, so those go in .env.example here and must be added to the server's real .env at deploy time (CANCERVERSE_PATH, CANCERVERSE_LOWRES_PATH).
  • nifti_processor.py does not have a get_panTS_id copy (spec was mistaken), so nothing to add there.

Testing

  • py_compile clean on all changed files
  • Helper unit smoke: get_cancerverse_id('123')→CV_00000123, get_cancerverse_id('CV_00000123')→CV_00000123, dataset dispatch + masks_available correct, DF_CV gracefully None with no CV data
  • 42/42 test_path_safety.py pass
  • Server verification needed (no CancerVerse data locally): set the env vars, confirm ?dataset=cancerverse search returns CV rows, a CV case's CT loads, and mask endpoints return masks_available:false.

No gunicorn restart in this PR — that's the separate post-merge deploy step.

CancerVerse is a second dataset (22.8k CT-only cases, CV_%08d ids, CSV
metadata) served alongside PanTS. Additive and dataset-dispatched — PanTS
paths/behaviour are unchanged; CancerVerse is disabled unless CANCERVERSE_PATH
is set.

- constants.py: CANCERVERSE_PATH, CANCERVERSE_LOWRES_PATH, DATASET_PREFIXES.
- utils.py: get_cancerverse_id / get_dataset_from_case_id / get_folder_id /
  get_case_nifti_paths; load CancerVerse CSV through the same _norm_cols into
  DF_CV (None if absent); select_dataset_df() switches search on ?dataset=.
- api_blueprint.py: get-main-nifti serves CV CT + low-res from CANCERVERSE
  paths; mask endpoints (get-segmentations, get-specific-segmentations,
  get-report-data) return {"masks_available": false} for CV instead of
  crashing; previews degrade gracefully; /api/search honours ?dataset=.
- services/npz_processor, preprocess_meshes: get_cancerverse_id parallels.
- .env.example + README: document the new env vars.

CancerVerse is CT-only for now, so any mask-dependent path is a graceful no-op.
…not inside

CancerVerse_dataset_metadata.csv lives in the parent of CANCERVERSE_PATH
(/mnt/.../data/CancerVerse_dataset_metadata.csv), alongside the CancerVerse/
image folder — not within it. The old path never found it, so DF_CV stayed
None and ?dataset=cancerverse always returned empty. Use dirname(normpath(...))
(also robust to a trailing slash) and correct the .env.example + README notes.
@aperson30

Copy link
Copy Markdown
Collaborator Author

Fixed the CSV path bug from review: CancerVerse_dataset_metadata.csv is now read from dirname(CANCERVERSE_PATH) (its real location, next to the image folder) rather than inside CANCERVERSE_PATH. Used os.path.dirname(os.path.normpath(...)) so a trailing slash on the env var is also handled, and corrected the .env.example + README notes. Pushed as 9856118.

@aperson30
aperson30 merged commit 2455a2c into main Jul 18, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant