Add CancerVerse dataset support to the viewer backend#78
Merged
Conversation
CancerVerse is a second dataset (22.8k CT-only cases, CV_%08d ids, CSV
metadata) served alongside PanTS. Additive and dataset-dispatched — PanTS
paths/behaviour are unchanged; CancerVerse is disabled unless CANCERVERSE_PATH
is set.
- constants.py: CANCERVERSE_PATH, CANCERVERSE_LOWRES_PATH, DATASET_PREFIXES.
- utils.py: get_cancerverse_id / get_dataset_from_case_id / get_folder_id /
get_case_nifti_paths; load CancerVerse CSV through the same _norm_cols into
DF_CV (None if absent); select_dataset_df() switches search on ?dataset=.
- api_blueprint.py: get-main-nifti serves CV CT + low-res from CANCERVERSE
paths; mask endpoints (get-segmentations, get-specific-segmentations,
get-report-data) return {"masks_available": false} for CV instead of
crashing; previews degrade gracefully; /api/search honours ?dataset=.
- services/npz_processor, preprocess_meshes: get_cancerverse_id parallels.
- .env.example + README: document the new env vars.
CancerVerse is CT-only for now, so any mask-dependent path is a graceful no-op.
…not inside CancerVerse_dataset_metadata.csv lives in the parent of CANCERVERSE_PATH (/mnt/.../data/CancerVerse_dataset_metadata.csv), alongside the CancerVerse/ image folder — not within it. The old path never found it, so DF_CV stayed None and ?dataset=cancerverse always returned empty. Use dirname(normpath(...)) (also robust to a trailing slash) and correct the .env.example + README notes.
Collaborator
Author
|
Fixed the CSV path bug from review: |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a second dataset — CancerVerse (22.8k CT-only cases,
CV_%08dids, CSV metadata) — alongside PanTS. Fully additive and dataset-dispatched: every PanTS path stays byte-for-byte unchanged, and CancerVerse is inert unlessCANCERVERSE_PATHis set.CancerVerse has no masks yet (no
combined_labels, segmentations, or organ intensities), so every mask-dependent endpoint degrades gracefully to{"masks_available": false}instead of 404/crashing.Changes (
flask-server/)CANCERVERSE_PATH,CANCERVERSE_LOWRES_PATH(default/home/visitor/cancerverse_lowres),DATASET_PREFIXES.get_cancerverse_id,get_dataset_from_case_id,get_folder_id,get_case_nifti_paths; loads the CancerVerse CSV through the same_norm_colsintoDF_CV(staysNoneif the path/CSV is absent — never raises);select_dataset_df()switches search on?dataset=.get-main-niftiserves CV CT +?res=lowfrom the CancerVerse paths;get-segmentations/get-specific-segmentations/get-report-datareturn{"masks_available": false}for CV; previews degrade gracefully;/api/searchhonours?dataset=cancerverse|all.get_cancerverse_idparallels.Notes / deviations from the spec
utils.pywas edited per explicit maintainer sign-off (normally off-limits). Kept surgical: one added entry tocase_cols, the new helpers, and theDF_CV/select_dataset_dfblock — no existing search logic changed.get_dataset_from_case_id+get_case_nifti_paths) rather than mechanically wrapping each scatteredget_panTS_idcall. PanTS branches are untouched; CV is a pureif. Same result, far lower regression risk..env—.envis gitignored, so those go in.env.examplehere and must be added to the server's real.envat deploy time (CANCERVERSE_PATH,CANCERVERSE_LOWRES_PATH).nifti_processor.pydoes not have aget_panTS_idcopy (spec was mistaken), so nothing to add there.Testing
py_compileclean on all changed filesget_cancerverse_id('123')→CV_00000123,get_cancerverse_id('CV_00000123')→CV_00000123, dataset dispatch +masks_availablecorrect,DF_CVgracefullyNonewith no CV datatest_path_safety.pypass?dataset=cancerversesearch returns CV rows, a CV case's CT loads, and mask endpoints returnmasks_available:false.No gunicorn restart in this PR — that's the separate post-merge deploy step.