feat: add split and merge modules and bump snapshots for latest versions - #12774
feat: add split and merge modules and bump snapshots for latest versions#12774chaitanyakasaraneni wants to merge 8 commits into
Conversation
a08419a to
3958256
Compare
piplus2
left a comment
There was a problem hiding this comment.
I've used Claude to help me with the review:
1. merge will fail on the most common real-world input
modules/nf-core/samplesheetparser/merge/main.nf:11
tuple val(meta), path(samplesheets)Per-project sheets are almost always all named SampleSheet.csv. Staging two files with the same basename raises Process ... input file name collision. The sibling diff module already handles this with stageAs: "old/*" / "new/*". The tests pass only because they use two differently-named files (flowcell_samplesheet.v1.csv / flowcell_samplesheet_b.v1.csv).
tuple val(meta), path(samplesheets, stageAs: "input*/*")Could you also add a test case with two identically-named sheets?
2. split silently converts V1 sheets to V2
modules/nf-core/samplesheetparser/split/main.nf:31-36
The module never passes --to. In cli.py, split's --to defaults to "v2" (the docs table saying "same as input" is inaccurate) and it's applied via SampleSheetSplitter(..., target_version=target). So the "Split V1 sheet by project" test is actually emitting V2 sheets — surprising for anyone splitting a sheet to feed bcl2fastq.
Either expose --to as an input/arg, or state the V2 default explicitly in meta.yml. merge passes --to correctly and is a good model here.
3. by case handling is inconsistent
modules/nf-core/samplesheetparser/split/main.nf:25,32 — by.toLowerCase() is validated, but raw ${by} is passed to the CLI. cli.py checks by not in ("project", "lane") — case-sensitive — so by = 'Project' clears the Groovy guard and then dies with exit 2. Normalise once and reuse:
def by_norm = by?.toLowerCase()
if (!['project', 'lane'].contains(by_norm)) {
error "by must be 'project' or 'lane', got: ${by}"
}
...
--by ${by_norm} \\merge/main.nf:25,30 has the same asymmetry, but it's harmless there since _resolve_version() lowercases — still worth making symmetric. Both also NPE on a null value instead of producing the intended error message, hence the ?. above.
4. merge requires ≥2 files, unguarded
modules/nf-core/samplesheetparser/merge/main.nf:11 — cli.py exits 2 with fewer than two inputs. Since the module already validates target_version in Groovy, a samplesheets.size() >= 2 guard would give a much clearer failure than a bare exit 2 when a channel happens to yield a single sheet.
Smaller:
meta.ymlJSON field lists are incomplete:mergealso emitsoutput_pathandsummary;splitalso emitssource_versionandsummary.documentation: https://illumina-samplesheet.readthedocs.ionow 301-redirects tosamplesheet-parser.readthedocs.io. Inherited from the existing modules, so optional here — but it's being copied into two new files.
0f4b11f to
3970523
Compare
|
You should be able to inside the nf-test rename the input files to make sure they have the same name |
921dca3 to
439c910
Compare
|
Thanks for the updates! Re-reviewed at 4.
|
…-to, normalize by/target_version
…dd same-name/warning tests, fix meta.yml
439c910 to
63184ba
Compare
Description of changes
Updates the
samplesheetparsermodules for the samplesheet-parser 2.5.1 release.samplesheetparser/splitandsamplesheetparser/merge, followingthe existing module conventions (shared meta,
versionstopic channel, stub block,nf-test with real + stub cases).
samplesheetparser/diff,info, andvalidatebumped1.3.0 → 2.5.1in
environment.yml(BioConda pin) and the BioContainers/Singularity image inmain.nf.samplesheet-parser=2.5.1is published on BioConda and BioContainers(
quay.io/biocontainers/samplesheet-parser:2.5.1--pyhdfd78af_0).nf-test --update-snapshot --profile docker— 16 tests passing. Onlyvalidateoutput md5s changed (validation JSON now emits an informational
noteschannel);diff/infooutput is byte-identical across versions.Tested locally with
nf-core modules test <module> --profile dockerfor all five modules.PR checklist
Closes #XXX
topic: versions- See version_topicslabelnf-core modules test <MODULE> --profile dockernf-core modules test <MODULE> --profile singularitynf-core modules test <MODULE> --profile condanf-core subworkflows test <SUBWORKFLOW> --profile dockernf-core subworkflows test <SUBWORKFLOW> --profile singularitynf-core subworkflows test <SUBWORKFLOW> --profile conda