Hybrid pangenome callers - #37
Merged
Merged
Conversation
Extract the optional pangenome callers into reusable stages so both pangenome pipelines can compose the same building blocks: `SegdupStage` (`stages/segdup.py`), `ExpansionHunterStage` (`stages/expansion.py`), `T1KStage` (`stages/t1k.py`, with the default HLA/KIR loci) and the composite `PangenomeCNVStage` in `stages/cnv.py` (CNVscope and CNVModelApply, indel2cnv.py on the PangenomeSV output, and combine_cnv.py over the two). `cmd_segdup_caller` gains optional `lr_alignments` / `lr_bundle` arguments for long-read calling; `cmd_pyexec_indel2cnv` and `cmd_pyexec_combine_cnv` move the two script invocations into `command_strings.py`. `util` gains `caller_sex_arg`, the shared `--sex male|female` mapping for ExpansionHunter and segdup-caller. No pipeline uses the new stages yet -- `sentieon_pangenome.py` keeps its inline builders, so the module-level version constants are duplicated for now -- and the DAG snapshots of all 88 configurations are identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`SentieonPangenome` now builds its optional callers from the stages added in the previous commit, and everything both pangenome pipelines will need moved up to `BasePangenome`. `BasePangenome` gains the shared caller arguments -- `par_bed`, `sample_sex`, `segdup_caller`, `expansion_catalog` and the six `t1k_*` (the two loci defaulting to the constants in `stages/t1k.py`) -- with the matching `__init__` attributes, `has_cnv_model`, and the DAG-2 stashes `ploidy_json` / `sr_alignments` / `sr_replace_rg` (`sr_replace_rg` is unused until the hybrid pipeline needs it). `validate_t1k` moved up unchanged, and four thin helpers build the stages: `output_path(suffix)`, `add_t1k(dag, ctx, inputs, upstream)`, `add_expansion(dag, ctx, alignment)`, `add_segdup(dag, ctx, sr_alignment, *, lr_alignment, lr_bundle, overrides)` and `add_pangenome_cnv(dag, ctx, sv_vcf, inputs, *, replace_rg, interval)`. `--call_svs` stays on `SentieonPangenome`. `hybrid-pangenome` inherits the new flags, which are accepted but build no jobs yet. Deleted from `sentieon_pangenome.py`: the `SEGDUP_MIN_VERSION` / `EXPANSION_MIN_VERSION` / `T1K_MIN_VERSION` constants (now `*_MIN_VERSIONS` next to their stages) and the default T1K loci, the moved params and `__init__` lines, `validate_t1k`, `build_segdup_job`, `build_t1k_jobs`, `build_expansion_job`, `_add_cnv_jobs`, `_build_indel2cnv_job`, `_build_combine_cnv_job`, and the dead `self.rw_bam` that `build_dag` assigned but nothing read. The DAG snapshot over all 88 configurations is identical, including `compare --with-ids`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Give `hybrid-pangenome` the argument surface, validation, and first-DAG jobs of the optional callers that `dnascope-pangenome` already has. The new `--call_cnvs` flag opts into CNVscope calling, and the shared `--sample_sex`, `--par_bed`, `--segdup_caller`, `--expansion_catalog` and `--t1k_*` arguments come from `BasePangenome`. `validate_bundle` now records whether the bundle carries a 'cnv.model' file and requires `diploid_model` when `--segdup_caller` is set, since segdup-caller reads it from the bundle passed as its `--lr_model`. `validate()` runs `validate_segdup`, `validate_expansion`, `validate_t1k` and `validate_cnv` after the readgroup checks. segdup-caller and ExpansionHunter accept a single `--sr_aln` file (segdup-caller also a single `--lr_aln`), `--segdup_caller` cannot be combined with `--skip_small_variants`, and `--call_cnvs` requires SV calling and a bundle CNV model. CNV runs resolve the PAR BED file during validation and exit when none is available; a run without `--bed` warns that CNVscope will cover every contig. `--bed` now restricts CNV calling as well as small-variant calling. The first DAG always adds the `PloidyStage`, writing `<output>_ploidy.json` whatever the enabled callers, and runs T1K HLA/KIR on the short reads. It also stashes the short-read alignments, their `--replace_rg` rows, and the single long-read alignment for the second DAG. `--call_cnvs`, `--segdup_caller` and `--expansion_catalog` are validated but build no second-DAG jobs yet; `build_second_dag` follows in the next commit. The DAG snapshots gain one `estimate-ploidy` row per hybrid-pangenome config and are otherwise identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd DAG `HybridPangenome.build_second_dag` builds the sex-aware callers once ploidy estimation has run, using the short-read alignments, the sample sex and the long-read alignment stashed by `build_dag`. With `--call_cnvs` the DAG holds the `PangenomeCNVStage` jobs: CNVscope over the short reads only -- restricted by `--bed` and carrying the short-read `--replace_rg` rows for `--sr_aln` input -- CNVModelApply, `indel2cnv.py` over the PangenomeSV VCF, and `combine_cnv.py` writing `<output>_cnv.vcf.gz`. `--expansion_catalog` adds ExpansionHunter over the short reads. `--segdup_caller` adds segdup-caller with both data types (`--short` and `--long`) and this pipeline's model bundle as both `--sr_model` and `--lr_model`, since it ships the `diploid_model`. The DAG-snapshot harness gains seven hybrid-pangenome configs for the new callers; every pre-existing config is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add optional variant callers to the hybrid pangenome pipeline.