Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
- name: Check that saved notebook outputs still match the package
# Runs pages that need no downloaded data, without saving them, and fails
# when what they print has changed. Add such pages to PAGES.
run: make -C docs check-outputs PAGES="quickstart differential_expression guides/export_cubes guides/bulk_rna_seq"
run: make -C docs check-outputs PAGES="quickstart differential_expression guides/export_cubes guides/bulk_rna_seq guides/composition"
- name: Store the site
if: ${{ !inputs.pages }}
uses: actions/upload-artifact@v7
Expand Down
12 changes: 8 additions & 4 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,8 @@ The pages of the documentation (`docs/source/`) read the datasets that `docs/dat
prepares in a cache outside the repository: two simulated with fixed random seeds by
that script, and the two below, GEO series GSE96583 and the MSigDB Hallmark
interferon-alpha response set. No file of these datasets is in the repository; the
script records the source, licence, citation and checksum of each.
script records the source, licence, citation and checksum of each. Two guides also
download the networks CollecTRI and PROGENy, as their sections below say.

### GEO series GSE96583 (Kang et al. 2018)

Expand Down Expand Up @@ -277,7 +278,9 @@ are distributed under the Creative Commons Attribution 4.0 International License

### CollecTRI (Mueller-Dott et al. 2023)

`collectri()` in `cytepool/networks.py` and `validation/activity.py` download
`collectri()` in `cytepool/networks.py`, called by `validation/activity.py` and by the
guides `docs/source/guides/gene_sets.ipynb` and
`docs/source/guides/pathway_activity.ipynb`, downloads
`CollecTRI_regulons.csv`, version 2.0 of Zenodo record 8192729
(https://zenodo.org/records/8192729, doi:10.5281/zenodo.8192729), into a cache outside
the repository. No row of it is in the repository: `tests/data/activity` holds rows in
Expand All @@ -298,10 +301,11 @@ activities. Nucleic Acids Research 51, 10934-10949. doi:10.1093/nar/gkad841

### PROGENy (Schubert et al. 2018; Holland, Szalai and Saez-Rodriguez 2020)

`progeny()` in `cytepool/networks.py` and `validation/activity.py` download the R
`progeny()` in `cytepool/networks.py`, called by `validation/activity.py` and by the
guide `docs/source/guides/pathway_activity.ipynb`, downloads the R
package progeny 1.28.0 from Bioconductor 3.20
(https://bioconductor.org/packages/3.20/bioc/src/contrib/progeny_1.28.0.tar.gz) into a
cache outside the repository and read its data frames `model_human_full` and
cache outside the repository and reads its data frames `model_human_full` and
`model_mouse_full`. No value of them is in the repository: `tests/data/activity` holds
a package in its layout whose data are simulated. progeny, its data included, is
distributed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0).
Expand Down
14 changes: 0 additions & 14 deletions docs/coverage_exemptions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,21 +63,7 @@
"cytepool.table.Table.select" = "guides/pool_cells: sample and group tables"
"cytepool.table.Table.filter" = "guides/pool_cells: sample and group tables"
"cytepool.table.Table.with_columns" = "guides/pool_cells: sample and group tables"
"cytepool.table.Table.to_pandas" = "guides/genes_per_cell_type: result tables as pandas DataFrames"
"cytepool.plotting.plot_barcode" = "guides/gene_sets: one set on the ranked statistics"
"cytepool.plotting.plot_cell_numbers" = "guides/pool_cells: cells pooled per sample and cell type"
"cytepool.plotting.plot_composition" = "guides/composition: cell-type proportions per sample"
"cytepool.plotting.plot_dispersions" = "guides/bulk_rna_seq: DESeq2's dispersion estimates"
"cytepool.plotting.plot_forest" = "guides/combine_studies: one gene across studies"
"cytepool.cube.CubeCollection.select" = "guides/combine_studies: the studies, cell types and genes of an analysis"
"cytepool.plotting.plot_gene_sets" = "guides/gene_sets: the results of camera, fry and mroast"
"cytepool.plotting.plot_md" = "guides/genes_per_cell_type: MD plot of a contrast"
"cytepool.plotting.plot_mds" = "quickstart and guides/genes_per_cell_type: samples on an MDS plot"
"cytepool.plotting.plot_mean_variance" = "guides/genes_per_cell_type: voom's mean-variance trend"
"cytepool.plotting.plot_p_values" = "guides/genes_per_cell_type: the distribution of p-values"
"cytepool.plotting.plot_proportions" = "guides/composition: proportions by condition with propeller's results"
"cytepool.plotting.plot_quasi_dispersions" = "guides/bulk_rna_seq: edgeR's quasi-dispersions"
"cytepool.plotting.plot_residual_variances" = "guides/normalization_and_quality: residual variances against expression"
"cytepool.plotting.plot_sample_weights" = "guides/normalization_and_quality: sample quality weights"
"cytepool.plotting.plot_significant_genes" = "guides/genes_per_cell_type: changed genes per cell type"
"cytepool.plotting.plot_volcano" = "quickstart and guides/genes_per_cell_type: volcano plot of a contrast"
8 changes: 6 additions & 2 deletions docs/source/differential_expression.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"metadata": {},
"source": [
"(differential-expression-tutorial)=\n",
"# Differential expression per cell type\n",
"# From cells to results\n",
"\n",
"This page runs a pseudobulk differential-expression analysis from cells to results:\n",
"it simulates single cells from six donors, each sampled before and after a\n",
Expand All @@ -19,7 +19,11 @@
"\n",
"Pooling the cells of each sample first makes the sample, not the cell, the unit of\n",
"replication: cells from one donor are not independent observations, and a test that\n",
"treated them as such would find differences that do not hold across donors."
"treated them as such would find differences that do not hold across donors.\n",
"\n",
"The guides of Compare conditions take each question further, with the choices it\n",
"involves; [Genes that change in each cell type](guides/genes_per_cell_type.ipynb)\n",
"runs this analysis on published data."
]
},
{
Expand Down
300 changes: 300 additions & 0 deletions docs/source/guides/composition.ipynb

Large diffs are not rendered by default.

420 changes: 420 additions & 0 deletions docs/source/guides/gene_sets.ipynb

Large diffs are not rendered by default.

568 changes: 568 additions & 0 deletions docs/source/guides/genes_per_cell_type.ipynb

Large diffs are not rendered by default.

508 changes: 508 additions & 0 deletions docs/source/guides/normalization_and_quality.ipynb

Large diffs are not rendered by default.

342 changes: 342 additions & 0 deletions docs/source/guides/pathway_activity.ipynb

Large diffs are not rendered by default.

460 changes: 460 additions & 0 deletions docs/source/guides/repeated_measures.ipynb

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ metafor. Its only required dependencies are numpy and numba, and every method is
compared with the package that defined it ([Validation](validation.md)).

[Installation](installation.md) · [Quick start](quickstart.ipynb) ·
[Differential expression per cell type](differential_expression.ipynb) ·
[From cells to results](differential_expression.ipynb) ·
[Choosing a method](statistics/choosing_a_method.md) ·
[Capability matrix](reference/capabilities.md) · [API reference](/api/index.html)

Expand All @@ -32,14 +32,14 @@ what CytePool offers for each today.

| Question | What CytePool offers | Where to read |
| --- | --- | --- |
| Which genes change in each cell type? | limma-voom, limma-trend and dream on every cell type of a cube; edgeR's quasi-likelihood tests and DESeq2 on one matrix at a time | [Differential expression per cell type](differential_expression.ipynb), [Bulk RNA-seq](guides/bulk_rna_seq.ipynb), [Choosing a method](statistics/choosing_a_method.md) |
| Which cell types expand or contract? | propeller. Not yet: precision-weighted mixed models or count models of composition | [Composition tests](statistics/composition.md) |
| Which pathways and regulators respond? | camera, fry, roast, mroast and romer on any contrast; gene set enrichment analysis (GSEA) of ranked genes; over-representation analysis of the genes that change; scores of pathways in every sample (GSVA, ssGSEA, combined z-scores, PLAGE); activities of transcription factors and pathways from the networks CollecTRI and PROGENy (decoupler's ULM, MLM and weighted aggregate); gene sets read from GMT files | [Gene-set tests](statistics/gene_set_tests.md) |
| What drives the variation between donors, and what must be adjusted for? | Design checks for confounded factors, sample quality weights, mixed models that estimate a variance per random effect for every gene, and variance partitioning: the share of each gene's variance that each variable explains, in every cell type | [Designs, linear models and contrasts](statistics/linear_models.md), [Mixed models, REML and dream](statistics/mixed_models.md) |
| Which genes change in each cell type? | limma-voom, limma-trend and dream on every cell type of a cube; edgeR's quasi-likelihood tests and DESeq2 on one matrix at a time | [Genes that change in each cell type](guides/genes_per_cell_type.ipynb), [Bulk RNA-seq](guides/bulk_rna_seq.ipynb), [Choosing a method](statistics/choosing_a_method.md) |
| Which cell types expand or contract? | propeller. Not yet: precision-weighted mixed models or count models of composition | [Cell-type composition](guides/composition.ipynb), [Composition tests](statistics/composition.md) |
| Which pathways and regulators respond? | camera, fry, roast, mroast and romer on any contrast; gene set enrichment analysis (GSEA) of ranked genes; over-representation analysis of the genes that change; scores of pathways in every sample (GSVA, ssGSEA, combined z-scores, PLAGE); activities of transcription factors and pathways from the networks CollecTRI and PROGENy (decoupler's ULM, MLM and weighted aggregate); gene sets read from GMT files | [Gene sets and pathways](guides/gene_sets.ipynb), [Pathway and regulator activity](guides/pathway_activity.ipynb), [Gene-set tests](statistics/gene_set_tests.md) |
| What drives the variation between donors, and what must be adjusted for? | Design checks for confounded factors, sample quality weights, mixed models that estimate a variance per random effect for every gene, and variance partitioning: the share of each gene's variance that each variable explains, in every cell type | [Normalization, filtering and sample quality](guides/normalization_and_quality.ipynb), [Repeated measures and donors](guides/repeated_measures.ipynb), [Designs, linear models and contrasts](statistics/linear_models.md), [Mixed models, REML and dream](statistics/mixed_models.md) |
| Does it replicate, and how should studies be combined? | Studies aligned by sample, gene and cell type, each analysed on its own and lined up gene by gene; meta-analysis of every gene across studies, with the between-study variance, Q and I²; combined p-values, the p-value that a change replicates in at least u studies, sign agreement, pi1 between studies and the genes found in k studies; SumRank, which asks that a gene rank high in most studies, and the prediction of cases in held-out studies. Not yet: shrinkage across studies | [Meta-analysis](statistics/meta_analysis.md), [The cube and its additive layers](statistics/cubes.md) |
| Can patients be stratified from their profiles? | Not yet; cubes convert to AnnData and MuData for other tools | [Capability matrix](reference/capabilities.md) |
| Which cell-cell interactions change? | Not yet | [Related software](reference/related_software.md) |
| Do effects depend on genotype, time, sex or context? | Interactions and F-tests in designs, likelihood ratio tests, random intercepts for repeated measures, with F-tests and small-sample (Kenward-Roger) tests in mixed models. Not yet: random slopes. QTL mapping is left to dedicated tools | [Designs, linear models and contrasts](statistics/linear_models.md), [Count models](statistics/count_models.md), [Mixed models, REML and dream](statistics/mixed_models.md) |
| Do effects depend on genotype, time, sex or context? | Interactions and F-tests in designs, likelihood ratio tests, random intercepts for repeated measures, with F-tests and small-sample (Kenward-Roger) tests in mixed models. Not yet: random slopes. QTL mapping is left to dedicated tools | [Repeated measures and donors](guides/repeated_measures.ipynb), [Designs, linear models and contrasts](statistics/linear_models.md), [Count models](statistics/count_models.md), [Mixed models, REML and dream](statistics/mixed_models.md) |
| How do cells respond to drugs and doses? | A dose as a factor or a covariate of a design. Not yet: dose-response fits, benchmark doses, synergy | [Capability matrix](reference/capabilities.md) |
| Can public studies be pooled into one cube? | The cubes of several studies stacked into one, of the genes and cell types that every study has, with sample and donor ids kept apart by study; cubes import the pseudobulk layouts of scanpy, decoupler and LIANA | [The cube and its additive layers](statistics/cubes.md), [Export a cube from another tool](guides/export_cubes.ipynb) |

Expand Down
16 changes: 14 additions & 2 deletions docs/source/myst.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,26 @@ project:
title: Installation
- file: quickstart.ipynb
title: Quick start
- file: differential_expression.ipynb
title: From cells to results
- title: Build and share cubes
children:
- file: guides/export_cubes.ipynb
title: Export a cube from another tool
- title: Compare conditions
children:
- file: differential_expression.ipynb
title: Differential expression per cell type
- file: guides/normalization_and_quality.ipynb
title: Normalization, filtering and sample quality
- file: guides/genes_per_cell_type.ipynb
title: Genes that change in each cell type
- file: guides/repeated_measures.ipynb
title: Repeated measures and donors
- file: guides/composition.ipynb
title: Cell-type composition
- file: guides/gene_sets.ipynb
title: Gene sets and pathways
- file: guides/pathway_activity.ipynb
title: Pathway and regulator activity
- title: Other designs and data
children:
- file: guides/bulk_rna_seq.ipynb
Expand Down
3 changes: 2 additions & 1 deletion docs/source/statistics/choosing_a_method.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ a variable that is constant within each donor, such as sex: that variable is con
with the donor effects, and {api}`Design.check` says so. A random intercept per donor
({api}`dream_voom`) keeps the correlation between the samples of a donor and still
estimates donor-level effects. [Mixed models, REML and dream](mixed_models.md) explains
the model.
the model, and [Genes that change in each cell type](../guides/genes_per_cell_type.ipynb) and
[Repeated measures and donors](../guides/repeated_measures.ipynb) show both designs on data.

## One matrix of counts

Expand Down
1 change: 1 addition & 0 deletions docs/source/statistics/composition.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ records how many cells of each type every sample holds ({api}`Cube.n_cells`), an
{api}`propeller` tests whether the share of each cell type differs between conditions,
as speckle's `propeller` does [@phipson2022]. Proportions estimated from single-cell
data are variable, so the test compares samples, the biological replicates, not cells.
[Cell-type composition](../guides/composition.ipynb) runs it on a simulated study.

## The steps

Expand Down
3 changes: 2 additions & 1 deletion docs/source/statistics/gene_set_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ ranking many sets, such as a pathway collection. The five tests of limma work on
linear model, so they allow any design and contrast and do not permute samples; all but
romer take voom's precision weights. Gene set enrichment analysis needs only a statistic
of every gene, and over-representation analysis only the genes that an analysis called
significant.
significant. [Gene sets and pathways](../guides/gene_sets.ipynb) and [Pathway and
regulator activity](../guides/pathway_activity.ipynb) run these methods on published data.

## The effects of each gene

Expand Down
3 changes: 2 additions & 1 deletion docs/source/statistics/mixed_models.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ correlation, but it also absorbs every variable that is constant within a donor,
disease status or sex, which then cannot be tested. A random intercept per donor models
the correlation instead and keeps donor-level variables testable. dream applies this to
gene expression [@hoffman2021], and dreamlet to the pseudobulk profiles of every cell
type [@hoffman2026a].
type [@hoffman2026a]. [Repeated measures and donors](../guides/repeated_measures.ipynb)
shows dream and variance partitioning on data.

## The model

Expand Down
4 changes: 3 additions & 1 deletion docs/source/statistics/multiple_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ cell type and contrast, so a call is judged within its cell type. With
type for the same contrast, one family per contrast. In simulations, the adjustment over
all cell types lowered both the false discovery rate and the power, and so was very
conservative [@crowell2020]. The adjustment across cell types is not larger for every
test: a family with more small p-values can lower an adjusted value.
test: a family with more small p-values can lower an adjusted value, as [Genes that
change in each cell type](../guides/genes_per_cell_type.ipynb#adjusting-within-or-across-cell-types)
shows on published data.

## Other families

Expand Down
4 changes: 3 additions & 1 deletion docs/source/statistics/precision_weights.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ Log counts are noisier at low counts than at high ones, and a linear model that
every observation alike gives the noisy ones too much say. voom estimates the
mean-variance trend of the log counts and gives every observation a precision weight
from it, so that the weighted linear model and the empirical Bayes moderation of limma
apply to counts [@law2014].
apply to counts [@law2014]. [Normalization, filtering and sample
quality](../guides/normalization_and_quality.ipynb) shows voom's trend and sample
quality weights on data.

## voom

Expand Down