Restructure the documentation as a short book, with one home for each fact

AGENTS.md

@@ -4,7 +4,7 @@ These instructions apply to the whole repository.
 
 ## Layout
 
-- Package code lives in `cytepool/`, tests in `tests/`, reference comparisons in
+- Package code is in `cytepool/`, tests in `tests/`, reference comparisons in
   `validation/`, and the test data those comparisons write in `tests/data/`.
 - Name modules, functions and files after what they compute, in terms a statistician or
   biologist would search for. Avoid project-internal labels.
@@ -16,10 +16,40 @@ These instructions apply to the whole repository.
   and are imported where they are used.
 - Use built-in generics and `X | None`. Do not add `typing.List`, `typing.Optional`, or
   `from __future__ import annotations`.
-- Avoid em dashes in public prose, docstrings and comments.
 - Validate arguments at the public boundary and raise errors that say what was wrong and
   what is expected.
 
+## Writing
+
+These rules apply to everything people read: pages, the README, docstrings, error
+messages, figure labels, commit messages and pull-request descriptions.
+
+- Start with the problem, explain the solution, then show its value. Lead with the
+  reader's task: "Test which genes change..." rather than "A library for testing...".
+- Give numbers context, round them where more digits add nothing, and keep the
+  precision that statistics need. Keep caveats in supporting detail: an opening stays
+  accurate, but conditions, versions and limits do not dominate it.
+- Use plain words, short sentences and one term for one thing, and explain a
+  statistic where the reader meets it.
+- No em dashes, and no en dashes used as punctuation; hyphenated words and terms such
+  as p-value are fine. No all-caps or labelled headings, no decorative numbering, no
+  centred dots between words.
+- No stock metaphors, such as "X lives in Y", "the X behind it" or "under the hood",
+  and no promotional claims: say what something does or where it is stored.
+
+## Documentation
+
+- Each fact has one home; other places give it in a clause and link to it. The
+  chapters of "Understand the methods" (`docs/source/statistics/`) explain the methods;
+  the Validation page gives the agreement with the reference tools, with the numbers
+  the validation scripts print; docstrings define behaviour; the guides show tasks on
+  data; and "Choosing a method" lists the questions, their defaults and what is not
+  implemented yet.
+- A change that adds, removes or limits a capability updates the summary of the
+  Validation page and "Choosing a method".
+- Link a section by its label, `(name)=`, rather than by its heading, and give a
+  docstring the address of the chapter section that explains it.
+
 ## Numerical routines
 
 - Compile numerical loops with `cytepool._jit.jit`. It caches machine code in a directory
@@ -50,9 +80,10 @@ These instructions apply to the whole repository.
 ## Checks
 
 ```bash
-uv sync --extra test --group dev
+uv sync --extra test --group dev --group docs
 uv run pytest
-uv run ruff check cytepool tests validation
-uv run ruff format --check cytepool tests validation
+uv run ruff check cytepool tests validation docs
+uv run ruff format --check cytepool tests validation docs
 uv run mypy cytepool
+make -C docs api
 ```

THIRD_PARTY_NOTICES.md

@@ -57,7 +57,7 @@ SOFTWARE.
 
 ### fgsea
 
-`cytepool/gsea.py` ports the algorithms of fgsea 1.32.2
+`cytepool/gene_set_enrichment.py` ports the algorithms of fgsea 1.32.2
 (https://bioconductor.org/packages/fgsea, https://github.com/ctlab/fgsea), keeping their
 structure, constants and order of operations, so that given fgsea's seeds it gives
 fgsea's results: from `src/fastGSEA.cpp`, the block prefix sums (`SegmentTree`), the
@@ -169,8 +169,9 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 ### SumRank
 
-`cytepool/sumrank.py` and `cytepool/held_out_prediction.py` implement SumRank and its
-held-out prediction from Nakatsuka et al. (2025), following conventions of the authors'
+`cytepool/stats/sumrank.py`, `cytepool/sumrank_across_studies.py` and
+`cytepool/held_out_studies.py` implement SumRank and its held-out prediction from
+Nakatsuka et al. (2025), following conventions of the authors'
 code (https://github.com/nathan-nakatsuka/scRNA_Reproducibility, commit 46392f66;
 `SumRank.R`, `CalibratePValueswithPermutations.R`, `PermutingCaseControlStatus.R`,
 `UCell_AUC_code.R`): ties ranked by their average rank, floor(p k) of the studies summed

cytepool/_analysis_records.py

@@ -2,7 +2,7 @@
 
 The result of a workflow on a cube, such as ``limma_voom`` or ``propeller``, holds a
 provenance record. These functions build the parts that every workflow records alike:
-the cube it read, identified by its fingerprint and the digests behind it; the design
+the cube it read, identified by its fingerprint and the digests it is made of; the design
 it fitted, with the contrasts it tested; and the versions of the software.
 """
 

cytepool/_expression_values.py

@@ -238,7 +238,7 @@ def _cube_values(
     kind = checked_choice(values, "values", VALUES)
     fewest = integral_number(min_cells, "min_cells")
     if fewest < 0:
-        raise ValueError(f"min_cells must be at least 0, got {fewest}")
+        raise ValueError(f"min_cells must be non-negative, got {fewest}")
     if isinstance(expression_filter, Mapping):
         filter_settings = filter_setting(expression_filter)
         check_sample_groups({"expression_filter": filter_settings}, cube.samples)

cytepool/_gene_sets.py

@@ -46,7 +46,10 @@
 IntArray = npt.NDArray[np.intp]
 
 # A pivot of the QR decomposition this small relative to the norm of its column is
-# zero up to rounding; see _effects_of_last_coefficient.
+# zero up to rounding; see _effects_of_last_coefficient. limma computes most such
+# zeros exactly, but leaves rounding noise in a few percent of genes, a different few
+# in camera and in fry, so that its bases change with the last bits of the data there
+# (found by probing); counted as zero, they do not here.
 _ZERO_PIVOT = 1e-12
 
 
@@ -88,12 +91,8 @@ def gene_set_data(
     gives its values (and a VoomResult its weights), and ``design``, ``contrast`` and
     ``feature_ids`` are as given.
     """
-    with_trend = (
-        None if trend is None else checked_flag(trend, "trend", expected="a bool")
-    )
-    with_robust = (
-        None if robust is None else checked_flag(robust, "robust", expected="a bool")
-    )
+    with_trend = None if trend is None else checked_flag(trend, "trend")
+    with_robust = None if robust is None else checked_flag(robust, "robust")
     if isinstance(data, DifferentialExpressionResult):
         for name, value, owner in (
             ("design", design, "the design of its group's fit"),

cytepool/_group_fits.py

@@ -186,14 +186,17 @@ def prepare_group(
             return fit.stop(
                 "fixed_effects_not_estimable",
                 "the samples of this group cannot separate the fixed effects: "
-                + "; ".join(relations),
+                + listing(relations, separator="; "),
             ), None
     else:
+        # limma's contrasts.fit refuses an estimable contrast that weighs an aliased
+        # column, and every contrast passed with it (found by probing); it is
+        # computed here, from the fit on the kept columns.
         violations = _contrast_violations(model, aliasing, contrasts)
         for position, relations in violations.items():
             fit.not_estimable[contrasts.names[position]] = (
                 "not estimable from the samples of this group, which cannot separate "
-                "the effects it depends on: " + "; ".join(relations)
+                "the effects it depends on: " + listing(relations, separator="; ")
             )
         estimable = [p for p in range(len(contrasts)) if p not in violations]
         if not estimable:
@@ -255,6 +258,8 @@ def left_out(
     counts: np.ndarray, cells: np.ndarray | None, min_cells: int, min_counts: float
 ) -> list[str]:
     """Why each sample is left out of a group's fit, or "" when it is used."""
+    # A sample with exactly min_cells cells or min_counts counts is kept, as
+    # decoupler's filter_samples keeps it (found by probing).
     has_counts = counts.any(axis=1)
     # Totals over every feature, exact for whole counts, as R's colSums gives them.
     totals = library_sizes(counts) if min_counts > 0.0 and counts.size else None
@@ -319,10 +324,10 @@ def group_positions(cube: Cube, groups: str | Sequence[str] | None) -> list[int]
 def min_cells_setting(value: Any) -> int:
     """The fewest cells a sample needs in a group, a whole number of at least 0."""
     if isinstance(value, bool | np.bool_):
-        raise TypeError("min_cells must be a whole number of cells, not bool")
+        raise TypeError("min_cells must be a whole number of cells, got bool")
     number = index_integer(value, "min_cells", expected="a whole number of cells")
     if number < 0:
-        raise ValueError(f"min_cells must be at least 0, got {number}")
+        raise ValueError(f"min_cells must be non-negative, got {number}")
     return number
 
 
@@ -369,8 +374,8 @@ def threads_setting(value: Any) -> int | None:
     if value is None:
         return None
     if isinstance(value, bool | np.bool_):
-        raise TypeError("n_threads must be a positive integer or None, not bool")
-    number = index_integer(value, "n_threads", expected="a positive integer or None")
+        raise TypeError("n_threads must be an integer or None, got bool")
+    number = index_integer(value, "n_threads", expected="an integer or None")
     if number < 1:
         raise ValueError(f"n_threads must be at least 1, got {number}")
     return number

cytepool/_jit.py

@@ -100,8 +100,8 @@ def thread_limit(n_threads: int | None) -> Iterator[int]:
         yield numba.get_num_threads()
         return
     if isinstance(n_threads, bool):
-        raise TypeError("n_threads must be a positive integer or None, not bool")
-    limit = index_integer(n_threads, "n_threads", expected="a positive integer or None")
+        raise TypeError("n_threads must be an integer or None, got bool")
+    limit = index_integer(n_threads, "n_threads", expected="an integer or None")
     if limit < 1:
         raise ValueError(f"n_threads must be at least 1, got {limit}")
     previous = numba.get_num_threads()

cytepool/_messages.py

@@ -29,15 +29,22 @@
 so that a warning is attributed to the first caller outside the package."""
 
 
-def listing(items: Sequence[Any]) -> str:
-    """Items as "a, b and c", or the first few and how many more."""
-    shown = [str(item) for item in items[:LISTED]]
+def listing(items: Sequence[Any], separator: str = ", ", limit: int = LISTED) -> str:
+    """Items as "a, b and c", or the first ``limit`` and how many more, as "a, b, c
+    and 3 more".
+
+    Items that hold commas themselves, such as relations between columns, take
+    another separator, such as "; ", which then comes before the "and" of the last
+    item too: "a; b; and c".
+    """
+    shown = [str(item) for item in items[:limit]]
     more = len(items) - len(shown)
+    last = " and " if separator == ", " else f"{separator}and "
     if more:
-        return ", ".join(shown) + f" and {more} more"
+        return separator.join(shown) + f"{last}{more:,} more"
     if len(shown) <= 1:
         return "".join(shown)
-    return ", ".join(shown[:-1]) + " and " + shown[-1]
+    return separator.join(shown[:-1]) + last + shown[-1]
 
 
 def quoted(items: Sequence[Any]) -> str:

cytepool/_options.py

@@ -4,8 +4,8 @@
 Each check takes the option as the caller gave it and its name, and returns it as a
 bool, str, float, int or numpy random generator, or raises TypeError for a value of
 the wrong type and ValueError for one out of range. The messages name the option, say
-what it must be and what it was; where the methods word a message differently, the
-check takes the phrase as an argument, so that every message stays as it was.
+what it must be and what it was: a TypeError ends with the type it got, and a
+ValueError with the value.
 """
 
 import math
@@ -18,20 +18,18 @@
 from ._masked_arrays import reject_masked
 
 
-def checked_flag(value: Any, name: str, *, expected: str = "True or False") -> bool:
+def checked_flag(value: Any, name: str) -> bool:
     """An option that is True or False, as a bool.
 
     Args:
         value: The option as given; a numpy bool is accepted.
         name: The option's name, for the message.
-        expected: What the message says the option must be: "True or False", or
-            "a bool" as the statistics' messages say.
 
     Raises:
-        TypeError: If ``value`` is not a bool.
+        TypeError: If ``value`` is not True or False.
     """
     if not isinstance(value, bool | np.bool_):
-        raise TypeError(f"{name} must be {expected}, got {type(value).__name__}")
+        raise TypeError(f"{name} must be True or False, got {type(value).__name__}")
     return bool(value)
 
 
@@ -83,14 +81,6 @@ def non_negative_number(value: Any, name: str) -> float:
     return number
 
 
-def number_at_least(value: Any, name: str, lower: float) -> float:
-    """A finite real number of at least ``lower``."""
-    number = real_number(value, name)
-    if not (math.isfinite(number) and number >= lower):
-        raise ValueError(f"{name} must be finite and at least {lower:g}, got {number}")
-    return number
-
-
 def positive_finite_number(value: Any, name: str) -> float:
     """A finite real number above 0."""
     number = real_number(value, name)