Scope stress correctly; add efficiency, real-data deployment, and agent/golden-dataset sections

AI_LOG.md

@@ -0,0 +1,56 @@
+# AI use log
+
+A dated, append-only record of AI-assisted planning/work sessions on this
+package and its paper (`paper/manuscript_marine.qmd`), in the spirit of the
+group's "always log AI use" practice. This is a summary log, not a raw
+transcript — one entry per session, capturing what was decided and why.
+
+## 2026-08-08 — Scope stress correctly; add efficiency, real-data deployment, and agent/golden-dataset sections to the manuscript
+
+**Tool**: Claude Code (Claude Sonnet 5).
+
+**Prompted by**: codameter has been tested against a larger-scale deployment
+(surfacing real computational bottlenecks, now fixed) and a real-data
+companion pipeline (`noisepy-dvv-cloud`, correlating CI.LJR via NoisePy on AWS
+Batch Fargate Spot) is now running. The manuscript needed to catch up, and
+the existing `§sec:stress` section needed a scope correction.
+
+**Decisions made** (all reviewed and approved by the author before any file
+was edited):
+- `§sec:stress` is *not* deleted, but its dv/v-to-stress conversion
+  methodology (the acoustoelastic equation, its best-practice table, and the
+  "companion framework, Denolle in prep." forward promise) is cut. The
+  section is rewritten as an explicit scope statement: this paper stops at
+  the depth-resolved velocity-change posterior and its covariance; it does
+  not convert that to stress, and does not promise to. Motivating mentions
+  of stress elsewhere (abstract, §bayes, §depth) are left as-is since they
+  don't have this problem — they say the covariance is useful input to such
+  work, not that this paper performs it.
+- A new section, "Toward deployment: a real-data retrospective pipeline"
+  (`§sec:deployment`), describes the `noisepy-dvv-cloud` architecture and its
+  validation protocol against the published Clements & Denolle (2023) CI.LJR
+  result. Written architecture-only at the time of this edit (the real
+  pipeline was, per live recon, an early scaffold with zero results
+  produced) — left an explicit placeholder for the dv/v(t) figure the author
+  is producing separately, with an instruction not to describe results ahead
+  of having them.
+- A short paragraph on the recent vectorized-fast-path speedups was added to
+  the end of `§sec:multiverse`. The cited numbers were independently
+  re-measured in this session rather than trusted from the commit
+  message/CHANGELOG, which turned out to be optimistic (measured ~2x/~3x/~3x
+  across repeated runs vs. previously documented ~2.3x/~4.9x/~4x) —
+  `CHANGELOG.md` was also corrected to match, since leaving it wrong would be
+  inconsistent with this paper's own thesis about honest, verified
+  reporting.
+- The Introduction's one-sentence description of codameter's agentic/skill
+  layer was reworded to match what `codameter.frugalmind`/`golden.py`/
+  `private_golden.py` actually do (RMS-recovery scoring against seeded
+  synthetic golden cases, including a hidden-truth variant), replacing an
+  inaccurate "evaluation against base models" framing.
+- Opportunistic fix: the Data Availability section referenced a stale
+  manuscript filename (`paper/manuscript.qmd`, pre-dating the switch to
+  `manuscript_marine.qmd` under `gji.cls`).
+
+**What the author did, not delegated**: the CI.LJR real-data run itself (in
+progress separately); the go/no-go on all of the above via an explicit
+plan-review step before implementation.

CHANGELOG.md

@@ -27,16 +27,18 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
   once per epsilon and applied to all days at once; trailing references come
   from a cumulative sum and the band-pass runs once over the whole matrix.
   `deviations._moving_reference` dispatches to it for the stretching
-  estimator (~4.9x on the 3-year volcano synthetic), keeping the generic
-  per-day loop for the other estimators.
+  estimator (~3x on the 3-year volcano synthetic, observed 3-4.5x across
+  repeated runs), keeping the generic per-day loop for the other estimators.
 
 ### Changed
 
 - **`_trailing_stack`** is now a difference of float64 cumulative sums —
   O(ndays x nlag) independent of the stack length instead of
-  O(ndays x k x nlag) (~2.3x at k=45). All three fast paths reproduce the
+  O(ndays x k x nlag) (~2x at k=45). All three fast paths reproduce the
   replaced per-day loops to ~1e-15 in dv/v, enforced by regression tests at
-  atol=1e-12; combined, a 5-member same-band ensemble drops ~4x in runtime.
+  atol=1e-12; combined, a 5-member same-band ensemble drops ~3x in runtime.
+  (Speedups are wall-clock, measured on one machine and noisy run to run —
+  re-benchmark before citing a more precise figure than "roughly Nx".)
 
 ## 0.3.0 — 2026-07-27
 

paper/manuscript_marine.qmd

@@ -112,8 +112,11 @@ depth profile (Section~\ref{sec:depth}). We only utilize synthetic examples for
 ground truthing on the signal processing parameters, since the concepts behind
 the observations of phase lags in scattered waves is well established [@Obermann2013].
 We package this new methodology in a Python software, ``codameter``, which we also
-recast as an agentic system with skill, examples of golden data sets, and its evaluation
-against base models.
+recast as an agentic skill: an AI agent can be asked to recommend a processing
+configuration or recover a \dvv(t) series, and its answer is scored against
+seeded synthetic golden cases with known ground truth, including a
+hidden-truth variant that withholds the answer from the public package so it
+cannot be reconstructed rather than measured.
 
 # Synthetic framework {#sec:methods}
 
@@ -709,6 +712,20 @@ ambiguity in significance (Fig.~\ref{fig:uncertainty}) \\
 \end{table}
 ```
 
+Running this many pipelines is only practical if each one is cheap. Testing
+codameter against a larger, multi-year deployment surfaced real bottlenecks in
+the per-day estimator loop, which we removed with three vectorized fast paths:
+a trailing stack built from a difference of cumulative sums instead of a
+per-day mean (roughly $2\times$ at a 45-day stack length), a vectorized
+moving-reference stretching estimator that computes the stretch-interpolation
+weights once per trial epsilon instead of once per day (roughly $3\times$ on a
+3-year synthetic), and, for ensembles that share a band, a single shared
+band-pass instead of one per pipeline member (roughly $3\times$ on a
+5-member ensemble). All three reproduce the loops they replace to within
+$10^{-15}$ in \dvv\ (regression-tested at $\mathrm{atol}=10^{-12}$); the
+reported ratios are wall-clock, vary with system load, and should be read as
+"roughly $N\times$," not exact.
+
 # A Bayesian measurement model and its data covariance {#sec:bayes}
 
 If the processing choice controls the answer, the principled response is not to
@@ -766,6 +783,30 @@ not the posterior of the averaged series, is what must be propagated.
   \label{fig:bayes}
 \end{figure}
 
+# Toward deployment: a real-data retrospective pipeline {#sec:deployment}
+
+Every result so far is on a truth-known synthetic, by design (Section~\ref{sec:methods}):
+it is the only way to separate a processing artefact from a real signal. The
+next step is to test whether the same measurement machinery holds up against
+real data at deployment scale, not just against a synthetic that was built to
+resemble it. We are building a companion pipeline, `noisepy-dvv-cloud`, that
+correlates real continuous waveforms for station CI.LJR (Lake Hughes,
+California) --- the reference station of @Clements2023 --- pulled from the
+public Southern California Earthquake Data Center S3 archive, using the
+NoisePy package [@Jiang2020] for the correlation step. Correlation and dv/v
+estimation each run as an AWS Batch job on Fargate Spot compute, following the
+same deployment pattern as QuakeScope; dv/v estimation itself is handed to
+`codameter.deviations.run_pipeline` and `codameter.uq_measurement`, with
+baseline processing configurations drawn from `codameter.use_cases.recommend`
+--- the same functions this paper's synthetic results are built on, not a
+separate reimplementation. The retrospective run is validated against the
+already-published @Clements2023 CI.LJR result before any new claim is drawn
+from it.
+
+<!-- TODO(author, before submission): drop in the CI.LJR dv/v(t) figure here
+once the pipeline run completes, with a caption describing only what it
+actually shows -- do not describe results ahead of having them. -->
+
 # Depth: propagating the measurement covariance through sensitivity kernels {#sec:depth}
 
 The remaining sections present a framework rather than new synthetics: they set
@@ -849,86 +890,31 @@ unstated model & Hidden systematic in the kernels and the moduli \\
 \end{tabularx}
 \end{table}
 
-# Stress: from velocity change to stress and strain at depth {#sec:stress}
+# Stress: a downstream step, not attempted here {#sec:stress}
 
 The quantity of interest is rarely the velocity change itself but the stress,
 strain, or pore-pressure change that produces it and --- increasingly --- the
 forcing responsible for it. Here the literature remains fragmented. Volcano and
 fault studies often interpret the velocity change qualitatively (a pre-eruptive
 pressurization, coseismic damage); hydrological studies fit an explicit
 poroelastic or thermoelastic model and separate the two by their seasonal phase
-lag [@Tsai2011; @Wang2017; @Clements2018]; and a growing body quantifies stress
-sensitivity through an acoustoelastic coefficient [@Brenguier2008; @Nakata2011].
-What is largely absent is the *uncertainty* on that final quantity and a
-reasonable apportionment of it among the competing forcings. Because the material
-properties entering the conversion are poorly constrained, the stress uncertainty
-is dominated by *epistemic* prior uncertainty rather than by the measurement, so a
-stress value reported without it is the least constrained quantity in the chain,
-and the one most often reported as a result.
-
-Velocity change is the acoustoelastic response to a stress change. Under isotropic
-load the response is scalar --- $\delta v/v = \beta\,\varepsilon_{kk}$ with the
-bridge relation $\beta=-\mu'\kappa/2\mu$ (Denolle, in prep.) --- but in general it
-is anisotropic, and the dominant fracture fabric selects *which* stress or strain
-component the velocity change tracks: a confining load reads as volumetric strain,
-a directional stress reads as one deviatoric component. Writing $\beta_{ij}(z)$
-for the acoustoelastic coefficient of the fabric-selected component, the
-depth-resolved meter is
-\begin{equation}
-\Delta\sigma_{ij}(z,t) = \beta_{ij}(z)\,\frac{\delta V_S}{V_S}(z,t),
-\label{eq:acoustoelastic}
-\end{equation}
-with $\beta_{ij}$ built from the layered moduli $\mu(z)=\rho V_S^2$,
-$\kappa(z)=\rho\,(V_P^2-\tfrac43 V_S^2)$ and the nonlinear sensitivity $\mu'(z)$;
-the forward physics, its drained/undrained regimes, and the fabric diagnostic that
-fixes $\beta_{ij}$ are the subject of the companion framework (Denolle, in prep.).
-These coefficients are layered and weakly constrained --- sometimes inferred from a
-tomographic $V_P,V_S$ model --- so they enter as *per-layer priors*, and the
-stress covariance is the Monte-Carlo pushforward of the depth posterior $C_m(z)$
-*through* those priors. This is the step where the aleatoric measurement chain and
-the epistemic material priors merge; taking either alone understates the stress
-uncertainty. Effective stress ($\Delta\sigma'=\kappa\,\varepsilon_{\rm vol}$ under
-isotropic load, poroelastically clean) is the primary quantity; total stress adds a
-poroelastic term $\alpha_B\,\Delta p$ and is an optional extension.
-
-Attribution --- how much of $\Delta\sigma(z,t)$ is thermoelastic, poroelastic,
-load, damage, or tectonic --- is deliberately *not* codameter's step. Each
-mechanism is a stress term that can act in isolation, and decomposing the total
-into those terms is the contribution of the companion unified-framework paper
-(Denolle, in prep.), which supplies the forward models and the fabric diagnostic
-that define $\beta_{ij}$. codameter's role ends at delivering $\Delta\sigma_{ij}(z,t)$
-with its posterior --- the input any such decomposition must consume. The
-division of labour is well defined: this paper is accountable for the measurement
-covariance and its propagation to stress; the framework is accountable for the
-physics that turns that stress into a named mechanism.
-
-\begin{table}
-\footnotesize
-\caption{Stress step: best practice versus the common deviation in converting a
-depth profile of velocity change to stress, with the consequence for the reported
-result. Attribution of the stress to a forcing mechanism is the companion
-framework's step and is not tabulated here.}
-\label{tab:bp-stress}
-\begin{tabularx}{\textwidth}{@{}>{\raggedright\arraybackslash}p{2.6cm} L L L@{}}
-\toprule
-\textbf{Choice} & \textbf{Best practice} & \textbf{Common deviation} &
-\textbf{Consequence} \\
-\midrule
-Stress conversion & Acoustoelastic meter $\Delta\sigma_{ij}=\beta_{ij}(z)\,\delta V_S/V_S$
-with layered $\beta_{ij},\mu'(z)$ priors & Single scalar coefficient, no prior &
-Understated (epistemic) stress error \\ \hline
-Stress component & Identify the fabric-selected component (isotropic vs deviatoric)
-before inverting & Assume the isotropic $\delta v/v=\beta\varepsilon_{kk}$
-everywhere & Wrong sign and magnitude under deviatoric load \\ \hline
-Uncertainty budget & Merge the aleatoric depth covariance with the epistemic
-material priors & Report the measurement error only & Stress error too small by
-its dominant term \\ \hline
-Stress measure & Report effective stress (poroelastically clean); flag any
-total-stress assumptions & Conflate effective and total stress &
-Non-comparable, ambiguous stress \\
-\bottomrule
-\end{tabularx}
-\end{table}
+lag [@Tsai2011; @Wang2017; @Clements2018]. What is largely absent across this
+literature is the *uncertainty* on that final quantity and a reasonable
+apportionment of it among the competing forcings.
+
+Converting a depth-resolved velocity-change posterior to stress or strain is a
+further step this paper does not attempt: it requires material priors --- an
+acoustoelastic, poroelastic, or thermoelastic coefficient, its depth dependence,
+and the fracture fabric or drainage regime that selects which stress or strain
+component the velocity change actually tracks --- that are outside this paper's
+scope. Because those coefficients are themselves poorly constrained, any such
+conversion is dominated by *epistemic* prior uncertainty rather than by the
+measurement, so a stress value reported without accounting for that is the least
+constrained quantity in the chain, and the one most often reported as a result.
+We stop deliberately at the depth-resolved posterior of Section~\ref{sec:depth}
+and its covariance: that object, not a stress or strain estimate, is what this
+paper delivers, and what any downstream stress or strain conversion must
+consume.
 
 # Discussion {#sec:discussion}
 
@@ -996,16 +982,16 @@ choice into a versioned, inspectable one, and lets a reader re-run a study's
 pipeline on the truth-known synthetic to see its bias before trusting it on data.
 
 **Propagate the covariance, do not truncate it.** The measurement covariance is
-not the end of the analysis but its first input. Sections~\ref{sec:depth}
-and~\ref{sec:stress} set out the rest of the chain: invert the per-band $C_d$
-through sensitivity kernels for a depth profile and its covariance, separate the
-shear-velocity and density contributions where the ground is partially saturated,
-and push the depth posterior through layered petrophysical priors to effective
-stress and strain at depth, which a companion framework then attributes to a
-forcing mechanism. At each step the honest error bar depends on the one before it,
-which is why a reliable stress at depth begins with a reliable $C_d$. Building
-and testing those propagation stages on truth-known synthetics is the natural
-continuation of this work, and is under way in the open codameter framework.
+not the end of the analysis but its first input. Section~\ref{sec:depth} sets out
+the next step of the chain: invert the per-band $C_d$ through sensitivity kernels
+for a depth profile and its covariance, separating the shear-velocity and density
+contributions where the ground is partially saturated. Converting that
+depth-resolved posterior to stress or strain (Section~\ref{sec:stress}) requires
+material priors this paper does not attempt to constrain, and is left to other
+work; what we deliver is the depth posterior and its covariance, the object any
+such conversion must consume. Building and testing the depth-propagation stage on
+truth-known synthetics is the natural continuation of this work, and is under way
+in the open codameter framework.
 
 # Conclusions {#sec:conclusions}
 
@@ -1018,12 +1004,12 @@ without touching the data. The remedy is not a single mandated pipeline but
 transparency: report every choice, sample the ones the physics does not fix,
 carry the resulting measurement covariance into the inference, and release the
 pipeline as executable code. The same discipline extends down the inference
-chain: the measurement covariance is the input a depth inversion consumes, that
-depth profile the input a stress conversion consumes, and only a pipeline that
-carries the covariance to the end can attach an honest uncertainty to a stress or
-strain change at depth --- and hand a reliable posterior to the companion
-framework that attributes it to a forcing. We offer codameter as one such record,
-and as the package in which the depth and stress stages are being built.
+chain: the measurement covariance is the input a depth inversion consumes, and
+only a pipeline that carries the covariance that far can attach an honest
+uncertainty to a depth-resolved velocity-change profile. Converting that profile
+to stress or strain is a further step this paper does not attempt
+(Section~\ref{sec:stress}). We offer codameter as one such record, and as the
+package in which the depth stage is being built.
 
 \appendix
 
@@ -1133,7 +1119,7 @@ openly available at <https://github.com/Denolle-Lab/codameter>. All figures are
 reproduced by `python literature/synthetic_dvv_demo.py`, the multiverse and
 deviation ranking by `python -m codameter.deviations`, and the Bayesian
 measurement model by `python -m codameter.uq_bayes`. The manuscript itself is
-rendered from `paper/manuscript.qmd` by `python paper/build.py`.
+rendered from `paper/manuscript_marine.qmd` by `python paper/build.py`.
 
 # Acknowledgements {.unnumbered}