Skip to content

Commit 1265833

Browse files
MapCreation GRID: hybrid time sampling for pp and MC mode
- Hybrid time sampling (pp): with MAKE_HYBRID_MAPS=true, SLOT_LENGTH_MIN ("fine") maps for the first FINE_WINDOW_MIN minutes of each fill and SLOT_LENGTH_MIN_COARSE ("coarse") maps for the rest. The cutoff is anchored to the fill start, read from optional <fill> <startTimestampMs> columns in the discoverResiduals.sh batch file. Per-slot mapMeta.txt in the output. - MC mode: discoverResiduals.sh --mc [--mc-cycle] searches /alice/sim/<year>/<prodTag>/<cycle>/<run>/ and reads the merged residuals. MC_MODE=true in submitMapCreation.sh (stageSlotsForGrid.py --mc) makes one map per run without the per-TF time cut, since MC residual files carry no time label. - Post-process: request 1 core (single-threaded macros), shorter c1D/c2D QA PNG names. - README: hybrid and MC usage, MC examples in the Quickstart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent dc51651 commit 1265833

6 files changed

Lines changed: 547 additions & 67 deletions

File tree

‎DATA/production/configurations/mapCreationGrid/README.md‎

Lines changed: 152 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,24 @@ left as-is.
2020
1. **Build the slot lists** (local, once per batch of runs):
2121

2222
```bash
23-
./discoverResiduals.sh <batch_file>
23+
./discoverResiduals.sh <batch_file> --calib-pass <cpass>
2424
```
2525

26-
`<batch_file>` lists one run per line: `<period> <run>`, e.g. `LHC26ak 572557`.
26+
`<batch_file>` lists one run per line: `<period> <run>`, e.g. `LHC26ak 572557`. (pp only, and only if
27+
you'll set `MAKE_HYBRID_MAPS=true` in step 2: add `<fill> <startTimestampMs>` — see "Hybrid time
28+
sampling (pp)" under step 2.)
29+
30+
**MC:** use `--mc` instead of `--calib-pass`, with the MC production tag in place of the period:
31+
32+
```bash
33+
printf 'LHC26j19_residuals %s\n' 568336 568337 568338 > batch_mc.txt # <prodTag> <anchorRun>
34+
./discoverResiduals.sh batch_mc.txt --mc
35+
```
36+
37+
This searches `/alice/sim/<year>/<prodTag>/0/<run>/ResidualTPCmerging/Stage_0/` for the production's
38+
merged residuals, falling back to the per-job files under `/alice/sim/<year>/<prodTag>/0/<run>/` with
39+
a warning when there are none (`--mc-cycle <n>` for a production cycle other than `0`). In step 2, also set `MC_MODE=true` and `BAD_RANGES_NAME=""`. Details: "MC productions" in
40+
"The chain, in order" below.
2741

2842
2. **Submit stage 1 (map creation).** In `submitMapCreation.sh`, set:
2943
- `BAD_RANGES_NAME` — the bad-time-ranges file for this data period (keep the default if it still
@@ -43,6 +57,39 @@ left as-is.
4357
This takes hours per map — watch progress on alimonitor and wait for it to finish (or mostly finish)
4458
before continuing.
4559

60+
**pp hybrid production** (fine-length maps early in each fill, coarse-length for the rest — see
61+
"Hybrid time sampling (pp)" below): also set `MAKE_HYBRID_MAPS=true`, `SLOT_LENGTH_MIN_COARSE`,
62+
`FINE_WINDOW_MIN`, and it's worth pinning `RUNS_FILTER` explicitly since `./lists/` accumulates across
63+
batches. For example:
64+
65+
```bash
66+
JOBNAME="mapcreation-pp-lowfield-v1" # unique per submission
67+
BAD_RANGES_NAME="<your pp bad-ranges file>" # or "" to disable -- the default is PbPb-specific
68+
RUNS_FILTER="566139,566159,566160,566161,566188,566189,566190,566191"
69+
MAKE_HYBRID_MAPS=true
70+
SLOT_LENGTH_MIN=5 # fine length (first FINE_WINDOW_MIN min of each fill)
71+
SLOT_LENGTH_MIN_COARSE=60 # coarse length (rest of the fill)
72+
FINE_WINDOW_MIN=60
73+
```
74+
75+
Needs step 1's batch file to have carried `<fill> <startTimestampMs>` for these runs (see step 1
76+
above) — `submitMapCreation.sh` fails fast, before submitting anything, if any run named in
77+
`RUNS_FILTER` is missing that.
78+
79+
**MC production** (lists from `discoverResiduals.sh --mc` in step 1): set `MC_MODE=true`. Each run
80+
then becomes one map over all of its files, with no time cut. `SLOT_LENGTH_MIN` is ignored, and
81+
`MAKE_HYBRID_MAPS` must stay `false`. For example:
82+
83+
```bash
84+
JOBNAME="mapcreation-MC-LHC26j19-v1" # unique per submission
85+
BAD_RANGES_NAME="" # the data's bad-ranges list does not apply to MC
86+
MC_MODE=true
87+
MAKE_HYBRID_MAPS=false
88+
```
89+
90+
It should report `MC_MODE=true -- one map per run, no time cut` and one slot per run. Step 3 is the
91+
same as for data.
92+
4693
3. **Submit stage 2 (smoothing + FastTransform).** In `submitPostProcess.sh`, set `STAGE1_JOBNAME_TAG`
4794
to the exact `JOBNAME` used in step 2, then:
4895

@@ -117,13 +164,21 @@ without redoing the expensive stage-1 work for that slot.
117164
./discoverResiduals.sh <batch_file> [--calib-pass cpass2_residuals]
118165
```
119166

120-
`<batch_file>` is one run per line: `<period_token> <run>`, e.g.:
167+
`<batch_file>` is one run per line: `<period_token> <run> [<fill> <startTimestampMs>]`, e.g.:
121168

122169
```
123170
LHC26ak 572557
124171
LHC26ak 572558
125172
```
126173

174+
The last two columns are **optional** and only needed for **hybrid map creation** (see below) — omit them
175+
for a normal, non-hybrid production. When given, both must be present together: `<fill>` is the LHC fill
176+
number (shared by every run in it) and `<startTimestampMs>` is that run's start-of-run (SOR) timestamp,
177+
UTC milliseconds (13 digits) — the same metadata columns the old SLURM-era hybrid tooling
178+
(`makeStaggerd.py`/`combinedSampling.py`) used. When present, `discoverResiduals.sh` writes them to a
179+
`<run>.fillinfo.txt` sidecar next to that run's slot-list index, which `stageSlotsForGrid.py --hybrid`
180+
reads later, at submission time.
181+
127182
This queries AliEn (`/alice/data/<year>/<period>/<run>/<calib-pass>/`) for `o2tpc_residuals_*.root`
128183
files and writes per-run slot lists under `./lists/<run>/`. Runs in parallel (`N_PARALLEL=8` in the
129184
script). Safe to re-run; each run is independent. The period is resolved **per line**, so a batch file
@@ -221,6 +276,44 @@ it adds to it, and Step 2 by default puts *everything* under `./lists/` into the
221276
silently recalibrates the earlier batch from scratch at hours per slot. Either use a fresh `listsDir`
222277
per batch, or set `RUNS_FILTER` in the Step-2 wrapper (see below).
223278

279+
#### MC productions
280+
281+
MC residuals live under `/alice/sim/<year>/<prodTag>/<cycle>/<anchorRun>/<NNN>/o2tpc_residuals.root`,
282+
not under a data period's calib pass. Use `--mc`, with the production tag in place of the period:
283+
284+
```bash
285+
cat > batch_mc.txt <<EOF
286+
LHC26j19_residuals 568336
287+
LHC26j19_residuals 568337
288+
EOF
289+
./discoverResiduals.sh batch_mc.txt --mc # cycle 0; --mc-cycle <n> for another one
290+
```
291+
292+
The year is taken from the tag (`LHC26…` → 2026). `--calib-pass` is rejected with `--mc`, since the MC
293+
path has no such level.
294+
295+
The production also merges the per-job files, about 150 jobs into one
296+
`<anchorRun>/ResidualTPCmerging/Stage_0/<NNN>/o2tpc_residuals.root` (same trees, ~15 GB). When any
297+
merged file exists, the run's list holds **only** the merged files. A search over the whole run
298+
directory would list every residual twice, once per-job and once merged, and each `alien://` open costs
299+
~7 s in stage 1, so a few dozen merged files beat thousands of per-job ones. A `Stage_0/<NNN>/` without
300+
the file is a merge job that failed or is still running: its share of the statistics is missing,
301+
and discovery prints a `WARNING: only <n>/<m> merge jobs produced o2tpc_residuals.root` for that run.
302+
Wait for the merging, or accept the smaller sample. A run with no merged file at all falls back to the
303+
per-job files, with a warning.
304+
305+
MC file names carry no `<firstTS>_<lastTS>` label, so there is nothing to cut windows from:
306+
`makeResidualLists.sh` writes one list per run (`residuals.all.<run>.txt`), and Step 2 needs
307+
`MC_MODE=true`. Each run then becomes **one** map over all of its files, with window bounds `0 0`, which
308+
switches off the macro's per-TF time cut (`firstTFTime > 0` is its condition). `MAX_TRACKS_PER_SLICE`
309+
still caps the statistics. MC paths carry no `_orbit` field, so `mapCreationJob.sh` leaves the file order
310+
as listed. Staging refuses to mix the two kinds: data lists with `--mc`, or `MAKE_HYBRID_MAPS=true` with
311+
`MC_MODE=true`, are errors, and MC lists without `--mc` are skipped with a warning naming the flag.
312+
313+
Two things work differently in stage 2: `mapMeta.txt` gets `tsStart=0`, `tsEnd=0`, and `fill=-1`. The
314+
bad-time-ranges list is applied by the anchor run's orbits, so set `BAD_RANGES_NAME=""` unless you want
315+
the data's bad ranges cut from the MC too.
316+
224317
### Step 2 — configure and submit stage 1 (map creation)
225318

226319
Edit the config block at the top of **`submitMapCreation.sh`**:
@@ -229,7 +322,11 @@ Edit the config block at the top of **`submitMapCreation.sh`**:
229322
|---|---|
230323
| `JOBNAME` | Identifies this production. Tags the AliEn manifest (`mapSlots.<JOBNAME>.tsv`) and the small `stage1Workdir.<JOBNAME>.txt` pointer file (see Step 3) — **must be unique per submission** so concurrent/repeated productions don't clobber each other's manifest or pointer. Stage 2 needs to know this value (see Step 4). |
231324
| `TOPWORKDIR` | Static top-level bucket under your GRID home for this job's working directory (replaces `grid_submit.sh`'s generic `selfjobs` default with something identifiable). |
232-
| `SLOT_LENGTH_MIN` | Target length of **one map**, in minutes. Each run is cut into whole steps of this length; a trailing remainder shorter than half a step is absorbed into the last window rather than left as a runt that would fail `MIN_TRACKS_PER_SLICE`. `0` = one map per run. A 20 min run at `5` gives 4 maps, a 60 min run gives 12 — the count follows the run length instead of being fixed. |
325+
| `SLOT_LENGTH_MIN` | Target length of **one map**, in minutes. Each run is cut into whole steps of this length; a trailing remainder shorter than half a step is absorbed into the last window rather than left as a runt that would fail `MIN_TRACKS_PER_SLICE`. `0` = one map per run. A 20 min run at `5` gives 4 maps, a 60 min run gives 12 — the count follows the run length instead of being fixed. With `MAKE_HYBRID_MAPS=true`, this is reused as the **fine** map length (see below). |
326+
| `MAKE_HYBRID_MAPS` | `true`/`false` (default `false`). Hybrid time sampling for pp: `SLOT_LENGTH_MIN`-length ("fine") maps for the first `FINE_WINDOW_MIN` minutes of each **fill**, `SLOT_LENGTH_MIN_COARSE`-length ("coarse") maps for the rest of that fill — see "Hybrid time sampling (pp)" below. |
327+
| `SLOT_LENGTH_MIN_COARSE` | Only used when `MAKE_HYBRID_MAPS=true`: target length of one **coarse** map, in minutes. Same `0`/trailing-remainder semantics as `SLOT_LENGTH_MIN`, applied independently within the coarse region. |
328+
| `FINE_WINDOW_MIN` | Only used when `MAKE_HYBRID_MAPS=true`: duration, in minutes, of the fine-sampling window measured from the fill's start (default `60`). |
329+
| `MC_MODE` | `true`/`false` (default `false`). `true` for lists from `discoverResiduals.sh --mc`: one map per run, no time cut, `SLOT_LENGTH_MIN` ignored — see "MC productions" under Step 1. |
233330
| `PACKAGESPEC` | O2/O2DPG software tag to run under — set this to the async-reco tag matching your data. Also determines whether `MACRO_SOURCE=official` can find the macros (see "Where the macros come from" above). |
234331
| `ASUSER` | AliEn account this job runs/is charged as, e.g. `pwg_pp` (your account must already be a member of that group) instead of your own personal account/quota. Determines both which quota pays for the job and where its `${ALIEN_JDL_OUTPUTDIR}` lands (see Step 3). Verify on alimonitor that the first job is really attributed correctly before relying on it for a big batch. |
235332
| `MACRO_SOURCE` / `MACRO_LOCAL_PATH` | See "Where the macros come from" above. |
@@ -259,6 +356,32 @@ Note: the macro no longer reads IDC scalers from CCDB directly (`useCTPLumi`/`nS
259356
are gone from its signature) — luminosity values are joined back offline by `SmoothingExtrapolate.C` in
260357
stage 2 instead (its own `USE_CTP_LUMI`, see Step 4).
261358

359+
#### Hybrid time sampling (pp)
360+
361+
pp fills need denser time sampling right after the fill starts, while space charge is still building up,
362+
than for the rest of the fill — the same physics motivation as the old SLURM-era
363+
`makeStaggerd.py`/`combinedSampling.py` hybrid tooling this mirrors. Set `MAKE_HYBRID_MAPS=true` and
364+
`stageSlotsForGrid.py` cuts `SLOT_LENGTH_MIN`-length ("fine") windows for the first `FINE_WINDOW_MIN`
365+
minutes of each **fill**, `SLOT_LENGTH_MIN_COARSE`-length ("coarse") windows for the rest of it — instead
366+
of `SLOT_LENGTH_MIN` uniformly everywhere.
367+
368+
The cutoff is anchored to the **fill's own start**, not each run's — deliberately, since a pp fill
369+
routinely spans several runs, and the build-up being sampled densely is a property of the fill, not of
370+
where a run happens to start within it. Concretely: the fill's start is the earliest `startTimestampMs`
371+
among that fill's runs present under `listsDir` for *this* submission (same "what's actually in this
372+
production" scoping `RUNS_FILTER` already uses elsewhere in this doc) — so a run that starts 70 minutes
373+
into its fill gets windowed entirely coarse, even though it is only 10 minutes old itself, while a run
374+
straddling the cutoff gets fine windows before it and coarse windows after, with no runt window on either
375+
side of the boundary (each region absorbs its own short trailing remainder independently, same as the
376+
non-hybrid case).
377+
378+
This needs every selected run's `discoverResiduals.sh` batch-file line to have carried the optional
379+
`<fill> <startTimestampMs>` columns (see Step 1) — that's what produces each run's `<run>.fillinfo.txt`
380+
sidecar. If any selected run is missing it, `stageSlotsForGrid.py` fails loudly and locally, before
381+
anything is submitted, listing exactly which run(s) — not a silent fall-back to per-run windowing, which
382+
would silently change the physics. A non-hybrid submission (`MAKE_HYBRID_MAPS=false`, the default) needs
383+
none of this — plain 2-column batch-file rows are enough, same as before this feature existed.
384+
262385
Then:
263386

264387
```bash
@@ -378,7 +501,32 @@ logical slot:
378501
FT_voxRes.<run>_<firstTS>_<lastTS>.TD<w>of<W>.it0.GausSmooth.Extrapolation.root
379502
voxResQA.voxRes.<run>_<firstTS>_<lastTS>.TD<w>of<W>.it0.GausSmooth.Extrapolation_1D.root
380503
voxResQA.voxRes.<run>_<firstTS>_<lastTS>.TD<w>of<W>.it0.GausSmooth.Extrapolation_2D.root
504+
stage1Source.txt
505+
mapMeta.txt
506+
```
507+
508+
`stage1Source.txt` is a one-line pointer to the stage-1 map this slot was built from (written right
509+
after smoothing, so it exists even if the FastTransform step fails). `mapMeta.txt` is the per-slot
510+
summary, one `key=value` per line, written only once `FT_*.root` is staged:
511+
381512
```
513+
stage1=<alien path of the stage-1 map>
514+
stage2=<alien path of FT_*.root>
515+
meanIDC=...
516+
medianIDC=...
517+
meanCTP=...
518+
medianCTP=...
519+
tsStart=<firstTS>
520+
tsEnd=<lastTS>
521+
fill=<fill number>
522+
```
523+
524+
The IDC values come from the smoothed map's UserInfo (offline IDC join in `SmoothingExtrapolate.C`) and
525+
the CTP values from stage 1. `tsStart`/`tsEnd` are the manifest window in ms, the same values as in the
526+
file names. `fill` is looked up in `GLO/Config/GRPLHCIF` at `tsStart`; it is `-1` if the CCDB lookup
527+
failed.
528+
529+
Both files are non-fatal: if writing or staging one fails, the slot still succeeds.
382530

383531
Stage 1's and stage 2's output for the same slot are *not* grouped under one tidy parent directory —
384532
that's the direct cost of letting the JDL own output placement instead of managing it ourselves (see

0 commit comments

Comments
 (0)