Skip to content

feat!: --explain reads back what a run said - #35

Merged
Ruby184 merged 1 commit into
mainfrom
feat/merge-unmatched
Oct 30, 2026
Merged

Ruby184 merged 1 commit into
mainfrom
feat/merge-unmatched

Conversation

@Ruby184

@Ruby184 Ruby184 commented Oct 30, 2026

Copy link
Copy Markdown
Owner

What

Four things a run said only as counts are now listed in full and read back:

on lists and reads
--explain unmatched epg merge, epg build every programme the merge nearly joined to another source's and did not the startToleranceMs, windowMs or normalizeTitle that would have joined each group
--explain empty epg grab, epg build channel-days a source answered with nothing a broken parse, a short horizon, a channel another source has, off air, a gap
--explain requests epg grab, epg build requests by site and label: count, time out, time queued, failures where each site's time went, what would change it, what its failures mean
--explain held epg grab, epg build sites the source told to slow down how each hold ended, and the pace that earned it beside the site's rateLimit

--explain can be repeated and writes to stderr once the run is done. It's refused with --dry-run and --format json, and a cancelled run explains nothing.

New on events:

  • merge:unmatched (debug), with a signed shiftMs and a why.
  • site:done.msBy: the site's wall clock split into source, held, rateLimit, working, queued and other. It's measured as the site runs, including in threads through shared memory.
  • RequestCounts.waitedMs.
  • pacing:held.retryAfterMs, and lengthened on a hold that was extended.

A reporter may now say which event kinds it wants. The run asks once, the first time it needs to. It then makes its frequent events only where one is wanted: entry:*, request:started/done, site:note, pacing:rateLimit and merge:unmatched. A site in a thread neither makes nor sends them.

  • The shipped reporters want what they show or count.
  • emitterReporter wants what its emitter has listeners for, or what { wants } says.
  • combineReporters wants what any of its reporters wants.
  • A config's events may now be a reporter, for an emitter that hears more than its listeners do.

Also:

  • merge:summary.combined no longer counts a source joining its own programme held over from the day before.
  • nearMiss counts only another source's programmes left unjoined.
  • A reporter given an event of a kind this version doesn't know filters it by the event's own level instead of throwing.

Breaking changes

  • ResolvedMatch.matches(a, b) is replaced by apart(a, b), which returns why two programmes are not one broadcast, or undefined when they are.
  • site:done.msBy and RequestCounts.waitedMs are required.
  • Emit carries a required hears set. Make one with emitter({ reporter }); sitePacing throws at the call on a plain function.
  • A run no longer makes its frequent events for kinds an emitterReporter's emitter has no listener for. An emitter whose own emit passes every event on needs { wants: () => true }.

Measured

Plain Node on the built dist, 1000 channels × 7 days, one request per channel-day, text reporter at info:

  • Skipping unwanted events: 675 → 625ms in-process (−7%), and 1990 → 1910ms in one thread (−4%).
  • The site clock: about 4µs per channel-day at concurrency 1 in the worst case. All four explanations together: about 2µs per channel-day.
  • A simpler near-miss pass (one scan after placement) gave the same answers but ran about 60% slower where retitled programmes sit inside the match window, so the current design stays.

Checks

Format, lint, typecheck and build are clean. Tests: 2329 passed, 2 skipped, on Node 22.18, 24.15 and 26.12.

Four things a run said only as counts are now said in full, and read:
`--explain unmatched` on `epg merge` and `epg build`, and `--explain empty`,
`requests` and `held` on `epg grab` and `epg build`. Repeatable, and written on
stderr once the run is done. Each lists what the run said and what it most
likely means, said as likely, never as found: it has the events and the
config, not the source. Refused with `--dry-run` and `--format json`; a
cancelled run explains nothing.

- unmatched: every programme the merge nearly joined to another source's and
  did not, grouped by channel, source pair and what kept them apart, with the
  `startToleranceMs`, `windowMs` or `normalizeTitle` that would have joined
  them. Each is a new `merge:unmatched` event at `debug`, with a signed
  `shiftMs` and a `why`: `start`, `title`, `both`, `runs`, or `matcher` for a
  config's own `match`.
- empty: the channel-days a source answered with nothing, read against the
  rest of the run: a parse that no longer reads its source, a source that has
  not published that far, a channel another source has, one every source
  asked is empty on, a gap. A day served from the cache, unchanged or failed
  is never taken as evidence.
- requests: every site's requests by label, with how long they were out, how
  long they sat in the site's queue (`RequestCounts.waitedMs`) and what failed
  them; then where each site's time went, what would change it, and what its
  failures mean, as its `backoff` has them.
- held: each site the source told to slow down, with what, how each hold
  ended, and the pace that earned it beside its `rateLimit`.

`site:done.msBy` says where a site's wall clock went: `source`, `held`,
`rateLimit`, `working`, `queued` and `other`, which add up to `ms`. A clock
measures it as the site runs, reading the time only when what holds changes.
A site in a thread writes it to the memory it shares with the run, and a cache
call's wait for a slot on the run's thread is told back to it. A streaming
site's pass reading its document is `source`. `pacing:held` says the
`retryAfterMs` the source asked for, and `lengthened` where it made a running
hold longer.

A reporter may say which kinds it wants (`wants`). A run asks once, the first
time it needs to, and makes its frequent events (`entry:*`,
`request:started`/`done`, `site:note`, `pacing:rateLimit`, `merge:unmatched`)
only where one is wanted; a site in a thread neither makes nor sends them. At
`info` that is 7% of a grab's time, and 4% in a thread. The shipped reporters
want what they show or count, `emitterReporter` what its emitter has
listeners for or what `{ wants }` says, and `combineReporters` what any of
them does. A config's `events` may now be a reporter, for an emitter that
hears more than its listeners do.

Also: merge:summary's `combined` no longer counts a source joining its own
programme held over from the day before, and `nearMiss` counts only another
source's left unjoined; a reporter given an event of a kind this version does
not know filters it by its own level.

BREAKING CHANGE: `ResolvedMatch.matches(a, b)` is replaced by `apart(a, b)`,
which answers why two programmes are not one broadcast, or `undefined` where
they are. `site:done.msBy` and `RequestCounts.waitedMs` are required. `Emit`
carries a required `hears` set: make one with `emitter({ reporter })`, as
`sitePacing` now insists. A run no longer makes its frequent events for kinds
an `emitterReporter`'s emitter has no listener for, so an emitter whose own
`emit` passes every event on needs `{ wants: () => true }`.
@Ruby184
Ruby184 merged commit fa54725 into main Oct 30, 2026
4 checks passed
@Ruby184
Ruby184 deleted the feat/merge-unmatched branch October 30, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant