Repository navigation
feat(merge)!: texts two sources spell differently are written once - #31
Merged
Merged
Conversation
When two programmes merged, their titles, descriptions and categories were
unioned by the exact text, within each language. Two sources rarely write one
the same to the letter, so a guide kept every spelling: "Dj Mixes Premiere"
beside "DJ Mixes premiere", "film" beside "Film", a description cut off with
"…" beside the whole of it. homepi carried 120 lines of transform to undo it.
`merge.union` says per field which texts are one, within a language, the
higher-priority source's kept. Titles and sub-titles are the same by the
match's own `normalizeTitle`, so a config's title rule loosens both what is
one broadcast and what is one title. A description another contains as whole
words, but for case, spacing and a trailing "…", is dropped, and the one
containing it takes its place. Categories, keywords and countries are the
same but for case, spacing, hyphens and underscores, as a profile finds a
genre. A field takes 'exact', 'caseless', 'match', 'contained',
`{ contained: fn }` or a key function of its own, and `union: 'exact'` sets
every field exact.
It applies as each programme is written, not as two are merged, so matching
still sees every title and a later source's spelling still finds its
broadcast. `mergeProgrammes`, `mergeInto` and `backfillInto` keep every text,
as before; `mergeProgrammeLists` takes a union and applies it to what it
returns, and `resolveUnion` settles one into a `ResolvedUnion`, whose `apply`
writes one programme. A loaded config carries its union settled, with the
union it was settled from as its own fields, and a misspelt field or mode is
refused as the config loads, before anything is grabbed. `ResolvedMatch`
carries the `normalizeTitle` it compares by, and `genreKey` folds Unicode
forms together.
BREAKING CHANGE: a guide built from several sources writes one of two texts
that say the same thing rather than both, so it changes on upgrade: fewer
titles, categories and descriptions, none of them rewritten. Set
`merge.union: 'exact'` to write every text as the sources gave them, as
before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A guide merged from several sources wrote every spelling of a programme's texts: "DJ Mixes premiere" beside "Dj Mixes Premiere", "Film" beside "film", a description cut off with "…" beside the whole of it. homepi carried ~120 lines of transform to undo it, and measured thousands of such programmes a day. One commit, breaking (guides change on upgrade).
Breaking
Migration:
merge: { union: 'exact' }writes every text as the sources gave them, as before.ResolvedMatch.normalizeTitleis required.resolveMatchalways set it; only a hand-builtResolvedMatchneeds the field.merge.unionPer field, which texts are one — within a language, with the higher-priority source's kept:
title,subTitle'match'normalizeTitlemakes them the same (case, accents and spacing by default), so a title rule that loosens what is one broadcast loosens what is one title toodesc'contained'…; the fuller one takes the shorter one's place.KrimibesideKrimiseriál o …stayscategory,keyword,country'caseless'Sci-FiandSci Fiare one)A field also takes
'exact',{ contained: fn }(the source's own suffix taken off first, then compared as'contained'does), or a key function of its own.union: 'exact'sets every field exact. Languages are never compared with each other; a missing tag and an empty one are the same, and tags are compared regardless of case.Applied as each programme is written
Not as two programmes merge. Matching decides what is one broadcast by its titles, so it has to see every one: a spelling dropped when the first two sources merged would leave a third source's shifted programme with nothing to match, and it would come out as a duplicate. So:
mergeProgrammes,mergeIntoandbackfillIntokeep every text, as before.transformthe next programme with the union applied too.mergeProgrammeLists(lists, strategy, match?, fillStopMs?, union?)applies it to what it returns.resolveUnion(union, resolveMatch(match).normalizeTitle).apply(programme)applies it to one programme.Settled at load
unionthat isn't a plain object, is refused as the config loads, before anything is grabbed.ResolvedUnionwhose own fields are the union it was settled from:merge.union's type stays what a config writes.normalizeTitlehas changed.Smaller changes
genreKeyfolds Unicode forms together, so a category written decomposed finds its genre in a profile too.src/merge/text.ts, which removes an import cycle.resolveMatchacceptsnull, andmerge: nullin a config loads as it did before.Testing
next, a loaded config's settled union, copies and another install's union, a stale match, serve's ETag, and every refusal.test/readme-examples.tsand typechecks.