Add files needed for maplibre unicode code geneation script.

[HarelM]

Similar to #101 these files/information is also required:

extracted/DerivedJoiningType.txt → Joining_Type.
UnicodeData.txt → Decomposition_Mapping.

I must admit I'm out of my elements here, so this is mainly my attempt at reasoning what Claude it telling me...

[HarelM]

Claude also told me that the following might be something worth changing:

node-unicode-data/scripts/utils.mjs

Line 107 in 144669f

const isCaseFoldingOrMapping = type === 'Case_Folding' || type === 'Simple_Case_Mapping' || type === 'Special_Casing';

To:

const isCaseFoldingOrMapping = type === 'Case_Folding' || type === 'Simple_Case_Mapping' || type === 'Special_Casing' || type === 'Decomposition_Mapping';

Let me know if this makes sense, again, sorry for pasting AI comments...

[1ec5]

For context, MapLibre GL JS has a custom text layout implementation because the built-in browser APIs don’t provide low-level access to the text layout engine. The implementation currently relies on downloading these tables directly from the Unicode website. We don’t actually decompose any Arabic characters, but the decomposition mappings indicate if a given presentation form is initial/medial/final/isolate. #102 is helpful, but it turns out that the ArabicShaping table is abridged and there’s a different unabridged table:

The file ArabicShaping.txt is also exceptional, because it omits the listing of many characters whose property value (jt=T) can be derived by rule. Adding an "@missing" line to that file would result in the wrong interpretation of Joining_Type values for omitted characters. The full explicit listing of Joining_Type values and the correct "@missing" line for the default Joining_Type value (jt=U) can be found in the file DerivedJoiningType.txt instead. The values of Joining_Type listed in DerivedJoiningType.txt should be taken as definitive, because of the difficulty of deriving the correct values for all characters based only on the entries in ArabicShaping.txt.

[FekyBaz]

I can take the Decomposition_Mapping part (parse field 5 of UnicodeData.txt, following the parse-bidi-class First/Last + value-alias pattern). One question before I start on DerivedJoiningType: it feeds the same Joining_Type output as the just-merged ArabicShaping parser — would you prefer it to replace that source, or coexist (e.g. as the explicit T-set complement)? Happy to implement either way once you confirm the direction.

[1ec5]

Hm, not sure. It kind of looks like MapLibre is downloading and parsing both, but I’m not sure why. The derived table is supposed to be a more comprehensive superset of the base table.

[FekyBaz]

Verified @1ec5's intuition against the 17.0.0 files: DerivedJoiningType (3,004 entries) covers every ArabicShaping Joining_Type value with zero mismatches, plus 2,224 explicit T entries that ArabicShaping leaves implicit. One nuance: 50 code points exist only in ArabicShaping — all Joining_Type=U entries (e.g. U+0600..U+060B), which the derived file omits as implicit default. So for pure Joining_Type consumers the derived file alone suffices; ArabicShaping remains the only source for Joining_Group. Up to @mathiasbynens whether the new parser should complement or migrate — either way my Decomposition_Mapping offer stands as independent work.

[HarelM]

@mathiasbynens any chance to add those files? I see you just released a new version, would be nice to have those files in the next one.

[mathiasbynens]

Hi @HarelM, I’m supportive of including this data in the packages. I haven’t gotten around to implementing this yet, but am happy to review patches.

[HarelM]

Not sure the below is helpful:

• Add missing joining types and decomposition file and parsers #107

But I hope it does.