[RNE Rewrite] feat: add tokenizer pipeline (#1248)

[msluszniak]

Description

Adds the tokenizer pipeline (issue #1248) using the new worklet-based architecture, with functional parity to the current TokenizerModule.

A new text extension exposes a loadTokenizer JSI primitive (top-level on __rnexecutorch_jsi__, like loadModel) returning a Tokenizer host object backed by tokenizers::HFTokenizer. On top of it sits a createTokenizer(config, runtime?) async factory (async + *Worklet variants + dispose) and a useTokenizer hook. Methods: encode, decode, getVocabSize, idToToken, tokenToId — same semantics as today (special tokens follow the tokenizer.json post_processor). The *Worklet variants let an upcoming text-embeddings task tokenize → build tensors → run forward within a single worklet.

• C++: cpp/extensions/text/{tokenizer,install}.{h,cpp}, wired into RnExecutorch.cpp.
• TS: src/extensions/text/{ops,tasks}/tokenizer.ts, src/hooks/useTokenizer.ts, exports in index.ts, example models.tokenizer.ALL_MINILM_L6_V2.
• Build: tokenizer include path added to android/CMakeLists.txt and the podspec (headers ship in the ExecuTorch llm bundle); documented in third-party/README.md.

Introduces a breaking change?

• Yes
• No

Type of change

• Bug fix (change which fixes an issue)
• New feature (change which adds functionality)
• Documentation update (improves or adds clarity to existing documentation)
• Other (chores, tests, code style improvements etc.)

Tested on

• iOS
• Android

Testing instructions

CI is TypeScript-only on this branch; native is not compiled in CI. Verified locally: yarn typecheck, root yarn lint, and yarn prepare (bob build) all pass. On-device native build requires provisioning the ExecuTorch third-party artifacts (see third-party/README.md).

Screenshots

Related issues

#1248, part of #1208

Checklist

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have updated the documentation accordingly
• My changes generate no new warnings

Additional notes

The C++ mirrors the proven current TokenizerModule calls against the same software-mansion-labs/tokenizers-cpp fork. Tokenizer download currently uses the temporary react-native-fs-based useResourceDownload introduced in #1264 (to be replaced by the ResourceFetcher in #1253).