Converted the Jina Tokenizer regex pattern to python.
Based on: https://x.com/_philschmid/status/1825121514816938105
SebastianBodza/jina_tokenizer_regex_python
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Converted the Jina Tokenizer regex pattern to python.
Based on: https://x.com/_philschmid/status/1825121514816938105