Skip to content

Conversation

@zharinov
Copy link

Adds from_raw_parts() for constructing models from pre-parsed components, from_pretrained() now delegates to it.

Also fixes a bug where loading would fail if the tokenizer doesn't define an unk_token (not all tokenizers have one).

- `from_pretrained` now delegates to `from_raw_parts`
- Fixes BPE tokenizer support (unk_token_id now optional)
Copy link
Member

@Pringled Pringled left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this PR @zharinov! This is a nice functionality to have I think, and good catch about the unk_token. I have two small (but nice to have) improvements; if you could implement those this is good to go. Thanks for updating the tests as well 👍

@Pringled
Copy link
Member

@zharinov one additional comment, could you also run clippy to fix the formatting issues?

@zharinov zharinov changed the title feat: Add from_raw_parts() constructor feat: Add from_borrowed() constructor Jan 26, 2026
@zharinov
Copy link
Author

Hey,

I wanted to support zero-copy initialization with include_bytes!, yet missed the array clone in my own code 🤦‍♂️

The second attempt transforms StaticModel to use Cow, allowing for both owned and static scenarios. This adds some performance penalty, but hopefully it's minimal thanks to CPU branch prediction.

Also, I applied the suggestion for unk_token.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants