Feature: any chance to convert to GGUF format

[kalle07]

any chance to convert to GGUF format ?
same like
https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF

[stephantul]

Hello @kalle07,

We've looked at this very briefly. I don't know how difficult it is right now, but I'll investigate.

[kalle07]

@stephantul ... no rush ...
i only know it should be llama.cpp compatible, so maybe better to change your code a bit instead of asking llama.cpp to add yours

[thiswillbeyourgithub]

Gguf format would mean that we can import it to ollama, which woumd be terrific