feat(llm): add Featherless AI provider#65
Closed
aviflombaum wants to merge 2 commits intospacedriveapp:mainfrom
Closed
feat(llm): add Featherless AI provider#65aviflombaum wants to merge 2 commits intospacedriveapp:mainfrom
aviflombaum wants to merge 2 commits intospacedriveapp:mainfrom
Conversation
9e3f00d to
cad3859
Compare
Add Featherless AI as a native LLM provider, giving access to 17,000+ open-source models via their OpenAI-compatible API. Registers featherless_key as a named key that auto-maps to a ProviderConfig with ApiType::OpenAiCompletions, following the same pattern as anthropic, openai, and openrouter. The generic provider dispatch handles routing without any model.rs changes. Changes: - config: featherless_key field, FEATHERLESS_API_KEY env var, FEATHERLESS_PROVIDER_BASE_URL constant, auto-mapping to ProviderConfig in both load_from_env and from_toml, onboarding menu entry, needs_onboarding env check - llm/routing: default model (Qwen/Qwen3-32B), provider_to_prefix - api/providers: get/update/delete endpoints, routing auto-switch - api/models: configured_providers discovery
cad3859 to
6cf69ba
Compare
The call_openai() function appends /v1/chat/completions to the base URL, so including /v1 in the constant caused double /v1 in the final URL (https://api.featherless.ai/v1/v1/chat/completions), resulting in 404.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Featherless AI as a native LLM provider. Featherless hosts 17,000+ open-source models behind an OpenAI-compatible API, giving spacebot users access to models like Qwen3-32B, Llama, Mistral finetunes, and others not available through existing providers.
Registers
featherless_keyas a named key that auto-maps to aProviderConfigwithApiType::OpenAiCompletions, following the same pattern as anthropic, openai, and openrouter. The generic provider dispatch inattempt_completionhandles routing without anymodel.rschanges.Changes
4 files changed, 55 insertions, 4 deletions
src/config.rsfeatherless_keyfield onLlmConfig,TomlLlmConfigFields,TomlLlmConfig;FEATHERLESS_API_KEYenv var inload_from_envandfrom_toml;FEATHERLESS_PROVIDER_BASE_URLconstant; auto-mapping toProviderConfigin both config paths;has_any_key()check;needs_onboarding()env check; onboarding menu entrysrc/llm/routing.rsdefaults_for_providerentry (default:featherless/Qwen/Qwen3-32B);provider_to_prefixentrysrc/api/providers.rsfeatherlessfield onProviderStatus; all three endpoint handlers (get_providers,update_provider,delete_provider); routing auto-switch inhas_key_for_currentsrc/api/models.rsconfigured_providersdiscovery lineUsage
Config (config.toml):
Environment variable:
Model routing format:
featherless/<org>/<model-name>— thefeatherless/prefix is stripped before sending to the API, so model names match what Featherless lists (e.g.Qwen/Qwen3-32B,moonshotai/Kimi-K2.5).Design notes
llm/model.rsorllm/manager.rs— the genericProviderConfigdispatch added in feat(llm): add custom providers and dynamic API routing #36 handles Featherless automatically once the key is registered withApiType::OpenAiCompletionsor_insert_withso a custom[llm.provider.featherless]block in config.toml takes precedence over the named key auto-mappingQwen/Qwen3-32B(32B parameter open-source model with tool calling support)Why Featherless
Featherless specializes in hosting open-source models that aren't available (or are expensive) on other providers. Users get access to thousands of community and frontier open-source models with a single API key, complementing the closed-model providers already supported.
Testing
main(no merge conflicts)