LLM Providers

Moltis supports multiple LLM providers through a trait-based architecture. Configure providers through the web UI or directly in configuration files.

Available Providers

API Key Providers

Provider	Config Name	Env Variable	Features
Anthropic	`anthropic`	`ANTHROPIC_API_KEY`	Streaming, tools, vision
OpenAI	`openai`	`OPENAI_API_KEY`	Streaming, tools, vision, model discovery
Google Gemini	`gemini`	`GEMINI_API_KEY`	Streaming, tools, vision, model discovery
DeepSeek	`deepseek`	`DEEPSEEK_API_KEY`	Streaming, tools, model discovery
Mistral	`mistral`	`MISTRAL_API_KEY`	Streaming, tools, model discovery
Groq	`groq`	`GROQ_API_KEY`	Streaming
xAI (Grok)	`xai`	`XAI_API_KEY`	Streaming
OpenRouter	`openrouter`	`OPENROUTER_API_KEY`	Streaming, tools, model discovery
Cerebras	`cerebras`	`CEREBRAS_API_KEY`	Streaming, tools, model discovery
MiniMax	`minimax`	`MINIMAX_API_KEY`	Streaming, tools
Moonshot (Kimi)	`moonshot`	`MOONSHOT_API_KEY`	Streaming, tools, model discovery
Venice	`venice`	`VENICE_API_KEY`	Streaming, tools, model discovery
Z.AI (Zhipu)	`zai`	`Z_API_KEY`	Streaming, tools, model discovery

OAuth Providers

Provider	Config Name	Notes
OpenAI Codex	`openai-codex`	OAuth flow via web UI
GitHub Copilot	`github-copilot`	Requires active Copilot subscription

Local

Provider	Config Name	Notes
Ollama	`ollama`	Local or remote Ollama instance
LM Studio	`lmstudio`	Local LM Studio or any OpenAI-compatible server
Local LLM	`local-llm`	Runs GGUF models directly on your machine

Custom OpenAI-Compatible

Any OpenAI-compatible endpoint can be added with a custom- prefix:

[providers.custom-myservice]
enabled = true
api_key = "..."
base_url = "https://my-service.example.com/v1"
models = ["my-model"]

Configuration

Via Web UI (Recommended)

Open Moltis in your browser.
Go to Settings → Providers.
Choose a provider card.
Complete OAuth or enter your API key.
Select your preferred model.

Via Configuration Files

Configure providers in moltis.toml:

[providers]
offered = ["anthropic", "openai", "gemini"]

[providers.anthropic]
enabled = true

[providers.openai]
enabled = true
models = ["gpt-5.3", "gpt-5.2"]
stream_transport = "sse"              # "sse", "websocket", or "auto"

[providers.gemini]
enabled = true
models = ["gemini-2.5-flash-preview-05-20", "gemini-2.0-flash"]
# api_key = "..."                     # Or set GEMINI_API_KEY / GOOGLE_API_KEY env var
# fetch_models = true                 # Discover models from the API
# base_url = "https://generativelanguage.googleapis.com/v1beta/openai"

[chat]
priority_models = ["gpt-5.2"]

Provider Entry Options

Each provider supports these options:

Option	Default	Description
`enabled`	`true`	Enable or disable the provider
`api_key`	—	API key (overrides env var)
`base_url`	—	Override API endpoint URL
`models`	`[]`	Preferred models shown first in the picker
`fetch_models`	`true`	Discover available models from the API
`stream_transport`	`"sse"`	`"sse"`, `"websocket"`, or `"auto"`
`alias`	—	Custom label for metrics
`tool_mode`	`"auto"`	`"auto"`, `"native"`, `"text"`, or `"off"`

Provider Setup

Google Gemini

Google Gemini uses an API key from Google AI Studio.

Get an API key from Google AI Studio.
Set GEMINI_API_KEY in your environment (or use GOOGLE_API_KEY).
Gemini models appear automatically in the model picker.

[providers.gemini]
enabled = true
models = ["gemini-2.5-flash-preview-05-20", "gemini-2.0-flash"]

Gemini supports native tool calling, vision/multimodal inputs, streaming, and automatic model discovery.

Anthropic

Get an API key from console.anthropic.com.
Set ANTHROPIC_API_KEY in your environment.

OpenAI

Get an API key from platform.openai.com.
Set OPENAI_API_KEY in your environment.

OpenAI Codex

OpenAI Codex uses OAuth-based access.

Go to Settings → Providers → OpenAI Codex.
Click Connect and complete the auth flow.
Choose a Codex model.

If the browser cannot reach localhost:1455, Moltis now supports a manual fallback in both Settings and Onboarding: paste the callback URL (or code#state) into the OAuth panel and submit it.

Docker and cloud deployments

The OAuth flow redirects your browser to localhost:1455. In Docker, make sure port 1455 is published (-p 1455:1455). On cloud platforms where localhost cannot reach the server, authenticate via the CLI instead:

# Docker
docker exec -it moltis moltis auth login --provider openai-codex

# Fly.io
fly ssh console -C "moltis auth login --provider openai-codex"

The CLI opens a browser on your machine and handles the callback locally. If automatic callback capture fails, the CLI prompts you to paste the callback URL (or code#state) directly in the terminal. Tokens are saved to the config volume and picked up by the gateway automatically.

GitHub Copilot

GitHub Copilot uses OAuth authentication.

Go to Settings → Providers → GitHub Copilot.
Click Connect.
Complete the GitHub OAuth flow.

Docker and cloud deployments

GitHub Copilot uses device-flow authentication (a code you enter on github.com), so it works from the web UI without extra port configuration. If you prefer the CLI:

docker exec -it moltis moltis auth login --provider github-copilot

Info

Requires an active GitHub Copilot subscription.

Ollama

Ollama auto-detects when running at http://127.0.0.1:11434. No API key needed.

[providers.ollama]
enabled = true
# base_url = "http://127.0.0.1:11434/v1"  # Override for remote Ollama

LM Studio

LM Studio auto-detects when running at http://127.0.0.1:1234. No API key needed. Also works with llama.cpp or any OpenAI-compatible local server.

[providers.lmstudio]
enabled = true
# base_url = "http://127.0.0.1:1234/v1"  # Override for different port/host

Local LLM

Local LLM runs GGUF models directly on your machine.

Go to Settings → Providers → Local LLM.
Choose a model from the local registry or download one.
Save and select it as your active model.

Switching Models

Per session: Use the model selector in the chat UI.
Per message: Use /model <name> in chat.
Global defaults: Use [providers].offered, provider models = [...], and [chat].priority_models in moltis.toml.

Troubleshooting

“Model not available”

Check provider auth is still valid.
Check model ID spelling.
Check account access for that model.

“Rate limited”

Retry after a short delay.
Switch provider/model.
Upgrade provider quota if needed.

“Invalid API key”

Verify the key has no extra spaces.
Verify it is active and has required permissions.

Keyboard shortcuts

Moltis Documentation