LLM Gateway
Connect QuickStack Sandbox Instances to a LiteLLM-compatible gateway for centralized model access and API keys.
An LLM Gateway is a LiteLLM-compatible endpoint that QuickStack Sandbox Instances use to reach language models. Centralizing access in a gateway means you manage model credentials in one place instead of distributing provider API keys to every Sandbox Instance.
Experimental / canary LLM gateways are part of the experimental Agent Sandbox feature and are only visible when the canary channel is enabled.
Add an LLM gateway
-
Go to Settings → Integrations → LLM Gateways.
-
Click Add LLM Gateway.
-
Fill in:
Field Description Name A label for the gateway LiteLLM Base URL The gateway endpoint, e.g. https://litellm.example.comLiteLLM Admin Key The admin key used to manage models on the gateway -
Click Save.
-
Click Test Connection to verify QuickStack can reach the gateway.
Deploy a LiteLLM instance
If you do not have a gateway yet, QuickStack can deploy one for you:
- In the LLM Gateway dialog, choose Deploy new LiteLLM instance.
- Select an Existing Project or create a New Project (the default project name is
LiteLLM). - QuickStack deploys LiteLLM together with its PostgreSQL and Redis dependencies and wires up the connection.
You can also deploy LiteLLM from the template catalog.
Use a gateway with an Agent Sandbox
When creating or editing an Agent Sandbox, choose the LLM Gateway and a Model Alias in its model configuration. The alias identifies which model the gateway should route to.
Model aliases Templates receive the gateway URL and alias as environment variables, so the harness in each Sandbox Instance talks only to the gateway. Available model aliases come from the gateway's LiteLLM configuration.