Agent Serve · Models
Pick a model per block, or let Agent Serve route each task by difficulty. Bring your own keys, or keep inference in your cloud.
Agent Serve, the AI workspace where teams build, deploy, and manage AI agents, supports models from providers including OpenAI, Anthropic, Google, Mistral, xAI, DeepSeek, Groq, Cerebras, AWS Bedrock, Azure and Vertex AI, self-hosted models through Ollama and vLLM, and speech-to-text and text-to-speech blocks.
How it works
Agent Serve sets the model where the work happens, in each Agent block, and records what every call used.
Choose a provider and model in each Agent block, or pick Auto and let Agent Serve choose by task difficulty.
Add your own provider keys in Settings → Model API keys, for one team space or the whole organization.
Access policies decide which providers each group may use and which models are denied.
Run history shows the model, input and output tokens, and cost for every step of every run.
Providers
Agent Serve reads these providers from its model definitions: first-party labs, cloud platforms and self-hosted runtimes.
Directory
Browse 155 AI models across 25 providers. Compare pricing, context windows, and capabilities.
Side-by-side comparison of top models across key metrics.
These providers load their model lists dynamically at runtime.
In depth
How Agent Serve decides which model runs, whose key pays for it, and where inference happens.
Choose Auto in the model picker on Agent Serve Cloud. A classifier rates the task’s difficulty, then picks a model that fits it and anything attached.
Agent Serve connects to self-hosted Ollama, vLLM and LiteLLM endpoints, and to models in Azure, AWS Bedrock and Vertex AI.
Agent Serve run history has a Cost column, and each run’s trace lists the model, input and output tokens for every step.
By the numbers
155
tracked models
25
model providers
3
self-hosted runtimes: Ollama, vLLM, LiteLLM
10
keys per provider, rotated in turn
Counts are read from the Agent Serve model definitions when the site is built.
Voice & speech
Workflows can transcribe audio with speech-to-text blocks and turn text into natural speech with text-to-speech blocks. These are the providers and models available today.
Speech blocks process audio inside workflows. Agent Serve does not offer live voice conversations with agents or Relays today; in Chat you can dictate a message.
Governance
Agent Serve puts model choice under the same access policies, keys and logs as the rest of the workspace.
Security and governanceAgent Serve access policies set the allowed model providers and the denied models for each group of people.
Agent Serve stores your provider keys encrypted. Organization keys apply to every team space unless a team space adds its own.
Admin, write and read roles per workspace, plus permission groups that limit which models, blocks and integrations a team can use.
Use your organisation’s own model provider keys at organisation or workspace level.
Zero data retention with supported model providers, backed by your own provider keys and agreements or self-hosted models. Some provider features keep data by design, such as Gemini Deep Research, so they are outside zero data retention.
Run Agent Serve in your own cloud or data centre, next to the systems it reads.
Every workflow run records each block’s inputs, outputs and errors, so a reviewer can trace how a result was reached.
Before and after
| Work | Before | With Agent Serve |
|---|---|---|
| Choosing a model | BeforeOne model hard-coded for every task, strong or simple. | With Agent ServeA model per Agent block, or Auto to match each task by difficulty. |
| Provider keys | BeforeKeys pasted into scripts and shared in chat threads. | With Agent ServeKeys stored encrypted in Agent Serve, at team space or organization level. |
| Sensitive prompts | BeforeEvery prompt leaves for a public endpoint. | With Agent ServeSelf-hosted models or your own cloud account keep inference inside your network. |
| Cost | BeforeA monthly invoice nobody can trace back to a task. | With Agent ServeTokens and cost on every run, step by step. |
Works with
Questions
The most important factors for agent tasks are reliable tool use (function calling), a large enough context window to track conversation history and tool outputs, and consistent instruction following. In Agent Serve, OpenAI GPT-4.1, Anthropic Claude Sonnet, and Google Gemini 2.5 Pro are popular choices, and each supports tool use, structured outputs, and context windows of 128K tokens or more. For cost-sensitive or high-throughput agents, Groq and Cerebras offer significantly faster inference at lower cost.
The context window is the total number of tokens a model can process in a single call, including your system prompt, conversation history, tool call results, and any documents you pass in. For agents running multi-step tasks, context fills up quickly, and each tool result and each retrieved document adds tokens. A 128K-token context window fits roughly 300 pages of text; models like Gemini 2.5 Pro support up to 1M tokens, enough to hold an entire codebase in a single pass.
Yes. Input, cached input, and output prices are all listed per one million tokens, matching how providers bill through their APIs. For agents that chain multiple calls, costs compound quickly: an agent completing 100 turns at 10K tokens each consumes roughly 1M tokens per session. Cached input pricing applies when a provider supports prompt caching, where a repeated prefix like a system prompt is billed at a reduced rate.
Tool use, also called function calling, lets an agent invoke external APIs, query databases, run code, or take any action you define. In Agent Serve, all first-party models from OpenAI, Anthropic, Google, Mistral, Groq, Cerebras, and xAI support tool use. Look for the Tool Use capability tag on any model card in this directory to confirm support.
Open Agent Serve, add an Agent block, and select your provider and model from the model picker inside that block. Every model listed in this directory is available in the Agent block. Swapping models takes one click and does not affect the rest of your agent, making it straightforward to test different models on the same task without rebuilding anything.
Enterprise access
Agent Serve is available to teams by invitation. Tell us what you want to build and we will set up your workspace with you.