Perplexity’s Orchestrator retries failed requests across providers
Perplexity's Orchestrator retries Claude and GPT requests across deployments, reports stronger success rates, and offers preview routing that can switch models between turns.

Why it matters
- Perplexity launched Orchestrator on November 17, 2026.
- Tests from October 4 to November 2, 2026 showed Orchestrator requests succeeded more often than any individual provider measured.
- Anthropic and AWS Bedrock each led Claude Opus 5.5 output speed on 15 days.
- Azure's median GPT-5.6 Sol output speed was 27% higher than OpenAI's.
- Tier 0 GLM-5.3 Flash costs $0.15 per million input tokens and $0.50 per million output tokens.
By Elena Vasquez
Perplexity launched Orchestrator on November 17, 2026, after reporting that routed Claude and GPT requests succeeded more often than requests to any individual provider it measured. The API serves proprietary and open-weight models on Perplexity's inference infrastructure, combining consolidated billing, same-model provider failover and optional automatic model routing in preview.
For developers, Orchestrator separates two decisions that often get conflated. Failover can move a request across providers without changing its model. Preview routing can choose another model for a later turn. The stakes are uptime, predictable behavior and the bill.
What did Perplexity launch?
Orchestrator gives developers a single API for model families from Anthropic, OpenAI, Google, xAI, Z.AI, DeepSeek and Moonshot AI, according to Perplexity's launch documentation. Google, xAI and open-weight models are available at every usage tier.
Access is tiered for the proprietary catalogs. Perplexity says calling OpenAI and Anthropic models requires Tier 1 or higher. Claude Opus 5, Opus 5.5 and Fable require Tier 2 or higher.
Perplexity's documentation says Orchestrator accepts OpenAI Chat Completions, OpenAI Responses and Anthropic Messages requests. Developers can keep their OpenAI or Anthropic SDK. They then switch to a Perplexity API key, the Orchestrator base URL and a full model ID from the catalog.
The quickstart describes the service as “stateless” and intended for “direct model access” with a customer's own prompts and tools. That separates Orchestrator from the Agent API, which returns web-grounded answers with built-in citations.
For developers, that arrangement targets uptime while preserving model behavior. The preview routing layer can still alter a conversation's model, capability and cost.
How does same-model failover work?
Orchestrator's routing documentation separates model selection from deployment selection. After a developer chooses a model, provider routing distributes traffic among that model's deployments.
Perplexity says the router uses observed error rates, capacity and latency. Customers do not choose the deployment that serves a request. The service decides where a selected model runs.
Provider errors, rate limits and timeouts can trigger a retry on another deployment of the same model. A Claude request can move from Anthropic to AWS Bedrock after a failure while remaining a Claude request.
That distinction preserves the application's model contract. It also lets Perplexity frame failover as “resilience without changing the selected model or its token rate.”
This narrower promise differs from automatic model routing. That preview feature can replace the selected model between turns, rather than merely move it to another provider deployment.
What did the reliability tests show?
Perplexity measured Claude and GPT requests sent through Orchestrator from October 4 through November 2, 2026. The company says those requests succeeded more often than requests to any individual provider it measured.
The same test produced smaller daily swings during provider incidents. Perplexity defines a failure as either exhausting every available deployment or receiving an error from the serving layer.
Client errors, including invalid requests and client rate-limit violations, were excluded from that success-rate comparison. The result therefore measures infrastructure and provider failures rather than malformed client traffic.
Speed results were more mixed across the same 30 days. Anthropic and AWS Bedrock each led Claude Opus 5.5 output speed on 15 days. Azure's median GPT-5.6 Sol output speed was 27% higher than OpenAI's.
Current provider routing already considers latency, error rates and capacity. Perplexity says adaptive routing that responds to those output-speed differences remains in development.
When does Orchestrator stop retrying?
Failing over does not mean Orchestrator retries every request. Perplexity documents HTTP 400 responses for invalid requests and prompts that exceed a model's context window, without another deployment attempt.
Exhausting every available deployment produces HTTP 429 and includes a Retry-After header. The response gives clients information about when to retry.
Perplexity also draws the billing boundary around delivered output. Requests that fail before producing output are not billed. A stream that fails after output begins still charges for the tokens already delivered.
These rules leave some failures for the developer to handle. HTTP 400 requires a corrected request. HTTP 429 requires pacing. A broken stream can leave a partial result and a charge for the portion that arrived.
How does automatic model routing work?
Automatic model routing changes the model itself, rather than only the deployment. Perplexity presents it as a separate decision from provider failover.
Setting the model to perplexity/auto calls a decision model. That model sees the conversation text and plain-language route descriptions. It does not receive route names or model IDs.
Developers can provide a custom routing object. It can define a default model and one to five task-specific routes. The route descriptions tell the decision model what kinds of work they handle.
The default model handles three cases: no route fits, the decision model fails or times out, or the conversation contains images, audio, video or files.
Routing runs independently for every request. As a task changes, separate turns in one conversation can use different models. That makes automatic routing more flexible than failover, but less consistent in capability and price.
Every request must carry the full conversation, and each model maintains a separate prompt cache. Stable session identifiers or prompt-cache keys can help keep related requests on one deployment.
Perplexity calls that deployment affinity best effort. It does not pin model selection across turns, even when session identifiers remain stable. A route can therefore change models without changing the session identifier.
Response headers describe the routing outcome and any selected route. The response's model field identifies the model that actually answered. Those signals let clients see when a turn crossed from the opening model to another one.
What changes for developers?
With failover, infrastructure selection stays hidden behind the existing model request. Developers can keep their prompts, tools and SDK while Perplexity shifts deployment after an error.
Model routing makes a larger choice visible in each response. A conversation may move among model families as its task changes. Perplexity's headers and model field report the outcome, but they do not make preview behavior constant across turns.
This preview changes the practical bargain. Customers gain task-based model selection and billing based on the model that answered, but they give up a fixed model for the life of a conversation. Perplexity itself qualifies the feature by noting that capability and cost can change between turns.
What will developers pay?
Perplexity's billing rules follow the model that answered, using that model's published per-token rate. The deployment that served the request does not change the rate.
Usage draws from the same prepaid credits as the rest of a Perplexity API project. In the catalog, Tier 0 GLM-5.3 Flash costs $0.15 per million input tokens and $0.50 per million output tokens.
Cache reads cost $0.03 per million tokens. Reasoning tokens carry the applicable output rate. Failover therefore leaves the selected model's price intact.
Preview routing can still change the bill by selecting another model for a turn. Customers need the response's model field to connect the answer with its charge.
Orchestrator's immediate promise is narrow and testable: keep one model, move failed traffic to another deployment, and preserve the model's rate. Its broader promise depends on preview routing and unfinished adaptive, speed-aware provider selection. Until that feature ships, resilience rests on same-model retries, while model, capability and cost can still change between turns.
Source: unite.ai
More from Elena Vasquez
Show full bio
Market editor covering technology and product news in artificial intelligence and machine learning at AI In Context.
708 articles
Related articles
- Perplexity Launches Multiagent Mode to Coordinate Frontier AI Models
- Baseten Lands Four Open Models in Microsoft Foundry as First-Party Preview
- Perplexity brings its full Computer agent to local Apple Silicon Macs
- OpenAI Brings GPT-5.5, Codex, and Managed Agents to AWS Bedrock
- OpenAI launches ChatGPT agent, folding Operator into ChatGPT