Routing profiles and auto mode: fallbacks and per-request overrides
Build a routing profile that prefers one provider and falls back to another, steer one request with X-Dz-Router, and read the decision record on a trace.
17 min readLast reviewed October 19, 2026Verified against gateway chart 0.10.0 (contract 0.6.0)
What you’ll have at the end
By the end you will have a routing profile that answers three questions for every keyed request: which model, which upstream, and which provider behind that upstream. You will also know how to steer one request with a header, how to force a failover on purpose, and how to read what the gateway recorded about it.
Before you begin
- A Nautili team with at least one issued key (
sk-dz-...). Create one under API keys if you have none. - A gateway host you can reach, written below as
https://<your-gateway-host>. - Permission to edit routing profiles for the team. Profiles live on Routing.
- Optional, for the failover demo in Step 6: a provider key of your own under Provider keys, and managed custody available on the same upstream.
Step 1: Read the catalog and try auto
Open Routing and scroll to the Path catalog section. Type a model id such as anthropic/claude-haiku-4.5 into the Model field. The table lists every serving path the team could reach for that model as <upstream>:<variant tag>@<custody>, each marked eligible or excluded with its exclusion reason in words (region unknown, retains data, provider denied, provider terms not cleared).
Then send a request that names the reserved model auto.
On a team where no rung of the ladder enables auto, the gateway refuses with 400 auto_not_enabled rather than picking a platform default. Once a profile enables it, auto resolves at the upstream: the decided model is recorded as the literal auto, the served model is whatever the upstream's auto-router picked, and the row carries routing_switch_reason: auto_selection.
Step 2: Create a profile on the Profiles wizard
In the Routing profiles section of Routing, start a new profile. The wizard has four zones: Name it, attach it, Default model & fallbacks, Providers, auto, provider choice, custody order, eligibility, and What this profile will do. Every field starts as inherit.

Fill it in to prefer one provider and fall back to another:
| Field on screen | Value |
|---|---|
| Profile name | prefer-anthropic |
| Applies to | Team default |
| Default model | anthropic/claude-haiku-4.5 |
| Fallback models | openai/gpt-5.6-luna |
| Rank providers by | Fastest to respond (latency) |
| Preferred providers | anthropic |
| Denied providers | deepinfra |
The same fields, as the API spells them, with one line each:
| Field | Meaning |
|---|---|
default_model | Model used when a request names none. Absent means this rung sets no default. |
fallback_models | Ordered chain, at most 4, tried only when the model above cannot be served. |
provider_sort | cost, latency or throughput. Ranks the providers that serve the decided model. |
provider_order | Explicit provider order, tried before any other eligible provider. |
provider_allow | Only these providers may serve. Empty means no restriction. |
provider_deny | These providers never serve. A provider cannot be both allowed and denied. |
auto_enabled | Whether auto may be used in this scope. Absent means inherit. |
auto_cost_quality | Dial 0 to 10. 0 is pure quality, 10 always takes the cheapest. Unset everywhere resolves to 5. |
auto_allow_models | Patterns auto may pick from, at most 32: author/slug, author/prefix*, author/* or *. |
auto_exclude_models | Patterns removed from whatever the allow line admits, at most 32. |
upstream | Which registered upstream connector serves this scope. Absent means no preference. |
delegation | delegated (the upstream's router picks the serving path) or router (Nautili's router picks). Default delegated. |
eligibility | Conditions a serving path must meet: custody, regions, zero data retention, terms classes, quantizations, price caps. |
custody_order | own_first (your key, then DevZero managed) or managed_first. Default own_first. |
Zone 4 echoes the server's own rules before you save, so a chain longer than four or a provider on both lists is named on screen. Save the profile.
Step 3: Choose who picks the provider
In zone 3, set Provider choice ("Who picks the provider") to the Nautili router option. This flips delegation from delegated to router.
With delegated, the gateway expresses your chain and provider preferences to the upstream, which performs the failover; the gateway records what served. With router, the gateway composes serving paths from the path catalog and the credentials your scope holds, ranks them, and walks them itself: at most 3 distinct serving paths per request across the decided model and its chain, each path retried at most once, and, on streaming requests, a 30 second first-byte deadline per attempt and a 60 second budget for the whole walk (both operator-configurable). A non-streaming attempt has no first-byte deadline, because its first byte is the whole answer. Only the attempt that served is charged.
What counts as a failure, and what the walker does next:
| Failure | Action |
|---|---|
| No response at all, or 500/502/503/504/529 | Retry the same path once, then move to the next path |
| 429, credit or quota exhausted, 401/403 on this credential | Move to the next path |
| No first byte before the deadline | Move to the next path |
| Context too long | Move to a path with a larger window, then to the next model in the chain |
| Content-policy refusal | Fail the request (or move to the next model in the chain if the gateway operator enabled ROUTER_FALLBACK_ON_CONTENT_POLICY) |
| Other 400/404/422 | Fail the request |
| Anything after the first byte | Fail; nothing can be re-sent once the caller holds part of an answer |
custody_order decides which credential arm is tried first when the key's custody is unstated: own_first tries your own provider key, then DevZero managed custody; managed_first reverses it. A key bound to one arm never crosses.
Step 4: Send a request and read the decision headers
Send a request that names no model. The merged default from the ladder resolves it. The OpenAI SDKs require the model field; send it empty and the gateway resolves the ladder default.
In router mode the response carries three routing headers beside X-Dz-Request-Id. Trimmed to the headers that matter:
X-Dz-Serving-Path names the path that served as <upstream>:<variant tag>@<custody>. X-Dz-Route-Attempts is the count, then each attempt as <path>=<result>, where the result is served, an HTTP status, timeout, transport, budget or unusable; an attempt on a fallback model names it in parentheses after the path. X-Dz-Route-Affinity is kept, kept:evicted, started or moved:<reason> with the reason failed, ineligible, evicted or saturated. All three are absent on a delegated profile and on a cache hit. Send X-Dz-Session-Id on every turn: the session pin and the path pin both key on it.
Step 5: Override one request
A caller may steer one request with the X-Dz-Router header (canonical, compact JSON, at most 8192 bytes) or the dz_router body field (compatibility alias). It is the fourth rung of the ladder and merges by the same rule: a field you omit is inherited, never cleared.
The request routes with cost ranking and novita excluded, and the chain for this request is replaced whole by openai/gpt-5.6-luna. The row records which override fields applied, by name only, in routing_override (for example fallback_models,provider_deny,provider_sort).
Three rules bound the override:
- Consent. An override steers inside a mode the profile enabled and cannot enable one.
fallback_modelswith no chain and no auto on the ladder, orautodials with auto not enabled, is400 routing_override_not_permitted. - Vocabulary. The marketplace's own knobs
route,transforms,presetandpluginsare refused at the top level of every keyed body with400 unsupported_routing_field. An unknown field insidedz_routeris400 invalid_dz_router, never ignored. - Size. Over 8192 bytes on the header is
400 dz_router_too_large. A large override is configuration and belongs in a profile.
Step 6: Force a failover and read the record
The deterministic way to see the walker move is a credential failure on the first arm. With delegation: router and custody_order: own_first, save a revoked provider key under Provider keys, with managed custody available on the same upstream, and send the request from Step 4 again.
A 401 on your credential is a credential fault, so the walker moves to the next path without retrying, the managed arm serves, and the pin moves with the reason failed. The served model is still the decided model, so routing_switch_reason stays empty: the path changed, the model did not.
A model fallback fires when every eligible path of the decided model fails within the budget, or a path refuses the prompt as too long with nothing larger left. In the header's notation a model fallback looks like this:
On that row decided_model is the primary, served_model is the fallback, and routing_switch_reason is upstream_fallback, the only reason a decided model changes outside auto mode. The session pin moves to the fallback and that turn is denied the cross-turn cache discount, because the new model's prompt cache is cold.
Verify
Open Traces and find the request by its X-Dz-Request-Id. The trace details panel shows Model and Provider as served, and for a router-mode request an attempts block: one line per attempt with its outcome (Served, Retried, Moved to next path, Fell back to next model, Failed), HTTP status, failure class and custody arm; a summary such as "Served after 1 failed attempt"; what the path pin did; and which arm was charged when the arms differ.
The full decision record is the telemetry row behind that trace. Trimmed to the routing fields:
requested_model is empty because the request named no model; decided_model is what the ladder resolved and the gate bound; served_model is what answered. Every keyed request writes this record, denials and cache hits included.
Options
provider_allow and provider_deny
Restrict which providers may serve. They are separate fields, not one signed list, and each merges down the ladder. On a router profile the allow list can only narrow what broader rungs allow.
A request under that profile with X-Dz-Router: {"provider_allow":["anthropic"]} narrows to one provider and is served on it. A request with X-Dz-Router: {"provider_allow":["novita"]} shares no provider with the profile's list, so no path can pass both:
eligibility
Consent to serving paths as conditions on their attributes rather than as a list of names. A profile stating a rule must set delegation: router; the control plane refuses to save it otherwise.
A per-request X-Dz-Router: {"eligibility":{"max_prompt_usd_per_1m":3}} tightens the cap and is served under it. A per-request {"eligibility":{"zero_data_retention":false}} would widen the rule, so it is ignored, the request is served under the stricter rule, and the row carries routing_degraded: eligibility_widening_ignored. A path whose attribute is unpublished fails a condition that needs it. On a delegating profile that inherits a rule from a broader rung, the request is refused with 400 unsupported_capability.
auto_enabled and the auto dials
A request with "model": "auto" now resolves at the upstream, and X-Dz-Router: {"auto":{...}} may pass per-request dials. Under a guardrail with a non-empty model allowlist, keep auto_allow_models explicit: the gate checks that set against the allowlist before forwarding, and a wildcard under a non-empty allowlist is refused. A delegated auto turn bypasses both caches once; the session pin then makes later turns gate and cache normally.
Troubleshooting
| Error or symptom | Cause | Fix |
|---|---|---|
Error or symptom400 auto_not_enabled | CauseA request named auto and no rung enables auto. | FixSet auto_enabled on a profile bound to the team, the identity or the key. |
Error or symptom400 no_default_model | CauseThe request named no model and no rung sets default_model. | FixSet a default on the team-default profile, or send model. |
Error or symptom400 invalid_model_id | CauseA malformed id, or a bare name the catalog does not resolve, such as gpt-4o. | FixUse author/slug, for example openai/gpt-5.6-luna. |
Error or symptom400 invalid_dz_router | CauseUnknown field in the override, malformed JSON, or the header sent twice. | FixCheck the field names against the table in Step 2; send the header once. |
Error or symptom400 dz_router_conflict | CauseOverride present on both the header and the body field. | FixDelete one carrier, even if the two are identical. |
Error or symptom400 dz_router_too_large | CauseHeader over 8192 bytes. | FixMove the configuration into a routing profile. |
Error or symptom400 unsupported_routing_field | Causeroute, transforms, preset or plugins at the top level of the body. | FixRemove them; dz_router is the routing vocabulary. |
Error or symptom400 routing_override_not_permitted | CauseOverride tried to enable fallbacks or auto that no rung enabled. | FixEnable the mode on a profile, then steer inside it. |
Error or symptom400 no_serving_path | CauseRouter mode, and every path for the decided model was excluded by a rule or a provider list. | FixRead the exclusion reasons in Path catalog on Routing; loosen the rule or the list. |
Error or symptom400 provider_allow_disjoint | CauseA narrower allow list shares no provider with a broader rung's. | FixMake the lists overlap, or drop the narrower one. |
Error or symptom400 unsupported_capability | CauseA restriction (allow, deny or eligibility) the serving upstream cannot enforce. | FixSet delegation: router, or remove the restriction from a delegating scope. |
Error or symptom502 unknown_upstream | CauseThe profile or override names an upstream the installation has not enabled. | FixFix the id, or enable the upstream on the installation. |
Error or symptom503 upstream_undetermined | CauseThe profile could not be read and more than one upstream is registered. | FixRetry; it clears when the control plane answers. |
Error or symptomNo X-Dz-Serving-Path on the response | CauseThe profile is delegated, or the response was a cache hit. | FixSet delegation: router; X-Cache: HIT means no path served. |
Error or symptomrouting_degraded is non-empty | CauseA preference could not be expressed, or a rung tried to widen eligibility. | FixRead the token; a degraded request still served. |
Next steps
- Routing profiles for the full merge rules and what the resolver reports back.
- Fallbacks and providers for how direct provider upstreams treat rankings and restrictions differently.
- Per-request override for the complete
dz_routerfield reference. - Request headers for the session id and attribution headers that make traces comparable across turns.
- Guardrails overview for the gate that runs on the decided model after routing.
- Nautili gateway quickstart if you have not sent a keyed request yet.
- Your first model trial to compare a candidate model against the profile's default before promoting it.
- Govern LLM traffic with guardrails for the gate that runs on the decided model after routing.