Self-host the Nautili gateway with Helm: what leaves your network
Install the Nautili gateway in your own cluster with the Helm chart, size the state store and ingress, and know exactly which bytes cross to the control plane.
23 min readLast reviewed October 19, 2026Verified against gateway chart 0.10.0 (image devzeroinc/nautili-gateway:v0.10.0)
At the end of this guide you will have a Nautili gateway running in your own Kubernetes cluster, reporting to the control plane, with a values file you can hand to a security reviewer. You will also have a one-page answer to the question that reviewer asks first: which bytes stay inside the cluster and which bytes leave it.
Before you begin
Every team already has the Shared Gateway, https://shared.gateway.nautili.ai, DevZero-run and multi-tenant, with nothing to install; the Install gateway screen says so at the top. Installing a gateway is for keeping prompts in your own environment or for a proof of concept. The screen lists the three install methods in this order.
| Method | When to choose it |
|---|---|
| Helm / K8s | Production (the card reads Recommended). The gateway runs in your cluster and prompts stay in your network. |
| DevZero-hosted | Proofs of concept, on compute a global admin grants your team (the card reads POC entitlement). DevZero runs the gateway and gives you a URL. Request bodies transit DevZero infrastructure by construction. |
| Docker Compose | The gateway and its Redis in one Compose file, for a laptop, VM or single server. |
Helm is the production default because two of the three methods keep prompts in your environment, and Helm is the one that ships a replica floor, a disruption budget, a shared state store and an upgrade operator. The architecture is one sentence long: the gateway runs inside your cluster, your clients talk to it over the cluster network or your ingress, and the gateway makes outbound HTTPS to the Nautili control plane to fetch configuration and send telemetry. There is no inbound path from DevZero into your VPC. Remote configuration is pulled, never pushed.
You need:
- A Kubernetes cluster you can deploy to, with
kubectlpointed at it. - Helm 3. The chart is an OCI artifact, so Helm must be able to pull from
registry-1.docker.io. - Outbound HTTPS from the gateway pods to the control plane endpoint. Nothing else needs to reach in.
- The pre-filled install command from Install gateway. Open it, choose Helm / K8s, name the installation (for example
prod-us-east), and press the button. That step mints the installation's auth token and installation id, and it runs only when you press it. Copy the command somewhere safe. The values in it are what the rest of this guide puts into avalues.yaml. - A
sk-dz-key from API keys under Access, for the final verification request.

Step 1: Write the values file
Create values.yaml with the identity settings from the pre-filled command.
What each setting does, in chart terms:
config.teamIDrendersTEAM_ID. Required. The render fails withconfig.teamID is requiredwithout it.config.backendEndpointrendersBACKEND_ENDPOINT, the control plane the gateway reports to. Required. Keep ithttps. A plain-httpendpoint is refused for credential material unless you setgovernance.insecureBackendOK: true, which exists for in-cluster test backends and nothing else.config.authTokenis the installation's token. The chart writes it into a Secret named<release fullname>-credentialsunder the keyauth-token, and the gateway reads it through asecretKeyRef. It never lands in the ConfigMap. The chart has no existing-Secret option for this token (that option exists only for the Redis password, underredis.auth.existingSecret). If your policy forbids tokens in values files, the proxy's reference to that Secret is markedoptional: true, so you can leaveconfig.authTokenempty, create the Secret yourself with that exact name and key, and the pod will still start. Helm will not manage that Secret.config.installationIdrendersINSTALLATION_ID. Leave it empty and the chart derives one fromconfig.teamID, the release namespace and the release name, formatted as a UUID. Those three inputs are identical for every replica and survive restarts and upgrades. Anything you supply wins over the derivation.config.installationNamerendersINSTALLATION_NAME. Cosmetic, nothing is keyed on it, but without it Sources shows the installation id instead of a name.config.publicUrlrendersGATEWAY_PUBLIC_URLand is reported on every heartbeat, so first-party clients such asntare told where to send inference. Empty, the chart derives it fromingress.hostwhen ingress is enabled; with your own ingress or load balancer, set it.config.postureisproxy(the default, one team's gateway) orshared(the fleet gateway DevZero runs). Leave it atproxy. The shared posture refuses a team id and an auth token and is not a self-hosting shape.
Step 2: Decide the state store and replica count
Add the replica and Redis sections. The defaults are safe; this step is about knowing why.
The rule that matters: more than one replica requires a Redis the replicas share. Governance counters (budgets, rate limits) and the per-session routing pin are cost-bearing state. If each replica kept its own copy, a conversation that moved between pods would be charged against two budgets and could be handed a different model on every turn. The chart enforces this at render time, and the gateway enforces it again at boot. With replicaCount above 1 and no store, helm template stops with:
The chart bundles two Redis instances rather than one because their eviction policies are opposites. The state instance (<release fullname>-redis-state, 128mb, noeviction) holds counters and pins and must never evict. The bodies instance (<release fullname>-redis-bodies, 256mb, allkeys-lru) holds the exact-match response cache and may evict freely. Both run as single replicas on emptyDir; every value in them is TTL-bounded and reconstructible, so there is no volume to back up. Both require a password by default, generated once at install and reused on every upgrade, and a NetworkPolicy restricts their ingress to this release's proxy pods where your CNI enforces policy.
To bring your own Redis, set redis.external.bodiesAddr and/or redis.external.stateAddr. Any non-empty address disables bundling for both purposes, whatever redis.enabled says. One address serves both purposes; two split them, and then stateAddr must point at a noeviction instance. For a managed service that generates a password per instance, redis.external.bodiesPassword and redis.external.statePassword override redis.auth.password for each.
replicaCount: 1 is supported for single-node clusters and development installs. The disruption budget (podDisruptionBudget.maxUnavailable: 1) and the hostname topology spread (topologySpread.whenUnsatisfiable: ScheduleAnyway) are written to stay correct at one replica.
For autoscaling, set autoscaling.enabled: true. The chart then renders a HorizontalPodAutoscaler on CPU, drops spec.replicas from the Deployment, and ignores replicaCount. The replica guard reads autoscaling.maxReplicas as the ceiling, so a storeless install cannot scale past one pod.
Step 3: Expose the surfaces
The gateway listens on four request ports and one probe port (plus 8888 when the Cursor tap is enabled).
| Port | Surface | Env var |
|---|---|---|
| Port8080 | SurfaceOpenAI-compatible surface and the management API | Env varLISTEN_ADDR |
| Port8081 | SurfaceNative Anthropic surface | Env varANTHROPIC_LISTEN_ADDR |
| Port8082 | SurfaceNative Gemini surface | Env varGEMINI_LISTEN_ADDR |
| Port8083 | SurfaceNative OpenAI / Codex surface | Env varOPENAI_LISTEN_ADDR |
| Port8090 | SurfaceProbes and the Prometheus exporter: /health, /ready, /metrics | Env varPROBE_LISTEN_ADDR |
The Service publishes 8080 to 8083. Port 8090 is deliberately not a Service port: the probes address the pod IP, and keeping the exporter off 8080 keeps your denial counters off the public hostname. Without ingress, in-cluster clients use http://<release fullname>.<namespace>.svc.cluster.local:8080/v1 and the matching port for each native surface.
Ingress is off by default. Turn it on with:
With routing: hostnames the chart derives four hostnames from the one base host:
| Hostname | Surface | Port |
|---|---|---|
| Hostnamellm.example.com | SurfaceOpenAI-compatible | Port8080 |
| Hostnameanthropic-llm.example.com | SurfaceNative Anthropic | Port8081 |
| Hostnamegemini-llm.example.com | SurfaceNative Gemini | Port8082 |
| Hostnameopenai-llm.example.com | SurfaceNative OpenAI / Codex | Port8083 |
Four hostnames rather than four paths, because the OpenAI-compatible surface and the native OpenAI surface serve the same paths (/v1/chat/completions, /v1/responses) to different upstreams. Only a hostname or a port can tell them apart. A surface gets a route only while its nativeSurfaces.<name>.enabled is true.
routing: paths is the single-hostname shape: one HTTPRoute sends /v1/messages to the Anthropic listener and everything else to the OpenAI-compatible one. It requires kind: httproute, nativeSurfaces.anthropic.enabled: true, and the native OpenAI and Gemini surfaces off; the render fails otherwise and names the value to change.
Step 4: Review the operator and governance defaults
Two more blocks ship enabled. Read them before you install, because both decide something a reviewer will ask about.
The upgrade operator. With operator.enabled: true the chart renders a second Deployment, <release fullname>-operator, with its own ServiceAccount and RBAC. Every operator.pollInterval (default 5m) it asks the control plane which version this installation should run and records that it asked. When the stable channel moves, the operator performs a verified helm upgrade --atomic --wait of the gateway release from inside the cluster, bounded by operator.maxUpgradeDuration (default 10m), and rolls back on failure. The control plane's release catalog and its stable channel are authoritative for which version a gateway should run; the installation runs the higher of the stable channel head and its own pin, so a pin can hold it ahead of the channel but never behind it. The operator's poll is its own liveness, not the gateway's: a gateway can be connected while its operator has stopped syncing.
To freeze a version at the chart level, set operator.enabled: false and pin image.tag. You then own upgrades. Do not pin image.tag to anything older than v0.10.0 with the default probes: the /ready route and the probe listener arrived with the drain-contract build, and an older image never becomes ready.
Governance. governance.enabled: true renders three env vars together: GOVERNANCE_MODE=enforce, GOVERNANCE_BYOK_FORWARD=true and GOVERNANCE_ISSUED_KEYS=true. That is one posture, not three settings, because the binary refuses to boot on a partial set: a keyed request carries no credential of its own, so issued-key detection needs the credential substitution it depends on. A governed gateway also needs a reachable Redis for its counters, which is why the bundled store is on by default. governance.enabled: false renders none of the three and gives you an ungoverned gateway that serves only the passthrough surfaces: no sk-dz- keys, no budgets, no routing.
Telemetry content. config.telemetryContentUplink is the residency switch. null (the default) omits the env var and lets the binary decide by deployment method: hosted keeps content uplink on, and every self-hosted method (Helm, Docker, binary) is content-free. Explicit true or false pins it. It renders LLM_TELEMETRY_CONTENT_UPLINK and is never manageable from the dashboard, which is why it is a first-class value and not an extraEnv entry. The boot log says which state you are in:
config.extraEnv pins any other binary env var. A set env var always wins over the dashboard and shows there as read-only. Values must be strings.
Step 5: Install
Render first, then install.
A render failure is a sentence naming the value to change. A successful render is a file your reviewer can read: grep it for Secret, ConfigMap, NetworkPolicy and HTTPRoute to see exactly what will be applied. During helm template the bundled Redis password prints as a throwaway value because there is no cluster to look it up in; that is expected and does not mean the password changed.
Helm creates the namespace, the gateway Deployment with two replicas, the two Redis Deployments, the Services, the ConfigMap, the credentials Secret and the Redis password Secret, a PodDisruptionBudget, a NetworkPolicy for the Redis pods, the operator Deployment with its RBAC, and the ingress objects if you enabled them. --wait returns when the readiness probe on /ready answers 200 on every replica.
Verify
Check that the pods are running and ready.
Pod names are trimmed; yours carry full hashes. Probe the gateway on the probe port.
/health carries the upstream breaker statistics after status (trimmed here) and answers 200 regardless of upstream state, by design: a provider outage must not restart working gateways. /ready is the only route that reports the drain and flips to 503 with {"status":"draining"} the instant the process observes SIGTERM.
Open Sources under Connections. Your installation appears by name, tagged Helm / K8s and Proxy. Before the first heartbeat its status reads Awaiting connection. Once the heartbeat has arrived the row carries a connectivity chip (Healthy, Degraded or Disconnected) and a traffic chip: No data while nothing has flowed in the last 24 hours, Live once it has. The Install gateway flow, if you left it open, shows Gateway connected at the same moment.

Send one request through the OpenAI-compatible surface with a sk-dz- key. Through a port-forward on 8080, or through your ingress host:
The response carries two headers worth reading (trimmed to the ones that matter):
X-Dz-Request-Id is the id the trace is filed under; search for it on Traces. The Install gateway flow, if you left it open, moves from Gateway connected to Traffic detected, and Sources turns the installation Live.
What stays inside and what crosses
This is the table to hand to the security reviewer. It describes a self-hosted gateway with config.telemetryContentUplink unset or false.
| Stays inside your cluster | Crosses to the control plane |
|---|---|
| Full request and response bodies | Content fingerprints only, never text |
| Prompt and response previews | Token counts (input, output, cache read, cache write) |
| Extracted user prompt text and assistant response text | Cost, latency and cache statistics |
| Tool-call argument summaries | Tool names, argument byte lengths and recurrence |
| The exact-match response cache (bodies Redis) | Billing regime, workload type and attribution (user, team, project, env, labels) |
| Governance counters and routing pins (state Redis) | Governance decisions: the authorize verdict, the decided and served model, the custody that resolved |
Provider credentials you supply under gateway.secrets or through BYOK provider keys | Heartbeats: team id, gateway version and commit, overall status, Redis enabled and connected, uptime since, installation id, name and type, deployment method, public URL |
| Heap profiles (the pprof listener binds to loopback because a profile contains prompts) | Operator polls: the version this installation runs and the outcome of each upgrade |
The scrub runs in the gateway before send, on copies, so the gateway's own records keep full content in-cluster. The list of blanked fields is the function ContentFreeSendFunc in gateway/telemetry/scrub.go: FullRequestBody, FullResponseBody, PromptPreview, ResponsePreview, UserPromptText, LatestUserPrompt, AssistantResponseText and every tool call's ArgsSummary.
Content-free covers every channel to the control plane, not only the telemetry uplink. The consequence is where work runs, not whether it runs: trials on a content-free installation execute gateway-side and only the pair scores are uplinked; an evals dataset seeded from traces can use only traces whose bodies reached the control plane, and names the installation that kept the rest. Spend, tokens, budgets, routing and guardrails are unaffected.
Under BYOK custody, the provider credential your team registered on Provider keys is resolved through the control plane as a short-lived forwarding credential and used by the gateway inside your cluster to call the provider. The provider request itself goes from your cluster to the provider, not through DevZero.
Options
Upgrading, and the old llm-proxy chart name
Upgrade with the same values file and a new chart version.
Pass the values file every time. A --reuse-values upgrade renders over the previous chart's computed values, which may predate blocks the new chart reads. The pod template carries a checksum of the ConfigMap, so a values-only change rolls the gateway pods; the operator pod is deliberately annotation-free and keeps the identity values it started with, so if you change backendEndpoint, teamID or installationId, restart the operator by hand.
The chart used to be published as llm-proxy. The name feeds app.kubernetes.io/name, which is part of the Deployment's immutable selector, so a release installed from an llm-proxy-named chart cannot be helm upgraded to nautili-gateway directly. Either reinstall, or keep the old runtime identity:
If your values file carries retired keys from that era (config.policySyncInterval, a top-level cache, state or sidecar block, governance.mode), the render fails and names the replacement. Delete the key or move it to config.extraEnv as the message says.
Your own Redis
Nothing is bundled, the addresses render verbatim as STATE_REDIS_ADDR and CACHE_REDIS_ADDR, and the per-store passwords ride a Secret of their own. With only stateAddr set there is no response cache; CACHE_ENABLED is true exactly when a bodies address exists. Setting the per-store passwords while the store is still bundled fails the render.
Pin a binary knob
The entry renders last on the proxy container, beats the ConfigMap, and rolls the pod on the next upgrade. The dashboard shows a pinned knob as read-only. Leave a knob out and the dashboard manages it over the binary's default, while config.remoteConfigEnabled is true. Set that to false and the gateway refuses dashboard-managed values entirely.
Disable the native surfaces you do not use
The Service stops publishing 8082, the ingress stops deriving gemini-<host>, and the gateway renders GEMINI_LISTEN_ADDR empty. The OpenAI-compatible surface on 8080 is unaffected.
Troubleshooting
| Symptom or exact message | Cause and fix |
|---|---|
config.teamID is required or config.backendEndpoint is required at render | The two required values are missing. Copy them from the pre-filled command on Install gateway. |
replicaCount > 1 (or autoscaling.maxReplicas > 1) requires a Redis the replicas share ... | You set redis.enabled: false with no external address. Leave the bundled store on, set redis.external.stateAddr, or run replicaCount: 1. |
governance.enabled=true requires a reachable Redis for its cost-bearing counters ... | Same cause as above, hit through the governance guard. Same fix, or governance.enabled: false for an ungoverned gateway. |
governance.mode is retired (ADR 0064) ... | Remove governance.mode, governance.issuedKeys and governance.byokForward. Chart 0.10.0 has one switch, governance.enabled. |
ingress.parentRefs is required when ingress.kind is httproute | A Gateway API route with no parent is accepted and never programmed. List the Gateway under ingress.parentRefs. |
ingress.routing=paths requires ingress.kind=httproute | Single-hostname routing is a Gateway API shape. Switch to kind: httproute or routing: hostnames. |
config.posture must be "proxy" or "shared" | Typo in config.posture. Self-hosted installs use proxy. |
Pod never becomes Ready, helm install --wait hangs | image.tag points at an image older than the drain-contract build, which has no /ready route. Use v0.10.0 or later, or set both probes.readiness.path: /health and probes.readiness.port: http for a pinned old image. |
Gateway pod in CrashLoopBackOff, log mentions the replica guard | The chart found an address but the process could not reach it. Check the Redis Service, the password Secret and your NetworkPolicy. The boot line redis placement: state=… bodies=… split=… names what it reached. |
NOAUTH errors in gateway logs for a few seconds after upgrade | Expected during the rollout that adopts Redis auth: an old pod without the password meets the restarted store until the same rollout replaces it. |
| Sources shows the installation but no heartbeat arrives | Outbound HTTPS from the pods to config.backendEndpoint is blocked, or config.authToken is wrong. kubectl logs on a gateway pod shows the heartbeat error. |
| Installation appears twice, or loses its dashboard config after an upgrade | The installation id changed. Set config.installationId explicitly to the value from the install screen and keep it in version control. |
| API keys shows no base URL for the gateway | config.publicUrl is empty and ingress is off. Set config.publicUrl to the address clients use. |
502 devzero_auth_unconfigured on nt requests | devzeroAuth.issuer and devzeroAuth.resource are empty. The dashboard's install command fills both; copy them into devzeroAuth. Setting only one is a render error. |
400 invalid_model_id on the verification request | The model id must be author/slug, for example anthropic/claude-haiku-4.5. A bare name the catalog does not resolve, such as gpt-4o, is refused; bare Claude names map to their catalog id. |
| A trial runs gateway-side, or a dataset seeded from traces reports missing bodies | The gateway is content-free. That is the default and the point. Opt in with config.telemetryContentUplink: true only if your policy permits prompt text to leave the cluster. |
Next steps
- Nautili gateway quickstart, the guide for the install flow and first verified request if you have not run it yet.
- Keep secrets and PII out of prompts, the guide for the content rules that run inside this gateway before anything reaches a provider.
- Quickstart, for the four-step install flow and the two connection tracks (coding tools and API access) that verify differently.
- Self-hosting reference, for the drain contract, the disruption budget form and the topology-spread hardening this guide did not cover.
- API access, to issue the
sk-dz-keys your applications will use and bind guardrails to them. - Content rules, to redact or block detector hits inside the gateway before anything reaches a provider.
- How it works, for the request path, the four questions the gateway asks the control plane, and how each one degrades instead of failing.