Managed serving — destination: modelrig-served

Serve a route's model on ModelRig's own credentials instead of yours. One less provider key to hold; the same route bundle, the same validate → repair → fall-through pipeline, the same per-call telemetry — served through the hosted gateway and paid from your organization's prepaid balance.

The honest default is direct. A route or run that declares no destination is served exactly as today: your own provider key, ModelRig not in the credential path, byte-identical behaviour. Nothing in this page changes a route you have not opted in.

Requires modelrig 0.9.1 or newer — the first published version that carries the hosted serving transport. On an older package a modelrig-served run is served through the local seam on whichever key the adapter already holds, so upgrade before you rely on the keyless path.

What "served" means

Which providers can be served is a fact in the capability table — the destination column in Route bundles → provider × knob support (and in llms.txt). A provider whose cell reads direct only cannot be modelrig-served; declaring it fails closed before any call.

Five steps

1. Declare the destination on the route (or the run)

# modelrig/routes/example.support_summarize.yaml
route: example.support_summarize
version: 3
destination: modelrig-served       # absent ⇒ direct (byte-identical)
candidates:
  - { provider: <a served-capable provider>, model: <model> }
# …schema, prompt, policy unchanged

A per-run override wins over the route:

await rig.run("example.support_summarize", {
  input: { ticket },
  destination: "modelrig-served",   // this run only
});

Precedence: RunOptions.destination → route destination → direct.

2. Enable managed serving in code

import { createRig, loadConfigFromEnv } from "modelrig";

const rig = createRig({ ...loadConfigFromEnv(), managedServing: true });

managedServing is a config field set in code, never an environment variable. It defaults to false: with it off, a modelrig-served route throws a teaching invariant_violation before any provider call rather than silently serving direct. This is the per-process switch; the per-organization switch is step 4.

3. Mint a key with the serve scope

Managed serving spends money, so it is a deliberate grant: in the console (Keys → new key) tick serve, and set that key as MODELRIG_API_KEY in the process that runs the rig. The SDK sends this rig_sk_ key as the only credential on a served call. A key without serve is refused with a 403 that names the scope.

4. Turn managed serving on for your organization

An owner flips it in the console — Account settings → Managed serving (off for every new organization). Or from a coding agent / script, through the settings API with an org-settings-scoped key:

PATCH /v1/orgs/<your-org-id>/settings
{ "managed_serving_enabled": true }

The server's owner-only gate decides; a member or admin is refused, and the setting applies to the next served call — no redeploy.

5. Fund the prepaid balance

Served calls are debited at serve time: the server-measured provider cost plus the flat fee, as two ledger entries anchored to the inference id (idempotent — a retry never charges twice). Before dispatch the gateway checks that a conservative reservation is available; otherwise it refuses with a 402 and nothing is served. Top up in the console (Billing). A direct route never needs a balance.

What the refusals teach

Every refusal is a RigFailureError of class invariant_violation — terminal, pre-dispatch where possible, and never a silent fall back to direct. The fixHint on the error is the text below.

A 404 here is a routing fact, not a permissions oracle: the hosted lane resolves slugs only inside the calling key's organization, so another organization's route is simply "not found".

What you see on the row

A served call's inference row carries destination: modelrig-served, the provider and model that actually served it, the served tier, the server-measured cost (which is what the ledger drew), and the inference id the charge is anchored to. It is the same row shape as every other call — the console, the analytics surface, exports and tenant statements all read it unchanged — so "which calls did ModelRig serve for me, and what did they cost" is one filter on the destination tag.

Data posture

On the served path your request reaches the serving provider on ModelRig's credentials, which makes that provider ModelRig's subprocessor for the request — the published security page states the posture, and the named list is part of the security-questionnaire pack available on request. Regulated or PHI data stays on direct (or managed custody) and is not sent through the served path until the subprocessor agreement chain is confirmed. Content custody is a separate axis with its own account setting (see Artifact content custody); the serving destination never changes what is stored.

See also

• Route bundles — the destination field and the provider × knob table.
• Routing & reliability — what fails closed and why.
• Quickstart — the direct path everything above is additive to.