Choosing a lane

The question this page answers: I have LLM calls in my code today. How should they reach ModelRig?

There are three ways in. They aren't exclusive: most codebases use more than one, call site by call site. Start with observe unless you already know you want routes. It changes nothing about what your calls do, and it gets your runs onto /runs the same day.

The three lanes

	Observe (wrap what you have)	The `runRaw` seam (one adapter for a router you already own)	Named routes (`rig.run`)
When	You want to see your pipeline's runs, costs and steps before changing anything.	Your code already picks the provider, model and prompt at run time: a `callProvider(provider, model, prompt)` dispatcher, env-resolved models, runtime-assembled prompts.	A call site has a fixed prompt, a fixed output shape and a model choice you'd like to make in config, not code.
What you write	A `run.scope` around each pipeline execution, a `stepKey` per call, `artifact.save` for the work products. Each model call either goes through `rig.runRaw` (same provider, same model, your own provider key) or stays a direct call that you report with `rig.recordCall` right after it returns; either way the step carries tokens and cost. Build the rig with `routesDir: null`. `npx modelrig observe` scaffolds the seam, wired with `rig.recordCall`.	Replace the body of your dispatcher with one `rig.runRaw({ provider, model, apiKey, systemPrompt, userPrompt, … })` call. Build the rig with `routesDir: null`.	A route YAML (prompt template, JSON Schema, candidate models, retry and budget policy) and `rig.run("task.name", { input, tags })` at the call site.
What you get	The run → step → artifact graph on `/runs`, cost and tokens per call, tags on `/costs`, grades, goldens you can bake off.	Everything observe gives, for every call the router makes, with no per-task YAML.	Schema-validated output, the candidate ladder with typed retries and fall-through, budget envelopes, bake-offs by `--replay-last`, live model pins and A/B without a redeploy.
Who picks the model	Your code passes a default per step; with `stepKey` set, you can pin another model, tier or Flex window for that step from the console or MCP, no redeploy. A call you report with `rig.recordCall` keeps your code's model.	Your code passes a default per step; pin a step's model the same way (step config).	The route's candidates in YAML; pin or A/B a model live from the console or MCP, no redeploy.
What you give up	No validation, no fallback, no A/B. The raw lane forwards your request as-is (apart from a step pin you set).	Same as observe: the raw lane has no ladder, no `require`, no repair, no envelope.	A migration per call site (proposed, proven with `modelrig verify-swap`, human-merged).
Cost	Your provider bill (your own key), plus ModelRig's fee under the current pricing terms.	Same.	Same, plus whatever extra attempts your retry and fall-through policy allows, capped by budget envelopes.

All three record into the same graph. A rig.runRaw call and a rig.run call inside one run.scope both become steps of that run.

The default: start with observe

1. Mint a key and confirm the org (Orgs and keys). 2. Wrap one pipeline execution in a run:

Wrap one pipeline execution in a run. Every pipeline execution is a run. Wrap each execution in run.scope (or run.start / run.end), name one step per model call with stepKey, artifact.save the work products, and pass environment so the Env column on /runs is filled. Inside the run the SDK stamps run_id and step on every call. run_id is reserved: don't set it yourself.


import { createRig, loadConfigFromEnv } from "modelrig";

const rig = createRig(loadConfigFromEnv());

// Every pipeline execution is a run. Inside the scope the SDK records each
// rig.run / rig.runRaw call as a step and stamps run_id and step on every row.
// run_id is reserved: don't set it yourself (put your own attempt id in a tag of
// your own, such as pipeline_attempt_id).
await rig.artifacts.run.scope(
  { pipeline: "support", episodeKey: ticketId, environment: process.env.NODE_ENV ?? "development" },
  async () => {
    const result = await rig.run("example.support_summarize", {
      input: { ticket, product, priorContext },
      tags: { subject: customer, feature: "summarize" },
      stepKey: "summarize", // names this step on /runs
    });
    rig.artifacts.artifact.save(result.output, { name: "summary", type: "step_output", stepKey: "summarize" });
  },
);

await rig.close(); // flushes the exporter — call once at shutdown

3. Run it once and open /runs. The run, its steps in order, and the artifacts you saved should be there.

The snippet calls a route, rig.run("example.support_summarize", …). In an observe-only codebase there are no routes yet, so build the rig raw-only and put each existing call behind rig.runRaw (same provider and model), or keep the call as it is and record it; the run wrap is identical:

No routes yet? Build the rig raw-only. Observing a pipeline, or recording calls your code makes itself, needs no route files. Construct the rig with routesDir: null: rig.runRaw, rig.runRawStream and rig.recordCall work at once, and rig.run / rig.runBundle refuse with a teaching error until you add routes. Without it, createRig loads ./modelrig/routes: a directory you configured that is missing throws, and a missing default directory falls back to raw-only with a one-line warning.


import { createRig, loadConfigFromEnv } from "modelrig";

// Raw-only on purpose: no route files needed. rig.runRaw, rig.runRawStream and
// rig.recordCall work; rig.run needs routes (add them later, drop routesDir: null).
const rig = createRig({ ...loadConfigFromEnv(), routesDir: null });

Record a call your code makes itself. A provider call your code makes directly (a client ModelRig has no lane for, a stream you read yourself, an embedding, a fallback) is invisible until you report it. Call rig.recordCall right after it returns, inside the run, with the tokens the provider reported: the call shows on /inferences as "Reported by your code", joins its run and step, and counts toward cost. It is synchronous and never throws; invalid input warns once and records nothing (inferenceId: null). Spread optional fields only when the provider reported them, so the call type-checks under exactOptionalPropertyTypes.


const started = Date.now();
const resp = await genai.models.generateContent({ model: "gemini-3.1-pro-preview", contents });
const u = resp.usageMetadata;
const finishReason = resp.candidates?.[0]?.finishReason;

const rec = rig.recordCall({
  provider: "gemini",
  model: "gemini-3.1-pro-preview",
  usage: {
    tokensIn: u?.promptTokenCount ?? 0, // total input, cached included
    tokensOut: (u?.candidatesTokenCount ?? 0) + (u?.thoughtsTokenCount ?? 0),
    tokensCached: u?.cachedContentTokenCount ?? 0,
    // optional fields: spread only when the provider reported them
    ...(u?.thoughtsTokenCount !== undefined ? { reasoningTokens: u.thoughtsTokenCount } : {}),
  },
  servedTier: "standard", // the tier that SERVED the call: pass what the provider reports (flex, priority); it is priced at that tier
  latencyMs: Date.now() - started,
  stepKey: "summarize", // the step on /runs (inside run.scope / run.start)
  ...(finishReason !== undefined ? { finishReason } : {}),
});
if (rec.inferenceId !== null) rig.grade({ kind: "inference", id: rec.inferenceId }, { score: 1, kind: "deterministic" });

Full API: Observe a pipeline.

When to graduate a call site to a route

Move one call site at a time, when one of these is true:

You want the output schema-validated and repaired, rather than validated by your own code.
You want a fallback model when the first one fails, or a cheaper model proven on your traffic first.
You want to A/B a model live, or fall back across candidates, from the console without a deploy. (Pinning one model per step works on the raw lane too: see below.)

Don't extract routes from a router that already assembles prompts and picks models at run time. Migrate one naive call site as a reference route and keep the router on the runRaw seam. The step-by-step procedure, including the checkpoints a coding agent stops at, is the migration playbook.

Change a raw step's model without routes

You don't need a route to change which model a step uses. Name the step with stepKey and pin it (needs modelrig 0.14.0 or newer):

Choose the model per step without a deploy. Name each rig.runRaw call's step with stepKey, and the model your code passes becomes that step's default. Pin a different model, tier or Flex window for the step on Routes & steps in the console (https://app.modelrig.ai/routes), or with set_call_config from the MCP oracle: no redeploy. A pin applies to runs that start after ModelRig's settings cache refreshes (at most about a minute after you save); a run never changes model mid-run. With no pin, the call is exactly what your code asked for. If ModelRig can't be reached, the step keeps the last pin it read, or uses your code's model if it never read one; the call never fails because of it. Keys never move: the call still goes out from your process on your own provider key, and a pin to another provider uses that provider's key from your environment, failing closed with an error naming the variable (for example OPENAI_API_KEY) when it is missing. A pin the call can't use (grounding or a responseSchema the pinned model can't honor) fails closed before dispatch, and a cache handle made for your code's model is dropped, so the pinned call runs uncached. Every change is a version you can roll back in one click, and each call records model_source (code or step-config). It needs MODELRIG_API_KEY; createRig({ ...loadConfigFromEnv(), stepConfig: false }) turns it off.


const overview = await rig.runRaw({
  stepKey: "business_overview", // the step's name: the handle a pin attaches to
  provider: "gemini",
  model: "gemini-3.1-pro-preview", // your code's default; a step pin replaces it for new runs
  apiKey: process.env.GEMINI_API_KEY ?? "",
  systemPrompt: "You are an equity analyst.",
  userPrompt: "Summarize the business in five bullets.",
});

The full semantics are in Routing & reliability.

The gateway (an OpenAI-compatible base URL)

Optional — the gateway (OpenAI-compatible base URL). Optional, and not where setup starts: an OpenAI-compatible client can point its base URL at https://api.modelrig.ai/v1 with your rig_sk_ key (MODELRIG_API_KEY) as its API key, naming the model as <provider>/<model> (such as openai/gpt-5.4-mini) for a BYOK passthrough on your own provider key. A human must first store that provider key on the Vault page (https://app.modelrig.ai/vault), and a coding agent never handles the value; without a stored key the call returns 402, and it never falls back to a ModelRig key. Serving a route over the gateway needs managed serving (ask us). Gateway calls show on Calls (/inferences) but carry no run context, so they never fill Runs (/runs): runs come from the SDK path, rig.runRaw or rig.recordCall inside run.scope.


# Optional: point an OpenAI-compatible client at ModelRig (needs a provider key stored in Vault).
base_url = https://api.modelrig.ai/v1
api_key  = $MODELRIG_API_KEY              # your rig_sk_ key, sent as the bearer
model    = "openai/gpt-5.4-mini"          # <provider>/<model>: BYOK passthrough on the key stored in Vault (402 without one)

What none of the lanes see

What ModelRig does not see unless you record it. ModelRig records the calls your code makes through the SDK (rig.run, rig.runBundle, rig.runRaw, rig.runRawStream) or the gateway, plus any call your code makes itself and reports with rig.recordCall. Nothing else is seen. Five things stay invisible or unattached unless you record them: (1) direct provider SDK calls, with or without a JSON schema; (2) streams you read from a provider client yourself; (3) a fallback your code makes after the raw lane refuses a responseSchema it cannot enforce (deepseek, deepinfra, fireworks and zai fail closed with invariant_violation); (4) calls made outside run.scope / run.start, which land on /inferences but on no run; (5) embeddings. A recorded call shows as "Reported by your code": ModelRig neither dispatched nor validated it, and its tokens are the numbers you reported. Details and the console tells: https://modelrig.dev/not-captured.html.