ModelRig Quickstart Routing & reliability Route bundles Probes Bake-offs & replay How it fits Grade protocol Optimization loop Caching lifecycle Bring your traces (OTLP) Observe a pipeline Migration playbook (T0–T2) Recognition playbooks Tenants & statements Published receipts Provenance & trust Artifact content custody The MCP oracle Use-case templates Template: ticket triage Template: document extraction Template: CS next action Template: lead qualification Template: compliance review Template: catalog cleansing Template: call disposition QA Template: financial classification Template: medical classification Leaderboard

_This quickstart is as of modelrig v0.9.0 (the published npm package)._

ModelRig quickstart — see, prove, and save on every model call

ModelRig is the operating layer for your AI workstream. Each LLM task becomes a versioned route file (prompt template + JSON schema + candidate models + policy) that you call with one line — rig.run("task.name", {input, tags}) — and three things happen at once:

The rest of this page is Step 0 (two minutes, once), then the two ways to instrument your project — point your coding agent at it, or do it by hand.

Step 0 — create your org and key (two minutes, human-only)

Account setup stays human: your coding agent never creates accounts, signs in, or holds credentials. Once, up front:

1. Sign up and create your organization at app.modelrig.ai — email and an org name is the whole form. 2. Issue an API key in the console (the rig_sk_… value is shown once — copy it). 3. Export it beside your provider keys: MODELRIG_API_KEY=rig_sk_….

That key is the only ModelRig credential your app ever holds — never a database URL, never a service-role key. Provider keys never leave your process.

No account needed for the open lane: with only your provider keys set, routes, probes and telemetry run entirely on your machine — the hosted console and synced telemetry are what the key unlocks. You can add it later.

There are two ways your calls are keyed and billed, and you pick per route:

Same flat 2% margin either way; your negotiated provider discounts stay yours.

Point your coding agent at it

The fastest migration is the one your agent performs. From inside your project, paste this into Claude Code, Codex, or Cursor:


Migrate this project to ModelRig routes (https://modelrig.dev).

0. If MODELRIG_API_KEY is not set and I want hosted telemetry, STOP and ask me
   to create an org and issue a rig_sk_ key at https://app.modelrig.ai. Never
   create accounts, sign in, or handle credentials yourself. Without the key,
   proceed in local-only mode — that is fully supported.
1. Read https://modelrig.dev/quickstart.html and
   https://modelrig.dev/route-bundles.html.
2. Find every place this codebase calls an LLM API directly.
3. For each call site, define a route bundle YAML (schema, candidate models,
   policy, prompt template) and replace the direct call with
   rig.run("<route.name>"). Keep prompt and schema semantics unchanged.
4. Before naming any model capability, check the probed registry — declared
   flags and probed behavior differ; trust the probed layer.
5. Run the project's existing tests and report every file you changed.

Your agent reads the docs, finds your LLM call sites, and turns them into routes. It won't create accounts, sign in, or touch your API keys (those stay in the environment you set in Step 0), and it swaps call sites to rig.run() without changing prompt or schema semantics — same model, human- reviewed. The full autonomy ladder and same-model guarantees are in the migration playbook.

Or set it up by hand

1. Install


pnpm add modelrig      # or npm i modelrig — Node >= 20

It installs a native module (better-sqlite3) — prebuilt for macOS/Linux on x64/arm64; an unusual platform compiles it and needs a C++ toolchain.

Deploying: modelrig installs prebuilt (no separate build step), but it needs a Node runtime (not Edge/Workers), and if your deploy bundles server code you must mark it external — Next.js serverExternalPackages: ['modelrig'], esbuild --external:better-sqlite3, webpack externals. Set MODELRIG_API_KEY on every service, worker, and cron that runs a rig.


import { createRig, loadConfigFromEnv } from "modelrig";

Environment (read once, in ModelRig's config layer only):


GEMINI_API_KEY=…        # only providers you configure get adapters
OPENAI_API_KEY=…
DEEPSEEK_API_KEY=…

# Where your telemetry goes — pick ONE lane:
MODELRIG_API_KEY=rig_sk_…                      # hosted: rows go to ModelRig,
                                               # scoped to the org that key names
                                               # SET IT ON EVERY service/worker/cron
# (omit it and everything below still works, locally)
MODELRIG_ROUTES_DIR=./modelrig/routes          # default
MODELRIG_TELEMETRY_DB=./.modelrig/telemetry.db # default
MODELRIG_ENVELOPE_BUDGET_USD=25                # default envelope budget

Provider keys never leave your process on either lane. A rig_sk_ key is issued in the console at app.modelrig.ai and is the only ModelRig credential your app ever holds — never a database URL, never a service-role key.

2. Define a route bundle

One YAML file per route under modelrig/routes/:


# modelrig/routes/example.support_summarize.yaml
route: example.support_summarize   # the task handle you pass to rig.run
version: 1                         # bump on ANY change
schema: ./schemas/support_summarize.schema.json  # null for unstructured routes
candidates:                        # THE candidate set — the invariant boundary.
  - provider: openai               # Nothing outside this list can ever serve
    model: gpt-5.4-mini            # the route: enforced by a branded type at
  - provider: gemini               # compile time and a runtime guard.
    model: gemini-3.1-flash-lite
require: [schema_conformant]       # hard constraints on candidate eligibility
prefer: [cost]                     # advisory ordering (cheapest first)
prompt:
  system: ./prompts/support_summarize.system.md
  variables: [ticket, product, priorContext]
policy:
  retries: { content_invalid: 2, network: 4, capacity_shed: 3 }
  timeout_ms: 60000
  tier: flex                       # requested tier; served tier is recorded
  json: native                     # native strict schema | json_mode (rung 2)
  sampling: { temperature: 0.1, max_output_tokens: 8192 }
  #                                  ^ preserve the original call's sampling;
  #                                    absent = the adapter's own defaults

Prompt templates support {{variable}} substitution and capability-conditional blocks resolved against the serving candidate's flags:


Summarise this ticket for the next agent: {{ticket}}

{{#if capability.grounded_native}}
Use web search to verify recent events.
{{/if}}
{{#unless capability.structured_native}}
CRITICAL: respond with a single JSON object, no markdown fences.
{{/unless}}

Variables must be declared in prompt.variables; referencing an undeclared variable fails at load time, not at run time.

The schema file is a JSON Schema. ModelRig validates output under JSON Schema 2020-12 as the primary dialect and also accepts draft-07 (a schema declaring either $schema compiles). Validation runs non-strict with union types allowed, and format is annotation-only — it is not asserted (an email/date-time format is documentation, not a constraint). Author 2020-12 unless you have a draft-07 schema already; both work.

3. Call it


const rig = createRig(loadConfigFromEnv());

const result = await rig.run("example.support_summarize", {
  input: { ticket: rawTicket, product: "Acme Cloud", priorContext: history },
  tags: { run_id: "run-123", step: "summarize", customer: "acme" },
  budget: { envelope: "run-123" },   // hard-stop cost envelope for the run
});

result.output;                 // JSON, already validated against your schema
result.meta.provider;          // which candidate actually served
result.meta.costEstimateUsd;   // priced from the pinned LiteLLM snapshot
result.meta.servedTier;        // vs meta.requestedTier — downgrades visible
result.meta.attemptsByClass;   // retries consumed, per failure class

rig.close();

4. Wrap it in a run — the standard path

Every pipeline execution is a run. A run is the standard way ModelRig records what your workstream did — not an appendix. Wrap the execution in a run context, name one step per model-call family, and save the work products as artifacts; then the console's Runs tab shows the run → step → artifact chain you can click through, and /projects groups it by tag.


const rig = createRig(loadConfigFromEnv());

// One run per pipeline execution. run.scope isolates concurrent runs; use
// run.start()/run.end() for one run at a time on a request path.
await rig.artifacts.run.scope(
  { pipeline: "support", episodeKey: ticketId },
  async () => {
    const result = await rig.run("example.support_summarize", {
      input: { ticket, product, priorContext },
      tags: { subject: customer, feature: "summarize" }, // run_id auto-stamped
      stepKey: "summarize",                              // names this step on /runs
    });
    // Save the work product — prompt/raw/parsed all save the same way.
    rig.artifacts.artifact.save(result.output, { name: "summary", type: "step_output" });
  },
);
rig.close();

The full artifact API — lineage (link), evaluations (evaluate), the reusable per-step seam, and run.scope vs run.start — is the instrumentation guide; this section is the standard path that links to it.

Change your route's model live (no redeploy)

A route's call-config — which model/provider serves it, and its sampling — can be changed while your app runs, without editing YAML or redeploying. You set it in the console (the route's call-config card), or your coding agent sets it for you over MCP (set_call_config, below); the SDK reads the current config just-before-call and merges it over your bundle. Two modes:

Read call-config just-before-call. Build a resolver once (it caches with a short TTL and revalidates by ETag — no per-call network cost when fresh), then overlay the current config onto a bundle and serve it through rig.runBundle():


import {
  createRig, loadConfigFromEnv,
  createApiCallConfigFetcher, createCallConfigResolver,
  getCallConfig,                 // pin-only convenience: route + bundle → runnable bundle
} from "modelrig";

const rig = createRig(loadConfigFromEnv());

// Build once, reuse across calls. TTL is the cache-freshness window (~30–60s).
const resolver = createCallConfigResolver({
  fetcher: createApiCallConfigFetcher({
    ingestUrl: process.env.MODELRIG_INGEST_URL ?? "https://api.modelrig.ai",
    apiKey: process.env.MODELRIG_API_KEY!,   // a read needs no special scope
  }),
  ttlMs: 30_000,
});

// `bundle` is the route's deploy-time bundle you already hold (see runBundle below).
// getCallConfig merges the live PIN over it; with no override it returns the SAME
// bundle reference — byte-identical to today.
const runnable = await getCallConfig("example.support_summarize", { resolver, bundle });
const result = await rig.runBundle(runnable, { input, tags });

Fail-open, always. The read never throws: on any error it serves the last-known-good config, and a route with no override resolves byte-identical to the deploy-time bundle (the same bundle reference, unchanged). Turning this on adds no failure mode to the hot path.

Where it applies. Call-config is overlaid onto a bundle you serve through rig.run() / rig.runBundle() — not rig.runRaw. The raw lane is BYOK and has no route or bundle to overlay, so a routeless raw-lane caller that wants live routing must first own a route/bundle (build one with bundleFromResolved, or resolve one from disk) and serve it through runBundle.

A/B — draw an arm locally, record it. For mode='experiment', resolve the config and draw the arm for this one call with resolveCallConfigForRun, then thread the arm name onto rig.runBundle(..., { arm }) so it lands on inferences.meta.arm:


const resolved = await resolver.getCallConfig("example.support_summarize");
const { bundle: runnable, arm } = resolveCallConfigForRun(bundle, resolved, {
  input,                         // stickyKey (if set) reads its unit from here…
  // unitKey: userId,            // …or pass the unit key explicitly (sticky assignment)
});
const result = await rig.runBundle(runnable, { input, tags, arm });
// arm is null for a pin / no-override / non-experiment call ⇒ no meta.arm (byte-identical).

Sticky assignment means the same unit always draws the same arm, so the A/B is measured honestly. The console reads per-arm outcomes at GET /v1/console/ab-readout?route=<route>. How meta.arm is populated and overlaid is in the instrumentation guide.

Set it from your coding agent (MCP). The modelrig-oracle MCP server exposes get_call_config, set_call_config (pin or experiment), and rollback_call_config, so the agent in your editor can pin a model or start an A/B on your behalf — a real, versioned, reversible control-plane change (not a draft). Writes need a rig_sk_ key with the route-config scope; reads need none. The full flow, the eligibility gate, and the rollback contract are in The MCP oracle → Setting a route's live config.

When it fails, it fails typed


import { RigFailureError } from "modelrig";

try {
  await rig.run("example.support_summarize", opts);
} catch (err) {
  if (err instanceof RigFailureError) {
    err.failure.class;   // content_invalid | capacity_shed | network | refusal
                         // | cache_invalid | timeout | config_auth
                         // | budget_exhausted | invariant_violation
                         // | quality_rejected
    err.failure.fixHint; // machine-readable remediation when available
  }
}

Retry budgets are per class and non-fungible — exhausting network retries never consumes the content_invalid budget. Terminal classes (budget_exhausted, invariant_violation) are never retried. config_auth (bad or missing credentials) is non-retryable by contract: it cannot appear in a bundle's retry budgets, is never retried on the same candidate, and the run loop advances straight to the next candidate — a dead key never burns backoff time. Failures that still billed tokens (refusals, truncations) carry their token usage, so envelopes and telemetry account the real spend.

Each class — what it means, whether it retries, and how it backs off — is on Routing & reliability; how the whole candidate ladder runs (ordering, validate → repair → fall-through, and what is not built) is on that page too.

Where the telemetry goes

Every attempt (not just every run) writes one row to package-owned SQLite, tagged with your tags — always, with no key. Setting MODELRIG_API_KEY turns on api export mode: a background exporter ships batches to the ingest API (api.modelrig.ai, override with MODELRIG_INGEST_URL), which the console reads — fire-and-forget, queue-on-failure; the inference path never blocks on it. Inspect live rows in your local SQLite file, or in the console once the key is set — never a database URL of your own.

MODELRIG_API_KEY gates the exporter per process. Set it on EVERY service, worker, and cron that constructs a rig — a process without it buffers to its own local SQLite and ships nothing. A multi-service deployment that sets the key on only the web tier silently loses every row a worker or cron produced.

Turning Optimization on for a route (capture: true) is the recommended default — your traffic is what makes the routing smarter and the bill smaller, and everything it keeps you can see, export and delete in the console. Keeping the prompts, outputs and evidence too is content custody, which is on by default — every new org is created managed, having accepted the published terms at signup — and captures into your managed store, scrubbed for the PII/PHI you classify. Either way the opt-out is one account setting (the console or the set_org_settings MCP tool): set posture off to capture nothing or metadata to keep hashes only, or use a zero_retention route. Pure router (capture off) stays real and selectable, at the same flat 2%.

Telemetry says what happened; a grade says whether it was good. Attach a 0–1 score (up = 1, down = 0) to a call or a whole run with rig.grade(...) — from the SDK or over HTTP for a non-SDK surface (see the grade protocol) — and every run gets an automatic run-outcome@v1 grade, with an explicit, bounded model judge available when you want one (see the grade protocol).

Runtime requirements

ModelRig assumes a long-lived Node process with a writable disk, and a few things follow from that. Check them before you deploy:

When it doesn't work

The first three failures below stop the bundle loading; the fourth lets it load and then misbehave, so it is the one to watch for. The full list, in the loader's own words, is in route-bundles.md.

Still stuck? Open an issue at <https://github.com/modelrig/modelrig/issues>.