Migrate a codebase to ModelRig — the T0–T2 playbook

This is the canonical playbook the ModelRig skill follows when you import it into your coding tool (Claude Code, Cursor, Codex, …) and ask it to onboard a codebase. It is a pointer skill: the skill file itself holds almost no facts and fetches this page, so one copy serves every ecosystem and a stale cached answer never overrides the current one.

The skill performs the full migration, not just detection: it finds LLM call sites, instruments work-product boundaries, and — one call site at a time — replaces direct provider calls with ModelRig routes. It does this on a four-tier autonomy ladder that treats behaviour-preserving work differently from behaviour-affecting work, and it keeps model optimization permanently out of its own hands.

The skill flags; it never forces. Where a call site does not fit a tier, the skill says so and routes it to the correct, lower tier — a raw-lane outcome is a correct result, not a failure. See "Stated limits" below.

The autonomy ladder

The dividing line is risk class, not effort: T0 and T1 preserve behaviour, so the agent applies them after a plan review; T2 changes what serves a call, so the agent only proposes it, proves it, and a human merges it. T3 is off the ladder — the skill's job ends at producing the evidence; a model swap is a separate, human-gated bake-off decision (see Tier 3).

Before you start — the one human handoff (F-6)

Everything upstream of the console an agent can do now. The one step it cannot is minting an API key, which needs a console login at <https://app.modelrig.ai> (/keys) — account/password auth. Plan that handoff rather than stalling on it:

export MODELRIG_API_KEY=<minted at app.modelrig.ai/keys by a human, once>
# The default ingest host (https://api.modelrig.ai) needs no other config.

A key is not required to run routes locally — it only decides where telemetry lands. But observing your runs in the console (Tier 1's payoff) does need it.

MODELRIG_API_KEY gates the exporter per process. Set it on EVERY service, worker, and cron that constructs a rig — a process without it buffers to its own local SQLite and ships nothing. In a multi-service deployment, the web tier alone having the key means every row a worker or cron produced is silently lost.

---

Before the ladder — which lane, and the two inventories

Read Routing & reliability first. It answers the capability questions — what falls through, what is stored per attempt, what is not built — so you never need to read the engine source to migrate correctly.

The autonomy ladder (T0–T2) is how you migrate one call site. The lane is which strategy fits the whole codebase — decided once, up front, from the shape of the code. Get this wrong and you either over-engineer a naive app or try to externalize a routing engine that already exists. You (the coding agent) work the lane assessment and the inventories yourself, then stop at the checkpoints (⏸) and present to your human.

The three lanes

• Lane A — naive call sites. Static prompts, a fixed model per call, no provider abstraction. This is the greenfield case the autonomy ladder was written for: extract routes (Tier 2), replace call sites with rig.run. First route in under an hour.
• Lane B — an existing router / orchestrator. The code already has a provider abstraction, runtime-assembled prompts, and env- or logic-resolved model choice (a callProvider(provider, prompt, model) dispatcher is the tell). **Do not write per-task routes for this.** Migrate ONE naive call site as a reference route, then integrate the router at the rig.runRaw adapter seam — replace the dispatcher's body with `rig.runRaw({ provider, model, apiKey, systemPrompt, userPrompt, … }). runRaw` is BYOK — pass your provider key; ModelRig holds it for the one call and never stores it. Already assemble one combined prompt string? Put it ALL in userPrompt and leave systemPrompt empty. ModelRig provides telemetry, typed failures, and accounting; your code keeps prompt assembly and cache lifecycle. The raw lane has no candidate ladder — a failure is yours to handle, surfaced as a typed class. BYOK, zero-margin, no route externalization.

runRaw-only construction (routesDir: null) — 0.4.0+. A Lane-B deployment that only ever calls runRaw/runRawStream has no routes. Construct the rig with routesDir: null so createRig skips route loading, the rig.yaml manifest, and the serveability gate — a stray route file it never calls can't down it, and the empty-temp-dir workaround is gone:

``ts const rig = createRig({ ...loadConfigFromEnv(), routesDir: null }); await rig.runRaw({ provider, model, apiKey, systemPrompt, userPrompt }); ``

loadConfigFromEnv() always yields a string routesDir (there is no env var that disables routes, by design) — the opt-out is the deliberate code-level override above. rig.run and every route-dependent surface then throws a terminal invariant_violation naming the fix; keep a reference route (a routed rig) if you also need run/bake-offs.

• Lane C — full route migration of a Lane-B system. Externalizing a dynamic prompt assembler into route templates is a scoped project (prompt externalization), never an onboarding step. Recognize it, name it, and leave it for later.

The lane picks which tiers apply: Lane A runs the full T0→T2 ladder; Lane B stops at the runRaw seam (T0/T1 plus the one reference route) and defers T2-for-everything to Lane C.

⏸ C1 — Fit assessment (present the verdict + lane)

Inventory the LLM call sites and classify the codebase (naive vs. router layer). Present which lane you chose and why. See "Call-site scope" for what counts.

⏸ C2 — Caching, grounding & sampling inventory (the stop-gate)

Before writing any route, grep the target for provider caching, provider-native grounding, and explicit sampling parameters — the three behaviours that vanish silently on a naive migration (a cost regression, a quality regression, and a consistency regression that only telemetry reveals). Search for:

cachedContent        # Gemini explicit cache resource
cache_control        # Anthropic marker cache
prompt_cache_key     # OpenAI / Grok key-hint cache
cached_tokens | cache_read | cachedContentTokenCount   # cache usage in responses
temperature | top_p | topP | max_output_tokens | maxOutputTokens | max_tokens
                     # explicit sampling — a classifier at temperature 0.1 is a
                     # deliberate choice, not a default
useGoogleSearch | googleSearch | grounding             # provider-native grounding
serviceTier | service_tier                             # flex / priority service tier
thinkingLevel | thinkingConfig | reasoning_effort      # per-step reasoning effort

Any hit is a STOP-gate. The route must carry the corresponding surface:

• Caching → the customer cache handle rides `rig.run(task, { cache: { key, provider } }) (or rig.runRaw({ …, apiKey, cache: { key } })`) — a one-field pass-through of the resource the customer already owns. ModelRig prices the hits but never manages the resource; a long job still needs the customer's TTL heartbeat. Full mechanics: Caching lifecycle.
• Grounding → on a route, require: [grounded] plus the route's grounding mode. On the Lane-B runRaw seam, pass grounding: { mode: "native" } for provider-native web grounding — gemini (Google Search); grok is fail-closed for now (xAI retired Live Search; its Responses-API web_search is tracked as CE-9b). A grounded step that also caches must bake the googleSearch tool into the cache resource (see the caching-lifecycle doc).
• Service tier → on the raw lane, serviceTier: "standard" | "flex" | "priority"; meta.servedTier reports what actually served, and a flex shed surfaces as capacity_shed for your own retry ladder. The raw-lane timeoutMs default is tier-aware since 0.5.0 (standard/priority 300 s, flex 900 s) — **flex ⇒ set timeoutMs in minutes** (default 900 s), and a timeout failure teaches the tier + budget in its fixHint. §3.2 flex degrade is customer-side: ModelRig does NOT replicate the gemini direct-path infra-retry wrapper on the raw lane, so a PIPELINE_SERVICE_TIER=flex capacity rejection is yours to retry at "standard" — a NAMED difference for your migration record, not a silent one.
• Reasoning / thinking → on the raw lane, reasoning: { level: "minimal" | "low" | "medium" | "high" }. Since 0.4.0, this generalizes beyond gemini WITHOUT a per-model allowlist — openai/deepinfra/fireworks forward the level as reasoning_effort and let the provider enforce (a rejection is classified teachably); grok coarsens it (minimal,low → low; medium,high → high); gemini stays thinking-model-gated. DeepSeek reasoning is model selection, not a knob: there is no effort control — select the deepseek-reasoner model and DON'T declare reasoning (it fails closed with exactly that fixHint). anthropic extended thinking is unwired (fails closed).
• Response format _(0.4.0+)_ → on the raw lane, responseFormat: "json_object" forces syntactic JSON on the OpenAI-dialect providers + gemini. This is MIME-shaping, NOT schema enforcement — the raw lane still never validates or repairs (F4), so meta.validated stays false; anthropic fails closed. On DeepSeek the prompt must contain the word "JSON" (the caller's responsibility on the raw lane).
• Which provider honors which knob is the single-source provider × knob matrix — the runtime guard and that table are generated from the same RAW_KNOB_SUPPORT. A knob a provider cannot honor FAILS CLOSED (invariant_violation, pre-dispatch) — never a silent drop. Read rawKnobSupport(provider) in code instead of duplicating the lists; GEMINI_THINKING_CAPABLE_MODELS is exported too (so you stop copying the §3.1 thinking-model set).
• Sampling → declare policy.sampling on the route: ```yaml policy: sampling: { temperature: 0.1, top_p: 0.9, max_output_tokens: 8192 } ``` All three fields are optional; carry over exactly what the original call set. Absent, every adapter keeps its own defaults (Gemini runs at temperature 1.0), so a call that relies on a specific temperature — a classifier at 0.1, say — MUST declare it or it silently changes behaviour. A declared value is sent to the provider as-is (no clamp); an out-of-range value is a load-time config error (temperature 0–2, top_p (0,1], max_output_tokens ≥ 1). Note for Gemini 3: Google recommends temperature 1.0 and warns sub-1.0 values can loop/degrade on complex reasoning tasks — the loader logs an advisory for a sub-1.0 Gemini-3 temperature and sends the request as declared (classification/extraction are typically fine).

If a surface cannot express what the code does yet, report the gap and the quantified cost/quality delta — never migrate past it silently. Present the inventory results and the decision (carry the handle, or a named, quantified regression). Silent is the only forbidden outcome.

The rig lifecycle (learned the hard way in the first Phase-0 swap)

• Create once, close always. One rig per server process (module singleton); per-invocation for one-shot jobs — and a one-shot caller MUST await rig.close() before exit, or telemetry rows are lost and the SDK's timers keep the process alive.
• tags is required; declare each tag key you rely on as a dimensions[].key in rig.yaml (tag-safe [A-Za-z0-9_]{1,64}) so the console can attribute cost by it. A required: true dimension missing from a run's tags warns once per process (and under-attributes that run) — it is never rejected. An example that shows tags: { run_id } assumes a run_id dimension exists.
• result.output is already parsed and schema-validated — delete the old call site's JSON.parse and fence-stripping rather than keeping both.
• Routes resolve from ./modelrig/routes relative to the process CWD. In a monorepo, place modelrig/ at the app root and run from there, or set MODELRIG_ROUTES_DIR to an absolute path.
• Load-check with modelrig validate — no key required. It structurally loads rig.yaml and every route bundle, prints per-route OK / config error, and exits nonzero on any failure — the dry authoring check to run before you wire any credential. (Serving still fail-closes keyless: a live createRig needs at least one provider key so a route has a serveable candidate.)

Package manager (detect, don't assume)

Install with the project's OWN package manager — read it from the lockfile: pnpm-lock.yaml → pnpm add modelrig; yarn.lock → yarn add modelrig; package-lock.json → npm i modelrig. Running npm i in a pnpm workspace corrupts the store — never hard-code npm. Node ≥ 20; provider keys stay in env vars.

Call-site scope (what "every LLM call" means)

• In scope: text/JSON generation calls, including the secondary extractor/repair follow-ups that are part of a generation flow (a JSON-extractor retry on a malformed response is in scope — it is the same logical call).
• Out of scope (v1, state it explicitly): embeddings, web-search tool calls, and rerankers. These are not generation routes; say so rather than guessing.

⏸ C3 — Candidate ratification · ⏸ C4 — Prove it

C3 (ratify the proposed candidate list) and C4 (tests + diff + the week-one cache-hit-rate/cost panel comparison) are the Tier 2 gates below — the human merges on a green report. A zero cache-hit rate on a route that cached before migration is a regression; say so at C4.

The cache-ratio-vs-baseline metric is a REAL-run measurement, not a probe. The cached-token ratio a migration must preserve comes from the pipeline running its actual traffic (a warm cache, real prompt prefixes) — a single-call probe cannot produce it and never closes this gate. At the checkpoint, mark cache-ratio-vs-baseline deferred-to-first-run and read it off the week-one telemetry panel; do not report it "verified" from a probe.

C3 — unprobed models are SERVABLE (ruled 2026-08-20). Probes gate claims, not serving: any model an adapter can reach may be a candidate, probed or not. Pin the model production uses today as candidate #1 — always, including models the registry has never probed — and never silently substitute a probed sibling for the one production runs. What UNPROBED actually means for a candidate (present these at C3, they are the honest cost of no evidence, not a punishment):

• json: native may NOT be claimed for it (that is a probed-capability claim; the emulation ladder serves structure instead);
• no leaderboard standing, no bake-off priors, no declared-vs-probed discrepancy protection;
• pricing may be missing until the next pricing sweep — an unpriced model charges budget envelopes at the conservative rate (never rides free).

Flag each unprobed candidate UNPROBED in the C3 presentation and offer the follow-ups: request a probe (probe-request page — customer traffic on a model is exactly how it earns a place in the next probe cycle), and/or a probed sibling as candidate #2 for the fallback slot. A third follow-up when you want a model but do not yet trust it live: declare it trial: true (the console's declare-first picker writes the YAML) so it is never served live, then bake it off on captured traffic and promote the winner — evidence before exposure, no leap of faith on an unprobed model.

C4 decision — JSON-mode originals. If the original call used mime-type JSON plus app-side validation, declaring a schema: on the route is an UPGRADE, not identity: the provider then enforces the schema and ModelRig's repair ladder engages. Usually the right call — but it is a deliberate behaviour change, so present it at C4 as one ("schema enforcement added"), or set policy.json: json_mode with schema: null for closest identity. Which wins is the human's ratification, never a silent default.

⏸ C5 — Keys & billing handoff (the setup a human must finish)

Everything above an agent does; placing keys and a payment method is the one part it must hand to a human. Do not stall on it and do not do it for them — present exactly what to place where, for the lane you chose at C1.

The hard rule (non-negotiable): you never touch a key VALUE. Not read beyond an existence check, not print, not paste into the console, not put in a manifest. Humans place keys directly. You check only whether a key EXISTS, by NAME:

# Existence only — prints the NAME and whether it is set, never the value.
for v in GEMINI_API_KEY OPENAI_API_KEY ANTHROPIC_API_KEY; do
  test -n "${!v}" && echo "$v: set" || echo "$v: MISSING"
done

Inventory the providers your written routes actually use (from the ratified candidates), then present lane-aware instructions:

• SDK lane (Lane A/B): keys stay in the customer's own env. List the exact env var each provider needs and which are missing. Say plainly: in the SDK lane ModelRig never receives your provider key — the key stays in your env and calls go direct to the provider.
• Hosted lane: keys go in the console vault at app.modelrig.ai → Keys (/vault), placed by the human through the console form.

State the pricing fact at this decision point: BYOK is 1,000,000 free requests per month (you bring the provider key; ModelRig meters, at zero per-request fee up to that ceiling). A payment method (app.modelrig.ai → Billing) is only needed for metered usage beyond the free tier.

Then, and only if MODELRIG_API_KEY is set, offer to pre-fill the human's console checklist with a metadata-only manifest — provider NAMES and booleans, never a key. Show the EXACT payload first and require an explicit yes:

# SHOW this to the human, ask "send this to pre-fill your console checklist?
# (metadata only — no keys)", and send ONLY on an explicit yes.
curl -sS https://api.modelrig.ai/v1/setup-needs \
  -H "authorization: Bearer $MODELRIG_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "manifest_version": 1,
    "lane": "sdk",
    "providers": [{ "provider": "gemini", "has_env_key": true }],
    "routes_count": 2,
    "needs": ["provider_key:gemini", "billing_card"],
    "notes": null
  }'

The server rejects any manifest carrying a secret-shaped value (400 + a teaching error) — a backstop, not a licence to send one. If MODELRIG_API_KEY is not set, do not send anything — print the instructions above and stop. On the human's next console login, the "Finish your setup" card shows the same items with live done/pending states (a key placed in the vault flips its item to done).

---

Tier 0 — Gateway pointing (transport-only)

The lightest possible change: point an existing OpenAI-compatible client at the ModelRig raw lane by swapping its base URL. Nothing about the request or the model changes — the traffic just flows through the gateway, so telemetry and cost accounting light up instantly with zero behaviour risk. (The raw lane is BYOK and carries no spend envelope — the org daily envelope caps only the managed spend ModelRig fronts, not your own key.)

Because it is transport-only, the agent applies it directly after a plan review. This is the correct home for call sites that resist template extraction (dynamically-assembled prompts) and for freeform chat / agentic loops — see "Stated limits".

Tier 1 — Instrumentation (behaviour-preserving)

Record each pipeline run as a run → step → artifact chain — the prompt, the raw model text, the parsed output, the evidence it grounded on — with lineage and an integrity hash per artifact. This is behaviour-preserving: the SDK contract is that instrumentation never changes a return value, and with the namespace off the pipeline is byte-for-byte identical.

Scaffold it with the CLI, then apply the emitted diffs:

modelrig observe [path] --scope <one pipeline>   # or: modelrig init --observe

modelrig observe generates a fresh, self-contained record-step seam and an OBSERVE.md whose per-call-site wiring is emitted as diff blocks. The instrumentation it produces is a code-level SDK seam — there is no zero-code observe — but you no longer hand-write it. The agent plans before editing, then applies (behaviour-preserving), and the validator warns when a stage is instrumented that should not be. Detect existing OpenTelemetry and extend it — never duplicate a span the customer already emits.

Full mechanics: Observe a pipeline.

Tier 1 is not optional for a Lane-B seam. A pipeline that reaches the model through rig.runRaw without a run.start around it records inferences but no run — /runs stays empty and /projects shows "(no run)". Open the run context around each pipeline execution (rig.artifacts.run.start({ pipeline, episodeKey }) … run.end()); inside it the SDK names the step and stamps the run_id tag — and the step tag too (a value you pass always wins) — you tag subject / feature yourself. Do not declare a migration done until one real execution shows a run with steps on /runs.

Tier 2 — Route extraction + same-model call replacement (behaviour-affecting)

Turn a direct provider call into a ModelRig route: the prompt becomes a systemTemplate + declared variables, the schema comes from the call site's Zod / TS types (or schema: null when it asked for no structure), and the call site becomes rig.run(<route>, …). The replacement is always same-model — the route pins the exact provider/model the original used. This is the C2-exit gateway-substitution ruling: T2 changes the plumbing, never the model. A model change is Tier 3, and never the skill's.

Because this is behaviour-affecting, the agent proposes, never applies:

1. Draft the route. modelrig init --scope <one pipeline> finds the call site and drafts a route bundle + a MIGRATION.md line for it. It writes only under modelrig/routes/drafts/ and never edits source. (A route bundle is validated at load — missing schema/prompt files, undeclared {{variables}}, and a bad json: native claim all fail loudly, so a drafted bundle either loads clean or tells you exactly what is wrong.) 2. Verify before proposing a merge. The bar is: - type-check passes, - the customer's own tests pass, and - modelrig verify-swap returns a passing shadow-equivalence report: ``bash modelrig verify-swap --route <route> --captures <fixture.json> --out pr-report.md ` It captures N calls of the original locally, replays the SAME inputs through the drafted route with the original model pinned, and diffs the outputs. Its verdict vocabulary is honest: EQUIVALENT, EQUIVALENT (modulo nondeterminism), DIVERGENT (do not merge), or REFUSED (not a T2 replacement — T0/T1 territory). Exit code is the gate: 0 pass, 1 divergent, 2 refused. 3. Embed the report in the PR. Paste the pr-report.md the verifier wrote verbatim into the PR body. It carries the plumbing-not-model-choice guarantee so a reviewer cannot mistake equivalence for a model-quality claim, and it states rollback in the same breath. 4. One call site per PR. This keeps git revert` a byte-for-byte restore and keeps the diff inside the customer's codebase legible — a product surface. 5. A human merges. The skill's autonomy ends at a green report and an open PR.

Replaced-call disposition (RULED, Matt 2026-08-16): DELETE the original, never comment it out

Rollback is git-native and the skill makes it explicit:

• (a) one call site per PR means git revert restores the original byte-for-byte;
• (b) the PR body quotes the original call verbatim with an explicit Rollback: revert this PR line;
• (c) a one-line breadcrumb comment stays at the site: ```ts // modelrig: migrated from <original call> — rollback: revert PR #N ```

Commented-out originals are banned — they rot against the live call, and our diffs inside customer codebases are a product surface. Because the replacement is same-model, behavioural rollback is additionally a route-YAML flip, not a code operation.

Tier 3 — Model optimization: NOT the skill's job

Choosing a cheaper or better model is permanently out of the skill's hands. The skill's T2 output — routed call sites with same-model parity — is precisely the evidence a bake-off needs. From there, optimization stays where it already lives: modelrig bakeoff replays your own traffic through a challenger, a proposal is raised, and nothing switches what serves a route without an explicit human approval (modelrig swap execute <id> --approve). "Actions live in your code" — the swap path emits route YAML, it never edits your source. The skill recommends; it does not rewrite your model choices.

Stated limits (the skill flags, never forces)

• Dynamically-assembled prompts resist template extraction → Tier 0/1 treatment (instrument or point the gateway; do not force a brittle template).
• Freeform chat / agentic loops don't fit task-shaped bundles → raw lane (a correct outcome, not a failure).
• Streaming call sites are recognized and routed per the batch-first deferral — not silently converted.

Each of these is a call the skill surfaces to you with a reason, never a silent downgrade.

Recognition playbooks (per provider)

How the skill recognizes a call site and what the before/after looks like — for OpenAI, Anthropic, the Vercel AI SDK, LangChain, and raw fetch — lives in the recognition playbooks, with canonical before/after examples for each.

Quick-reference gotchas

The nine things a migrating agent gets wrong from memory. Each is stated in full in the tier above; this is the cheat sheet to re-check before proposing a diff.

• Routes resolve from the process CWD (./modelrig/routes) — in a monorepo, put modelrig/ at the app root and run from there, or set MODELRIG_ROUTES_DIR.
• await rig.close() before the process exits, or telemetry rows are lost and the pending flush timers hang the process.
• json: native is a probe-gated CLAIM, not a preference — an unprobed candidate cannot assert it; the emulation ladder still serves structure.
• **The googleSearch grounding tool must be baked into a cache resource at creation** — it cannot be attached to an already-created cache handle later.
• Gemini defaults to temperature 1.0 (other adapters use their own defaults) — carry an explicit classifier temperature over exactly, or it silently changes.
• The raw lane is BYOK: pass the provider key on the apiKey field of the runRaw call; ModelRig never receives or stores it.
• A combined system+user prompt goes in userPrompt on runRaw, not split across separate fields.
• tags require declared dimensions: declare each tag key you rely on as a dimensions[].key in rig.yaml (tag-safe [A-Za-z0-9_]{1,64}). A required: true dimension missing from a run's tags **warns once per process and under-attributes that run on /costs** — the call is not rejected, so the only signal is the warning.
• Knob support lives in the matrix, not memory: which provider honors reasoning / grounding / responseFormat comes from rawKnobSupport() and the raw-lane support matrix — never assume.