Route bundle reference

One YAML file per route under the routes directory (default modelrig/routes/). Files are loaded, structurally validated, and frozen at createRig() time; any invalid bundle throws a RouteConfigError naming the file and every problem.

Top-level fields

dimensions live in modelrig/rig.yaml, not the bundle. Project-level cost dimensions ({ key, label, required }) that back /costs and the required-tag hygiene warning are declared once in the rig manifest — see Tags, dimensions, and the graph.

Prose and markdown output (output: text)

By default a route's output is JSON: the response is JSON.parsed, then checked against schema when one is declared. A route whose task returns prose — an editorial pass, a memo, a markdown report — declares output: text instead. The response is then returned verbatim as result.output (a string, untrimmed): no JSON parse, no schema, no json-mode coaching sent to the model, and the JSON repair rung is never entered. A prose route is therefore never rejected as content_invalid for "not being JSON" (the 2026-10-11 incident shape — output is not valid JSON: Unexpected token '#' — on a route that was correct about its model and wrong about its contract).

# modelrig/routes/report.editorial_polish.yaml — a prose route
route: report.editorial_polish
version: 2
output: text                      # ← prose/markdown, returned verbatim (no schema:)
candidates:
  - provider: gemini
    model: gemini-3.1-pro-preview
require: []                       # no schema ⇒ never schema_conformant
prompt:
  system: ./prompts/report.passthrough.system.md
  variables: [assembled_prompt]
policy:
  retries: { content_invalid: 1, network: 4, capacity_shed: 3 }
  timeout_ms: 600000
  tier: flex
  sampling: { max_output_tokens: 65536 }

The only check a prose route may declare is a length floor:

output: { kind: text, min_chars: 400 }   # a response shorter than 400 chars (untrimmed) is content_invalid

A min_chars miss is a real, billed content_invalid attempt: it is retried from the content_invalid budget (same candidate, then the next) and carries a fixHint; it is never sent for JSON repair. With no min_chars, any string — including an empty one — is accepted verbatim; use a qualityGate or grades for judgement beyond length. Quality gates receive the string; masking rehydrates it; capture, zero_retention, grounding citations and the run graph all behave exactly as for JSON routes.

Rules the loader enforces (each a RouteConfigError that names the fix):

• output: text and schema: — a prose route has no JSON Schema to conform to; remove one.
• output: text with require: [schema_conformant] — there is no schema.
• output: text with a repair: block (route- or variant-level) — repair is the JSON repair rung and would never run; remove it.
• output must be json, text, or { kind, min_chars? } (min_chars an integer ≥ 1, text only). Unknown keys are errors, so a typo cannot silently serve JSON.

output: json may be written explicitly; it is identical to omitting the key, and an existing route's bundle (and bundle_sha) is unchanged by this release. Flipping a route to text changes its bundle_sha, so the hosted mirror and any call-config pin see the new contract; a mirrored text route is validated as prose on the hosted lane too (/v1/chat/completions returns the string as the assistant content). A JSON route that receives prose keeps today's content_invalid and now carries a fixHint pointing here. Every dispatched route-lane attempt stamps the contract it was held to on inferences.meta.output_kind (json | text); raw-lane rows carry no such key.

JSON Schema dialect

The schema file is validated with Ajv, configured for JSON Schema 2020-12 as the primary dialect with draft-07 also accepted — a schema declaring "$schema": "…/2020-12/schema" or "…/draft-07/schema" both compile (the draft-07 meta-schema is registered explicitly). Author 2020-12 for new schemas; existing draft-07 schemas keep working.

Two properties of the configuration are load-bearing when you author a schema:

• Non-strict, union types allowed. Unknown keywords do not fail compilation, and "type": ["string", "null"] is accepted.
• format is annotation-only — it is NOT asserted. Neither dialect enforces format without ajv-formats, which is not registered, so an email, uri, or date-time format documents intent but never rejects a value. If you need a value constrained, express it with pattern / enum / minimum / maxLength, not format.

The gateway and the probe harness (packages/modelrig-probes) build an identical Ajv instance by design, so a fixture scores exactly as it serves.

policy

policy:
  timeout_ms: 60000
  tier: standard
  json: native
  sampling: { temperature: 0.1, max_output_tokens: 8192 }   # preserve the original call's sampling

Gemini 3 note. Google recommends temperature: 1.0 for Gemini 3 and warns that sub-1.0 values can loop / degrade on complex reasoning tasks (classification and extraction are typically fine). Declaring a sub-1.0 temperature on a Gemini-3 candidate loads and is sent as declared, with a load-time advisory naming the model and value.

Variants (bundle format v2 — Phase 3)

A route may declare variants — named bindings served via rig.run(route, { variant: "name" }). Omitting variant (or passing "default") serves the route's base config exactly as v1 did; v1 bundles remain valid unchanged. Every telemetry row records served_variant.

variants:
  - name: cheap
    candidates:               # narrowing ONLY — must be ⊆ the route's list
      - provider: openai
        model: gpt-5.4-mini
    json: json_mode           # rung override for this variant
    repair:
      max_repairs: 2
      repair_model: openai/gpt-5.4-mini
  - name: scaffolded
    scaffold: |               # reasoning scaffold, appended before coaching
      Think through the drivers step by step before emitting JSON.
  - name: reworded
    prompt_override: ./prompts/echo-v2.system.md   # inline text also accepted

repair (the repair rung of the candidate ladder — Phase 3)

repair:
  max_repairs: 2                     # 1 = retry-with-errors only; 2 adds the repair model
  repair_model: deepseek/deepseek-chat   # required when max_repairs is 2; must be a declared candidate

This is the repair rung of the candidate ladder — the ladder's rungs (resolve → render → dispatch → validate → repair → fall-through) are named in full on Routing & reliability.

On a schema-invalid output with repair enabled: attempt 1 re-asks the SAME model with the ajv error summary appended; attempt 2 hands {invalid output, errors, schema} to the designated repair model with a fixed repair prompt. Repair is JSON-only: an output: text route cannot declare it (load error) and never enters this rung. Repair attempts draw from their own repair budget (never the content_invalid budget), and repaired rows carry repaired_by in telemetry — repair cost is visible, not hidden.

Requirement → capability mapping (registry-wired since Phase 3)

Capability flags are resolved from registry facts (the packaged registry/registry.json, env-overridable via MODELRIG_REGISTRY_PATH) with per-flag precedence capabilityOverrides > probed > declared — a probed-false trumps a declared-true (DeepSeek's declared schema support is revoked because every probed sample served via json_mode coaching). Models absent from the registry fall back to the adapter-static baseline below. This table is the registry-absent fallback for gemini, openai, and deepseek; a provider not shown here falls back to its own adapter's declared capability set.

Overridable per provider via RigConfig.capabilityOverrides (replaces the whole flag set for that provider — useful for pinning a route to one candidate in tests).

Templates

• {{var}} — substituted from RunOptions.input. Strings verbatim; objects and arrays are JSON.stringifyed; missing values throw.
• {{#if capability.X}}…{{/if}} / {{#unless capability.X}}…{{/unless}} — resolved against the SERVING candidate's flags before variable substitution. Flat only (nesting is a load-time error); unknown flags are load-time errors.
• The rendered prompt.system template is sent as the system prompt. The user turn is a fixed terse trigger — unless the route declares prompt.user, in which case its rendered template is the user turn (next section). With a Gemini cache handle on the call the system slot is unavailable and everything rendered is delivered as the user turn (caching lifecycle).

Per-call user turn (prompt.user)

By default the user turn of every attempt is a fixed sentence ("Produce the required output now…") and the whole task lives in prompt.system. A route that needs per-call content in the user turn declares a second template:

prompt:
  system: ./prompts/analyst.system.md      # optional once prompt.user is declared
  user: ./prompts/analyst.user.md          # rendered per call: "Analyse {{ticker}} using {{step_instructions}}"
  variables: [ticker, step_instructions]   # ONE declared list, shared by both templates

• Same renderer as prompt.system: {{var}} substitution, {{#if capability.X}} blocks, and x-trust: untrusted removal (an untrusted variable renders empty in either template and is delivered once via the spotlighted untrusted block, which is appended to the user turn).
• The rendered user template replaces the fixed trigger on every primary attempt; a same-model repair re-ask sends it followed by the repair directive. The repair-model path (which re-frames invalid output + errors + schema) is unchanged.
• prompt.system may be omitted when prompt.user is declared (the system template is then empty and no system instruction is sent).
• Undeclared {{var}} references in prompt.user are load errors, as for the system template; an empty prompt.user file is a load error.
• Declaring prompt.user changes the route's bundle_sha (the mirror and any pin see the new contract); userTemplate rides the resolved bundle so the hosted lane renders the same turn.
• Why this exists: a Gemini cache handle (`rig.run(…, { cache: { key, provider: "gemini" } })`) forbids a system instruction on the request, so a cached route carries its per-call delta here while the cache holds the shared prefix — see caching lifecycle → A cache handle changes where the template is delivered. Every attempt records how its prompt reached the model on inferences.meta.prompt_delivery (system \| user \| user+cache).

Versioning discipline

The (route, version) pair keys the compiled validator cache, telemetry rows, and the console's routes mirror. Editing a bundle without bumping version makes telemetry lie across the change — bump every time.

When it doesn't work

These are the load-time and first-run failures a bundle actually hits, in the loader's own words. The first three fail LOUDLY (the bundle won't load); the last fails SILENTLY (it loads and then behaves unlike the code it replaced) — which is why it is the one to be most careful about.

• schema file not found: <path> — schema: names a JSON Schema file that isn't at that path. Paths resolve relative to the bundle file. Create the file there, or set schema: null for a genuinely free-form task.
• prompt.system file not found: <path> — same shape for the prompt template. Point prompt.system at where the file actually is.
• output is not valid JSON: Unexpected token '#' … (content_invalid, with a fixHint) — the model answered in prose/markdown and the route expects JSON (the default, even with schema: null). If the task IS prose, declare output: text; the response is then returned verbatim and never parsed. If the task is JSON, the model ignored the format — add a schema and let the repair rung / coaching do its job. Each retry of this failure is a billed attempt, so decide before raising retries.content_invalid.
• references undeclared variable "<name>" — the template (or grounding.query) uses a {{name}} that prompt.variables doesn't declare. Add it to the list, or fix the typo. This is caught at load so an empty slot can't ship silently. Note the ONLY conditional blocks the template engine supports are {{#if capability.X}} / {{#unless capability.X}} — a {{#if <variable>}} is not a feature and will not render.
• A wrong json: native — the bundle loads, then the run behaves unlike the code it replaced. json: native is a claim about the MODEL: set it only for a candidate whose registry entry shows probed native strict enforcement (structured_native). Otherwise omit it and let the emulation ladder handle structure. Check the probed registry (llms.txt or the MCP oracle) rather than recalling — declared and probed behaviour differ, which is the whole point.

Still stuck? See the Troubleshooting guide.

Raw-lane defaults (rig.runRaw)

What the raw lane sends when you omit a knob. Each knob is additive — absent ⇒ byte-identical dispatch. The one default with teeth is timeoutMs, which is tier-aware (since 0.5.0): a passthrough should not be shorter-fused than the tier it advertises, so standard/priority target 5 min and flex targets the 15-minute budget ModelRig advertises. An explicit timeoutMs always wins (no pre-dispatch rejection — a short deadline is a choice), and a timeout failure on the raw lane carries a fixHint naming the tier, its budget, and the two fixes (raise timeoutMs, or drop to serviceTier: "standard"). On flex, set timeoutMs in minutes. This table is generated from the RAW_DEFAULTS constant, so it cannot drift from what the runtime resolves.

Raw-lane provider × knob support (rig.runRaw)

Since 0.4.0: reasoning beyond gemini, responseFormat, and the DeepInfra/Fireworks prompt_cache_key mapping. The openai/grok prompt-cache hints were already true on 0.3.0.

The Lane-B rig.runRaw seam forwards a set of provider knobs beyond the core prompt/sampling ones. Each is ADDITIVE — absent ⇒ byte-identical dispatch — and a knob a provider cannot honor FAILS CLOSED pre-dispatch (RigFailureError, class invariant_violation; no meter, no provider call, no telemetry row), never a silent drop. This table is the single source of truth: the runtime guard (assertRawKnobsSupported) and this matrix are both generated from the same RAW_KNOB_SUPPORT table, so what you read here is exactly what the guard enforces.

responseSchema — structured output on the raw lane (validate, never repair). Pass a JSON Schema object and the raw lane forwards it as the provider's native structured-output directive — gemini responseJsonSchema + responseMimeType: application/json, openai and grok response_format: { type: "json_schema", strict: true } (strict needs an object root; a non-object root is warned pre-dispatch and the provider rejects it), anthropic a forced tool call carrying the schema (strict-first) — and validates the returned text client-side with the same Ajv the routed lane uses (2020-12, draft-07 accepted). A miss is a billed, terminal content_invalid: one lane: "raw" row with that class (the provider served and charged) and a thrown RigFailureError whose fixHint names the first errors. Nothing is repaired, retried, or fallen through — there is no ladder on this lane, so meta.validated stays false; if you want a schema miss recovered, graduate the call to a route (policy.retries.content_invalid + a repair rung). Providers whose adapter has no native schema path on this lane (deepseek; the OpenAI-compat hosts deepinfra / fireworks / zai, whose raw profile has no probed structured_native) FAIL CLOSED pre-dispatch — the schema would otherwise be dropped or degraded to json_object. Declare one of responseSchema / responseFormat. The row's meta.schema_sha is sha256: of the canonical (sorted-key) schema JSON, so declared-vs-served drift is a meta query; the schema also joins input_hash exactly as a route schema does. A customer cache.key composes with it (the handle is a Gemini content resource; the schema is request config). SDK-direct only: responseSchema is a rig.runRaw knob; the hosted raw lane (/v1/chat/completions) has no responseSchema and ignores response_format — a schema-checked hosted call means a route (the mirror lane honours a route's schema:).

const r = await rig.runRaw({
  provider: "gemini", model: "gemini-3.1-pro-preview", apiKey,
  systemPrompt, userPrompt,
  cache: { key: cachedContentName },          // optional — composes
  responseSchema: earningsFactsSchema,         // a JSON Schema object
  serviceTier: "flex",
});
// r.output is schema-conformant JSON text, or a content_invalid was thrown.

Fail-closed is discoverable, not a surprise. A knob a provider cannot honor throws RigFailureError invariant_violation PRE-DISPATCH (no meter, no provider call, no telemetry row) — read this table before you call, not after. reasoning has NO per-model allowlist outside gemini. openai/deepinfra/fireworks forward level verbatim and let the provider enforce; a rejection surfaces as a classified failure that names the fix. grok coarsens (disclosed above). deepseek reasoning is model selection (deepseek-reasoner); anthropic extended thinking is unwired. responseFormat is mime-shaping, not schema enforcement. json_object forces syntactic JSON only — the raw lane still never validates or repairs, and meta.validated stays false (F4). responseSchema validates, never repairs. A JSON Schema on rig.runRaw is forwarded as the provider's NATIVE structured-output directive AND checked client-side by the same Ajv the routed lane uses: a miss is thrown as content_invalid (a billed row with that class, fixHint naming the first errors) — the raw lane never repairs, never retries, never falls through, and meta.validated stays false (no ladder stands behind it). The row's meta.schema_sha is the sha256 of the canonical (sorted-key) schema JSON, so declared-vs-served drift is queryable. Declare ONE of responseFormat / responseSchema, not both. §3.2 flex degrade is customer-side. ModelRig does not replicate the gemini direct-path infra-retry wrapper on the raw lane; under flex a capacity rejection surfaces classified (capacity_shed) for your own → standard retry ladder — a NAMED difference for your migration record, not a silent one. destination (managed serving) is RELEASED (GA). The serving-destination column names which providers can be modelrig-served (reached on ModelRig's own key) vs direct only. Managed serving is gated behind a config flag (config.managedServing, default off, set in code — not an env var); it defaults off, so until you enable it every call is direct (ABSENT ⇒ byte-identical).

Ask it in code instead of duplicating this table: rawKnobSupport(provider) returns the row above, and GEMINI_THINKING_CAPABLE_MODELS is the exported set the gemini reasoning gate uses (both from the package root).

What native grounding costs

A grounded call's search surcharge rides its own rung:"search" cost row (model web_search, zero tokens) beside the token row, on rig.runRaw with grounding: { mode: "native" } and on rig.recordCall({ webSearchCalls }). Since modelrig 0.15.0 (GROUND-COST) the rate is model-aware: Gemini 3 and newer bill each search query, Gemini 2.x bills one grounded prompt however many queries it ran. webSearchCost(provider, model) returns the rate below (a WebSearchRate: unit is a WebSearchUnit, "query" or "prompt", plus usd), GEMINI_GROUNDING_RATES holds Gemini's two rates, and webSearchFeeUsd(provider, model, searches) the row's cost. These are list rates: free allowances are not netted out. A Gemini call that reported no query count is billed as 1, marked meta.grounding.search_count_source: "assumed". On rig.runRaw the result's RunMeta.searchCostUsd (and RunMeta.searchCalls) report what that row recorded; it is not in meta.costEstimateUsd, so add it once if you keep your own total. A Gemini call on a cache handle whose resource carries the googleSearch tool writes the row too when the response shows searches, even without grounding.

Hosted proxy — /v1/chat/completions message handling

The OpenAI-compatible proxy accepts a messages[] array. Two rules keep an untrusted caller turn from being mistaken for a trusted instruction:

• Role enum. Each message role must be one of system, developer (normalised to the system slot), user, assistant, or tool. Any other value is rejected before the request reaches the pipeline with a teaching message (HTTP 422, the gateway's validation status) — an arbitrary role is never forwarded.
• Unambiguous turn encoding (raw lane). When a raw-lane request carries a genuine multi-turn transcript, the non-system turns are flattened into the adapter's single user prompt as a **JSON-encoded block — one JSON object per line** ({"role":"user","content":"…"}, `{"role":"tool","tool_call_id":"…", "content":"…"}), not as role: content text lines. JSON.stringify` escapes any embedded newline, so a crafted turn whose content contains `"\n\nsystem: …" stays inside its content` field and can never read as a synthetic system line. system turns are still joined into the system prompt as before, and the common single-user-turn case is passed through verbatim (byte-identical).