ModelRig Quickstart Route bundles Probes Bake-offs & replay How it fits Leaderboard

Route bundle reference

One YAML file per route under the routes directory (default modelrig/routes/). Files are loaded, structurally validated, and frozen at createRig() time; any invalid bundle throws a RouteConfigError naming the file and every problem.

Top-level fields

fieldtyperequirednotes
routestringyesThe task handle passed to rig.run(). Unique across the directory (duplicates are a load error).
versioninteger ≥ 1yesBump on ANY change to the bundle, schema, or prompt. Recorded on every telemetry row.
schemarelative path \nullyesJSON Schema file for the output. null = unstructured route (output must still be parseable JSON). Path resolves relative to the bundle file.
candidateslistyes, non-emptyTHE candidate set. Each entry is {provider, model}. Providers: gemini, openai, deepseek (anthropic, grok reserved). Nothing outside this list can serve the route — enforced structurally (branded CandidateRef) and at runtime.
requirelistnoHard constraints: schema_conformant, grounded, zero_retention, trace_visible (v2 — reasoning trace exposed to the caller; zero_retention remains declared intent). grounded is satisfied natively OR — when a search provider is configured — via the grounding-inject rung (search results injected into the prompt, citations normalized onto RunMeta.citations, search cost accounted).
preferlistnoAdvisory ordering: cost (ascending via the pricing snapshot; unknown cost sorts last), latency (no-op until probe data exists, Phase 2).
prompt.systemrelative pathyesTemplate file (see below).
prompt.variablesstring listyesEvery {{var}} referenced by the template must be declared here.
policyobjectyesSee below.
capturebooleanno (default false)v2. Local-only replay capture opt-in: every attempt's rendered variables + output text are written to the local SQLite captures table. Captures NEVER leave the machine — the exporter is structurally unable to read them (export-isolation test).
repairobjectnov2. Repair rung for the default variant — see below.
variantslistnov2. Named serving variants — see below. Absence = pure v1 semantics.

policy

fieldtypenotes
retriesmapPer-failure-class retry budgets, e.g. { content_invalid: 2, network: 4, capacity_shed: 3 }. Missing class = 0 retries. Budgets are non-fungible. Terminal classes (budget_exhausted, invariant_violation) are rejected here at load time.
timeout_mspositive integerWall-clock deadline per attempt; breach maps to the timeout class.
tierstandard \flex \priorityRequested service tier. The tier actually served is recorded separately (servedTier) — silent downgrades become visible in telemetry.
jsonnative \json_modeEmission rung. native: candidates need native strict schema enforcement (structured_native); the schema goes to the provider natively. json_mode: candidates need either mechanism; the schema is additionally coached into the prompt with strict formatting guidance.

Variants (bundle format v2 — Phase 3)

A route may declare variants — named bindings served via rig.run(route, { variant: "name" }). Omitting variant (or passing "default") serves the route's base config exactly as v1 did; v1 bundles remain valid unchanged. Every telemetry row records served_variant.


variants:
  - name: cheap
    candidates:               # narrowing ONLY — must be ⊆ the route's list
      - provider: openai
        model: gpt-5.4-mini
    json: json_mode           # rung override for this variant
    repair:
      max_repairs: 2
      repair_model: openai/gpt-5.4-mini
  - name: scaffolded
    scaffold: |               # reasoning scaffold, appended before coaching
      Think through the drivers step by step before emitting JSON.
  - name: reworded
    prompt_override: ./prompts/echo-v2.system.md   # inline text also accepted
fieldnotes
nameUnique per route; "default" is reserved for the base config.
candidatesSubset of the route's declared candidates — variants narrow, never widen (load error otherwise; the candidate-set invariant holds through variants).
prompt_overrideReplaces the system template (rendered with the same variables + capability conditionals). Mutually exclusive with prompt_append.
prompt_appendAppended to the rendered system prompt.
scaffoldReasoning-scaffold text appended after prompt_append, before json coaching. Manual authoring only this phase.
jsonRung override (native \json_mode).
repairVariant-level repair rung, overrides the route-level repair.

repair (ladder rung 4 — Phase 3)


repair:
  max_repairs: 2                     # 1 = retry-with-errors only; 2 adds the repair model
  repair_model: deepseek/deepseek-chat   # required when max_repairs is 2; must be a declared candidate

On a schema-invalid output with repair enabled: attempt 1 re-asks the SAME model with the ajv error summary appended; attempt 2 hands {invalid output, errors, schema} to the designated repair model with a fixed repair prompt. Repair attempts draw from their own repair budget (never the content_invalid budget), and repaired rows carry repaired_by in telemetry — repair cost is visible, not hidden.

Requirement → capability mapping (registry-wired since Phase 3)

requirementsatisfied by
schema_conformant + json: nativestructured_native
schema_conformant + json: json_modestructured_native OR json_mode
groundedgrounded_native (native directive on dispatch) OR a configured search provider (grounding-inject)
trace_visibletrace_visible (e.g. deepseek-reasoner's exposed reasoning_content)

Capability flags are resolved from registry facts (the packaged registry/registry.json, env-overridable via MODELRIG_REGISTRY_PATH) with per-flag precedence capabilityOverrides > probed > declared — a probed-false trumps a declared-true (DeepSeek's declared schema support is revoked because every probed sample served via json_mode coaching). Models absent from the registry fall back to the adapter-static baseline below:

capabilitygeminiopenaideepseek
structured_native
json_mode
grounded_native
context_cache_explicit
prompt_cache_key
prefix_cache_implicit
tier_flexflex-eligible models only (no mini/nano)

Overridable per provider via RigConfig.capabilityOverrides (replaces the whole flag set for that provider — useful for pinning a route to one candidate in tests).

Templates

Versioning discipline

The (route, version) pair keys the compiled validator cache, telemetry rows, and the console's routes mirror. Editing a bundle without bumping version makes telemetry lie across the change — bump every time.