ModelRig Leaderboard

Probed, dated, reproducible capability facts — sampled statistics with 95% confidence intervals, never single-shot verdicts. Ranked by effective cost per 1,000 schema-conformant outputs. Reproduce any row: npx modelrig-probes run --model <model>. ⚠ badges mark declared-vs-probed discrepancies.

Coverage: 24 fixtures, mixed-domain (a finance-seeded probe-suite plus a domain-general demo-rig family), in 2 families (20 probe-suite, 4 demo-rig) — schema: 9 general, 7 finance, 1 technical, 1 e-commerce, 1 legal · grounding: 2 finance, 1 technical · caching: 1 general · image: 1 e-commerce. The demo-rig family is authored synthetic / public-domain tasks (example.support_summarize, invoice.extract, docs.qa, content.classify) with deterministic ground truth — strong for capability ranking, and explicitly not a claim about any customer's workload. Per-fixture stats (with domains and families) are in each published result file; the rates in this table aggregate the whole schema corpus, and the Corpus column names which families produced each row — a model probed only by demo-rig fixtures is marked so, never mixed silently. The Hard conf. column is conformance on the authored hard-tier subset — the same fixtures for every model — where capable models separate; the aggregate stays the ranking basis. A native or coached tag beside it names the serving path those strict fixtures were probed through: a model without probed structured_native is coached via json_mode, so a coached miss is that path's limit — read "coached, missed", not a flat fail. Every model still runs every fixture; the tag labels, it never excludes. The probe-kit's headline ask is fixtures from your domain — contribute one.

parity-50: 23/28 of the top-by-usage yardstick (source, as of 2026-08-08) are PROBED — 82%. Their coverage is declared; ours is verified — and our gaps are named, not hidden: