Template: compliance review

Check a document or message against your own policy, mark which of your rules it implicates, and send the uncertain and adverse ones to a human reviewer. This page is one of the use-case templates — a starting point, not a finished route. It is a first-pass triage that routes work to your reviewers, never a compliance control. The machine-readable source is node_modules/modelrig/templates/compliance-review.json.

Does this task fit?

Compliance review fits the shipped teach → calibrate → grade → optimize loop when four conditions hold. (1) Your policy and the material you review are stable enough that last quarter's examples still describe this quarter's — if your rules change constantly, a golden set goes stale, so pin the version you review against. (2) Your compliance or legal team can write down what a correct determination is, rule by rule, which is the grading guide below. (3) A reviewer who knows the policy can grade a batch of past items against the calls they would have made, so a grader can be calibrated against them. (4) You review enough items that calibration and optimization pay back. This template is a first-pass triage that routes work to your reviewers, never a substitute for them: it makes no claim to be a compliance control, and every adverse or uncertain call is meant to reach a person. Fill in your own determinations and rule set before you rely on it.

Starter schema

Replace every TODO with your own taxonomy; the enum slots ship without values on purpose, so an unfilled template refuses to look finished.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$comment": "STARTER SCHEMA. Replace every TODO with your own policy before use. The template runs as-is: free-text fields validate immediately. TODO markers show where adding your own enum values tightens grading to your review outcomes; do not copy example words from the docs.",
  "title": "ComplianceReview",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "determination",
    "needs_human"
  ],
  "properties": {
    "determination": {
      "type": "string",
      "description": "The review outcome for this item. TODO(enum): constrain this to YOUR own outcomes by adding an \"enum\": [ ... ] here (whatever labels your team already uses). It runs as free text immediately; add your own enum values here to tighten validation and grading to your process.",
      "$comment": "TODO(enum): add your review outcomes as the enum for this field."
    },
    "implicated_rules": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "The ids or names of the rules in YOUR policy that this item implicates. Leave empty when nothing is implicated. Lets the grader check that a flag cites a rule that actually applies rather than a general impression."
    },
    "needs_human": {
      "type": "boolean",
      "description": "True when the item is ambiguous, borderline, or adverse and a reviewer should decide. Pairs with the escalation band in the grading guide."
    },
    "rationale": {
      "type": "string",
      "description": "One or two sentences grounding the determination in the item's own text and the rule it cites. Optional; keep it if you want the grader to check the call is explained."
    }
  }
}

Starter grading guide

The starter criteria run as written — sharpen each one in your team's own words as you grade. They are in the exact jsonb shape the set_task_settings write path validates — binary checks, each with a one-sentence pass and fail. There is no pass_threshold here: that lives in the route's grade: block.

determination-correct — Determination is correct

• Pass: the item gets the determination a reviewer on your team would have reached under your policy.
• Fail: The determination is wrong for the item — a clear violation passed, or a clean item marked adverse.

rule-actually-applies — Cited rules actually apply

• Pass: every rule named in implicated_rules is one the item genuinely engages, and each cited rule exists in your policy.
• Fail: The output cites a rule the item does not implicate, or names a rule that is not in your policy at all.

no-missed-violation — Clear violations are not passed

• Pass: an item that plainly breaks a rule is marked, not waved through.
• Fail: An item that clearly violates a rule in your policy is returned as clean.

human-flag-when-unsure — Ambiguous or adverse items are flagged for a human

• Pass: an item that is borderline, novel, or adverse sets needs_human rather than being auto-cleared or auto-flagged with false confidence.
• Fail: An item a reviewer would have hesitated on is decided automatically instead of being sent to a person.

The guide's escalation band starts at epsilon: 0.1: a grade within that distance of the route's pass threshold is treated as judge-uncertain and sent to a human. It is a starting band — tune it as your grader calibrates.

Golden checklist

Roughly twenty examples that cover:

• One clear example per determination in your process — the ordinary, unambiguous item for each outcome.
• One clean example and one clear-violation example per rule you review against, so the guide learns each rule's boundary.
• The two or three rules your team most often argues over, side by side, so the guide learns your house reading.
• A borderline item that could plausibly go either way — the golden shows needs_human set rather than a confident call.
• An item that implicates more than one rule at once — the golden records every rule that applies, not just the first.
• An edge case: an empty, off-topic, or unintelligible item — the golden shows it flagged for a human rather than force-decided.
• Aim for roughly twenty goldens total, weighted toward the borderline and multi-rule items rather than the clear-cut centre.

Escalation starter

Start by escalating to a reviewer whenever the model sets needs_human, and treat every adverse determination as review-bound regardless of confidence — a first-pass triage should raise borderline calls, not resolve them. The machine-side band lives in starter_guide.escalation.epsilon: a grade within that distance of your route's pass threshold is treated as judge-uncertain and sent to the reviewer queue. The 0.1 here is a starting band — keep it wide, and only narrow it once a reviewer has graded enough items to trust the grader.

Apply it

By hand: copy the starter schema and replace each TODO with your own review outcomes and rule set; sharpen each starter criterion in your team's words, keeping the flag-everything-adverse posture that sends borderline calls to a person; gather the goldens the checklist describes; set the escalation band.

From a coding agent: the template feeds through the tools that already exist — create_route drafts the route bundle from the starter schema (files only, never your source), and set_task_settings (the task-settings scope) authors the grading guide as-is — the starter criteria run as written; sharpen them to your own rules as you grade. No new tool is involved. See the use-case templates overview for the full two-path walkthrough.