Template: medical classification

Classify a clinical document into your own categories — document type, specialty, coding bucket — and hand off the ones a qualified person must review. This page is one of the use-case templates — a starting point, not a finished route. It sorts documents; it does not make clinical decisions. Clinical documents carry protected health information: read the custody terms in the fit note below, and until a BAA or DPA is in place run this on a zero-retention route so ModelRig holds no content. The machine-readable source is node_modules/modelrig/templates/medical-classification.json.

Does this task fit?

Medical classification fits the shipped teach → calibrate → grade → optimize loop when four conditions hold. (1) Your categories and the documents you classify are stable enough that last quarter's examples still describe this quarter's — if the taxonomy is still moving, settle it before you calibrate. (2) Someone who owns the process can write down what a correct classification is, which is the grading guide below. (3) A qualified reviewer can grade a batch of past documents against the calls they would have made, so a grader can be calibrated against them. (4) You process enough documents that calibration and optimization pay back. This task handles protected health information, so the custody terms come first — quoted verbatim from the published content custody terms: "Do not send protected health information or other regulated data through ModelRig until a BAA or DPA is in place. This service is not offered for regulated data today, and nothing on this page should be read as agreeing to handle it." Until those terms are in place, the today-path is to run this on a zero-retention route (or with content custody turned off), so ModelRig holds no content and grading is limited to the scores and metadata your own systems send. The full terms are at https://modelrig.ai/terms/content-custody. This template sorts documents into your categories; it is not a diagnostic or clinical-decision tool, and it makes no claim about how well any model classifies your documents — that is what your goldens and grades measure — and no compliance claim of any other kind.

Starter schema

Replace every TODO with your own taxonomy; the enum slots ship without values on purpose, so an unfilled template refuses to look finished.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$comment": "STARTER SCHEMA. Replace every TODO with your own taxonomy before use. The template runs as-is: free-text fields validate immediately. TODO markers show where adding your own enum values tightens grading to your categories; do not copy example words from the docs.",
  "title": "MedicalClassification",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "classification",
    "needs_human"
  ],
  "properties": {
    "classification": {
      "type": "string",
      "description": "The category this clinical document belongs in. TODO(enum): constrain this to YOUR own category labels by adding an \"enum\": [ ... ] here. It runs as free text immediately; add your own enum values here to tighten validation and grading to your taxonomy.",
      "$comment": "TODO(enum): add your document categories as the enum for this field."
    },
    "needs_human": {
      "type": "boolean",
      "description": "True when the document is ambiguous, clinically significant, or otherwise a call a qualified person must make. Pairs with the escalation band in the grading guide."
    },
    "rationale": {
      "type": "string",
      "description": "One or two sentences grounding the classification in what the document actually contains. Optional; keep it if you want the grader to check the decision is explained."
    }
  }
}

Starter grading guide

The starter criteria run as written — sharpen each one in your team's own words as you grade. They are in the exact jsonb shape the set_task_settings write path validates — binary checks, each with a one-sentence pass and fail. There is no pass_threshold here: that lives in the route's grade: block.

classification-correct — Classification is correct

• Pass: the document gets the category a qualified reviewer on your team would have chosen for it, by your own rules.
• Fail: The document is classified into a category that does not match what it actually is, or into a catch-all when a specific category fits.

grounded-no-invention — Decision is grounded in the document

• Pass: every part of the classification is supported by something the document actually contains; no clinical detail is invented.
• Fail: The output asserts a diagnosis, a code, or a finding that the document does not contain.

stays-a-classification — Output stays a classification, not a clinical judgment

• Pass: the output sorts the document into your categories and stops there, without offering a diagnosis or a treatment decision.
• Fail: The output makes a clinical or diagnostic call the task is not meant to make, rather than classifying and deferring to a person.

human-flag-when-unsure — Ambiguous or clinically significant documents are flagged for a human

• Pass: a document that is genuinely ambiguous or clinically significant sets needs_human rather than being auto-classified.
• Fail: A document a qualified reviewer would have handled is classified automatically with false confidence.

The guide's escalation band starts at epsilon: 0.1: a grade within that distance of the route's pass threshold is treated as judge-uncertain and sent to a human. It is a starting band — tune it as your grader calibrates.

Golden checklist

Roughly twenty examples that cover:

• One clear example per category in your taxonomy — the ordinary, unambiguous document for each category.
• The two or three categories your team most often confuses, side by side, so the guide learns the boundary between them.
• A document that spans two categories at once — the golden fixes which one wins and why.
• A clinically significant document that should reach a qualified person regardless of category — the golden shows needs_human set.
• A document that tempts a clinical judgment — the golden shows it classified and deferred, not diagnosed.
• An illegible, truncated, or off-type document — the golden shows it flagged for a human rather than force-classified.
• Aim for roughly twenty goldens total, weighted toward the ambiguous, cross-category, and clinically significant documents rather than the routine centre.

Escalation starter

Start by escalating to a qualified person whenever the model sets needs_human, and always for clinically significant documents regardless of the classification. The machine-side band lives in starter_guide.escalation.epsilon: a grade within that distance of your route's pass threshold is treated as judge-uncertain and sent to the review queue. The 0.1 here is a starting band — keep it wide until a qualified reviewer has graded enough documents to trust the grader, then narrow it.

Apply it

By hand: copy the starter schema and replace each TODO with your own document categories; sharpen each starter criterion in your team's words, keeping the guard that stops the output at a classification rather than a clinical judgment; gather the goldens the checklist describes; set the escalation band. Settle the custody posture — a zero-retention route or content custody turned off — before you send any document.

From a coding agent: the template feeds through the tools that already exist — create_route drafts the route bundle from the starter schema (files only, never your source), and set_task_settings (the task-settings scope) authors the grading guide as-is — the starter criteria run as written; sharpen them to your own rules as you grade. No new tool is involved. See the use-case templates overview for the full two-path walkthrough.