Template: ticket triage

Classify an incoming support ticket into your own categories and a priority, and flag the ones a human should look at. This page is one of the use-case templates — a starting point, not a finished route. The machine-readable source is node_modules/modelrig/templates/ticket-triage.json.

Does this task fit?

Ticket triage fits the shipped teach → calibrate → grade → optimize loop when four conditions hold. (1) Your ticket stream is stable enough that last month's examples still describe this month's — if your categories churn weekly, tighten them before you calibrate. (2) Your support team can write down what a correct routing decision is, which is the grading guide below. (3) Someone who knows the queue can grade a batch of past tickets so a grader can be calibrated against them. (4) The queue carries enough volume that calibration and optimization pay back. This template is a starting point for each of those, not a claim that triage is solved — fill in your taxonomy and your definitions before you rely on it.

Starter schema

Replace every TODO with your own taxonomy; the enum slots ship without values on purpose, so an unfilled template refuses to look finished.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$comment": "STARTER SCHEMA. Replace every TODO with your own taxonomy before use. The template runs as-is: free-text fields validate immediately. TODO markers show where adding your own enum values tightens grading to your taxonomy; do not copy example words from the docs.",
  "title": "TicketTriage",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "category",
    "priority",
    "needs_human"
  ],
  "properties": {
    "category": {
      "type": "string",
      "description": "Which queue or team the ticket belongs to. TODO(enum): constrain this to YOUR own category labels by adding an \"enum\": [ ... ] here. It runs as free text immediately; add your own enum values here to tighten validation and grading to your taxonomy.",
      "$comment": "TODO(enum): add your category labels as the enum for this field."
    },
    "priority": {
      "type": "string",
      "description": "How urgently the ticket needs attention. TODO(enum): add an \"enum\": [ ... ] with YOUR priority levels (whatever your team already uses).",
      "$comment": "TODO(enum): add your priority levels as the enum for this field."
    },
    "needs_human": {
      "type": "boolean",
      "description": "True when the model is not confident enough to route this automatically and a person should decide. Pairs with the escalation band in the grading guide."
    },
    "reason": {
      "type": "string",
      "description": "One short sentence grounding the category and priority in the ticket text. Optional; keep it if you want the grader to check that the decision is explained."
    }
  }
}

Starter grading guide

The starter criteria run as written — sharpen each one in your team's own words as you grade. They are in the exact jsonb shape the set_task_settings write path validates — binary checks, each with a one-sentence pass and fail. There is no pass_threshold here: that lives in the route's grade: block.

category-correct — Category is correct

• Pass: the ticket is assigned to the category a human on your queue would have chosen for it.
• Fail: The ticket is assigned to a category that does not match its actual topic, or to a catch-all when a specific category fits.

priority-justified — Priority matches the stated impact

• Pass: the priority reflects the urgency and impact the ticket actually describes, by your team's own priority rules.
• Fail: The priority is higher or lower than the ticket's stated impact warrants.

grounded-no-invention — Decision is grounded in the ticket

• Pass: Every field is supported by something the ticket actually says; nothing is invented.
• Fail: The output asserts a detail (a product, an account state, an error) that the ticket text does not contain.

human-flag-when-unsure — Ambiguous tickets are flagged for a human

• Pass: a ticket that is genuinely ambiguous, mixed-topic, or high-risk sets needs_human rather than guessing a single category.
• Fail: A ticket that a human would have hesitated on is auto-routed with false confidence.

The guide's escalation band starts at epsilon: 0.1: a grade within that distance of the route's pass threshold is treated as judge-uncertain and sent to a human. It is a starting band — tune it as your grader calibrates.

Golden checklist

Roughly twenty examples that cover:

• One clear example per category in your taxonomy — the ordinary, unambiguous case for each queue.
• The two or three categories your team most often confuses, side by side, so the guide learns the boundary between them.
• A multi-topic ticket that could plausibly land in two categories — the golden records which one wins and why.
• A high-priority case and a low-priority case that use similar words, so priority is judged on impact, not vocabulary.
• An edge case: an empty, spam, or unintelligible ticket — the golden shows it flagged for a human rather than force-categorized.
• A sensitive case (security, legal, billing dispute, or churn risk) that should escalate regardless of category.
• Aim for roughly twenty goldens total, weighted toward the boundaries above rather than the easy centre of each category.

Escalation starter

Start by escalating to a human whenever the model sets needs_human, and whenever a ticket is sensitive (security, legal, or a churn/at-risk signal) regardless of its category. The machine-side band lives in starter_guide.escalation.epsilon: a grade within that distance of your route's pass threshold is treated as judge-uncertain and sent to the human queue. The 0.1 here is a starting band — widen it while you build trust, narrow it as your grader calibrates.

Apply it

By hand: copy the starter schema and replace each TODO with your own categories and priorities; sharpen each starter criterion in your team's words; gather the goldens the checklist describes; set the escalation band.

From a coding agent: the template feeds through the tools that already exist — create_route drafts the route bundle from the starter schema (files only, never your source), and set_task_settings (the task-settings scope) authors the grading guide as-is — the starter criteria run as written; sharpen them to your own rules as you grade. No new tool is involved. See the use-case templates overview for the full two-path walkthrough.