Template: document extraction

Pull the fields you care about out of a document — an invoice, a form, a contract clause — into a structured object your code can use. This page is one of the use-case templates — a starting point, not a finished route. The machine-readable source is node_modules/modelrig/templates/document-extraction.json.

Does this task fit?

Document extraction fits the shipped teach → calibrate → grade → optimize loop when four conditions hold. (1) The documents you extract from are stable in kind — the same layouts and field set recur, so past examples still describe new ones. (2) Someone can write down what a correct extraction is field by field, which is the grading guide below. (3) You have corrected extractions on hand — records where a person already fixed the fields — so a grader can be calibrated against ground truth. (4) You process enough documents that calibration and optimization pay back. This template gives you a schema shell and a grading guide to fill in; it does not assume your fields, and it makes no claim about how well any model extracts them — that is what your goldens and grades measure.

Starter schema

Replace every TODO with your own taxonomy; the enum slots ship without values on purpose, so an unfilled template refuses to look finished.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$comment": "STARTER SCHEMA. The fields below are placeholders. Replace them with the fields YOU extract, delete the ones you do not need, and mark each field required or optional to match your documents.",
  "title": "DocumentExtraction",
  "type": "object",
  "additionalProperties": false,
  "required": [],
  "properties": {
    "document_type": {
      "type": "string",
      "description": "What kind of document this is. TODO(enum): add an \"enum\": [ ... ] listing the document types you handle, so the model cannot invent a type.",
      "$comment": "TODO(enum): add your document types as the enum for this field."
    },
    "fields": {
      "type": "object",
      "description": "The extracted values. TODO: replace this placeholder with one named property per field you extract (e.g. invoice_number, total_amount, issue_date), each with its own type and description. Keep additionalProperties false once you have listed them so nothing extra is invented.",
      "additionalProperties": true,
      "$comment": "TODO(fields): replace with your real field set; add a required list once you know which fields must always be present."
    },
    "missing_fields": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Names of fields you asked for that were genuinely absent from the document. Lets the model say \"not present\" explicitly instead of guessing a value."
    },
    "needs_human": {
      "type": "boolean",
      "description": "True when the document is too unclear to extract confidently and a person should review. Pairs with the escalation band in the grading guide."
    }
  }
}

Starter grading guide

The starter criteria run as written — sharpen each one in your team's own words as you grade. They are in the exact jsonb shape the set_task_settings write path validates — binary checks, each with a one-sentence pass and fail. There is no pass_threshold here: that lives in the route's grade: block.

values-match-source — Extracted values match the document

• Pass: every extracted value is exactly what the document says, in the format you asked for.
• Fail: A value differs from the document — a wrong number, a transcription slip, a mis-parsed date.

no-fabrication — Absent fields are marked, never invented

• Pass: A field that is not in the document is reported as missing rather than filled with a plausible-looking value.
• Fail: The output supplies a value for a field the document does not actually contain.

completeness — Present fields are all captured

• Pass: every field that IS in the document and IS in your schema is extracted; none is silently dropped.
• Fail: A field that is clearly present in the document is left out of the output.

human-flag-when-illegible — Unreadable documents are flagged for a human

• Pass: A document that is illegible, truncated, or the wrong kind sets needs_human rather than returning a confident-looking but unreliable extraction.
• Fail: An unreadable or off-type document is extracted as if it were clean.

The guide's escalation band starts at epsilon: 0.1: a grade within that distance of the route's pass threshold is treated as judge-uncertain and sent to a human. It is a starting band — tune it as your grader calibrates.

Golden checklist

Roughly twenty examples that cover:

• One clean example per document type in your set — the ordinary, fully-legible case for each layout.
• A document that is missing one or more of your fields — the golden shows those fields reported as missing, not invented.
• A document with a field in an unusual but valid format (a date written differently, a total with a currency symbol) so the guide learns your normalization rules.
• A multi-page or multi-section document where the field you want appears late or twice — the golden fixes which occurrence is correct.
• An ambiguous or partially-legible document — the golden shows it flagged for a human rather than guessed.
• An off-type document (something that is not one of your document types at all) — the golden shows it rejected or flagged, not force-extracted.
• Aim for roughly twenty goldens total, weighted toward missing fields, odd formats, and illegible cases rather than the clean centre.

Escalation starter

Start by escalating to a human whenever the model sets needs_human, and whenever a required field comes back missing on a document that should contain it. The machine-side band lives in starter_guide.escalation.epsilon: a grade within that distance of your route's pass threshold is treated as judge-uncertain and sent to the human queue. The 0.1 here is a starting band — widen it early, narrow it as your grader calibrates against your corrected extractions.

Apply it

By hand: replace the placeholder fields object with one named property per field you extract, each with its own type; add a required list for the fields that must always be present; sharpen each starter criterion in your team's words; gather the goldens the checklist describes; set the escalation band.

From a coding agent: the template feeds through the tools that already exist — create_route drafts the route bundle from the starter schema (files only, never your source), and set_task_settings (the task-settings scope) authors the grading guide as-is — the starter criteria run as written; sharpen them to your own rules as you grade. No new tool is involved. See the use-case templates overview for the full two-path walkthrough.