Template: catalog cleansing

Normalize a product-catalog record into your own taxonomy and attribute shape, correct what is clearly wrong, and flag the ones a person should check. This page is one of the use-case templates — a starting point, not a finished route. Cleansing usually runs as a batch over a backlog of raw records; you grade a representative sample and let the grades stand in for the rest. The machine-readable source is node_modules/modelrig/templates/catalog-cleansing.json.

Does this task fit?

Catalog cleansing fits the shipped teach → calibrate → grade → optimize loop when four conditions hold. (1) Your catalog taxonomy and attribute set are stable enough that a rule that cleaned last month's records still cleans this month's — if the taxonomy is still being redrawn, settle it before you calibrate. (2) Someone who owns the catalog can write down what a correctly cleansed record is, field by field, which is the grading guide below. (3) You have corrected records on hand — rows a person already fixed — so a grader can be calibrated against that ground truth. (4) You process enough records that calibration and optimization pay back. Cleansing usually runs as a batch over a backlog of raw records rather than one at a time; the loop still applies — you grade a representative sample of the batch and let the grades stand in for the rest. This template gives you a taxonomy shell and a grading guide to fill in; it does not assume your fields, and it makes no claim about how well any model cleanses a record — that is what your goldens and grades measure.

Starter schema

Replace every TODO with your own taxonomy; the enum slots ship without values on purpose, so an unfilled template refuses to look finished.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$comment": "STARTER SCHEMA. Replace every TODO with your own taxonomy and attributes before use. The template runs as-is: free-text fields validate immediately. TODO markers show where adding your own enum values tightens grading to your catalog; do not copy example words from the docs.",
  "title": "CatalogCleansing",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "category",
    "needs_human"
  ],
  "properties": {
    "category": {
      "type": "string",
      "description": "The catalog category this record belongs in. TODO(enum): constrain this to YOUR own category labels by adding an \"enum\": [ ... ] here. It runs as free text immediately; add your own enum values here to tighten validation and grading to your taxonomy.",
      "$comment": "TODO(enum): add your catalog categories as the enum for this field."
    },
    "corrected_attributes": {
      "type": "object",
      "description": "The cleansed attribute values. TODO: replace this placeholder with one named property per attribute you normalize (e.g. brand, unit_size, color), each with its own type and description. Keep additionalProperties false once you have listed them so nothing extra is invented.",
      "additionalProperties": true,
      "$comment": "TODO(fields): replace with your real attribute set; add a required list once you know which attributes must always be present."
    },
    "duplicate_of": {
      "type": "string",
      "description": "The id of an existing record this one appears to duplicate, when you run dedupe. Leave absent when the record is not a suspected duplicate; the model should flag a suspected merge, never perform it."
    },
    "needs_human": {
      "type": "boolean",
      "description": "True when the record is too ambiguous or conflicting to cleanse confidently and a person should review. Pairs with the escalation band in the grading guide."
    }
  }
}

Starter grading guide

The starter criteria run as written — sharpen each one in your team's own words as you grade. They are in the exact jsonb shape the set_task_settings write path validates — binary checks, each with a one-sentence pass and fail. There is no pass_threshold here: that lives in the route's grade: block.

category-correct — Category is correct

• Pass: the record is placed in the category a person who owns your catalog would have chosen for it.
• Fail: The record is placed in a category that does not match what it actually is, or in a catch-all when a specific category fits.

corrections-grounded — Corrections are grounded in the record

• Pass: every corrected attribute is supported by what the record actually contains, normalized to your format; nothing is invented.
• Fail: A corrected value contradicts the record, or an attribute is filled with a plausible-looking value the record does not support.

preserves-correct-data — Already-correct data is preserved

• Pass: an attribute that was already correct is left intact, not overwritten or dropped.
• Fail: The output alters or removes a field that was already valid, losing data that did not need cleansing.

human-flag-when-unsure — Ambiguous or conflicting records are flagged for a human

• Pass: a record with conflicting attributes, an unrecognizable category, or an uncertain duplicate match sets needs_human rather than being force-cleansed.
• Fail: A record a person would have paused on is cleansed with false confidence, or a suspected duplicate is merged instead of flagged.

The guide's escalation band starts at epsilon: 0.1: a grade within that distance of the route's pass threshold is treated as judge-uncertain and sent to a human. It is a starting band — tune it as your grader calibrates.

Golden checklist

Roughly twenty examples that cover:

• One clean example per category in your taxonomy — the ordinary, well-formed record for each category.
• A record that is already correct — the golden shows it left untouched, so the guide learns not to over-correct.
• The two or three categories your catalog most often confuses, side by side, so the guide learns the boundary.
• A record with a conflicting or contradictory attribute — the golden fixes which value is right and why.
• A near-duplicate pair — the golden shows the duplicate flagged for review, not silently merged.
• An edge case: a garbled, truncated, or unrecognizable record — the golden shows it flagged for a human rather than force-cleansed.
• Aim for roughly twenty goldens total, weighted toward the conflicting, duplicate, and already-clean records rather than the easy centre.

Escalation starter

Start by escalating to a person whenever the model sets needs_human, and whenever a suggested duplicate merge would combine records that a person has not confirmed. The machine-side band lives in starter_guide.escalation.epsilon: a grade within that distance of your route's pass threshold is treated as judge-uncertain and sent to the review queue. The 0.1 here is a starting band — widen it while you build trust on a fresh batch, narrow it as your grader calibrates against your corrected records.

Apply it

By hand: replace the placeholder corrected_attributes object with one named property per attribute you normalize, each with its own type; replace the TODO category taxonomy with your own labels; sharpen each starter criterion in your team's words; gather the goldens the checklist describes; set the escalation band.

From a coding agent: the template feeds through the tools that already exist — create_route drafts the route bundle from the starter schema (files only, never your source), and set_task_settings (the task-settings scope) authors the grading guide as-is — the starter criteria run as written; sharpen them to your own rules as you grade. No new tool is involved. See the use-case templates overview for the full two-path walkthrough.