magnus919_agent-skills/training-data-annotation/references/task-and-label-design.md

Task and label design

Record the decision the labels support, the unit of annotation, eligible inputs, label definitions, allowed combinations, abstention reasons, and prohibited inferences. Define boundary examples before production labeling and version the guide whenever a rule changes.

For each label, state inclusion criteria, exclusion criteria, evidence required, counterexamples, and whether disagreement is expected. Separate an unknown or ambiguous state from a negative label. For multilabel, span, ranking, or continuous tasks, define overlap, tie, scale anchors, and missingness explicitly.

The annotation manifest should link task version, guide version, data revision, sampling batch, annotator role, calibration status, adjudication authority, privacy/rights record, and downstream consumer. Avoid collecting fields that do not serve a declared quality or governance question.