Walks a PM through Anthropic's Step 6 ritual — reading transcripts from many trials to diagnose every failed eval task as one of three things — model mistake, grader mistake, or task-spec error. Implements the practice Anthropic describes as "critical" — without it, badly-calibrated graders mask rea
Walks a PM through Anthropic's Step 6 ritual — reading transcripts from many trials to diagnose every failed eval task as one of three things — model mistake, grader mistake, or task-spec error. Implements the practice Anthropic describes as "critical" — without it, badly-calibrated graders mask real model improvements. Use when the user has a /run-eval result with failures, asks "why did this fai