Evals for summarisation

12-03-2026

• https://eugeneyan.com/writing/product-evals/

• https://www.anthropic.com/research/statistical-approach-to-model-evals

• https://eugeneyan.com/writing/evals/

• open standards for eval config

  • https://github.com/evaleval/every_eval_ever?tab=readme-ov-file

• science of evals

  • https://www.apolloresearch.ai/blog/we-need-a-science-of-evals/

• https://www.mager.co/blog/2026-03-08-claude-code-eval-loop/

interestingly, creating a skill using claude’s interactive mode on the web app allows you to setup evals by default, which links back to this underlying skill repo - https://github.com/anthropics/skills/blob/main/skills/skill-creator/SKILL.md, and has a whole bunch of scripts on setting up and running and viewing evals (see scripts/ or references/ or eval-viewer/), nice and lightweight, good to constrat for our own harness