← All 25 termsAI proof
Eval harness
AIA test rig that runs a model or agent through a fixed set of inputs, scores the outputs against a rubric, and persists the scores for trend analysis. The AI equivalent of a quant backtest.
Full definition
An eval harness is the AI-engineering equivalent of a quant backtest: a reproducible test rig that pins a model version, runs it through a fixed input set, scores outputs against a rubric (JSON-schema validators, BLEU/ROUGE, LLM-as-judge panels, ground-truth references), and persists the scores for trend analysis. The harness makes model behaviour auditable. On this site, the eval-mcp-server ships a reference harness that hits 100% round-trip parity on JSON-schema outputs across 20 prompts; the numerical-faithfulness-eval hits similar accuracy on quantitative reasoning prompts.
Related terms
Each link opens the long-form definition of a closely related term.
See this term in action.
The /proof page maps every term to the gate or artifact that implements it on this site.