> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and validate evaluators

> Interpret traces, define success criteria, and build code and LLM-as-a-judge evaluators calibrated against human judgment.

Evaluation turns raw traces into measurable judgments of quality. Interpret what your traces reveal, define the criteria that separate a good response from a bad one, and build evaluators, both code-based and LLM-as-a-judge, whose scores you can rely on.

## Before you write evals for agents, read your data | Ep. 5

<Frame>
  <iframe width="100%" height="420" src="https://www.youtube.com/embed/ksD69LzYr50" title="Before You Write Evals for Agents, Read Your Data | Ep. 5" frameborder="0" allow="clipboard-write; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />
</Frame>

Read what your traces reveal and establish the criteria that distinguish a good response from a bad one.

## Your first code eval for agents: catch bugs in 5 lines of Python | Ep. 6

<Frame>
  <iframe width="100%" height="420" src="https://www.youtube.com/embed/rW2c_wmKk3I" title="Your First Code Eval for Agents: Catch Bugs in 5 Lines of Python | Ep. 6" frameborder="0" allow="clipboard-write; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />
</Frame>

Score outputs with deterministic checks for criteria that don't require a model to judge.

## LLM-as-a-judge for agents: build a custom eval rubric that works | Ep. 7

<Frame>
  <iframe width="100%" height="420" src="https://www.youtube.com/embed/t-tR9lqG_ok" title="LLM-as-a-Judge for Agents: Build a Custom Eval Rubric That Works | Ep. 7" frameborder="0" allow="clipboard-write; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />
</Frame>

Use an LLM to assess qualities that are difficult to capture in code, and write judge prompts that produce consistent scores.

## 5 LLM and agent eval mistakes that turn metrics into noise | Ep. 8

<Frame>
  <iframe width="100%" height="420" src="https://www.youtube.com/embed/HtZHFGmT7Ks" title="5 LLM and Agent Eval Mistakes That Turn Metrics Into Noise | Ep. 8" frameborder="0" allow="clipboard-write; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />
</Frame>

The common mistakes that make evaluations misleading, and how to avoid them.

## Is your LLM judge right? Calibrate with meta-evaluation | Ep. 9

<Frame>
  <iframe width="100%" height="420" src="https://www.youtube.com/embed/K3_-KhwQzI8" title="Is Your LLM Judge Right? Calibrate with Meta-Evaluation | Ep. 9" frameborder="0" allow="clipboard-write; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />
</Frame>

Calibrate an LLM judge against human judgment so you can rely on its scores.

## Up next

<Card title="Experiment, monitor, and export traces" href="/ax/learn/getting-started-with-ax/ongoing-improvement" icon="chart-line">
  Prove changes with experiments, evaluate production traffic, and turn failures into fixes.
</Card>
