> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Datasets

> Version controlled examples to run your experiments

**Datasets are the backbone of effective LLM experimentation**, providing structured collections of examples for evaluation and iteration. Datasets allow you to test models consistently across any real-world scenarios and edge cases, quickly identify regressions, and track measurable improvements.

In Arize, datasets are fully integrated, allowing you to run experiments in the UI or programmatically via the SDK.

<Frame>
  <video src="https://storage.googleapis.com/arize-phoenix-assets/assets/videos/create-dataset-csv.mp4" controls />
</Frame>

# Common Types of Datasets

## Golden Datasets: Compare Against Ideal Outputs

Curating **golden datasets** allows you to establish a reliable benchmark. A golden dataset provides a consistent and trusted "ground truth" for LLM outputs. By meticulously hand-labeling ideal responses, you create a stable benchmark that allows you to objectively measure and compare the performance of different models and prompt versions over time.

## Regression Datasets: Focus on Areas of Improvements

A **regression dataset** captures examples where your application previously failed or performed poorly. These datasets are crucial for ensuring that fixes or improvements persist over time and don’t reintroduce bugs or regressions. Examples are often pulled from user feedback or logs with problematic behavior.

***

# Flexible Dataset Format

Arize AX supports flexible dataset formats so you can structure data in the way that best fits your LLM application:

**1. Key-Value Pairs:** Flexible for multi-input/multi-output tasks such as function calls, agents, or classification, ensuring complex workflows can be tested consistently.

<table>
  <thead>
    <tr>
      <th>Input</th>
      <th>Context</th>
      <th>Output</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><code>What is Paul Graham known for?</code></td>
      <td>"Paul Graham is an investor, entrepreneur, and computer scientist known for..."</td>
      <td><code>"Paul Graham is known for co-founding Y Combinator..."</code></td>
    </tr>
  </tbody>
</table>

**2. Prompt-Completion (String Pairs):** Simple format for validating single-turn completions, making it easy to measure correctness against expected outputs.

<table>
  <thead>
    <tr>
      <th>Input</th>
      <th>Output</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><code>"do you have to have two license plates in ontario"</code></td>
      <td><code>"True"</code></td>
    </tr>
  </tbody>
</table>

**3. Messages or Chat Format:** Purpose-built for conversational agents, allowing you to evaluate multi-turn interactions in context.

```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
Input:
{"messages": [{"role": "system", "content": "You are an expert SQL assistant"}]}
Output:
{"messages": [{"role": "assistant", "content": "SELECT * FROM users;"}]}
```

# Learn More

<Frame>
  <iframe width="100%" height="420" src="https://www.youtube.com/embed/4PNU5mTbGec" title="YouTube video player" frameborder="0" allow="clipboard-write; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen />
</Frame>

<CardGroup cols={3}>
  <Card title="Quickstart" icon="rocket" href="/docs/ax/get-started/get-started-improve-your-agent">
    Start experimenting with datasets in UI or code
  </Card>

  <Card title="Dive into a guide" icon="book" href="/docs/ax/cookbooks/improve/summarization">
    Explore end-to-end walkthroughs on building robust datasets for experimentation
  </Card>

  <Card title="Learn more about evals" icon="clipboard-check" href="/docs/ax/concepts/evaluators/overview">
    Read our evaluation concepts page
  </Card>
</CardGroup>