List evaluator templates

Retrieve the built-in LLM-as-a-judge evaluator templates. This is the same catalog the product offers when creating an evaluator, spanning response quality, code quality, trajectory, RAG, security, and session evals.

Each template carries the judge prompt, the labels it returns, the score for each label, its optimization direction, and the granularity it evaluates at.

Creating an evaluator from a template. Pick a template, then call POST /v2/evaluators with its fields mapped onto the request:

Template field	Where it goes in POST /v2/evaluators
column_name	version.template_config.name
template	version.template_config.template
classification_choices	version.template_config.classification_choices
direction	version.template_config.direction
data_granularity	version.template_config.data_granularity. Send SPAN, or omit it, when the template’s value is null
display_name	a label for your own use; reuse it for the evaluator’s name or description
rails	no destination; classification_choices already carries the same labels

Then add the fields a template doesn’t carry: space_id, name, and type: TEMPLATE on the evaluator; a version.commit_message; and the execution settings template_config.include_explanations, use_function_calling_if_available, and llm_config. Finally, create a task to run the evaluator.

A complete request built from the hallucination template:

{
  "space_id": "U3BhY2U6NDkzOkJaSkc=",
  "name": "hallucination",
  "description": "Built from the hallucination template",
  "type": "TEMPLATE",
  "version": {
    "commit_message": "Initial version from built-in template",
    "template_config": {
      "name": "hallucination",
      "template": "You are evaluating whether an answer is factual given reference text...\n{input}\n{output}",
      "classification_choices": { "hallucinated": 1, "factual": 0 },
      "direction": "MINIMIZE",
      "data_granularity": "SPAN",
      "include_explanations": true,
      "use_function_calling_if_available": true,
      "llm_config": {
        "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
        "model_name": "gpt-4o",
        "invocation_parameters": { "temperature": 0 },
        "provider_parameters": {}
      }
    }
  }
}

Scope: this returns only the built-in catalog, which is identical for every caller and contains no space, account, or user data. It does not include the evaluators that already exist in your space. List those with GET /v2/evaluators.

Pagination: not paginated. The catalog is a small fixed list (28 templates, roughly 32 KB of JSON) and the full set is always returned, so there is no cursor or limit.

This endpoint is in alpha, read more here.

GET /v2/evaluator-templates

List evaluator templates

cURL

curl --request GET \
  --url https://api.arize.com/v2/evaluator-templates \
  --header 'Authorization: Bearer <token>'

200

{
  "evaluator_templates": [
    {
      "column_name": "hallucination",
      "display_name": "Hallucination",
      "template": "You are evaluating whether an answer is factual given reference text.\n\n[Input]: {input}\n[Reference]: {context}\n[Answer]: {output}\n\nRespond with a single word: factual or hallucinated.",
      "rails": [
        "hallucinated",
        "factual"
      ],
      "classification_choices": {
        "hallucinated": 1,
        "factual": 0
      },
      "direction": "MINIMIZE",
      "data_granularity": "SPAN"
    },
    {
      "column_name": "session_frustration",
      "display_name": "Session Frustration",
      "template": "You are given a multi-turn session between a user and an AI assistant.\n\n{conversation}\n\nRespond with a single word: frustrated or ok.",
      "rails": [
        "frustrated",
        "ok"
      ],
      "classification_choices": {
        "frustrated": 1,
        "ok": 0
      },
      "direction": "MINIMIZE",
      "data_granularity": "SESSION"
    }
  ]
}

Authorizations

Authorization · string · header · required

Most Arize AI endpoints require authentication. For those endpoints that require authentication, include your API key in the request header using the format

Response

200

application/json

The full list of built-in evaluator templates.

evaluator_templates · object[] · required

Every built-in template, ordered by category as the product presents them (response quality, code quality, trajectory, RAG, security, session).

Show child attributes