Create an annotation queue

Create a new annotation queue.

Payload Requirements

• The annotation queue name must be unique within the given space (among active queues).
• At least one annotation_config_id is required, and all configs must belong to the specified space.
• Do not include system-managed fields on input: id, created_at, updated_at.
• If assignment_method is not provided, it defaults to "all".

Valid example

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["[e-mail]"],
  "assignment_method": "all"
}

Valid example with records

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["[e-mail]"],
  "records": [
    {"record_type": "SPAN", "project_id": "prj_abc", "start_time": "2024-01-15T00:00:00Z", "end_time": "2024-01-16T00:00:00Z", "span_ids": ["span_001"]},
    {"record_type": "EXAMPLE", "dataset_id": "ds_xyz", "example_ids": ["ex_001", "ex_002"]}
  ]
}

Invalid example (missing required annotation_config_ids)

{
  "name": "My Queue",
  "space_id": "spc_xyz789"
}

POST /v2/annotation-queues

Create an annotation queue

cURL

curl --request POST \
  --url https://api.arize.com/v2/annotation-queues \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": [
    "ac_abc123"
  ],
  "annotator_emails": [
    "[e-mail]"
  ],
  "assignment_method": "ALL"
}
'

201

{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "annotation_configs": [
    {
      "id": "ac_001",
      "name": "accuracy",
      "type": "CONTINUOUS",
      "space_id": "spc_xyz789",
      "minimum_score": 0,
      "maximum_score": 10,
      "optimization_direction": "MAXIMIZE",
      "created_at": "2024-01-10T08:00:00Z"
    },
    {
      "id": "ac_002",
      "name": "sentiment",
      "type": "CATEGORICAL",
      "space_id": "spc_xyz789",
      "values": [
        {
          "label": "positive",
          "score": 1
        },
        {
          "label": "neutral",
          "score": 0
        },
        {
          "label": "negative",
          "score": -1
        }
      ],
      "optimization_direction": "MAXIMIZE",
      "created_at": "2024-01-10T08:00:00Z"
    }
  ],
  "annotators": [
    {
      "id": "usr_001",
      "email": "[e-mail]"
    },
    {
      "id": "usr_002",
      "email": "[e-mail]"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-20T14:45:00Z"
}

Authorizations

Authorization · string · header · required

Most Arize AI endpoints require authentication. For those endpoints that require authentication, include your API key in the request header using the format

Body

application/json

Body containing annotation queue creation parameters

name · string · required

The name of the annotation queue. Must be unique within the space for active queues.

Maximum string length: 255

Example:

"Quality Review Queue"

space_id · string · required

The space ID that the annotation queue belongs to

Example:

"spc_xyz789"

annotation_config_ids · string[] · required

IDs of annotation configs to associate with this queue. All configs must belong to the same space.

Minimum array length: 1

Example:

["ac_abc123", "ac_def456"]

annotator_emails · string<email>[] · required

Email addresses of annotators to assign to the queue. Emails are resolved to user IDs server-side.

Minimum array length: 1

An email address

Example:

[
  "[e-mail]",
  "[e-mail]"
]

instructions · string

Instructions for annotators working on this queue

Maximum string length: 5000

Example:

"Review each response for accuracy and helpfulness"

assignment_method · enum<string> · default: ALL

How records are assigned to annotators. Defaults to ALL when omitted.

Available options: ALL, RANDOM

record_sources · object[]

Record sources to add to the annotation queue on creation. At most 2 record sources (projects or datasets) may be provided in a single create request. The total number of records resolved from all sources must not exceed 500. Additional records from other sources can be added after creation.

Maximum array length: 2

Option 1

Option 2

Option 3

Show child attributes

column_allowlist · string[]

The record column names annotators assigned to this queue are allowed to see. Omit or send an empty list to leave the queue unrestricted, so annotators see every column. Because this is an allowlist, columns introduced by records added to the queue later stay hidden until an admin allows them.

Maximum array length: 1000

Maximum string length: 512

Example:

[
  "context.trace_id",
  "attributes.llm.input_messages",
  "column with spaces"
]

Response

201

application/json

An annotation queue object

id · string · required

The unique identifier for the annotation queue

Example:

"aq_abc123"

name · string · required

The name of the annotation queue

Example:

"Quality Review Queue"

space_id · string · required

The space id the annotation queue belongs to

Example:

"spc_xyz789"

annotators · object[] · required

Users assigned as annotators to this queue

Show child attributes

created_at · string<date-time> · required

The timestamp for when the annotation queue was created

Example:

"2024-01-15T10:30:00Z"

updated_at · string<date-time> · required

The timestamp for when the annotation queue was last updated

Example:

"2024-01-20T14:45:00Z"

instructions · string | null

The instructions for the annotation queue

Example:

"Review each response for accuracy and helpfulness"

annotation_configs · object[]

The annotation configs associated with this queue

Option 1

Option 2

Option 3

Show child attributes

column_allowlist · string[] | null

The record column names annotators assigned to this queue are allowed to see. Absent or null means the queue is unrestricted and annotators see every column.

Example:

[
  "context.trace_id",
  "attributes.llm.input_messages",
  "column with spaces"
]