Skip to main content

Filter dataset examples and experiment runs with SQL-like queries

September 29, 2026 New  SDKs and REST APIs The Python, TypeScript, and Go SDKs now take an optional filter expression when listing dataset examples and experiment runs, so you can request only the records that match a condition instead of paging through an entire dataset or experiment. Filters use a SQL-like syntax over example and run columns, evaluation scores and labels (eval.<name>.score, eval.<name>.label), and annotations (annotation.<name>.*). The code example below returns only the dataset examples and experiment runs that match a filter:
For more information, refer to the datasets and experiments client documentation.

Break down LLM spend on every project with the Cost tab

September 29, 2026 New  Dashboards and Visualizations The Cost tab is now available on every project overview. It surfaces the built-in Cost agent’s analysis of your LLM spend: charts of spend over time and by model, and cost-savings issues that flag expensive models doing low-complexity work with quantified recommendations for cheaper routing. Projects that have not run the Cost agent yet show a preview of what it will surface. Learn more about the Cost agent.

Build and edit dashboards with Alyx

September 29, 2026 New  Alyx  Dashboards
Alyx can now build and maintain dashboards for every account with Alyx enabled. Describe what you want to track, and Alyx creates the dashboard and its widgets, checks that each widget returns data, and fixes the ones that come back empty.
  • Build a dashboard: Alyx can create a dashboard from a template or from scratch, then create and edit its time series, distribution, statistic, pivot table, and text widgets. Split a time series into one line per value, such as latency per model.
  • Redesign your dashboards: Retheme a dashboard, rename several charts, or add new widgets in one request. Alyx can also move, resize, and delete widgets you name.
  • Understand a dashboard: Ask Alyx to summarize what a dashboard or widget shows. Dashboard and Widget context pills attach what you are looking at to your message.
Learn more about Alyx dashboard capabilities and setting up dashboards.

Add entire sessions to a dataset from the Sessions view

September 29, 2026 New  Datasets and Experiments You can now select whole sessions on the Sessions page and add them to a dataset in one step, rather than collecting spans one at a time. Use the checkbox column to pick sessions (or select all), then choose “Add to Dataset” from the selection toolbar; each session is copied into the dataset as examples. The same toolbar can send the selected sessions to a labeling queue. Learn more about curating a dataset.

Inspect traces and spans inline in the session conversation view

September 29, 2026 Improvement  Tracing and Sessions The session conversation tab now lets you drill from a conversation into its underlying telemetry without leaving the session. Select a turn to open its trace tree, then select a span to open span details, all in the same view. A new Session Overview panel summarizes each turn’s latency, token usage, and errors, and shows a pass/fail grid of evaluation results where selecting a result opens the turn it applies to. Learn more about sessions.

Create LiteLLM, Fireworks, and Together AI integrations from the CLI

September 29, 2026 New  CLI Commands The ax integrations create llm command now supports three more providers: LiteLLM, Fireworks, and Together AI. You can register their credentials and models directly from the command line instead of configuring them in the Playground. The code example below creates one integration for each new provider:
Learn more about the integrations CLI commands.

Fixes and improvements

September 28–29, 2026 Datasets and Experiments
  • Fix Page through experiments that have repeated runs and receive complete, per-run results, with a separate evaluation result recorded for each repeated run.
  • Fix Attach custom code evaluators that use the reserved dataset_row parameter to an online task without mapping it to a data column.
  • Fix Run an evaluator on an experiment after running it on the parent dataset, so the experiment evaluation completes.
Webhooks and Events
  • Improvement See currently assigned webhooks preselected and replace them when editing webhook notifications in bulk from the Monitor Config editor.
Alyx
  • Improvement Turn on “Always accept” directly in the Alyx human-in-the-loop drawer to accept the current response and all future responses in that category.

Run high-volume structured evals with Jev-as-a-Judge

September 26, 2026 New  Evaluators You can now evaluate traces with TypeSafe AI’s Jev decision model from Evaluator Hub. Add a TypeSafe AI key under Settings → AI Providers, create a Jev-As-A-Judge evaluator with a shared state template and typed boolean, choice, or score questions, then attach it to an online eval task. Each record sends one System One call and publishes a label and confidence (or rubric) score per question, without free-text explanations, so you can cover structured checks at higher volume than a chat-model judge. Learn more about Jev-as-a-Judge and TypeSafe AI provider setup.

Connect Anthropic Messages API gateways as Custom Model Endpoints

September 23, 2026 New  Models and Integrations You can now add a Custom Model Endpoint that speaks the Anthropic Messages API, so you can connect a gateway that exposes only /v1/messages and authenticates with its own custom headers. Choose the API format when you create the endpoint, then use it in the prompt playground, Test Integration, and online evaluations exactly like an OpenAI-compatible endpoint.
  • API Format selector: pick OpenAI or Anthropic on the Custom Model Endpoint form. The format is fixed at creation and shown read-only when you edit the endpoint, because changing an endpoint’s wire protocol would affect every evaluator and task that uses it.
  • Custom headers: an Anthropic-format endpoint sends the headers you configure on the integration, so a gateway with its own authentication scheme works without a separate API key.
Learn more about Custom Model Endpoints.

Filter runs in the new experiment compare view

September 23, 2026 Improvement  Datasets and Experiments The query filter bar and the hide-all-null-outputs toggle now sit above the stacked runs table in the new experiment compare view, so you can narrow a comparison to the examples you care about, for example question contains "water". The filter matches on example groups, which keeps every run under a matching example visible, and the filter text, URL parameters, and shared links behave the same as they do in the classic view. Learn more about comparing experiments.

Run Claude Opus 5.5 in the Playground and evaluators

September 22, 2026 New  Models and Integrations Select Claude Opus 5.5 from the model picker to prototype prompts, run experiments, and score traces with Anthropic’s latest Opus model. It runs with adaptive extended thinking for complex reasoning and is available through the direct Anthropic API, AWS Bedrock, and Google Vertex AI.
  • Anthropic: claude-opus-5-5
  • AWS Bedrock: global.anthropic.claude-opus-5-5 and us.anthropic.claude-opus-5-5
  • Vertex AI: claude-opus-5-5
Learn more about model integrations.

Select OpenAI GPT-6 Sol and Luna in the prompt playground

September 22, 2026 New  Models and Integrations Two new GPT-6 models, Sol and Luna, are now in the prompt playground model picker for both OpenAI and Azure OpenAI, so you can prototype prompts and run them against either one. Both support reasoning-effort levels from none through max, defaulting to medium, and remain available through the Chat Completions API. GPT-5.6 Sol and Luna stay available as separate models.
  • OpenAI and Azure OpenAI: gpt-6-sol and gpt-6-luna
Learn more about the prompt playground.

Connect Together AI as an LLM integration with the SDK

September 22, 2026 New  SDKs and REST APIs You can now create a Together AI LLM integration programmatically with the Python and Go SDKs, so you can manage Together AI access alongside every other supported provider without opening the UI. Pass your Together AI API key, and optionally the model names to expose, to create_llm. The code example below creates a Together AI integration from your provider API key:
Learn more about the integrations client.

Fixes and improvements

September 21–23, 2026
  • New  SDKs and REST APIs Set a project type of application, harness, or experiment when you call register() in arize-otel, or with the ARIZE_PROJECT_TYPE environment variable, so new projects are classified by type as they are created.
  • Improvement  Data Fabric Attach up to 40 embedding features per record when logging data, up from 30.

Get Alyx help tailored to your experience

September 17, 2026 New  Alyx Alyx now adapts how it helps based on your experience level in each part of Arize AX. It can be an onboarding buddy that explains concepts and teaches while it works, or an advanced, focused agent that concentrates on completing your tasks. The experience is catered to you, and your familiarity is considered separately across product areas.
  • When you are onboarding: Alyx explains unfamiliar concepts, walks through important decisions, shows you where to find the result, and suggests a useful next step. For example, while creating an evaluation, it can explain query filters, variable mappings, classification choices, sampling, and run settings as it prepares the configuration.
  • When you are experienced: Alyx skips introductory walkthroughs, uses AX terminology and configuration details directly, and focuses on completing the request. For example, it can move directly to a multi-span trace analysis or configure an evaluation task, then report the result and any important caveats.
  • When your experience differs by area: Alyx can stay concise and technical in Tracing while providing onboarding guidance in Evals and Tasks, so learning a new workflow does not mean repeating introductions in areas you already know.
Combined with long-term memory, experience awareness helps Alyx remember your goals and preferences while providing the right level of guidance for the work in front of you. Learn more about experience awareness in Alyx.

Pick up where you left off with Alyx long-term memory

September 17, 2026 New  Alyx Alyx now remembers durable context across conversations, including your preferences, ongoing goals, project and asset learnings, team conventions, and important decisions. Instead of starting each chat from scratch, you can continue an investigation, measure progress, and reuse what you learned elsewhere in Arize AX. For example, if you are working with Alyx to reduce errors in an agent, it can:
  • Continue your goal: Ask, “How are we looking today?” and Alyx can connect the question to your error-reduction goal and compare the latest data with prior checkpoints.
  • Return to the signals that matter: If you focused on TransportQueryError or another recurring pattern, Alyx can revisit it in a later conversation.
  • Connect your AX workflow: Alyx can remember how a project, evaluator, dataset, prompt, or experiment relates to your goal—for example, which evaluator you created to track the errors.
  • Carry forward discoveries: Context such as test traffic, expected retries, deployment effects, and known schema quirks can inform the next investigation instead of being rediscovered.
  • Work the way you prefer: Alyx can continue grouping results by root cause, deployment, or customer impact when that is how you like to investigate.
You can also ask Alyx to remember a specific durable detail. For instructions and context you want to set yourself, fill in the Profile card in Alyx Settings; your user-entered profile takes priority over learned memory. Learn more about long-term memory in Alyx.

Edit a managed agent automation’s prompt without recreating it

September 15, 2026 Improvement  Agents You can now update the prompt on an existing managed agent automation from the Edit prompt action in the automation detail menu, so you can refine how the agent runs without deleting and rebuilding the automation. The edit changes only the prompt, so the automation keeps its schedule, connected agent integration, repository, and run history in place. Learn more about managed agents.

Populate datasets with every matching record, faster

September 14, 2026 Improvement  Datasets and Experiments Populating a dataset from your traces now writes every matching record instead of applying a sampling budget that could drop rows, and appends complete two to seven times faster. This change reaches all account tiers, so building or growing a dataset from spans captures the full set you selected. For more information, refer to the curate a dataset from traces documentation.

Fixes and improvements

September 14–16, 2026 Alyx
  • Fix Save dashboards and widgets through Alyx reliably, without a stalled or interrupted session, even when a widget uses an unsupported time-range value.
  • Fix Resume an Alyx session with your connected integrations still available.
  • Fix Reopen an Alyx thread without re-running an action you already auto-accepted.
  • Fix Read Alyx answers as plain text instead of raw JSON.
Custom Metrics
  • Fix Load APPROX_QUANTILE custom metric widgets across large time ranges.
Dashboards and Visualizations
  • Fix Configure the Attribute and split-by pickers on dashboard widgets for tracing-only projects.
Agents
  • Fix Delete a custom agent role even after a managed agent has used it.

Add and annotate entire sessions and traces in annotation queues

September 8, 2026 New  Annotations You can now send full multi-turn sessions and complete traces to an annotation queue, not just individual spans and dataset examples, and review them at session or trace granularity. Add sessions to a queue from the Sessions table or a session’s detail view in the app, or add sessions and traces through the REST API and the Python, TypeScript, and Go SDKs, so you can build session-level and trace-level review workflows end to end. One queue can hold a mix of record types, so a whole session, a whole trace, an individual span, and a dataset example can sit side by side, and the annotator completes each record as its own unit of work. A queued session renders as the full session for the annotator, who labels it without leaving the queue. The code example below adds a session to an existing queue for annotation at session granularity:
Learn more about assigning records to a labeling queue.

Inspect remote evaluator runs as traces

September 8, 2026 New  Evaluators Remote evaluator runs now emit a trace for each evaluated row into your space’s evaluations project, with every span capturing the request, response, status code, and latency, so you can inspect and debug remote scoring calls and see retries as sibling spans in one trace. Turn it on with the Enable Tracing toggle under Advanced settings when you create or edit a remote evaluator. Learn more about remote evaluators.

Select OpenAI GPT-6 in the prompt playground

September 8, 2026 New  Models and Integrations You can now select OpenAI’s gpt-6-astra in the prompt playground model picker and run prompts against it. The model supports reasoning-effort levels from low through max, a verbosity control, and structured output, and it routes through the OpenAI Responses API for tool calling. Learn more about the prompt playground.

See GPT model reasoning in Alyx with thinking mode

September 4, 2026 Improvement  Alyx Alyx now streams reasoning summaries for OpenAI GPT models through its thinking view, which previously showed reasoning only for Claude models. When Alyx runs on a supported GPT model, you can follow its reasoning as it works through a request rather than seeing only the final answer. Learn more about Alyx.

Fixes and improvements

September 3–9, 2026 Evaluators
  • Improvement Run evaluations continuously against your space’s evaluations tracing project; tracing is automatically kept off for these tasks so they do not feed their own results back in.
Annotations
  • Improvement Choose which columns annotators see when you create a labeling queue from a dataset’s Examples tab, a trace, a session, or any of their selection toolbars.
Tracing and Sessions
  • Fix See spans that behave like errors in the trace errors tab, including those whose name or status message contains terms such as timeout, failure, or exception, alongside spans that carry a formal error status.
Agents
  • Improvement Start an automation right away: creating an automation now runs it immediately and opens its detail view instead of waiting for the next scheduled run.
SDKs and REST APIs
  • New Detach tags from monitors, evaluators, and prompts through the REST API with DELETE /v2/monitors/{monitor_id}/tags, DELETE /v2/evaluators/{evaluator_id}/tags, and DELETE /v2/prompts/{prompt_id}/tags.

Run Agent-as-a-Judge evaluators on every plan

September 1, 2026 Improvement  Evaluators Agent-as-a-Judge evaluators, which use an agentic judge to inspect traces, find the relevant spans, and label issues, are now available on every plan instead of Enterprise only. Any account can create one from the New Evaluator menu in the evaluator gallery and let the judge adapt its criteria to the failures that emerge in production, so you can evaluate agent behavior that a fixed template cannot anticipate. Learn more about Agent-as-a-Judge.

Run Claude Fable 5.1 in the Playground and evaluators

September 1, 2026 New  Models and Integrations Select Claude Fable 5.1 from the model picker to prototype prompts, run experiments, and score traces with Anthropic’s latest Fable model. It is available through the direct Anthropic API as well as AWS Bedrock and Google Vertex AI, and runs with adaptive extended thinking for complex reasoning.
  • Anthropic: claude-fable-5-1
  • AWS Bedrock: us.anthropic.claude-fable-5-1 and global.anthropic.claude-fable-5-1
  • Vertex AI: claude-fable-5-1
Learn more about model integrations.

Deep dive on Signal issues with Alyx

September 1, 2026 Improvement  Alyx Alyx can now read the issues Signal has recorded for your space, so you can ask what Signal has found across a project or single out one issue and have Alyx analyze it in depth. Signal reports on cost, token waste, latency, prompt quality, and tool choice, much of it on traces that never returned an error.
  • Context from the Signal tab: the issue you have open is attached to each message as a pill in the composer, and Alyx answers questions about issue counts and timing from it instead of querying traces.
  • Attach an issue from any page: add an issue to the conversation through the @ menu, where it is listed under Signal Insights, the same way you attach projects, datasets, and experiments.
  • A pointer to prior analysis: when you ask Alyx to debug a trace that Signal already reviewed, it links you to the matching issue instead of repeating that analysis.
Learn more about Alyx.

Open the filtered traces behind an Alyx answer

September 1, 2026 Improvement  Alyx When you select Apply filter in Alyx or ask it for a link to a set of traces, Alyx now opens the tracing view showing exactly those traces, with the correct filter expression, environment, and time range already applied. This works across projects and for rolling time windows such as the last 15 minutes, which now resolve to live data instead of a fixed time range. Learn more about Alyx.

Annotate entire sessions from the session view

August 31, 2026 New  Annotations Select Annotate Session in a session’s detail view in the tracing page, then apply a label or score to the session as a whole through the annotation pane. Each annotation is recorded as session_annotation.<name>.label, .score, and .text, so you can query and filter on a session annotation the way you would any other attribute.
Session 20405 open over the Sessions tab, showing its total duration, tokens, cost, and trace count, with the Session Conversation tab listing Trace #1 and Trace #2 and a Session Annotations pane on the right where an Accuracy config is set to accurate
Session annotations appear in the sessions table, the session trace view, the tracing table, the trace slideover annotations table, and the dataset examples table and example page, and the query filter dropdown offers them as filter terms. The REST API and the Python, TypeScript, and Go SDKs annotate at session granularity as well. The code example below annotates one session by its session ID:
Learn more about annotating traces.

Switch between trace and session results in the evaluator preview

August 29, 2026 Improvement  Evaluators After choosing a filter, you can now switch between trace and session results in the Preview Matching Data step of the evaluator’s Configuration column, and select any trace or session from the results table to preview its data for easier variable mapping. Choose a row with its checkbox, or move through the results with the up and down arrows beside the Table and Single Trace toggle. Whichever record you select fills the Single Trace view and the per-variable value previews. A session-scoped evaluator leads with the Sessions tab, and the Traces and Spans tabs narrow to the session you selected. The Single Trace view shows one of that session’s traces at a time, newest first, and the arrows step through the rest. Your selection resets when you change the project, filter, scope, or preview window. Learn more about trace and session evals.

Fixes and improvements

August 27 – September 2, 2026 Datasets and Experiments
  • Improvement Large dataset uploads now complete reliably, with transient chunk failures retried automatically before the upload returns.
  • Fix The Experiments REST API (beta) now reports accurate created and updated timestamps for each experiment.
Evaluators
  • Fix Template and LLM evaluators now keep the model’s explanation even when a provider returns free-form text instead of structured output.
Tracing and Sessions
  • Fix Spans from Bedrock native InvokeModel calls now include the output message role, so you can filter, group, and evaluate on it.
Playground
  • Fix Prompt Playground experiment runs now finish when the worker queue is busy, giving each run time to start before any timeout applies.
Agents
  • Fix Signal issue deep links now open the exact issue, even when it is resolved or older than the 30-day issue window.
SDKs and REST APIs
  • Fix The Python SDK now resolves a space name to that exact space before listing resources, so a name that is a substring of another space no longer returns the wrong space’s resources.
  • Fix The TypeScript and Go SDKs now let you switch a task between span and trace or session query shapes when you update it.