Filter dataset examples and experiment runs with SQL-like queries
September 29, 2026 New SDKs and REST APIs The Python, TypeScript, and Go SDKs now take an optionalfilter expression when listing dataset examples and experiment runs, so you can request only the records that match a condition instead of paging through an entire dataset or experiment. Filters use a SQL-like syntax over example and run columns, evaluation scores and labels (eval.<name>.score, eval.<name>.label), and annotations (annotation.<name>.*).
The code example below returns only the dataset examples and experiment runs that match a filter:
- Python
- TypeScript
- Go
Break down LLM spend on every project with the Cost tab
September 29, 2026 New Dashboards and Visualizations The Cost tab is now available on every project overview. It surfaces the built-in Cost agent’s analysis of your LLM spend: charts of spend over time and by model, and cost-savings issues that flag expensive models doing low-complexity work with quantified recommendations for cheaper routing. Projects that have not run the Cost agent yet show a preview of what it will surface. Learn more about the Cost agent.Build and edit dashboards with Alyx
September 29, 2026 New Alyx Dashboards- Build a dashboard: Alyx can create a dashboard from a template or from scratch, then create and edit its time series, distribution, statistic, pivot table, and text widgets. Split a time series into one line per value, such as latency per model.
- Redesign your dashboards: Retheme a dashboard, rename several charts, or add new widgets in one request. Alyx can also move, resize, and delete widgets you name.
- Understand a dashboard: Ask Alyx to summarize what a dashboard or widget shows. Dashboard and Widget context pills attach what you are looking at to your message.
Add entire sessions to a dataset from the Sessions view
September 29, 2026 New Datasets and Experiments You can now select whole sessions on the Sessions page and add them to a dataset in one step, rather than collecting spans one at a time. Use the checkbox column to pick sessions (or select all), then choose “Add to Dataset” from the selection toolbar; each session is copied into the dataset as examples. The same toolbar can send the selected sessions to a labeling queue. Learn more about curating a dataset.Inspect traces and spans inline in the session conversation view
September 29, 2026 Improvement Tracing and Sessions The session conversation tab now lets you drill from a conversation into its underlying telemetry without leaving the session. Select a turn to open its trace tree, then select a span to open span details, all in the same view. A new Session Overview panel summarizes each turn’s latency, token usage, and errors, and shows a pass/fail grid of evaluation results where selecting a result opens the turn it applies to. Learn more about sessions.Create LiteLLM, Fireworks, and Together AI integrations from the CLI
September 29, 2026 New CLI Commands Theax integrations create llm command now supports three more providers: LiteLLM, Fireworks, and Together AI. You can register their credentials and models directly from the command line instead of configuring them in the Playground.
The code example below creates one integration for each new provider:
Fixes and improvements
September 28–29, 2026 Datasets and Experiments- Fix Page through experiments that have repeated runs and receive complete, per-run results, with a separate evaluation result recorded for each repeated run.
- Fix Attach custom code evaluators that use the reserved
dataset_rowparameter to an online task without mapping it to a data column. - Fix Run an evaluator on an experiment after running it on the parent dataset, so the experiment evaluation completes.
- Improvement See currently assigned webhooks preselected and replace them when editing webhook notifications in bulk from the Monitor Config editor.
- Improvement Turn on “Always accept” directly in the Alyx human-in-the-loop drawer to accept the current response and all future responses in that category.
Run high-volume structured evals with Jev-as-a-Judge
September 26, 2026 New Evaluators You can now evaluate traces with TypeSafe AI’s Jev decision model from Evaluator Hub. Add a TypeSafe AI key under Settings → AI Providers, create a Jev-As-A-Judge evaluator with a shared state template and typed boolean, choice, or score questions, then attach it to an online eval task. Each record sends one System One call and publishes a label and confidence (or rubric) score per question, without free-text explanations, so you can cover structured checks at higher volume than a chat-model judge. Learn more about Jev-as-a-Judge and TypeSafe AI provider setup.Connect Anthropic Messages API gateways as Custom Model Endpoints
September 23, 2026 New Models and Integrations You can now add a Custom Model Endpoint that speaks the Anthropic Messages API, so you can connect a gateway that exposes only/v1/messages and authenticates with its own custom headers. Choose the API format when you create the endpoint, then use it in the prompt playground, Test Integration, and online evaluations exactly like an OpenAI-compatible endpoint.
- API Format selector: pick OpenAI or Anthropic on the Custom Model Endpoint form. The format is fixed at creation and shown read-only when you edit the endpoint, because changing an endpoint’s wire protocol would affect every evaluator and task that uses it.
- Custom headers: an Anthropic-format endpoint sends the headers you configure on the integration, so a gateway with its own authentication scheme works without a separate API key.
Filter runs in the new experiment compare view
September 23, 2026 Improvement Datasets and Experiments The query filter bar and the hide-all-null-outputs toggle now sit above the stacked runs table in the new experiment compare view, so you can narrow a comparison to the examples you care about, for examplequestion contains "water". The filter matches on example groups, which keeps every run under a matching example visible, and the filter text, URL parameters, and shared links behave the same as they do in the classic view.
Learn more about comparing experiments.
Run Claude Opus 5.5 in the Playground and evaluators
September 22, 2026 New Models and Integrations Select Claude Opus 5.5 from the model picker to prototype prompts, run experiments, and score traces with Anthropic’s latest Opus model. It runs with adaptive extended thinking for complex reasoning and is available through the direct Anthropic API, AWS Bedrock, and Google Vertex AI.- Anthropic:
claude-opus-5-5 - AWS Bedrock:
global.anthropic.claude-opus-5-5andus.anthropic.claude-opus-5-5 - Vertex AI:
claude-opus-5-5
Select OpenAI GPT-6 Sol and Luna in the prompt playground
September 22, 2026 New Models and Integrations Two new GPT-6 models, Sol and Luna, are now in the prompt playground model picker for both OpenAI and Azure OpenAI, so you can prototype prompts and run them against either one. Both support reasoning-effort levels from none through max, defaulting to medium, and remain available through the Chat Completions API. GPT-5.6 Sol and Luna stay available as separate models.- OpenAI and Azure OpenAI:
gpt-6-solandgpt-6-luna
Connect Together AI as an LLM integration with the SDK
September 22, 2026 New SDKs and REST APIs You can now create a Together AI LLM integration programmatically with the Python and Go SDKs, so you can manage Together AI access alongside every other supported provider without opening the UI. Pass your Together AI API key, and optionally the model names to expose, tocreate_llm.
The code example below creates a Together AI integration from your provider API key:
- Python
- Go
Fixes and improvements
September 21–23, 2026- New SDKs and REST APIs Set a project type of
application,harness, orexperimentwhen you callregister()inarize-otel, or with theARIZE_PROJECT_TYPEenvironment variable, so new projects are classified by type as they are created. - Improvement Data Fabric Attach up to 40 embedding features per record when logging data, up from 30.
Get Alyx help tailored to your experience
September 17, 2026 New Alyx Alyx now adapts how it helps based on your experience level in each part of Arize AX. It can be an onboarding buddy that explains concepts and teaches while it works, or an advanced, focused agent that concentrates on completing your tasks. The experience is catered to you, and your familiarity is considered separately across product areas.- When you are onboarding: Alyx explains unfamiliar concepts, walks through important decisions, shows you where to find the result, and suggests a useful next step. For example, while creating an evaluation, it can explain query filters, variable mappings, classification choices, sampling, and run settings as it prepares the configuration.
- When you are experienced: Alyx skips introductory walkthroughs, uses AX terminology and configuration details directly, and focuses on completing the request. For example, it can move directly to a multi-span trace analysis or configure an evaluation task, then report the result and any important caveats.
- When your experience differs by area: Alyx can stay concise and technical in Tracing while providing onboarding guidance in Evals and Tasks, so learning a new workflow does not mean repeating introductions in areas you already know.
Pick up where you left off with Alyx long-term memory
September 17, 2026 New Alyx Alyx now remembers durable context across conversations, including your preferences, ongoing goals, project and asset learnings, team conventions, and important decisions. Instead of starting each chat from scratch, you can continue an investigation, measure progress, and reuse what you learned elsewhere in Arize AX. For example, if you are working with Alyx to reduce errors in an agent, it can:- Continue your goal: Ask, “How are we looking today?” and Alyx can connect the question to your error-reduction goal and compare the latest data with prior checkpoints.
- Return to the signals that matter: If you focused on
TransportQueryErroror another recurring pattern, Alyx can revisit it in a later conversation. - Connect your AX workflow: Alyx can remember how a project, evaluator, dataset, prompt, or experiment relates to your goal—for example, which evaluator you created to track the errors.
- Carry forward discoveries: Context such as test traffic, expected retries, deployment effects, and known schema quirks can inform the next investigation instead of being rediscovered.
- Work the way you prefer: Alyx can continue grouping results by root cause, deployment, or customer impact when that is how you like to investigate.
Edit a managed agent automation’s prompt without recreating it
September 15, 2026 Improvement Agents You can now update the prompt on an existing managed agent automation from the Edit prompt action in the automation detail menu, so you can refine how the agent runs without deleting and rebuilding the automation. The edit changes only the prompt, so the automation keeps its schedule, connected agent integration, repository, and run history in place. Learn more about managed agents.Populate datasets with every matching record, faster
September 14, 2026 Improvement Datasets and Experiments Populating a dataset from your traces now writes every matching record instead of applying a sampling budget that could drop rows, and appends complete two to seven times faster. This change reaches all account tiers, so building or growing a dataset from spans captures the full set you selected. For more information, refer to the curate a dataset from traces documentation.Fixes and improvements
September 14–16, 2026 Alyx- Fix Save dashboards and widgets through Alyx reliably, without a stalled or interrupted session, even when a widget uses an unsupported time-range value.
- Fix Resume an Alyx session with your connected integrations still available.
- Fix Reopen an Alyx thread without re-running an action you already auto-accepted.
- Fix Read Alyx answers as plain text instead of raw JSON.
- Fix Load APPROX_QUANTILE custom metric widgets across large time ranges.
- Fix Configure the Attribute and split-by pickers on dashboard widgets for tracing-only projects.
- Fix Delete a custom agent role even after a managed agent has used it.
Add and annotate entire sessions and traces in annotation queues
September 8, 2026 New Annotations You can now send full multi-turn sessions and complete traces to an annotation queue, not just individual spans and dataset examples, and review them at session or trace granularity. Add sessions to a queue from the Sessions table or a session’s detail view in the app, or add sessions and traces through the REST API and the Python, TypeScript, and Go SDKs, so you can build session-level and trace-level review workflows end to end. One queue can hold a mix of record types, so a whole session, a whole trace, an individual span, and a dataset example can sit side by side, and the annotator completes each record as its own unit of work. A queued session renders as the full session for the annotator, who labels it without leaving the queue. The code example below adds a session to an existing queue for annotation at session granularity:- Python
- TypeScript
- REST
Inspect remote evaluator runs as traces
September 8, 2026 New Evaluators Remote evaluator runs now emit a trace for each evaluated row into your space’s evaluations project, with every span capturing the request, response, status code, and latency, so you can inspect and debug remote scoring calls and see retries as sibling spans in one trace. Turn it on with the Enable Tracing toggle under Advanced settings when you create or edit a remote evaluator. Learn more about remote evaluators.Select OpenAI GPT-6 in the prompt playground
September 8, 2026 New Models and Integrations You can now select OpenAI’sgpt-6-astra in the prompt playground model picker and run prompts against it. The model supports reasoning-effort levels from low through max, a verbosity control, and structured output, and it routes through the OpenAI Responses API for tool calling.
Learn more about the prompt playground.
See GPT model reasoning in Alyx with thinking mode
September 4, 2026 Improvement Alyx Alyx now streams reasoning summaries for OpenAI GPT models through its thinking view, which previously showed reasoning only for Claude models. When Alyx runs on a supported GPT model, you can follow its reasoning as it works through a request rather than seeing only the final answer. Learn more about Alyx.Fixes and improvements
September 3–9, 2026 Evaluators- Improvement Run evaluations continuously against your space’s evaluations tracing project; tracing is automatically kept off for these tasks so they do not feed their own results back in.
- Improvement Choose which columns annotators see when you create a labeling queue from a dataset’s Examples tab, a trace, a session, or any of their selection toolbars.
- Fix See spans that behave like errors in the trace errors tab, including those whose name or status message contains terms such as timeout, failure, or exception, alongside spans that carry a formal error status.
- Improvement Start an automation right away: creating an automation now runs it immediately and opens its detail view instead of waiting for the next scheduled run.
- New Detach tags from monitors, evaluators, and prompts through the REST API with
DELETE /v2/monitors/{monitor_id}/tags,DELETE /v2/evaluators/{evaluator_id}/tags, andDELETE /v2/prompts/{prompt_id}/tags.
Run Agent-as-a-Judge evaluators on every plan
September 1, 2026 Improvement Evaluators Agent-as-a-Judge evaluators, which use an agentic judge to inspect traces, find the relevant spans, and label issues, are now available on every plan instead of Enterprise only. Any account can create one from the New Evaluator menu in the evaluator gallery and let the judge adapt its criteria to the failures that emerge in production, so you can evaluate agent behavior that a fixed template cannot anticipate. Learn more about Agent-as-a-Judge.Run Claude Fable 5.1 in the Playground and evaluators
September 1, 2026 New Models and Integrations Select Claude Fable 5.1 from the model picker to prototype prompts, run experiments, and score traces with Anthropic’s latest Fable model. It is available through the direct Anthropic API as well as AWS Bedrock and Google Vertex AI, and runs with adaptive extended thinking for complex reasoning.- Anthropic:
claude-fable-5-1 - AWS Bedrock:
us.anthropic.claude-fable-5-1andglobal.anthropic.claude-fable-5-1 - Vertex AI:
claude-fable-5-1
Deep dive on Signal issues with Alyx
September 1, 2026 Improvement Alyx Alyx can now read the issues Signal has recorded for your space, so you can ask what Signal has found across a project or single out one issue and have Alyx analyze it in depth. Signal reports on cost, token waste, latency, prompt quality, and tool choice, much of it on traces that never returned an error.- Context from the Signal tab: the issue you have open is attached to each message as a pill in the composer, and Alyx answers questions about issue counts and timing from it instead of querying traces.
- Attach an issue from any page: add an issue to the conversation through the
@menu, where it is listed under Signal Insights, the same way you attach projects, datasets, and experiments. - A pointer to prior analysis: when you ask Alyx to debug a trace that Signal already reviewed, it links you to the matching issue instead of repeating that analysis.
Open the filtered traces behind an Alyx answer
September 1, 2026 Improvement Alyx When you select Apply filter in Alyx or ask it for a link to a set of traces, Alyx now opens the tracing view showing exactly those traces, with the correct filter expression, environment, and time range already applied. This works across projects and for rolling time windows such as the last 15 minutes, which now resolve to live data instead of a fixed time range. Learn more about Alyx.Annotate entire sessions from the session view
August 31, 2026 New Annotations Select Annotate Session in a session’s detail view in the tracing page, then apply a label or score to the session as a whole through the annotation pane. Each annotation is recorded assession_annotation.<name>.label, .score, and .text, so you can query and filter on a session annotation the way you would any other attribute.

- Python
- TypeScript
Switch between trace and session results in the evaluator preview
August 29, 2026 Improvement Evaluators After choosing a filter, you can now switch between trace and session results in the Preview Matching Data step of the evaluator’s Configuration column, and select any trace or session from the results table to preview its data for easier variable mapping. Choose a row with its checkbox, or move through the results with the up and down arrows beside the Table and Single Trace toggle. Whichever record you select fills the Single Trace view and the per-variable value previews. A session-scoped evaluator leads with the Sessions tab, and the Traces and Spans tabs narrow to the session you selected. The Single Trace view shows one of that session’s traces at a time, newest first, and the arrows step through the rest. Your selection resets when you change the project, filter, scope, or preview window. Learn more about trace and session evals.Fixes and improvements
August 27 – September 2, 2026 Datasets and Experiments- Improvement Large dataset uploads now complete reliably, with transient chunk failures retried automatically before the upload returns.
- Fix The Experiments REST API (beta) now reports accurate created and updated timestamps for each experiment.
- Fix Template and LLM evaluators now keep the model’s explanation even when a provider returns free-form text instead of structured output.
- Fix Spans from Bedrock native InvokeModel calls now include the output message role, so you can filter, group, and evaluate on it.
- Fix Prompt Playground experiment runs now finish when the worker queue is busy, giving each run time to start before any timeout applies.
- Fix Signal issue deep links now open the exact issue, even when it is resolved or older than the 30-day issue window.
- Fix The Python SDK now resolves a
spacename to that exact space before listing resources, so a name that is a substring of another space no longer returns the wrong space’s resources. - Fix The TypeScript and Go SDKs now let you switch a task between span and trace or session query shapes when you update it.