> ## Documentation Index > Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt > Use this file to discover all available pages before exploring further. # October 2026 ## Find and fix eval gaps with the Eval Health agent October 21, 2026 NewEvaluators Eval Health is a managed agent that reviews the evals on a tracing project and proposes changes for you to accept. It reads the project's recent spans, tasks, and evaluators, then files an issue for each coverage gap or broken eval it finds, along with a proposed action: * **Create:** add an eval for a part of your app that isn't covered yet. * **Fix:** repair an eval in place, such as its column mappings, query filter, or template and labels. * **Delete:** remove an eval that's no longer producing useful signal. * **Optimize:** cut eval cost by converting an LLM judge to a code eval or suggesting a different judge model. Nothing changes until you accept it. Every proposal is ready to run the moment you approve it, and a run that finds sufficient coverage proposes nothing. Find it by adding the Eval Health Agent on any tracing project. ## Score whole conversations with trace and session remote evaluators October 21, 2026 ImprovementEvaluators At trace and session scope, a remote evaluator now receives each mapped variable as an array with one value per span instead of a single comma-joined string, so your endpoint can score an entire conversation in one call. Spans are ordered by start time, each trace's spans stay together, and every variable's array covers the same spans in the same order, so index `i` refers to the same span across all variables. This is the default for every trace and session remote evaluator; span scope is unchanged. * Spans with no value appear as `null`, so each variable's array stays aligned with the others. * Variables that exceed the size cap are truncated, and `metadata.truncated_variables` lists which ones were cut. The code example below shows the `input` payload a trace-scoped remote evaluator now receives, with one array entry per span in the trace: ```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} { "input": { "user_input": ["Can you tell me my deductible?", null, null], "agent_response": ["Let me check, one moment.", null, "Your individual deductible is $1,500, and $780 is met."] } } ``` Learn more about [remote evaluators](/docs/ax/evaluate/remote-evaluators.md). ## Correlate evaluation scores with their task and run October 21, 2026 NewSDKs and REST APIs Listing spans and traces now returns each evaluation's `metadata`, including the `task_id` and `run_id` of the evaluation task that produced the score, so you can tie a score back to the task and run that generated it without a separate Flight export. The field is returned on both `/v2/spans` and `/v2/traces`, and it is omitted when an evaluation has no metadata. The code example below lists spans and reads the task and run behind each evaluation: ```typescript theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} import { listSpans } from "@arizeai/ax-client"; const { data: spans } = await listSpans({ project: "My Project", space: "my-space", limit: 100, }); for (const span of spans) { for (const evaluation of span.evaluations) { // metadata is populated for scores written by an evaluation task if (evaluation.metadata) { console.log(evaluation.name, evaluation.metadata.task_id, evaluation.metadata.run_id); } } } ``` Learn more about [listing spans](/docs/api-clients/typescript/version-1/client-resources/spans.md). ## Run trace and session evaluators on datasets October 20, 2026 NewDatasets and Experiments Trace-level and session-level evaluators can now run over a dataset, not only over live projects and experiments, so you can score multi-span traces and full sessions from data you have already collected. The evaluator groups the dataset's rows into trace or session units and writes one verdict per unit instead of scoring each row on its own. If you created such a task through the API or SDK, it switches from per-row to per-unit scoring. Learn more about [trace and session evaluations](/docs/ax/evaluate/trace-and-session-evals.md). ## Connect Perplexity and Cloudflare as AI providers October 20, 2026 NewModels and Integrations Perplexity and Cloudflare are now available as AI providers in Arize AX, so you can use them for prompts and evaluations alongside your existing providers. Cloudflare is also available as a Decision Judge provider, which means you can run Decision Judge evaluations on Cloudflare models. Learn more about [AI provider integrations](/docs/ax/security-and-settings/integrations-playground/overview.md). ## Track Claude Code spend by model and user October 20, 2026 NewModels and Integrations A new Anthropic Console cost connector pulls Claude Code usage and spend from Anthropic's Claude Code Analytics API, so you can track what your organization spends on Claude Code and break it down by model and by user. Connect it with an Anthropic Admin key to import daily token usage and estimated cost. Learn more about [tracking costs](/docs/ax/instrument/track-costs.md). ## Restrict Alyx to one AI provider and model October 20, 2026 NewAlyx Space admins can now lock Alyx to a single AI provider and model from Space Settings, so everyone in the space works through an approved model and Alyx rejects requests for any other one. Once a model is set, members see it fixed in the Alyx model picker. Learn more about [Alyx](/docs/ax/alyx.md). ## Edit a saved dashboard's scope and name October 20, 2026 ImprovementDashboards and Visualizations You can now rename an account dashboard and change its saved scope after creating it, using the Edit button on the dashboard. Point it at the entire account, specific organizations, or specific spaces without rebuilding the dashboard, and its default time range is preserved. Learn more about [dashboards](/docs/ax/observe/dashboards.md). ## Fixes and improvements October 20–21, 2026 **Evaluators** * Improvement See how many rows an evaluation skipped and why: run summaries now count rows skipped for null input values and name the columns responsible, so successes, skips, and failures account for every row fetched. **Datasets and Experiments** * Fix Stop a task run and get an accurate status: the run is marked cancelled, keeps the counts it scored before stopping, and a fully-skipped run reports the reason. **Playground** * Fix Saving a prompt version to the hub now persists the full set of model parameters, including unchanged defaults, so a reloaded prompt reproduces the original configuration. * Fix Open prompts saved from spans that contain tool calls in the prompt playground. **Tracing and Sessions** * Fix Build trace filters that hold: NULL checks commit as is empty or exists, OR connectors stay OR after you clear a row, pasted SQL filters parse correctly, and View Traces from the agent graph keeps your filter. **Annotations** * Fix View a session's traces and compare them from an annotation queue: View All Traces uses the session's own time window, and trace comparison works on queue routes. **Custom Metrics** * Fix Trigger a managed agent on any custom metric in its space, not only a metric defined in the trigger's own project. **Agents** * Improvement Disconnect a connected Slack workspace from your account on the Slack Agents settings page, rather than only removing individual channel bindings. **Alyx** * Fix Accept an Alyx custom metric proposal on production-only and multi-environment models: Alyx now validates a proposed metric against the environment the target model actually uses. * Improvement Attach up to 16 custom request-metadata pairs to Alyx calls on an AWS Bedrock integration, so your own Bedrock logs attribute usage to Arize AX users. **Models and Integrations** * Improvement Track OpenAI spend using the per-API-key costs reported by OpenAI's Costs API, so per-key and per-project totals match your actual bill. ## Build and run Decision Judge evaluators with Alyx October 19, 2026 NewAlyx You can now ask Alyx to create, edit, and run Decision Judge evaluators from chat, so you can go from a description of what to check to scored spans. Alyx fills in the state template and typed questions in the Decision Judge editor, and nothing saves until you accept it. It works with any Decision Judge provider in your space, including Jev-as-a-Judge on TypeSafe AI; name one, such as OpenAI, to use it. ![Alyx chat responding to a request to create a Decision Judge evaluator with a Create Decision Judge Evaluator proposal, showing the evaluator name Jev Journey Classification, the AI Provider set to gpt-6-luna, a state template with user request and response variables, and a choice question named primary\_journey with its instructions](https://storage.googleapis.com/arize-assets/DecisionJudgeEval.png) * **Run**: Alyx reuses a matching evaluator or proposes a new one, creates a project task, checks its column mappings, and runs it over historical LLM spans. * **Edit**: Accepted changes to the state template, questions, or description save as a new version. * **Explain**: Alyx shows an evaluator's model, integration, state template, and the criteria for each question. Learn more about [Decision Judge](/docs/ax/evaluate/decision-judge.md). ## Run Decision Judge evals on OpenAI decision models October 16, 2026 ImprovementEvaluators Jev-as-a-Judge is now **Decision Judge**, a provider-neutral evaluator type. The same shared state and typed boolean, choice, and score questions can run on TypeSafe AI's `jev-latest` or OpenAI's `gpt-6-luna` through your existing OpenAI integration. Pick the provider from the AI Provider dropdown in the evaluator editor; only integrations in your space that can run Decision Judge evaluations are listed. Most Arize eval templates now have a Decision Judge version, loaded by selecting a TypeSafe AI model on the template card. Existing Jev-as-a-Judge evaluators keep working unchanged. Links to the old docs page redirect to the new one. Learn more about [Decision Judge](/docs/ax/evaluate/decision-judge.md). ## Reference a range monitor's second threshold in alert notes October 13, 2026 ImprovementWebhooks and Events When you compose a monitor's alert note, you can now insert the second threshold of a range monitor with the `{{threshold2.operator}}` and `{{threshold2.value}}` variables, so an alert can state both bounds of the range that triggered it. For example, a note of `Metric is {{threshold2.operator}} {{threshold2.value}}` renders the evaluated second bound in each alert. The two variables appear in the variable picker on the **Monitor Notes** field, in the **Notifications** section of the monitor configuration, and only when the monitor's threshold mode is set to range. The existing `{{threshold.operator}}` and `{{threshold.value}}` variables continue to reference the first threshold. For more information, refer to the [monitor notifications](/docs/ax/machine-learning/machine-learning/how-to-ml/monitors/configure-monitors/notifications-and-integrations.md) documentation. ## Filter traces by clicking a bar on an eval chart October 12, 2026 ImprovementTracing and Sessions On the **Evals and Metrics** tab in LLM tracing, you can now click a bar on an eval chart card to filter the traces by that eval label. The full column is clickable, so you can also filter by short bars and labels such as N/A that were previously hard to target. The same action is available as a **Filter by label** submenu in each card's overflow menu. Learn more about [viewing and managing traces](/docs/ax/observe/tracing/view-and-manage-traces.md). ## Turn a Signal issue into a running evaluator from the issue October 9, 2026 ImprovementAlyx Click **Create Evaluator** on a Signal issue, and Alyx builds an evaluator for that failure pattern and starts scoring new traces with it. Alyx creates a continuous task with no historical backfill and sets up its filters and column mappings automatically. If the chat is closed, Alyx works in the background; when it finishes, **View Task** in the notification opens the task on the current page. ![Signal tab for a support agent project with a critical issue open, process\_refund executes duplicate refunds without idempotency check, and the Create Evaluator button highlighted next to Add to Dataset and GitHub above the issue's overview, evidence, and proposed fix](https://storage.googleapis.com/arize-assets/SignalEvalAlyxCreation.png) Learn more about [Signal](/docs/ax/observe/signal.md). ## Fixes and improvements October 8–14, 2026 * Fix  Tracing and Sessions Keep building span filters without losing your in-progress text: **Add Filter** in the trace filter bar now appends the next filter instead of replacing the one you are editing. * Fix  Tracing and Sessions Display and filter tracing columns whose names contain dots, such as a `workflow.name` metadata key, a dotted prompt-template variable, or an annotation like `quality.v2`. * Fix  SDKs and REST APIs Call the Python SDK against a server of a different version without errors when a REST response omits fields the client expects. * Fix  CLI Commands Run `ax ai-integrations list` to list a space's AI integrations even when the space contains a TypeSafe AI integration. ## Create and manage webhooks from the REST API, SDKs, and CLI October 7, 2026 NewWebhooks and Events You can now create webhooks and subscribe them to prompt and evaluator events from the REST API, the Python, TypeScript, and Go SDKs, or the `ax` CLI, so you can script event delivery to your own systems, such as a CI job that runs when a prompt version gets a new label. A webhook is an HTTPS endpoint in your organization that authenticates each delivery with a bearer token or an HMAC-SHA256 signature. A subscription connects one webhook to one event on one prompt or evaluator. These endpoints are in alpha and may change without notice. * **Events**: `PROMPT_VERSION_CREATED`, `PROMPT_VERSION_LABELED`, and `PROMPT_VERSION_UNLABELED` for prompts; `EVALUATOR_VERSION_CREATED` for evaluators. * **Tests and delivery history**: Send a test delivery to a bearer-token webhook, and list each webhook's delivery attempts, including retries. * **Signing secrets**: Arize AX returns an HMAC-SHA256 webhook's signing secret only once, in the create response, so store it when you create the webhook. * **Versions**: Python `arize` 8.54.0 or later (`client.webhooks`), TypeScript `@arizeai/ax-client` 1.30.0 or later, Go SDK v2 v0.24.0 or later, and `arize-ax-cli` 0.39.0 or later (`ax webhooks`). The code example below creates a bearer-token webhook and subscribes it to new versions of a prompt: **Python** ```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} import os from arize import ArizeClient from arize.webhooks.types import ( WebhookAuthType, WebhookEventType, WebhookSourceType, ) client = ArizeClient() # reads ARIZE_API_KEY from the environment webhook = client.webhooks.create( organization="your-org-name-or-id", name="prompt-release-hook", url="https://hooks.example.com/arize", auth_type=WebhookAuthType.BEARER, # Sent as the full Authorization header value on each delivery auth_token=f"Bearer {os.environ['WEBHOOK_AUTH_TOKEN']}", ) subscription = client.webhooks.create_subscription( webhook=webhook.id, source_type=WebhookSourceType.PROMPT, source_id="your-prompt-id", event=WebhookEventType.PROMPT_VERSION_CREATED, ) print(subscription.id) ``` **TypeScript** ```typescript theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} import { createWebhook, createWebhookSubscription } from "@arizeai/ax-client"; // The client reads ARIZE_API_KEY from the environment const webhook = await createWebhook({ organization: "your-org-name-or-id", name: "prompt-release-hook", url: "https://hooks.example.com/arize", authType: "BEARER", // Sent as the full Authorization header value on each delivery authToken: `Bearer ${process.env.WEBHOOK_AUTH_TOKEN}`, }); const subscription = await createWebhookSubscription({ webhook: webhook.id, sourceType: "PROMPT", sourceId: "your-prompt-id", event: "PROMPT_VERSION_CREATED", }); console.log(subscription.id); ``` **Go** ```go theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} package main import ( "context" "fmt" "log" "os" "github.com/Arize-ai/client-go-v2/arize" "github.com/Arize-ai/client-go-v2/arize/webhooks" ) func main() { // An empty Config reads ARIZE_API_KEY from the environment client, err := arize.NewClient(arize.Config{}) if err != nil { log.Fatal(err) } ctx := context.Background() wh, err := client.Webhooks.Create(ctx, webhooks.CreateRequest{ Organization: "your-org-name-or-id", Name: "prompt-release-hook", URL: "https://hooks.example.com/arize", AuthType: webhooks.WebhookAuthTypeBearer, // Sent as the full Authorization header value on each delivery AuthToken: "Bearer " + os.Getenv("WEBHOOK_AUTH_TOKEN"), }) if err != nil { log.Fatal(err) } sub, err := client.Webhooks.CreateSubscription(ctx, webhooks.CreateSubscriptionRequest{ Webhook: wh.Id, SourceType: webhooks.WebhookSourceTypePrompt, SourceID: "your-prompt-id", Event: webhooks.WebhookEventTypePromptVersionCreated, }) if err != nil { log.Fatal(err) } fmt.Println(sub.Id) } ``` **CLI** ```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} # BEARER is the default auth type ax webhooks create \ --organization "your-org-name-or-id" \ --name "prompt-release-hook" \ --url https://hooks.example.com/arize \ --auth-token "Bearer $WEBHOOK_AUTH_TOKEN" # A webhook name needs --organization to resolve ax webhooks subscriptions create \ --webhook "prompt-release-hook" \ --organization "your-org-name-or-id" \ --source-type PROMPT \ --source-id "your-prompt-id" \ --event PROMPT_VERSION_CREATED ``` Learn more about [webhooks](/docs/ax/security-and-settings/webhooks.md), the [`ax` CLI](/docs/api-clients/cli/overview), and the [Python](/docs/api-clients/python/version-8/client-resources/webhooks.md), [TypeScript](/docs/api-clients/typescript/version-1/client-resources/webhooks.md), and [Go](/docs/api-clients/go/version-2/client-resources/webhooks.md) SDK references. ## Filter spans by nested JSON attributes with path typeahead October 6, 2026 NewTracing and Sessions When you build a span filter on the tracing page, Arize AX now suggests nested JSON attribute paths as you type, so you can drill into structured span attributes without recalling the exact path. Clicking a nested attribute value in a span adds it to your filter directly. The feature is available on every account tier across cloud and self-hosted deployments. For more information, refer to the [view and manage traces](/docs/ax/observe/tracing/view-and-manage-traces.md) documentation. ## Export full datasets and experiments to Parquet from the Python SDK October 5, 2026 NewDatasets and Experiments The Python SDK can now stream an entire dataset or experiment to a Parquet file on disk, batch by batch, so exports larger than available memory complete without loading every row into memory at once. The file holds the same rows and columns as `list_examples(all=True)` and `list_runs(all=True)`, and is written to a temporary `.partial` file and renamed on completion, so a failed export leaves any existing file untouched. The code example below streams a full dataset export and a full experiment export to Parquet files: ```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} from arize import ArizeClient client = ArizeClient(api_key="YOUR_API_KEY") # Stream every example in a dataset to a Parquet file on disk. client.datasets.export_to_parquet( dataset="my-dataset", # dataset ID or name space="my-space", # required when dataset is passed by name path="examples.parquet", ) # Stream every run in an experiment to a Parquet file on disk. client.experiments.export_to_parquet( experiment="my-experiment", # experiment ID or name path="runs.parquet", ) ``` Learn more about [exporting datasets](/docs/api-clients/python/version-8/client-resources/datasets.md) and [experiments](/docs/api-clients/python/version-8/client-resources/experiments.md). ## Select GPT-6.1 Sol and Claude Sonnet 5.5 in the Playground October 1, 2026 NewModels and Integrations Two new models are available for prompts and evaluations in Arize AX: OpenAI's GPT-6.1 Sol and Anthropic's Claude Sonnet 5.5. Select either from the model picker in the Playground to run prompts, build evaluators, or power online tasks. * **GPT-6.1 Sol**: available on OpenAI and Azure OpenAI, with a 128,000-token completion limit and configurable reasoning effort. * **Claude Sonnet 5.5**: available on Anthropic, AWS Bedrock, and Google Vertex AI, with a 128,000-token completion limit and adaptive reasoning effort. Learn more about [supported model providers](/docs/ax/security-and-settings/integrations-playground/overview.md). ## Fixes and improvements September 29 – October 6, 2026 **Tracing and Sessions** * Fix View harness projects, such as `claude-code`, on the Tracing Projects page and through the `GET /v2/projects` REST endpoint again; each is marked with a Harness label so it is not mistaken for a production project. * Fix See a consistent score on task-less evaluator cards in the Tracing project Metrics tab, whether the card is pinned or unpinned. * Fix Read accurate session metrics, such as total tokens, for sessions whose traces contain more than one root span. **Evaluators** * Fix See accurate evaluator scores on experiments, where nested metadata columns are no longer counted toward an evaluator's score. * Improvement Filter evaluation tasks by whether they run continuously, using the new run-mode dropdown on the Tasks page and the Running Eval Tasks tab of the Evaluators page. **Alyx** * Fix See the true dataset row count when Alyx previews a dataset, so it recognizes when a preview covers only part of the data and filters server-side. **Dashboards and Visualizations** * Fix Export dashboard widget data and view pivot tables with your active filters applied, so the exported and displayed rows match what you filtered to. **Annotations** * Fix Open annotation queue and dataset records that include an annotation named `notes`, which previously failed to load. **Webhooks and Events** * Fix Convert webhook list responses to a DataFrame with `to_df()`, which now returns one row per webhook, delivery attempt, or subscription. **Models and Integrations** * Fix Run Claude models selected through AWS Bedrock inference-profile ARNs or custom profile names, including regional profile IDs (`eu.`, `apac.`, `jp.`, `au.`, and `us-gov.`), in the Playground and LLM-as-a-judge evaluators, with the required token limits sent on every request.