iAdvizeDocs

List the holdout tests of a site

Returns every holdout test of the site, the running one included, newest first: what the runs table of the Holdout tab shows. Each test has its dates, planned duration, the agent version live when it started (`null` for a test started before it was recorded), the sessions and the add-to-cart share of each group, the effect in points, its `outcome` (`collecting`, `treatment_higher`, `control_higher`, `not_conclusive`, `no_verdict` or `invalidated`), the effect on mobile sessions and its validity (`unevenSplit`, `splitDrifted`, `changedDuringRun`). The figures of an ended test are the ones it froze when it ended. An invalidated test carries no figure. Groups are per browser session. An outcome never says there is no difference: past the floor without a verdict it is `not_conclusive`. `meta.matching` counts the matching tests up to 10,000 and is `null` after the first page; the cursor reads forward only.

GET
/api/v1/sites/{siteId}/holdouts

Authorization

ApiKeyAuth
AuthorizationBearer <token>

A WorkOS org-scoped API key, created from the dashboard's organization settings page. Identifies the tenant every read and write is scoped to. Missing, malformed, unknown, revoked, expired, or non-organization-owned keys all return 401 (no distinction, so the response can't be used to probe which keys or org ids exist).

In: header

Path Parameters

siteId*string

The site id (prefixed site_). A site id that isn't yours is a 404, same as any other cross-tenant id.

Query Parameters

status?string

Only running or only ended tests.

Value in

  • "running"
  • "ended"
outcome?string

Only tests with this outcome.

Value in

  • "collecting"
  • "treatment_higher"
  • "control_higher"
  • "not_conclusive"
  • "no_verdict"
  • "invalidated"
from?string

Only tests started at or after this instant (ISO 8601).

Formatdate-time
to?string

Only tests started at or before this instant (ISO 8601).

Formatdate-time
cursor?string

The meta.nextCursor of the previous page, with the same filters.

limit?integer

Tests per page, 1 to 50 (default 50).

Range1 <= value <= 50

Response Body

application/json

application/problem+json

application/problem+json

application/problem+json

application/problem+json

curl -X GET "https://example.com/api/v1/sites/string/holdouts"
{  "data": [    {      "id": "hold_01kvnwmagvf27rc9cze5rm9e2s",      "status": "running",      "startedAt": "2026-11-02T08:41:29.000Z",      "endedAt": null,      "plannedEndAt": "2026-11-09T08:41:29.000Z",      "plannedDurationDays": 7,      "endedReason": null,      "durationShare": null,      "agentVersion": null,      "outcome": "collecting",      "control": {        "assignedSessions": 886,        "convertedSessions": 83,        "rate": 0.0937      },      "treatment": {        "assignedSessions": 831,        "convertedSessions": 51,        "rate": 0.0614      },      "effectPoints": -3.2,      "mobileEffectPoints": -3.5,      "validity": {        "unevenSplit": false,        "splitDrifted": false,        "changedDuringRun": true      }    },    {      "id": "hold_01kvnwmagvf27rc9cze5rm9e1r",      "status": "ended",      "startedAt": "2026-10-28T19:40:37.000Z",      "endedAt": "2026-10-31T06:11:14.000Z",      "plannedEndAt": "2026-11-04T19:40:37.000Z",      "plannedDurationDays": 7,      "endedReason": "merchant_stopped",      "durationShare": 0.35,      "agentVersion": {        "id": "agtv_01kvnwmagvf27rc9cze5rm9e1r",        "number": 9      },      "outcome": "not_conclusive",      "control": {        "assignedSessions": 13952,        "convertedSessions": 1139,        "rate": 0.0816      },      "treatment": {        "assignedSessions": 13530,        "convertedSessions": 1030,        "rate": 0.0761      },      "effectPoints": -0.6,      "mobileEffectPoints": -0.5,      "validity": {        "unevenSplit": true,        "splitDrifted": false,        "changedDuringRun": true      }    }  ],  "meta": {    "nextCursor": null,    "matching": 2  }}

Get what a holdout start would do

Reads, before starting, what a holdout test would do on the site: the planned durations (3, 7, 14 and 28 days; 3 only when it reaches the precision target) with the sessions each group would receive and the smallest gap the test could confirm for this site's traffic and baseline add-to-cart rate, the recommended duration, and why a start is refused when it is. Estimated from the last 7 days of traffic, so traffic that changes during the run changes the figures. A read, open to every member.

Get a holdout test in detail

Returns one holdout test of the site with what the test view shows: its reading (`analysis.result`, with the effect and its always-valid `interval` once both groups reach the floor), the fixed-sentence `summary`, the add-to-cart events and the value added to carts per group and currency (median and mean per session with a cart, descriptive, never revenue), the effect by device (a lead, never a verdict; `sameSign` counts the site's other tests that reached the floor), the sessions per hour and the cumulative share every 6 hours, what the group with the assistant did (funnel, widgets, starter questions, conversations by type and resolution), the checks on its validity (the split with its p-value, the pixel, widgets that did not show, an early stop) and the audited changes made while it ran. Only changes made through the dashboard, the REST API or the MCP server are audited, so `changes` is a floor, never proof that nothing else changed. `analysis` is `null` for an invalidated test. `404` for a test that does not exist, belongs to another site or to another organization.