Start a holdout test
Starts a holdout test on the site: from now on half of its sessions, chosen by session, are served no assistant, until the planned end, when every session is offered the assistant again. The split is always 50/50 and a body that names a share is refused with `400`. `durationDays` is the planned duration, 3, 7, 14 or 28; without it the recommendation of `GET /holdout/start-options` is used, else 14. Exposure counts in reports drop by the same share while the test runs, and a conversation in progress can lose the assistant at its next page view. The storefront must not open the assistant from its own code (`iadvize('open')` or a `data-iadvize-open` attribute): confirm it with the merchant before calling. A test starts only when the site's add-to-cart counting is on and the Shopify pixel recorded a storefront add-to-cart in the last 7 days (`holdout_measurement_unavailable` otherwise), when no widget experiment runs on the site (`holdout_experiment_running`), and a second one cannot start while one runs (`holdout_already_running`). Records an audit entry (`site.holdout_started`) attributed to the API key.
Authorization
ApiKeyAuth A WorkOS org-scoped API key, created from the dashboard's organization settings page. Identifies the tenant every read and write is scoped to. Missing, malformed, unknown, revoked, expired, or non-organization-owned keys all return 401 (no distinction, so the response can't be used to probe which keys or org ids exist).
In: header
Path Parameters
The site id (prefixed site_). A site id that isn't yours is a 404, same as any other cross-tenant id.
Request Body
application/json
TypeScript Definitions
Use the request body type in TypeScript.
No share: the split is always 50/50, so a body that names a share or anything else is refused with 400.
Response Body
application/json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
curl -X POST "https://example.com/api/v1/sites/string/holdout/start" \ -H "Content-Type: application/json" \ -d '{ "durationDays": 14 }'{ "data": { "id": "hold_01kvnwmagvf27rc9cze5rm9e1r", "siteId": "site_01kvnwmagvf27rc9cze5rm9e1r", "status": "running", "state": "no_assignment", "endedReason": null, "startedAt": "2026-10-22T08:00:00.000Z", "endedAt": null, "controlPercent": 50, "trafficFloor": 5000, "control": { "assignedSessions": 0, "convertedSessions": 0, "rate": null, "addToCartEvents": 0, "eventsWithoutAmount": 0, "valueAdded": [], "valuePerAssignedSession": [] }, "treatment": { "assignedSessions": 0, "convertedSessions": 0, "rate": null, "addToCartEvents": 0, "eventsWithoutAmount": 0, "valueAdded": [], "valuePerAssignedSession": [] }, "progress": 0, "effectPoints": null, "winner": null, "detectionLimitPoints": null, "lastStorefrontAddAt": "2026-10-26T09:41:00.000Z", "plannedEndAt": "2026-11-05T08:00:00.000Z", "forecastGapPoints": null, "earlyResult": false, "engagementShare": null, "effectPerEngagedPoints": null }}Get a site's holdout test
Returns the site's holdout test, as the Holdout tab of the site's settings shows it: the running test, otherwise the most recent ended one. A holdout serves half of the sessions no assistant (a fixed 50/50 split, chosen by browser session) and compares the share of sessions with an add-to-cart between the two groups, counting an add-to-cart from the assistant's panel or from the merchant's page. No verdict is returned before both groups reach the traffic floor, and an inconclusive result gives the smallest gap the test could confirm (`detectionLimitPoints`), never a claim of no difference. While a test runs, reading it checks that its split is still even and ends it as invalid when it is not. `404` when the site has never had a holdout.
Stop a holdout test
Stops the site's running holdout test: every session is offered the assistant again from its next page view, and the figures stay readable with `GET /holdout`. A test stopped before both groups reached the floor has no verdict; one stopped after keeps the verdict it had. `holdout_not_running` when no test runs. Records an audit entry (`site.holdout_stopped`) attributed to the API key.