Get what a holdout start would do
Reads, before starting, what a holdout test would do on the site: the planned durations (3, 7, 14 and 28 days; 3 only when it reaches the precision target) with the sessions each group would receive and the smallest gap the test could confirm for this site's traffic and baseline add-to-cart rate, the recommended duration, and why a start is refused when it is. Estimated from the last 7 days of traffic, so traffic that changes during the run changes the figures. A read, open to every member.
Authorization
ApiKeyAuth A WorkOS org-scoped API key, created from the dashboard's organization settings page. Identifies the tenant every read and write is scoped to. Missing, malformed, unknown, revoked, expired, or non-organization-owned keys all return 401 (no distinction, so the response can't be used to probe which keys or org ids exist).
In: header
Path Parameters
The site id (prefixed site_). A site id that isn't yours is a 404, same as any other cross-tenant id.
Response Body
application/json
application/problem+json
application/problem+json
application/problem+json
application/problem+json
curl -X GET "https://example.com/api/v1/sites/string/holdout/start-options"{ "data": { "siteId": "site_01kvnwmagvf27rc9cze5rm9e1r", "canStart": true, "blockedBy": null, "controlPercent": 50, "trafficFloor": 5000, "observedSessionsLast7Days": 80000, "estimable": true, "baselineRate": 0.078, "targetGapPoints": 0.78, "recommendedDays": 7, "recommendedReachesTarget": true, "options": [ { "days": 7, "sessionsPerGroup": 40000, "confirmableGapPoints": 0.6, "confirmableGapRelative": 0.77, "reachesTarget": true, "recommended": true }, { "days": 14, "sessionsPerGroup": 80000, "confirmableGapPoints": 0.5, "confirmableGapRelative": 0.64, "reachesTarget": true, "recommended": false }, { "days": 28, "sessionsPerGroup": 160000, "confirmableGapPoints": 0.4, "confirmableGapRelative": 0.51, "reachesTarget": true, "recommended": false } ], "note": "Estimated from the last 7 days of traffic: traffic that changes during the run changes these figures." }}Extend a holdout test
Extends the site's running holdout test by 7, 14 or 28 days: its planned end moves later and its identity stays, so the sessions already assigned keep their group and the group without the assistant stays without it until the new end. `holdout_not_extendable` once the test has ended, reached its planned end, or when it has no planned end (it was started before planned ends existed): start a new test instead. Records an audit entry (`site.holdout_extended`) attributed to the API key.
List the holdout tests of a site
Returns every holdout test of the site, the running one included, newest first: what the runs table of the Holdout tab shows. Each test has its dates, planned duration, the agent version live when it started (`null` for a test started before it was recorded), the sessions and the add-to-cart share of each group, the effect in points, its `outcome` (`collecting`, `treatment_higher`, `control_higher`, `not_conclusive`, `no_verdict` or `invalidated`), the effect on mobile sessions and its validity (`unevenSplit`, `splitDrifted`, `changedDuringRun`). The figures of an ended test are the ones it froze when it ended. An invalidated test carries no figure. Groups are per browser session. An outcome never says there is no difference: past the floor without a verdict it is `not_conclusive`. `meta.matching` counts the matching tests up to 10,000 and is `null` after the first page; the cursor reads forward only.