Holdout tests now have a planned duration and end by themselves

A holdout test used to run until someone stopped it, with no way to know beforehand how long it needed. It now starts from a duration, and it tells you what that duration can show for your site.

Choose a duration at start. The start dialog offers 3, 7, 14 or 28 days. Each option shows the sessions each group should get and the smallest gap in add-to-cart rate the test could confirm, in points and as a share of your own rate. The figures come from your last 7 days of traffic. The dialog recommends the shortest option that confirms gaps of 10% of your rate or less, and says so when no option gets there. 3 days only appears when it reaches that target.

The test ends at its planned end. From that moment every session sees the assistant again, and the figures stay on the page. You can extend a running test by 7, 14 or 28 days, and stop it at any time. While it runs, the card shows a forecast of the smallest gap the test should confirm by its planned end. A test that reaches a conclusive result early says so and keeps running until you stop it: an effect read early may overstate the true effect. A test started before this change has no planned end and runs until you stop it.

A test and a widget experiment no longer run together. Both would compete for the same sessions. You cannot start a test while a widget experiment is running on the site, nor an experiment while a test is running. Paused and stopped experiments do not block anything.

More than a count. Each group now shows its add-to-cart events and the value added to carts, per currency and per assigned session. That is what was added to carts, including what was later removed: not revenue. Amounts are recorded only from late October 2026, so older events show as without an amount. A conclusive result also shows an estimated effect among the sessions that engage with the assistant.

A new tab from the panel keeps the session. A product link in the panel opens in a new tab, which used to start a new session and could put the add-to-cart in the other group. The tag now keeps the session id in your site's localStorage for 15 seconds so that tab continues it. Nothing is written for a shopper who objected to measurement. See What the tag stores in the browser.

The REST API and the MCP server follow: a start takes an optional durationDays, there is a new extension route and an extend_site_holdout tool, and a start refused because of a running widget experiment returns holdout_experiment_running.