Quality > Conversations
How labeled conversations ended and which quality criteria failed, from the AI judge's labels. The resolution rate, the resolution ring, trust failures, pass rate by criterion, and the minimum sample behind every percentage.
The Conversations tab of the Quality report shows how your assistant's conversations ended and where they went wrong. Every figure counts the labels the AI judge wrote (see Conversation labeling). Open Reports, then Quality, then Conversations, or go to /dashboard/sites/{siteId}/reports/quality/conversations.
Beta
The tab is marked Beta. The labels are judged by a model, not counted from shopper behavior, and their validation isn't finished. Use the figures to find conversations worth reading, not as a score to report.
Before you read a number
Filters and period
The tab uses the same header and filter bar as the Satisfaction tab: date range, Source, and Agents, Agent versions, Panel language, Subject or keyword and Device behind Add a filter. There is no Customer feedback filter. The filters you set carry over when you switch Quality tabs. Compare to previous period adds a change in percentage points to the tiles that have one.
Coverage line
The first line of the page says how many conversations the figures rest on, for example "Based on 412 of 450 conversations. 38 are not labeled yet."
- The first number is the labeled conversations in your filters.
- The second is labeled plus not yet labeled.
- A conversation counts as not labeled yet when it has no current label. Most are recent: a conversation is labeled about 30 to 90 minutes after its last message (see When it runs).
- Conversations whose shopper objected to measurement are left out of both numbers. The page never says why a conversation isn't labeled.
The 10,000 cap
A page reads at most the 10,000 most recent labeled conversations in your filters. When more match, a notice under the coverage line says "Figures cover the most recent 10,000 labeled conversations". Narrow the period to read all of them. The not-labeled count is capped the same way.
Minimum sample
A percentage rests on a count of conversations, and each figure has its own count. When that count is under 30, the page shows the sample size and no percentage. The 30 applies to each figure's own denominator, so one tile can show a rate while another next to it doesn't.
- A tile under the minimum reads "Not enough labeled conversations yet. Rates show from 30."
- The resolution ring needs 30 labeled conversations in total. Under that it shows "Not enough labeled conversations yet" and how many there are so far.
- A criterion judged on fewer than 30 conversations reads "Not enough judged conversations".
Every tile that does show a rate carries its sample, written n = 412.
If no conversation in the period is labeled, the page shows "No labeled conversation yet" instead of the figures.
Resolution rate
The Resolution rate tile is the share of conversations where the shopper had a need and it was resolved.
- Numerator: conversations the judge labeled Resolved.
- Denominator: labeled conversations the taxonomy doesn't exclude from the rate. It excludes a Visitor left or No real need conversation, except when the judge also flagged it for something else, such as a frustrated visitor or a failed tone criterion. Every other resolution stays in.
- Partly resolved counts as not resolved.
The tile's note spells out the counts: "310 resolved of 380. 20 labeled conversations left out." Labels written before the exclusion was recorded count in neither term.
It's a diagnostic of the assistant's conversations. It is not a satisfaction score: it says what the judge found, not what the shopper felt. For that, read the Satisfaction tab. The delta badge, when comparison is on, is never colored good or bad.
Resolution
A ring shows how all labeled conversations ended, in a fixed order: Resolved, Partly resolved, Unresolved, Visitor left and No real need. Each conversation holds exactly one value. Each part shows its count and its share of the labeled conversations, and opens the Conversations list filtered on that Resolution.
Trust and service tiles
Four tiles, each a share over its own denominator:
| Tile | Counts | Out of |
|---|---|---|
| Trust failure | Labeled conversations where at least one trust criterion failed | All labeled conversations |
| Facts not sourced | Conversations where a hard fact (such as a price, stock, a link, a delivery, return or warranty term) came from no tool result or site information | Conversations judged on that criterion |
| False action claims | Conversations where the assistant claimed an action it cannot perform | Conversations judged on that criterion |
| Loops | Conversations where the assistant repeated itself without progress | Conversations judged on that criterion |
"Judged" means the criterion came out pass or fail. A conversation where it didn't apply is in neither count.
Facts not sourced reads Stuck to known facts, which covers hard facts only since taxonomy v1.2: the full list is in the Label reference. A product description, a usage tip or a benefit the sources don't back isn't counted, and a conversation where the assistant stated no hard fact isn't judged on it. The rate isn't comparable with what the tile showed under v1.1.
Pass rate by criterion
Two lists, Service and Trust, show for each criterion the share of judged conversations where it passed, with its sample size. The lowest pass rate comes first. A criterion under the minimum sample goes last and shows no rate.
On-brand tone and Followed your instructions carry a Your reference badge. The judge compares them to your own tone of voice and instructions, so the rate says nothing about other brands.
A single criterion has no link of its own, because the Conversations list can't filter on one criterion. Each list heading has a See conversations with a failure link instead. It opens the list on the Issues filter's "A service criterion failed" or "A trust criterion failed", which matches a failure on any criterion of the group.
Conversations with a trust failure
The last card lists the labeled conversations in your filters where a trust criterion failed, as a table you can sort and page through. See all conversations opens the same set in the Conversations list (Issues filter, "A trust criterion failed").
What the links open
Every link on this tab opens the Conversations list with your period, Source, Agents, Agent versions, Panel language, Device and keyword filters carried over, plus one of:
- a Resolution value (the ring parts);
- "A service criterion failed" or "A trust criterion failed" on the Issues filter (the criteria headings, the table).
Nothing else is linked. The tiles themselves don't open anything.
What it doesn't tell you
- Only labeled conversations. A conversation not yet labeled, or one whose shopper objected to measurement, isn't in any figure.
- Beta labels. A judge can be wrong on a single conversation. Open the ones behind a figure before acting on it.
- No revenue or priority. Nothing here estimates what a failure cost you.
Quality report
Answer quality over a date range you choose, in five tabs. Satisfaction reads the thumbs-up and thumbs-down ratings; Conversations, Needs review and Knowledge read the AI judge's labels; Feedback lists every rating.
Quality > Knowledge
What the assistant lacked to answer shoppers, from the AI judge's labels. Content gap share, answer rate, the five gap types with where to fix each, the topics behind them, and who can close capability gaps.