Conversation Quality

Conversation Quality answers the questions every owner of a messaging agent eventually asks: how often does it actually help, how much is it handling, and through which channels? Instead of scrolling the inbox to form an impression, you get outcomes and volumes for a period you choose, each shown beside the same period before it — so «better than last month» becomes a number rather than a feeling.

A typical use: you turned on a new channel two weeks ago and want to know whether it brought real conversations or just noise. Pick the last 30 days, choose that agent, and the channel breakdown answers it directly.

It also lets your agents get better by themselves. From the questions an agent couldn’t resolve, it learns what to change — a line in its instructions or an article in its knowledge base — checks the change, and either makes it or asks you first, then keeps an eye on whether answers got better. See How your agent improves itself.

Where to find it

AI Agents → Conversation Quality. You need the Conversation Quality permission; ask your workspace administrator if the item is not in your menu.

Choosing what you are looking at

Two controls at the top of the page:

Agent — a single messaging agent, or All agents for the whole workspace. Your choice stays in the page address, so you can send a colleague a link to exactly the view you are looking at.
Period — Today, This Month, the last 7, 30 or 90 days, the last year, or a custom range from the calendar button.

Dates and hours follow the timezone on your profile.

The four tabs

The page is split by the question you came with, so each view opens straight to its answer:

Overview — how the agent did at a glance: what it resolved on its own, what went to your team, who is still waiting for a reply, what the agent has learned, and how sessions ended.
Learning & improvements — how your agents learn and whether the weekly report goes out, the questions the agent didn’t resolve, the changes waiting for your decision, and every change your agent received and how it worked. The number beside the tab is how many changes are waiting for your decision.
Handovers & team — what became of the conversations handed to your team, and how your team compares with the agent.
Volume — how much came in, through which channels and when, and which messages went unanswered.

The open tab stays in the page address together with the agent, so a link you send opens on the same tab. A link to a conversation — an example, a suggestion’s evidence, a missed reply — opens it in a new browser tab, in that agent’s Conversations, where you can reply or pause the AI for that conversation, so you keep your place here. Wherever a list names a customer, their name is the link. The links in the weekly report open this page on the report’s week.

The three numbers at the top

Three numbers open the Overview tab and tell you at a glance how the agent did and whether anyone is waiting for your team:

Resolved by the agent — of the conversations that were real requests, how many the agent resolved on its own, without your team taking part — for example «61 of 233 requests». It carries a small trend of the last eight weeks.
Handed to team — of the same requests, how many ended in your team’s hands: the agent handed over, a colleague stepped in, or the customer was sent to your team by email or phone.
Customers waiting — customers who wrote while the agent was paused or switched off and have had no reply since, in the conversation or by email, with how many of them asked a question. See who opens them under Missed replies on the Volume tab.

The first two count the same requests as How sessions ended below them. Handed to team is the same share as its segment there; Resolved by the agent counts only what the agent resolved without your team, so it can read lower than the resolved part of the bar, which also holds conversations your team took part in. Each is compared with the same period before it — «up from 50%», «down from 45%» or «same as before» — and each number carries its exact definition behind the small info icon.

What your agent has learned

The card under the three numbers tells you at a glance whether self-learning is paying off and whether anything needs you. For example, 2 improvements are ready for your agent with Drafted from conversations not resolved: Order status (27), Contact request (16). means the agent prepared two changes from conversations it couldn’t finish — 27 about order status, 16 about contact requests — and they wait for your decision. Review 2 improvements opens Learning & improvements, where they are listed under Needs your decision.

While nothing waits, the heading says whether the agent is learning — Your agent is learning, or Your agent isn’t learning when Self-learning is Off — with a tally such as 6 improvements · 1 taken back · questions it learned: 31% → 58% resolved by the agent alone. Before the first change, it says so instead, for example No changes yet — they appear here as the agent learns from conversations it didn’t resolve.

Four numbers follow:

Learned so far — every change listed under Changes, with how many were taken back automatically because answers got worse. A change you undid yourself, or one taken back because a tool it relies on was turned off, is not counted as taken back.
Waiting for you — the changes waiting for your decision, with the date the first of them closes by itself.
Still to learn — how many questions the agent didn’t resolve in the period you picked; see Questions the agent didn’t resolve.
Room in instructions — how many more lines fit in the agent’s instructions, with how many characters they hold of the 40,000 allowed. With All agents and two or more agents, Agents learning counts those whose Self-learning isn’t Off instead.

Under them, Most asked, not resolved names the question the agent left unresolved most often, for example Holster compatibility (Fit and choice) · 43 conversations · 15 gave up, and See the questions opens the full list. The next line names the latest change, for example Latest: Order status (Orders and delivery) — Look the order up before handing over, with its state, such as Checking or Working; with several agents it also names the agent. All changes opens Learning & improvements at Changes.

«Questions it learned» («questions they learned» with several agents) compares, for the changes whose check is over, how often the agent resolved their questions on its own before and after. Changes that tighten a rule, such as handing more questions to your team, are left out, since they are expected to leave more to your team, and so are changes with too few conversations before them to credit them. Until those changes cover at least five requests before and five after, this part of the tally is not shown.

The card counts every change to date, not only those in the period you picked, so it shows even when the period has no finished conversations. With an agent selected it describes that agent; with All agents, all of them together.

Changing the Self-learning setting here. The bottom of the card holds the same Self-learning setting as the top of Learning & improvements, so you can switch the mode right from Overview — it works exactly as described in How your agent improves itself, including the confirmation for Full autopilot. With one agent selected, or with All agents if you have only one, it shows the agent’s Mode with what that mode does, and in Automatic the option to add what it learns to the knowledge base by itself. If that agent has no instructions of its own yet and its mode would change them by itself, it adds Acts as Suggest only until this agent has its own instructions. With All agents and two or more agents, Self-learning on for 2 of 3 agents counts those learning, and each agent is listed with its own mode, which you can change there.

Changing the setting needs permission to manage messaging agents. People who can open Conversation Quality but can’t view messaging agents see the mode named instead, for example Self-learning: Automatic.

Outcomes

Every conversation is reviewed once it ends, and the outcomes are gathered on Overview, under the three numbers — so «the agent handles most things» becomes a share you can watch week by week.

How sessions ended splits every real request by outcome: resolved, likely resolved, handed to team, left unresolved or abandoned — each name carries what it means. A conversation also counts as handed to team when the agent sent the customer to your team another way, such as by email or phone, or promised that someone would follow up using contact details the customer had already left. Only asking for their contact details does not count. Story mentions, bots and lone reactions are counted underneath and left out of every percentage, and so are threads your team started itself and the agent never replied in — an outreach message or a campaign is not a question the agent was asked. Beside it, Who handled it splits the same requests by who carried them: The agent alone, Your team took part — a person wrote at some point, and the agent may have answered there too — and Nobody answered. Its last line, Left to the agent, says how many of the requests the agent was left to finish on its own it resolved, so a week in which your team took more over does not make the agent look worse than it was.

The last line of the card is the Review queue: whether any outcome is waiting for someone to confirm it. While something waits, a Review queue card follows with the newest of them, whichever period you picked; it appears for people who can read conversations. Press Review to confirm or correct an outcome right there, or open the conversation from the customer’s name to read it in full first. Correcting works as described in Correct an outcome, and needs permission to manage messaging agents.

Handovers

The Handovers & team tab follows a handover from start to end. Five tiles first: Transfer rate — how many of the conversations the agent answered reached a person, whether it handed over or someone stepped in; Escalation rate — how many of them it handed over itself; Escalations picked up — how many of those a person replied to, in the conversation or by email; Time to first human reply — how long the first reply in the conversation took; and Operator takeovers — how often a person stepped in without being asked. A lot of takeovers usually means the team does not yet trust the agent; a conversation someone on your team started, and the agent only joined later, is not one of them.

Below the tiles, three cards explain the period’s handovers. Why the agent handed over lists the reasons found in the conversations — an answer the agent was never given, an action outside its reach, a customer who wanted a person. What happened after shows how your team’s part ended; Sent to email or phone counts the handovers where the agent sent the customer to your team by email or phone, or promised your team would get in touch that way, so nobody was asked to reply in the conversation itself. Answered by email counts the handovers nobody answered in the conversation that your team answered from your connected mailbox, in an email thread the customer wrote in. Why your team stepped in lists what prompted the takeovers. All three point at what to improve: an answer to add, or a kind of conversation your team does not yet trust the agent with.

Questions the agent didn’t resolve

This section, near the top of the Learning & improvements tab, turns the outcomes into a to-do list: what your customers keep asking that the agent does not resolve on its own. If «return shipping cost» tops it week after week, that is the answer to add to the agent’s knowledge, or the kind of question to hand to your team sooner.

Questions are grouped by kind, such as Orders and delivery or Fit and choice, the busiest kind first, so a dozen similar questions read as one thing to fix. Each kind shows how many sessions raised it, how many of them were Handed to team — the agent handed them over, a colleague stepped in, or the customer was sent to your team by email or phone — and in how many the Customer gave up: nobody from your team stepped in, and the customer left dissatisfied with the agent’s replies or stopped answering its question. The rest were resolved by your team: those are included on purpose, as the ones the agent could learn to finish itself. The five busiest kinds show first; Show all lists the rest. A kind with a change waiting for your decision on one of its questions says so, for example 1 change waits.

Open a kind to see its questions, each with the same numbers. The same question asked in different words is combined into one automatically, so it is counted once instead of competing with itself for your attention. A question is listed once it came up in at least three sessions; a session about two questions counts under both, but once in their kind. People who can read conversations can open a question to see up to three recent examples; open one to read the conversation in full. When a change is waiting for your decision on that question, Show it takes you to it under Needs your decision.

The Overview tab shows the three busiest kinds and how many changes need your decision, with a link to Learning & improvements, and its self-learning card names the single question left unresolved most often.

How your agent improves itself

Your agent learns from the conversations it couldn’t resolve, so the same question stops reaching your team week after week. Every week it looks at those questions, drafts a fix, checks it, and — as far as you allow — makes it by itself; everything else waits for your decision. Then it measures whether answers got better, and takes the change back if they got worse.

For example: before, the agent handed every «Where is my order?» to your team, although it can look orders up. After a week it has added a line to its instructions telling itself to look the order up first. You see the change under Changes, and a few weeks later how many of those questions it now resolves on its own.

Each agent has its own Self-learning setting, which decides how much of this the agent may do by itself and how much waits for you. For example, you might keep a new agent on Suggest only while you get to know its suggestions, and switch a well-tuned one Off during a seasonal campaign whose instructions you don’t want touched.

You find it at the top of the System Prompt card, on the agent’s AI Reply tab under Messaging Agents. The line under its name says since when the agent has been learning. While one agent is in view, the same setting is also on the Overview tab, beside what your agent has learned. The modes:

Off — learns nothing new and closes the suggestions waiting for you. Changes already made stay, and are still taken back if answers get worse.
Suggest only — prepares improvements, and each one waits for your OK.
Automatic (recommended) — improves its instructions and its knowledge base by itself, and asks you before changing text you wrote, rules like discounts, refunds, handover and personal data, or retrying a change you said no to or took out. Its option Add what it learns to the knowledge base by itself is on by default; switch it off and New facts wait for your OK instead.
Extended — also rewrites text you wrote, and always adds what it learns to your knowledge base. It asks you only about rules like discounts, refunds, handover and personal data, and before retrying a change you said no to.
Full autopilot — decides everything by itself, including rules like discounts, refunds, handover and personal data. Choosing it opens a confirmation that lists what still protects you; tick I understand the agent may change these rules by itself and press Turn on Full autopilot.

In every mode, each change is checked and taken back automatically if answers get worse. Every change is listed under Changes, with Undo, and a change to the instructions is in History too. A change you undo is never made again. Articles wait when your knowledge base is full or their document is turned off, and an agent without its own instructions asks before every change.

What the agent changes by itself. In Automatic, the agent adds new lines to its instructions, and rewrites or removes lines it added itself or that came with its role (when it has a single role), without waiting for you. For example, if the agent keeps handing order-status questions to your team although it can look orders up, it may add a line telling itself to look the order up first, and you find the change under Changes.

Extended also rewrites or removes text you wrote. If your instructions say «Answer in a friendly tone» and customers keep asking the agent to get to the point, it may reword your line to «Answer in a friendly tone, in two or three sentences». It also tries again a question whose change you took out while editing the instructions more widely. Full autopilot goes further and also changes rules like discounts, refunds, handover and personal data, and it never comes back to a question you said no to. Either way, the agent changes at most one line you wrote in any seven days.

Articles. From Automatic up, the agent also adds what it learns to your knowledge base by itself — in Automatic only while its option Add what it learns to the knowledge base by itself is on. When your team keeps answering the same question — who pays for return shipping, say — the agent writes an article from their answers and adds it to the Learned from conversations folder of your knowledge base, in the agent’s document for that kind of question. It looks such an article up when a customer asks about it, rather than reading it with every reply, and your own documents take priority over it. See Facts your agents learned.

When your team later answers the question differently, the agent updates its article rather than adding a second one. An article your team approved or reworded counts as text you wrote: Automatic asks before updating it, and Extended and Full autopilot update it within the same limit of one of your texts in any seven days. Articles count toward your plan’s knowledge-base limit; when the knowledge base is full, or the document an article belongs in is turned off, the article waits for you instead under Needs your decision.

Before it makes a change, the agent checks it against its instructions and the changes you said no to:

A change that would relax a rule, letting the agent do more on its own, waits for you — except in Full autopilot, which makes it.
A change that contradicts a line you wrote waits for you in Automatic. In Extended and Full autopilot it rewrites your line instead, unless the agent already changed one of your lines in the last seven days — then it is not made. One that contradicts a line the agent added itself, or a line from its role, rewrites that line instead.
A change to the instructions that repeats a line already there or a change you said no to, or that states a price, a policy or another fact about your business, or someone’s personal details, is not made — facts belong in articles.
An article with someone’s personal details, or one about a single customer’s order or situation rather than an answer for everyone who asks, is not added, and neither is one that repeats an article you said no to — unless it corrects a figure in it, such as a new price or deadline. One that offers a discount, a refund or another exception waits for you in Automatic and Extended, like any change to such a rule.
A change that can’t be checked is not made, and its question is looked at again next time.

The agent makes at most two changes by itself in any seven days, and only one in its first seven days of doing so; the weekly run and Find improvements now share this limit. Once it has made them, it drafts nothing new — no change and no suggestion — until the seven days free up. Each change appears under Changes and is checked like any other. If answers get worse, it is taken back — an article is then removed from its document, or the earlier article it replaced is put back, unless someone reworded it since. A change to the instructions is marked Improved automatically in the agent’s History.

The agent still asks you first, under Needs your decision, before it:

rewrites or removes a line you wrote, or adds one that contradicts it — in Automatic;
changes a rule like discounts, refunds, handover or personal data — including rewriting or removing any line with a price, a percentage or another number in it, or a line that hands customers to your team — in Automatic and Extended;
adds or updates an article — in Automatic while its option is off, and in any mode when the knowledge base is full or the article’s document is turned off;
updates an article your team approved or reworded — in Automatic;
makes a change for a question where you said no to an earlier change or undid one — in every mode but Full autopilot, which leaves that question alone for good;
makes a change for a question whose change you took out in a larger edit of the instructions — in Automatic. Taking one or two of the agent’s changes out while editing counts as saying no to them.

If you turn off a tool that a change the agent made by itself relies on, the agent notices at its next weekly run and, if the tool is still off a week later, takes the change back automatically. This is not a no from you, so the agent may learn that question again.

Taking a change back. Every change, whoever made it, is listed under Changes with Undo, and an instruction change also in the agent’s History. A change you undo is never made again. Editing the instructions yourself counts the same way: if a save takes out one or two of the agent’s changes — by deleting their lines, restoring an earlier version, or pressing Reset to role default and saving — the agent reads it as a no to those changes. A larger rewrite that takes out more is simply your new text, and the agent may learn those questions again; so may applying a role, which replaces the changes (Replaced under Changes) without saying no to them.

What can’t be measured. A change whose question gets too few conversations to compare is kept, and Changes says Kept — too few conversations to tell: there is nothing to judge it by.

Every week the weekly report tells you what your agents changed by themselves and what waits for your decision.

Good to know:

Switching to any mode but Off leaves the suggestions already waiting for you to decide. A suggestion Off closed stays closed when you switch back, though its question may be suggested again from newer conversations.
Agents created before this setting existed start on Suggest only; new agents start on Automatic.
Changing the setting needs permission to manage messaging agents. While Conversation Quality is turned off, the setting is shown but can’t be changed.

Suggestions

Suggestions turn the questions your agent keeps failing into fixes you can apply in one step. For a question the agent didn’t resolve, you get one of two drafts:

A knowledge-base article, when someone on your team gave customers the answer the agent was missing. If your team explained who pays for return shipping five times last week, the draft puts that explanation where the agent will find it next time.
A change to the agent’s instructions, when the agent had what it needed and still handled the question wrongly — for example, it passed delivery questions to your team although its knowledge base answers them, or guessed an order’s status instead of looking the order up. The change adds a line, or rewrites or removes one already there when that line is what led the agent astray.

Suggestions take the agent’s enabled tools into account. Something a tool looks up live in a connected service, such as your store’s stock, an order’s status or free calendar slots, never becomes an article; when the agent should have used a tool it has, the suggestion is an instruction saying when to use it.

New suggestions arrive every Wednesday, drafted from the questions of the week before; to get them sooner, use Find improvements now. Each agent gets at most three in any seven days, however they were asked for. An article is written only from what your team told customers, and an instruction describes how the agent should behave, never prices or policies. A question neither would help with gets no suggestion and stays under Questions the agent didn’t resolve. Suggestions are written in the language of your agent’s instructions. Depending on the agent’s Self-learning setting, the agent makes some changes by itself (see How your agent improves itself); the rest wait for you under Needs your decision.

Suggestions go where there is something new to learn:

A question isn’t looked at again while a change for it waits for you or is still being checked, and an instruction change already waiting for you is not suggested a second time for another question.
A question the agent already looked at without finding a useful change waits about eight weeks, unless it comes up at least twice as often.
A question you said no to waits about three months — in Full autopilot, for good — and a question whose change you took out in a larger edit of the instructions, or whose learned fact you removed in a larger clean-up of the knowledge base, waits about eight weeks.
A question whose change was taken back because answers got worse waits about two months, or about six months after the second time.
A question your team is meant to take — most of its conversations handed over because of a rule, the customer’s wish or their feelings — gets no suggestion.
A disabled agent, and an agent with Self-learning set to Off, get no suggestions. An agent with AI Reply turned off still does: it learns from how your team answered.

The weekly report tells you how many changes wait for your decision and what your agents changed by themselves.

Needs your decision

A change listed here reaches your agent only once you say yes, so this list is where you decide. Under Suggest only every suggestion waits here; from Automatic up, only the changes that need a person do — which ones depends on the mode (see How your agent improves itself). On Learning & improvements it follows the questions the agent didn’t resolve, newest first, and its header says how many changes wait for you. While nothing waits, it holds only Find improvements now.

Each change is one row: what applying it does, its title, the question it is about, and a line such as 27 conversations not resolved · the week of Sep 7 · closes Oct 21. Details shows why it was suggested, the change itself and a few example conversations; a link to a change, such as Review in the weekly report, opens with its details shown. Each change tells you:

What applying it does — for example This adds a line to the agent’s instructions, This adds an article to the knowledge base or This updates an article in the knowledge base. A change to your own wording says so instead: This rewrites a line you wrote, This removes a line you wrote, This adds a line that contradicts one you wrote or This rewrites an article you wrote.
Why it waits for you, when there is a particular reason:
- It touches a rule like discounts, refunds, handover or personal data, so it waits for you. When the agent’s own check found what the change would loosen, it says so in a line under the headline.
- New articles wait for your OK before they join the knowledge base.
- You said no to an earlier change for this question.
- You edited out an earlier change for this question.
- Your knowledge base is full. Remove an article you no longer need, or move to a larger plan, to add this one. It comes with Open the knowledge base.
- The document it belongs in is turned off in the knowledge base. Turn it on to add this article. It also comes with Open the knowledge base.
- This agent uses your shared instructions; applying gives it its own copy. For an article: This agent uses your shared instructions, so it asks before every change.
What it is about — the question and the kind it is listed under, and, with All agents selected, the agent. Learned from your team means the agent answered none of the conversations it came from, and the change follows how your team handled the question.
What it rests on — how many conversations on the question ended without the agent resolving them in the days it was drafted from, and in one sentence what customers kept asking and what went wrong. People who can read conversations also see a few of those conversations under Examples.
The change itself — the line it removes, marked −, and the line it adds, marked +, for people who can read the agent’s instructions; for an article, its title and opening.
When it closes — a change nobody decides on closes by itself about four weeks after it arrived, on the date shown, and leaves the list.

When a change doesn’t fit. A messaging agent’s instructions hold up to 40,000 characters. If applying a change would take them past that, the row says so — for example Doesn’t fit: the instructions are 39,850 of 40,000 characters. Shorten them and this can be applied. — and Apply stays unavailable. Shorten them opens the agent, where you can trim its instructions; once there is room, the change can be applied. You can still say no to it, and it closes by itself on its date like any other.

If the agent’s instructions changed after a suggestion was written — for example, after you applied another change for the same agent — it says so; check that it still fits them before applying it.

To apply an instruction change:

Read the change; press Details to see the line it adds or changes.
Press Apply. The agent’s instructions are saved at once, and the change moves to Changes.

If the line the change touches was edited since, or someone saved the agent’s instructions a moment before, the change is not applied: the review opens instead, showing the change against the instructions as they are now. There you can edit the line — a new line shows as Instruction, a rewrite as Replaces this line and New line, a removal as Removes this line — and press Add to instructions, Replace in instructions or Remove from instructions. If the line a rewrite or a removal points at is no longer there, the change cannot be applied: say no to it, or edit the instructions yourself.

An article, and a line too long to read in the list, show Review instead of Apply and open that review first. For an agent that uses your shared instructions rather than its own, Apply opens the review too: applying gives the agent its own copy of those instructions, with the change in it.

To add an article:

Press Review. The review says where the article goes — for example Adds to “Shop Assistant · Orders and delivery” in Learned from conversations, or, for an update, Replaces “Return shipping” in Learned from conversations with What changes below.
Read the Title and the Article, and edit either if needed.
Press Add to knowledge base.

The article becomes a question in that agent’s document in Learned from conversations, and only that agent uses it; an update replaces the earlier article on the same question. If your plan’s knowledge-base limit is reached, or the document is turned off, the article is not added and the change stays in the list until it closes, so you can add it once there is room or the document is on again.

To say no, press Keep mine on a change that rewrites or removes a line, Keep the current article on one that updates an article, or Don’t add on one that adds a line or an article. The change leaves the list, and its question waits about three months before anything new is suggested for it; in Full autopilot, nothing new is suggested for it again. An instruction change you said no to is not suggested again in the same or similar words, and an article you said no to is not added again unless it corrects a figure in it.

A change that would leave the agent without instructions, or push them past 40,000 characters, is not applied; edit the instructions yourself instead. If someone changes the agent’s instructions while you are reviewing, the change is not applied either: check the updated comparison, then press the button again.

An applied change is saved as a new version of the agent’s instructions, marked Suggestion you applied in their History, so you can compare it with the wording before. Applying a role to the agent later replaces its instructions, applied changes included; see Restore an earlier version of the instructions.

Deciding needs permission to manage messaging agents, and adding an article also needs permission to manage the knowledge base.

Find improvements now

When your team has just handled a run of questions the agent couldn’t, you don’t have to wait for Wednesday to learn from them. Find improvements now looks at the conversations of the last seven days right away and drafts suggestions from them, the way the weekly run does. For example, after a busy weekend of delivery questions, press it on Monday morning and decide on the suggestions the same day.

It sits in Needs your decision on Learning & improvements, below the changes waiting there. With an agent selected it looks at that agent’s conversations; with All agents, at every agent’s.

Open Learning & improvements.
Press Find improvements now.
While it works, the line beside the button reads Looking at recent conversations…. You can keep working meanwhile.

When it finishes, a message says how many new improvements it found — for example Found 2 new improvements — or Nothing new to suggest from the last seven days. If the agent has already made as many changes by itself as a week allows, or its weekly allowance is used up, the message says so instead. The ones that need your decision appear under Needs your decision. From Automatic up, the agent may also make some of them by itself, as the weekly run does (see What the agent changes by itself); those appear under Changes.

Good to know:

Its suggestions count toward the same limit as the weekly ones — at most three per agent in any seven days — so using it early leaves fewer for Wednesday rather than adding more.
It can be used once a day per agent. When it can’t be used yet, the line beside the button says when it can.
It looks only when conversations were reviewed since the last time; otherwise the line reads No newly reviewed conversations since the last run.
It isn’t available for a few minutes before and after the weekly run on Wednesday morning.
A disabled agent, or one with Self-learning set to Off, gets no suggestions, so it can’t be used for one.
Using it needs permission to manage messaging agents.

Changes

Changes, under Needs your decision, answers whether what your agent learned is working. It lists every change applied to the agent — newest first, with All agents naming the agent of each — and says what became of it. For example, you may see that return shipping questions went from 13% to 56% resolved by the agent after you added an article.

Each change shows the day it was applied, its title, what it did — Added a line, Rewrote a line, Removed a line, Added an article or Updated an article, with you approved this when someone applied it and nothing more when the agent made it by itself — its question, and how it went. Details shows, for people who can read the agent’s instructions, the line it added or removed, an updated article’s old and new text, and Undo. Its state is one of:

Checking — the change is being measured: Checking until the date it will be judged. A change is judged about two weeks after it was applied, and up to four weeks when conversations about its question are few. A change drafted from a week in which the agent answered nothing on its question says Checking starts when the agent answers this question.
Working — the change stays. The line says how its question went: Resolved by the agent alone before and after, or No clear change, or Kept when there were too few conversations to tell, too few before the change to credit it, or nothing earlier to compare with. For a change that tightens a rule, such as handing more questions to your team, the line names how many customers were left without an answer instead, since fewer questions resolved by the agent alone is expected.
Taken back — the change is no longer in the instructions: Taken back automatically because answers got worse, with the numbers before and after; Taken back automatically: a tool it relies on was turned off, for a change the agent made by itself; or Undone by you.
Taking back — answers got worse after the change, but it could not be taken back by itself, usually because its line was edited since. It is tried again every week; to settle it now, edit or remove the line in the agent’s instructions.
Replaced — a newer change rewrote the same line or article, a role was applied to the agent, your edits to the instructions moved past it, you removed several learned facts at once, or you reworded the line yourself and it is part of your own text now.

The measurement compares how often the agent resolved that question on its own, in the conversations it answered, in the four weeks before the week the change came from with the weeks since it was applied. If a change makes things worse — more customers left with neither an answer nor a person, or, for a change that doesn’t tighten a rule, fewer questions resolved by the agent — it is taken back automatically, whoever applied it. A change first looks for this harm a week after it is applied. Once the check is over and there were enough conversations to compare, the result may also appear in the Self-learning part of the weekly report, which lists up to three changes and links to all of them.

Read the numbers as a signal, not proof: anything else that changed in those weeks — a busy season, other changes to the agent — shows up in them too. They are taken when the change is checked and then kept, so a later correction of a conversation’s outcome doesn’t change them.

To undo a change:

Press Details on the change, then Undo.
Press Undo again to confirm.

The change comes out of the agent’s instructions, or the article comes out of its document in Learned from conversations — an update puts back the article it replaced — and every change made after it stays. An instruction change you undid is not made again in the same or similar words; an article you undid is not added again, and its question waits about three months before anything new is suggested for it (in Full autopilot, for good). You can also undo an instruction change from the agent’s History; see Undo a change you applied from a suggestion. Undo is offered while the change is in effect and not reworded by you, to people who can manage the agent — and, for an article, the knowledge base too.

The list shows the ten most recent changes, and a link at its end, such as Show 12 older changes, lists the rest. It does not depend on the period you choose, and is visible to anyone who can open Conversation Quality.

The weekly report

Once a week you get a short report on your messaging agents, so you can see how they did and act on what needs you without opening this page. For example, a report may say your agents handled 212 requests and resolved 52% of them, up from 44% the week before, that one change waits for your decision, and that an agent learned one thing by itself.

It goes out on Wednesdays, for every week your agents had conversations, to the channels chosen on the Conversation Quality digest card of Notification Channels. Until you choose there, it goes wherever your account alerts go. The Weekly digest line at the top of Learning & improvements says whether it is on and where it goes, for example Every Wednesday to email and Telegram, for weeks with conversations; Change opens Notification Channels for people who can manage them.

The report has four parts, each with a link that opens this page on the same week:

How your agents did — requests, the share resolved by the agents and the share handed to your team, each against the week before; how many customers are still waiting for a reply; and your busiest agents. Open the overview opens the Overview for that week; in the email, an agent’s name opens it for that agent.
Waiting for you — how many changes wait for your decision, and the newest of them. Review opens it under Needs your decision.
Self-learning — what your agents changed and took back by themselves in the week’s run, and how earlier changes worked out, up to three of them. See all changes opens Changes.
What customers asked that the agents couldn’t resolve — the five most frequent questions. See all questions opens them on Learning & improvements.

A week with no news in a part leaves that part out. The report’s week runs Monday to Sunday in UTC, so a link from it shows Report week with those dates, in UTC, in place of the period, and the page’s numbers match the report’s. Select the × beside it to choose a period again.

The bottom of Learning & improvements says whether the report is on. To stop it, change where it goes, or send a test to see what it looks like, use its card; Change in Notification Channels takes you there if you have permission to manage notification channels.

AI vs your team

This section, under the handovers on Handovers & team, compares the agent with the people who work alongside it. Two things are being measured, and they answer different questions.

The section opens with the headline of the period — «AI answers first 35× faster than your team» — and then puts both sides of each metric next to each other. Where one side is at least twice as good, its number stands out and carries what it wins by; above a thousandfold it reads «more than 1,000× faster», since the exact multiple stops meaning anything. Whoever is ahead is the one marked, so a week your team answered faster than the agent says so. Where the two are close, or one of them never answered, nothing is marked. Messages per session are two bars instead, since lengths of one size compare honestly; there too the side that asked less of the customer is the one marked.

Response times are measured per phase. In a conversation the agent started and a person finished, the agent’s speed is judged on its part and the team’s on theirs. After a handover the team’s clock starts at the handover, not at the customer’s original message — the team is not charged for the time the agent was still trying.

Handling time and customer effort are measured per conversation. Here the whole conversation belongs to whoever handled it, so the comparison is between conversations the agent handled alone and those your team took part in — the agent may have answered in those too, but once a person joined, the conversation counts as theirs. That is why those rows carry different labels from the rows above them. Customer messages per session counts how many messages the customer had to send before it was over, so fewer means they got there with less work.

Handovers are covered just above it. Here the comparison is about speed and effort.

Team lists each operator with their own response times and how many handovers they answered in the conversation. A handover answered by email, or not at all, belongs to no operator — Escalations picked up counts the email replies apart and shows how many handovers were answered at all. When your team answers a customer outside the inbox — in the channel’s own app, such as Instagram or WhatsApp Business, or from a CRM — nobody’s name comes with the reply, so those conversations are listed together as Replied outside the inbox.

Each metric name carries its exact definition; open it from the name itself.

Times are counted around the clock. A handover at 6 pm answered at 9 am next morning counts the whole night, so if your team does not work nights, read the team’s times with that in mind.

Volume

The Volume tab shows how much your agents handle and where it comes from.

Conversations, sessions, messages and new customers, each with its change against the previous period. A session is one continuous exchange: if the same customer comes back a week later, that is a second session in the same conversation.

Sessions by day. Daily volume split by channel, so a spike can be traced to where it came from.

Channels. Which channels carry the load, with each one’s share.

Sentiment. How the customer felt by the end — Positive, Conditionally positive (no clear satisfaction or complaint) or Negative — judged from their own messages together with the outcome. Conversations that were not a request, or that nobody answered at all, carry no sentiment, so this total is smaller than the one under Outcomes.

Busiest hours. When your customers actually write, by day of the week and hour. Useful for deciding when the team should be available.

Returning customers. How many conversations took one session, and how many customers came back for more. A new session starts when a customer writes again after the agent’s session timeout — a follow-up question, a new order or a thank-you — so coming back is not in itself a sign of a problem.

Missed replies. Messages that were owed an answer and did not get one, grouped by cause: the agent was paused or off, the agent decided not to reply, a reply was never delivered, a silent handover, and so on. Each cause carries a one-line explanation. This is the section that explains a gap between messages received and messages answered — and, opened, who is still waiting. Messages that needed no answer at all — a story mention, a sticker, an empty button tap — are counted in one line under the list, so they do not make the gap look bigger than it is.

Open a cause to see the conversations behind it, newest first, each with the customer’s last message; open a conversation from the customer’s name. A Silent handover also shows the agent’s reason for handing the conversation to your team without replying, when the agent recorded one. Show 10 more loads the next ones. AI paused or disabled is split into Nobody answered — customers who wrote while the agent was paused and got no reply at all, with how many of them end on a question above the list — and Answered later, where someone replied afterwards, in the conversation or from your connected mailbox in an email thread the customer wrote in. The agent chose not to reply opens into the agent’s own reasons, the most common first, each with a few of the conversations it gave them in. The conversations are listed for people who can read conversations; everyone else sees the counts, including how many nobody answered, and the agent’s reasons.

What is counted, and when

A session is counted from the moment it starts, so conversations still in progress are already included and today’s numbers keep growing through the day.

Sentiment, outcomes and unresolved questions are the exceptions. They are worked out after a session ends — after a quiet period equal to the agent’s session timeout — so an ongoing conversation has none of them yet and does not appear in those sections.

Suggestions follow from those outcomes, once a week. If an agent’s session timeout is longer than about two days, conversations from the last days of a week may not have ended in time to count toward that week’s suggestions.

Messages are counted by when they were sent, not by when their session began. A conversation that started yesterday evening and continued this morning shows up in today’s message counts, busiest hours and missed replies even though no new session started today — so a period can show activity with zero new sessions.

Comparisons against the previous period appear on every period except Last 1 year, which already reaches back as far as your history goes and has nothing earlier to compare with.

Every change is shown with the value it is measured from. A share or a rate says where it came from: if Resolved by the agent went from 40% to 50%, it reads «up from 40%». For a count, the change is relative: if Sessions went from 400 to 500, it reads «+25.0% from 400». If the period before holds fewer than five sessions to share — for example, right after reviews started — the tile says «too few sessions before to compare» instead, since a couple of conversations would swing the share.

Missed replies can appear on their own. Content the agent cannot answer — a story mention, a sticker, an empty button tap — is counted there as needing no answer without ever starting a conversation, so a quiet period can show nothing but that section.

Demo conversations from the agent preview are never counted — the page reports real customers only, and they are not reviewed either, so they get no outcome badge. When a period holds nothing to show, each tab says so: Overview and Handovers & team when no session in the period has finished yet — Customers waiting still shows, since a customer can be waiting before any conversation has finished — and Volume when nothing at all happened in it. Learning & improvements always shows the changes waiting for your decision and every change made, and Overview the card of what your agent has learned, because they do not depend on the period.

Where an outcome comes from

Every completed session is reviewed once and given an outcome — Resolved, Assumed resolved, Handed to team, Left unresolved, Abandoned or Not a request. The same outcome is shown as a badge above the conversation in the Inbox and in the agent’s own inbox, where anyone who can manage messaging agents can correct it. A corrected outcome is what this page counts from then on. The conversation’s short summary and the names of the questions it raised are written in the language of your agent’s instructions.

Conversations that ended before the review existed are left unjudged rather than reviewed retroactively. Ask your administrator if you want a past period brought up to date.

Turning it off

Conversation Quality reviews every finished conversation and drafts weekly suggestions, and each review is an AI request on your account. If you would rather not have that for a while — a quiet season, a trial you are not ready for — a workspace administrator can turn it off with the Reviews switch at the top of the page.

Off, nothing is reviewed or drafted, and the page shows a notice instead of How sessions ended, the card of what your agent has learned, the handover reasons and everything on Learning & improvements; the outcome badge disappears from the Inbox. Customers waiting on Overview, the handover tiles, AI vs your team and Volume keep working, since they need no review. Everything already collected is kept and comes back the moment you turn it on again.

Conversations that ended while it was off are not reviewed later: only sessions that end after the switch comes back on get an outcome. If you need a past period brought up to date, ask support.

If the switch is missing and the notice says to contact support, the feature was turned off for your workspace on our side — write to support to have it turned back on.

Messaging Agents — setting up the agents these numbers describe
Messaging Conversations — reading the individual conversations
Inbox — replying to customers yourself, and what the outcome badge means
Roles and instructions — the System Prompt the agent learns into, its History and Undo
Knowledge Base — the articles an agent adds
Billing — your plan’s knowledge-base limit
Notification Channels — where the weekly report goes