Support · Ticket QA
DashboardCustomer supportScoring every closed support ticket against your QA rubric
Ticket QA usually means a lead reading five tickets per agent a month. This dashboard scores every closed ticket against a written rubric and sends the policy misses to a person the same day.
Valnora & Elric team5 min read
Latest failed checks
| Ticket | Agent | Check | Quote | Status |
|---|---|---|---|---|
| #71842 | Elif | Policy | I have refunded the annual plan in full. | Policy miss |
| #71857 | Ben | Accuracy | Exports are limited to 1,000 rows. | In review |
| #71866 | Dev | Tone | As I already said twice... | Coached |
| #71871 | Elif | Resolution | Closing this for now. | In review |
5% → 100%
Closed tickets checked (illustrative)
Every solved ticket is scored, not a monthly sample.
Same day
Policy miss to QA review (illustrative)
The monitor puts the ticket in front of a person.
4 min
Review time per flagged ticket (illustrative)
The reviewer starts from the failing quote.
Illustrative example. Figures show how the workflow is measured; they are not customer results.
A support team closes a few hundred tickets a day across email, chat and messaging. QA is a lead reading five tickets per agent each month, in a spreadsheet, a few weeks after the fact. A full refund given against policy, or a confident answer that contradicts the docs, is found by chance if at all.
This post walks through a ticket QA dashboard built with Agent Serve Dashboards. It scores every closed ticket against a rubric your team writes in plain language, shows the results by agent, and alerts the QA lead the day a policy miss happens. The dashboard and figures shown are illustrative.
Writing the rubric in Rules
The rubric is the part that matters, so start there. Most QA forms boil down to four questions:
- Tone. Was the agent clear and polite, and did they avoid blame?
- Accuracy. Did every factual answer match the help docs and the product as it is today?
- Policy. Did refunds, credits, discounts and escalations follow the written policy?
- Resolution. Was the customer's actual problem solved, or was the ticket closed without an answer?
Each one becomes a plain-language check in Rules, with a score from 0 to 2 and a weight. A fifth rule combines them into a QA score out of 100, and a policy-miss rule flags any ticket that scores 0 on policy. Because the definitions live in one place, anyone can read exactly what was checked, and a change to the wording applies to every panel and alert at once.
Getting closed tickets into a table
A workflow starts from the Zendesk or Intercom trigger each time a ticket is solved. It fetches the full thread, masks card numbers and passwords with a Guardrails block, and passes the text to an Agent block. For each criterion the agent records a score, a one-line reason and a short quote as evidence. For accuracy and policy, it searches a Knowledge Base holding your help docs and refund policy, so it judges against your rules rather than general knowledge.
The workflow writes one row per ticket to a table: ticket ID, agent, team, channel, topic and the criterion results. The dashboard reads that table, so panels update as rows land.
What the solution produces
Scored ticket row (illustrative)
Record- Ticket
- #71842 · billing · email · agent Elif · team EMEA
- Tone
- 2 / 2 · polite and clear throughout
- Accuracy
- 2 / 2 · plan details match the pricing article
- Policy
- 0 / 2 · full refund on an annual plan in month 7. Policy allows a pro-rata refund only, after month 3, without team lead approval.
- Resolution
- 2 / 2 · customer confirmed and ticket solved
- QA score
- 70 / 100 · policy miss flag set
What each panel shows
- KPI tiles. Tickets scored, average QA score, policy misses and resolution pass rate, each against the previous period.
- Score by agent. A bar per agent, this period against last. It answers who improved and who needs coaching, and it is easy to drill from team to agent.
- Weekly trend. The average score by week and team, so a drop after a policy change or a new hire cohort is visible.
- Evidence table. The latest failed checks with the ticket, agent, criterion, quote and review status. This is where the QA lead works.
Define the QA score and the policy-miss rate as named metrics, so the tiles, the chart and the monitor all count the same way.
The monitor that catches policy misses
A monitor on the policy-miss rule checks every few minutes and fires when any ticket matches. The alert goes to the QA lead in the app and by email or Slack, with the ticket, the agent and the quote. A second monitor can watch the average QA score per team and alert when it drops by more than a set amount week on week.
Policy misses are the alert worth waking up for, because they often cost money or set an expectation the next agent cannot meet. Tone and accuracy trends are better reviewed weekly from a scheduled report.
Keeping it fair to agents
Scoring every ticket changes how agents feel about QA, so be open about it.
- Share the rubric with the team before turning it on, and keep the wording in Rules where everyone can read it.
- Let the QA lead overturn any score, and keep the overturned results. A criterion that is often overturned needs rewriting.
- Review a week of scores by hand against your current sample before anyone is coached on them.
- Use row-level access so agents see their own results and team leads see their team.
How to measure it
5% → 100%
Closed tickets checked (illustrative)
Every solved ticket is scored, not a monthly sample.
Same day
Policy miss to QA review (illustrative)
The monitor puts the ticket in front of a person.
4 min
Review time per flagged ticket (illustrative)
The reviewer starts from the failing quote.
Illustrative example. Figures show how the workflow is measured; they are not customer results.
Access and records
Controls and governance
Row-level access
Row-level policies narrow what each person sees in tables, dashboards, exports and scheduled reports.
Roles and permission groups
Admin, write and read roles per workspace, plus permission groups that limit which models, blocks and integrations a team can use.
Run logs
Every workflow run records each block’s inputs, outputs and errors, so a reviewer can trace how a result was reached.
Audit trail
Proposals, approvals, declines and executed actions are written to the workspace audit log, which admins can export.
PII detection
A Guardrails block detects personal data and blocks or masks it before text reaches the next step.
SSO and SCIM
Sign in through your identity provider with SAML 2.0 or OIDC, and provision people with SCIM.
More on how Agent Serve handles access, data and audit on the security page.
Every scoring run is in the logs, so when an agent asks why a ticket failed you can show the reason, the quote and the policy passage it was checked against. For the same approach applied to phone calls, see call QA on every call.
Questions teams ask
Does the score replace the QA lead?
No. It decides which tickets a person reads first. The QA lead confirms or overturns every policy miss, and overturned results are kept so you can see where the rubric needs clearer wording.
Can agents see their own scores?
Yes, if you want them to. Row-level access can limit each agent to their own rows and each team lead to their team, on the dashboard, in exports and in scheduled reports.
From a closed ticket to a scored row and a policy alert
Workflow · Score each ticket
- 01Ticket solvedThe Zendesk or Intercom trigger fires when a ticket is solved or a conversation is closed.
- 02Read the threadAn Agent block reads every message and records a score, a reason and a quote for each criterion.
- 03Check accuracyAnswers are compared with the help docs and refund policy in the Knowledge Base.
- 04Ticket QA tableOne row per ticket: agent, team, channel, topic, criterion scores and quotes.
Dashboard · Ticket QA
- 05QA rubric in RulesTone, accuracy, policy and resolution, each a plain-language check with a weight.
- 06QA dashboardScores by agent and team, a weekly trend and an evidence table of failed checks.
- 07Policy-miss monitorFires when any ticket fails the policy check and alerts the QA lead.
- 08QA lead reviewsConfirms or overturns the result and agrees coaching with the team lead.
What you need
- A Zendesk or Intercom account, or a help desk with an API the API block can call
- Your current QA form, rewritten as four or five plain-language criteria with a weight each
- The refund, discount and escalation policy as documents in a Knowledge Base
- Agent, team and channel on every ticket, so the dashboard can group results
- A named QA lead who reviews every policy miss and can overturn a score
- Row-level access so each team lead sees their own team only