Support · Cited replies
MixedB2B SaaS supportCited reply drafts for a SaaS help desk, approved by an agent
Most SaaS tickets ask something the help centre already answers. This architecture drafts a cited reply for each one, checks it against the source, and lets a support agent approve it before the customer sees a word.
Valnora & Elric team6 min read
- Knowledge BasePolicies and procedures agents search before they answer.
- WorkflowsThe visual builder: blocks, models and integrations wired into agent logic.
- TablesA database, built in, for cases, queues and decisions.
- DashboardsCharts, tables and KPI tiles over governed metrics.
- MetricsNamed, reusable measures every chart and agent reads the same way.
- MonitorsWatch a metric and alert the right people when it moves.
- LogsEvery run traced block by block.
A mid-sized SaaS company gets a few hundred tickets a day. Most ask something the help centre already answers: how to reset SSO, why an invite link expired, where to find invoices. Tier 1 agents spend their mornings finding the right article, pasting it and rewording it. The hard tickets wait.
This post sets out an architecture in Agent Serve that drafts a reply for every new ticket, cites the help articles it used, checks the draft against those sources and hands it to an agent to approve. Nothing reaches a customer without a person.
The shape of the solution
From a new ticket to an approved, cited reply
Data · Help docs
- 01Help Center articlesSynced from Zendesk or Intercom with the Knowledge Base connector, on a schedule.
- 02Product docs Knowledge BaseArticles, release notes and known-issue pages, searched by meaning, each chunk tied to its source URL.
Workflow · Draft and check
- 03New ticketThe Zendesk or Intercom trigger starts a run for each new ticket or conversation.
- 04Classify and draftAn Agent block tags the topic, searches the Knowledge Base and writes a reply that cites the articles it used.
- 05Hallucination checkA Guardrails block scores the draft against the Knowledge Base. Below the threshold, no draft is offered.
- 06Agent approvalA Human in the loop step shows the draft and sources. The agent approves, edits or rejects it.
- 07Reply postedThe approved text is added to the Zendesk ticket or sent as an Intercom reply.
Dashboard · Deflection and CSAT
- 08Reply outcomes tableOne row per ticket: topic, check score, approved or edited, reopened, CSAT rating.
- 09Support dashboardDeflection rate, edit rate, CSAT by topic and the articles cited most.
- 10Quality monitorAlerts the support lead when CSAT on drafted replies drops below a set level.
There are three parts: a Knowledge Base kept in step with your help docs, one workflow per ticket, and a table and dashboard that show whether the drafts actually help.
Keeping the Knowledge Base in step with the help centre
The Knowledge Base has connectors for Zendesk and Intercom that sync Help Center articles on a schedule. If your docs live elsewhere, connectors for Notion, Confluence, Mintlify and others work the same way. Each article is split into chunks that keep a link back to the source, which is what makes citations possible.
Keep the Knowledge Base narrow. Public help articles, release notes and a short list of known issues are enough. Internal runbooks and old macros tend to produce drafts that are accurate for an agent but wrong to send to a customer.
The workflow, block by block
The workflow starts from the Zendesk or Intercom trigger when a ticket or conversation is created.
- Classify. An Agent block tags the topic and priority and decides whether the ticket is a question at all. Bug reports, billing disputes and anything that mentions a security incident skip the draft and go straight to the right queue.
- Search and draft. The same agent searches the Knowledge Base and writes a short reply in your tone. It must cite the article for each step, as numbered sources at the end. The instructions say plainly: if the sources do not answer the question, say so and do not guess.
- Hallucination check. A Guardrails block set to Hallucination Check scores the draft against the Knowledge Base on a 0 to 1 scale. Below your threshold, the workflow posts no draft. It adds an internal note with the closest article and a docs-gap tag, so the agent starts from facts, not a guess.
- Approval. A Human in the loop block shows the agent the draft, the sources and the score. The agent approves, edits or rejects.
- Post. On approval, the workflow adds the reply to the Zendesk ticket as a comment, or sends it with Intercom's reply action. It also writes a row to the outcomes table.
What the solution produces
Draft shown to the agent (illustrative)
Record- Ticket
- #48213 · "SSO login loops back to the sign-in page"
- Topic
- Authentication · SAML
- Draft reply
- This usually means the ACS URL in your identity provider does not match your workspace. Copy the ACS URL from Settings → Security → SSO into your IdP app, then sign out fully and try again. [1] [2]
- Sources
- [1] Set up SAML SSO (updated 3 weeks ago) · [2] Fix SSO redirect loops
- Hallucination check
- Score 0.91 · passed (threshold 0.80)
- Not included
- No account-specific changes. No promise of a fix date.
Low-score case (illustrative)
MessageTicket #48240 · "Can I export audit logs to our SIEM?"
Hallucination check score 0.42 · below threshold.
No draft offered. Closest article: "Export audit logs as CSV". It does not mention SIEM export.
Routed to Tier 2 and tagged docs-gap.
A Guardrails block set to PII Detection can sit before the Agent block to mask card numbers or passwords that customers paste into tickets.
Measuring deflection honestly
Deflection is easy to overstate. Here it has a strict meaning: a ticket solved on the approved first reply, with no reopen and no further agent message within seven days. A second scheduled workflow checks each ticket after seven days and updates its row in the outcomes table, along with the CSAT rating your help desk already collects.
The dashboard reads that table and shows:
- deflection rate by topic, week on week
- edit rate: how often agents change the draft before sending
- CSAT on drafted replies next to CSAT on replies written from scratch
- the articles cited most, and the docs-gap tags that came up most
The edit rate is the most useful number early on. A topic where agents rewrite every draft points at an article that needs work, not a model that needs changing. Define deflection rate and edit rate as named metrics so every panel counts them the same way.
How to measure it
38%
Solved on the first reply (illustrative)
No reopen and no further agent message within seven days.
1 in 5
Drafts edited before sending (illustrative)
Edits point at articles that need rewriting.
Illustrative example. Figures show how the workflow is measured; they are not customer results.
A monitor checks CSAT on drafted replies every few hours and alerts the support lead by email or Slack if it falls below the level you set.
Controls
Controls and governance
Approvals before action
Relays never send email or book meetings without an approval, under any autonomy setting. An approval covers the exact payload the reviewer saw and expires if nobody decides.
Run logs
Every workflow run records each block’s inputs, outputs and errors, so a reviewer can trace how a result was reached.
Deployed versions
Workflows run from a deployed version, so a change is reviewed before it reaches production runs.
PII detection
A Guardrails block detects personal data and blocks or masks it before text reaches the next step.
Roles and permission groups
Admin, write and read roles per workspace, plus permission groups that limit which models, blocks and integrations a team can use.
Bring your own keys
Use your organisation’s own model provider keys at organisation or workspace level.
More on how Agent Serve handles access, data and audit on the security page.
Every run is in the logs, block by block, so when a customer questions an answer you can see which articles were retrieved, what the check scored and who approved it. Workflows run from a deployed version, so a change to the prompt or the threshold is reviewed before it reaches live tickets.
Where to start
Pick the five topics with the most tickets and turn the workflow on for those only. Run it for a week with a threshold of about 0.8, read every rejected draft, and fix the articles behind them before widening the scope. For checking the finished replies against a written rubric, the same tickets can feed a QA review; see the related posts below.
Questions teams ask
Does any reply go to a customer without a person?
Not in this design. The Human in the loop step waits for an agent to approve, edit or reject each draft. Some teams later let one narrow topic skip approval, but that is a separate decision you make on the evidence from the dashboard.
What does the hallucination check actually test?
The Guardrails block searches the same Knowledge Base and scores how well the draft is supported by what it finds. It catches claims that have no source. It does not prove a source is current, so stale articles still need an owner.
Can this run on a help desk other than Zendesk or Intercom?
Zoho Desk, Jira Service Management and ServiceNow also have blocks. The pattern is the same: a trigger or schedule, the draft and check, an approval, then an update on the ticket.
What you need
- A Zendesk or Intercom account with Help Center articles you trust
- The Knowledge Base connector set to sync those articles on a schedule
- A tone and policy note for replies: what agents may promise and what goes to Tier 2
- A hallucination threshold to start with, tuned after a week of reviews
- A reply outcomes table and the CSAT rating your help desk already collects
- Named Tier 1 agents who approve drafts, and a support lead who owns the dashboard