Reference sheet · lessons 2, 4, 5, 6

Answer pipeline

FAQ → approved sources → honest fallback → human. The flow, the shapes, the codes, the policy.

Flow

Shapes

interface Answer {
  tier: "faq" | "sources" | "fallback";
  text: string;
  citations: { title: string; url?: string; quote?: string }[];
  confidence: number;      // FAQ score, 1 for grounded answers, 0 for fallbacks
  handoff: boolean;        // UI should offer or trigger escalateToHuman()
  reason: string;          // analytics key, see below
}

Reason codes and what they tell you

reason	Meaning	Weekly action
`visitor-requested-human`	Tier 0 matched a phrase	Review phrases for false positives (“human resources”)
`faq:<id>`	Tier 1 hit	Spot-check a sample; watch for near-threshold scores
`no-approved-chunks`	Retrieval found nothing allowed	Crawl gap or FAQ gap: add pages to the index or variants to the FAQ
`model-declined`	Claude replied `NOT_IN_SOURCES`	The pages do not answer it, or the chunks were poor; check chunking and top-k
`insufficient-citations`	Text without allowed citations	Prompt drift or a citation pointing at a disallowed URL; inspect
`grounded`	Tier 2 answered with citations	Celebrate; sample for quality

FAQ dials

strategy (exact | jaccard | tfidf) · threshold (minimum top score, 0..1) · margin (minimum gap to runner-up). Tune on a test set: zero wrong first, then fewest missed.

Grounding prompt (rules)

1. Use only facts that appear in the documents. Do not use outside knowledge, even if you are confident.
2. Cite the documents for every factual sentence.
3. If the documents do not contain the answer, reply with exactly: NOT_IN_SOURCES
   and nothing else. Saying so is the correct answer, not a failure.
4. Be brief and concrete: two to five sentences, plain language, no preamble.
5. Never invent URLs, prices, dates or policy details.

Giving the model explicit permission to say it does not know is Anthropic’s first listed hallucination-reduction technique.Claude docs: reduce hallucinations

Handoff policy (summary)

Trigger	UI
Explicit request	Open the widget now
Any fallback	Fallback text + “Talk to a person” button; auto-open on the second fallback in a session
Escalation vocabulary	Answer if possible; always show the button
Near-threshold FAQ hit	Answer + “Did that help?”
Status offline	Ticket or email path; never an empty chat

Travels with the visitor: login(userId, hash); attributes bot-last-tier, bot-last-reason; tag and event bot-handoff; drafted summary ending with a session reference (payloads from tawk.to do not include attributes, so the transcript must carry the reference itself).

Where each piece lives

Piece	File	Lesson
FAQ matcher	`examples/src/faq-match.ts`	4
Allow-list guard	`examples/src/allowed-sources.ts`	5
Claude with citations	`examples/src/grounded-answer.ts`	5
Orchestration	`examples/src/pipeline.ts`	6
Secure mode hash	`examples/src/tawk-secure-mode.ts`	2
Webhook handler	`examples/src/tawk-webhook.ts`, `examples/api/tawk-webhook.ts`	3
Graph prototype	`examples/src/graph-lite.ts`	7