Reference sheet · lessons 4, 5, 7

Retrieval & Graph RAG

The compressed essence of the answer-engine lessons. Print it; pin it next to your editor.

The tiers, in order

Tier	Answers when	Returns	Guard
0 · Human requested	Visitor asks for a person	Handoff into tawk.to	Phrase list; never argue
1 · FAQ	Top score ≥ threshold and gap to runner-up ≥ margin	Canonical answer verbatim + source URL	Test set: zero wrong first
2 · Approved sources	Retrieval finds allowed chunks and the model cites them	Grounded text + citations	`isAllowedUrl()` ×3, `NOT_IN_SOURCES` sentinel
3 · Fallback	Everything else	Honest “not in my sources” + offer of a person	`handoff: true`

FAQ tier dials (lesson 4)

Scores are 0..1 for exact, jaccard, tfidf; one threshold works across strategies.
threshold: minimum top score. margin: minimum gap to the runner-up (catches sibling entries).
Tune order: zero wrong → fewest missed → add variants from real transcripts.
Step up to BM25 (MiniSearch, BM25+ with fuzzy/prefix) past a few hundred entries; embeddings after that (Anthropic recommends Voyage AI; Anthropic has no embedding model).MiniSearch Claude docs: Embeddings
Precedent: Rasa retrieval intents + rules; Dialogflow CX generative fallback only on no-match.Rasa: Chitchat and FAQs Dialogflow CX: Generative fallback

Citations request shape (lesson 5)

{ "type": "document",
  "source": { "type": "text", "media_type": "text/plain", "data": "chunk text" },
  "title": "Support channels",
  "context": "Source URL: https://help.harbor.example/support",   // read, never cited
  "citations": { "enabled": true } }

Enable on all or none of the documents in a request. All active models support citations.Claude docs: Citations
Plain text and PDF are chunked into sentences; custom-content documents are cited block by block with no re-chunking.
Response: several text blocks; cited ones carry citations[] with cited_text, document_index, document_title, and char_location (0-indexed start, exclusive end) / page_location (1-indexed) / content_block_location.
cited_text is not billed as output tokens, nor as input when passed back.
Incompatible with structured outputs (output_config.format): 400.

Domain filter rules (web search / web fetch)

Rule	Example
No scheme	`example.com`, not `https://example.com`
Bare domain includes subdomains	`example.com` covers `docs.example.com`
Named subdomain restricts to itself	`docs.example.com` excludes `example.com`, `api.example.com`
Paths: web search only	`example.com/blog` matches `/blog/post-1`; a path entry never matches a web fetch
allowed XOR blocked	Both in one request → 400
Wildcards only in paths	`example.com/` ok; `.example.com` rejected
ASCII only	Homograph domains look identical and are not

Source: server tools docs.Claude docs: Server tools, domain filtering Web fetch extras: latest type web_fetch_20260318; params max_uses, allowed_domains, citations, max_content_tokens, use_cache; only URLs already in the conversation are fetchable (url_not_in_prior_context); no JavaScript rendering; no extra fee beyond tokens (≈2,500 tokens per 10 kB page).Claude docs: Web fetch tool Web search costs $10 per 1,000 searches and always cites.Claude docs: Web search tool

Prompt caching facts

cache_control: { type: "ephemeral" } on a block, or top-level for automatic placement; up to 4 breakpoints; prefix order is tools → system → messages.
Minimum cacheable prefix: 512 tokens on Claude 5.x models (Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1); 1,024–4,096 on older models. Shorter prefixes are silently not cached.
TTL 5 minutes by default, refreshed on every hit; ttl: "1h" available. Writes cost 1.25× (5 min) or 2× (1 h); reads 0.1× (0.05× on Opus 5.5 / Sonnet 5.5).
Verify with usage.cache_read_input_tokens.Claude docs: Prompt caching

Retrieval quality levers (when you move past the FAQ)

Contextual retrieval: prepend a 50–100 token, document-aware context to each chunk before embedding and BM25-indexing. Anthropic measured top-20 retrieval failure falling from 5.7% to 3.7% (embeddings), 2.9% (+BM25), 1.9% (+reranking). Below ~200k tokens, skip retrieval and cache the whole corpus.Anthropic: Contextual Retrieval
Summary indexing + reranking: Anthropic’s RAG cookbook raised end-to-end accuracy from 71% to 81% on its own docs.Claude cookbook: RAG guide
Stores reachable from Vercel: Postgres + pgvector (Neon, Supabase; <=> is cosine distance, HNSW index), Upstash Vector (REST, hybrid dense+BM25), Pinecone (Node ≥ 22).Neon: pgvector Upstash Vector Pinecone Node SDK
Vercel AI SDK: embed, embedMany, cosineSimilarity from ai; the Anthropic provider exposes webFetch_20260318/webSearch_20260318 and cacheControl, but no embedding model.AI SDK: Anthropic provider

Graph RAG decision table (lesson 7)

Question shape	Chunks	Graph (local)	Graph (global)
One fact on one page	best	equal, costlier	wasteful
Two facts joined across pages (multi-hop)	misses the bridge at small k	best	n/a
Themes across the corpus	cannot	cannot	best, prone to answering null questions
Unanswerable question	refuses well	ok with sentinel	hallucinates

Definition: “GraphRAG is RAG, where the Retrieval path includes a knowledge graph.”Neo4j: GraphRAG manifesto
Microsoft pipeline: TextUnits (1200 tokens) → entities/relationships/claims → hierarchical Leiden → community reports → embeddings; modes Local, Global, DRIFT, Basic.Microsoft GraphRAG: default dataflow
Measured: NQ F1 64.78 (RAG) vs 63.01 (GraphRAG); MultiHop-RAG 70.27 vs 67.02; null queries 19.27 (global) vs 96.01; indexing 135 s vs 5,560–7,702 s.arXiv 2502.11371 Prompt tokens per query 879 (vanilla) vs ~38.7k (local) vs ~331k (global).GraphRAG-Bench
JS tooling: graphology (Louvain), neo4j-driver, @langchain/neo4j; LlamaIndex graphs are Python-only.

Prototype API (examples/)

matchFaq(faq, question, { strategy: "tfidf", threshold: 0.5, margin: 0.1 })  // → { hit, entry|reason, ranked }
isAllowedUrl(url, ["harbor.example", "docs.partner.example/help"])               // → boolean
groundedAnswer(question, chunks)                                                 // → { text, citations, grounded }
answer(question, { faq, allowedSources, retrieve, generate })                    // → { tier, text, citations, handoff }
new GraphIndex(parseTriples(text)).neighborhood(seeds, 2)                        // → { nodes, triples }