RAG From Raw Email To Grounded Answers

Search a collection and select a small amount of information that may answer the question.

question: "What was the Book Nook reference?"

10,000 messages
      ↓ local search
3 plausible messages

Retrieval is an information-retrieval problem. Its output is evidence candidates, not prose.

Ask a language model to compose a useful answer from that bounded context.

The transaction was INR 805 at Book Nook on 24 Aug 2026.
Reference: TXN-HC-100005. [source]

Generation is a conditional language-modelling problem. The model predicts an answer using the question, instructions, and retrieved text.

The separation matters because the final answer can be wrong for three quite different reasons:

Increasing the language model’s size addresses none of these automatically. A brilliant writer cannot quote a page that was never handed to it.

MIME—Multipurpose Internet Mail Extensions—is the convention that lets one email carry several body alternatives and attachments. A simplified wire message might look like this:

Content-Type: multipart/alternative; boundary="b1"

--b1
Content-Type: text/plain; charset="utf-8"

Paid INR 805 at Book Nook.
--b1
Content-Type: text/html; charset="utf-8"

<p>Paid <strong>INR 805</strong> at Book Nook.</p>
--b1--

The boundaries are transport structure, not prose. The plain and HTML parts are two presentations of one fact, not two independent confirmations. A parser must decode the declared character set, identify alternatives, sanitize or render HTML, and choose a canonical text path. Indexing both alternatives as separate messages would double-count the evidence.

Now place that mail inside a reply:

The corrected amount is INR 905.

On Monday, Aster Bank wrote:
> Paid INR 805 at Book Nook.

If quoted history receives the same weight as the new body, searching for 805 may return the correction thread without making clear that 805 is obsolete. The system needs source regions: new authored text, quoted text, signature, and attachment extraction. Whether quoted text should be searched depends on the question, but its identity must not disappear.

Attachments introduce another source layer. OCR text from an image and extracted text from a PDF are useful retrieval derivatives. They may contain recognition errors, so the provenance should say attachment OCR rather than pretending the characters came from the authored body. The ingestion contract determines what a later citation can honestly claim.

Text begins as characters. A tokenizer groups character sequences into vocabulary units called tokens. One illustrative tokenizer might split:

"Book Nook transaction TXN-HC-100005"

→ ["Book", " Nook", " transaction", " TX", "N", "-", "HC", "-100", "005"]

The exact split depends on the model’s vocabulary. Each token is replaced by an integer ID, then an embedding table maps each ID to a learned vector. A transformer repeatedly mixes those token vectors using attention and feed-forward layers. A pooling rule finally produces one fixed-size vector for the whole query or passage.

The five boxes above hide most of the interesting work. Let us open them before using the word “embedding” as if it explained itself.

Suppose a toy encoder uses three-dimensional rows:

Looking up a row is only the beginning. At this point Book has the same initial row whether the text says “Book Nook charged me” or “book a flight.” Retrieval needs a contextual meaning. A transformer encoder creates it by repeatedly letting tokens exchange information.

An embedding model begins with weights that do not yet organize our retrieval task usefully. Nobody assigns coordinate 17 to “merchant” or coordinate 208 to “payment.” Instead, training changes the model so that relationships between whole vectors become useful.

Picture a map with the query at the centre. A good transaction passage should point in nearly the same direction. An unrelated deployment email should point elsewhere. A bookstore newsletter is the difficult case: it shares the topic Book Nook, but it does not answer what was spent.

For retrieval training, one labelled batch might contain:

query                 money spent buying books
positive passage      INR 805 was debited at Book Nook
easy negative         Atlas deployment moved to Friday
hard negative         Book Nook announces its reading festival

The labels define the desired relationships:

query ↔ positive       pull their directions together
query ↔ easy negative  keep their directions apart
query ↔ hard negative  learn that topic overlap is not enough

The hard negative is important because a model trained only against obviously unrelated text can succeed by learning crude topics. “Bookstore” versus “software deployment” is easy. “A bookstore transaction” versus “a bookstore newsletter” forces the model to learn the intent of the question.

From similarities to a training error

Suppose the model currently produces these cosine similarities:

positive transaction       0.45
hard-negative newsletter   0.43
easy-negative deployment  -0.10

The raw ranking is barely correct. Contrastive training converts the similarities into logits, usually divides them by a temperature, and applies softmax. The positive passage should receive most of the probability mass.

At temperature $\tau=0.2$, the positive logit is $0.45/0.2=2.25$, while the hard negative is $0.43/0.2=2.15$. Their exponentials are nearly equal, so the positive probability remains low and the loss remains high. Backpropagation calculates how each model weight contributed to that loss. An optimizer takes a small step that tends to increase the positive score and decrease competing scores.

This does not mean the training system drags one saved vector around a map. It updates shared encoder weights. On the next forward pass, those changed weights produce new vectors for every example. Millions of small updates gradually make the geometry generalize to passages that were never in the training batch.

The three modes expose three different lessons:

• Only easy negatives: the loss falls, but the model may learn only broad topics. Evaluation against similarly worded distractors can still fail.
• Hard negatives: a near-miss receives strong probability at the beginning, creating the pressure needed to learn relevance rather than vocabulary alone.
• False negatives: a genuinely relevant second answer is accidentally labelled negative and pushed away. Better mining is not automatically better; labels and deduplication decide what the loss teaches.

Temperature changes the strength of the competition. A small $\tau$ magnifies small similarity differences and produces sharper probabilities. That can create strong learning signals, but also makes incorrect labels more destructive. A large $\tau$ softens the distribution and may under-emphasize difficult rivals. It is a training hyperparameter, not a confidence calibration for end users.

Where training examples come from

Positive pairs might come from query-click logs, question-answer passages, document titles paired with bodies, human labels, or synthetic queries generated from passages. Negatives might be random documents, other examples in the same batch, BM25 near-misses, dense-retrieval near-misses, or deliberately mined hard cases.

Each source carries bias. Clicks reflect position and interface effects. Synthetic queries reflect the generator. In-batch negatives can accidentally include another valid answer. Hard-negative miners over-sample whatever the previous model found confusing. A credible training recipe documents those choices and evaluates separate slices: exact identifiers, paraphrases, multilingual text, short passages, long passages, and adversarial near-misses.

Why useful geometry is distributed

After training, we may draw a two-dimensional projection where purchases cluster near purchase questions. The real geometry lives across hundreds of coordinates. No single axis needs a stable human name. What matters is that the chosen similarity operation ranks useful neighbours ahead of misleading ones.

Rotating every vector by the same distance-preserving transformation would change all coordinates while leaving retrieval unchanged. This is why reading individual embedding dimensions rarely explains a result. Neighbourhood tests, retrieval metrics, counterexamples, and ablations are the right level of evidence.

Some retrieval encoders are asymmetric. They are trained with a query role and a passage role, sometimes expressed through prefixes such as query: and passage:. The same sentence encoded with the wrong role can occupy a less useful location. Model name alone is therefore insufficient provenance; role, preprocessing, pooling, dimensions, normalization, and version also belong to the contract.

When no dedicated learned embedder is selected, Thunderbird can build a local 32-dimensional fallback vector:

1. lowercase the text;
2. split it into words;
3. hash each word;
4. choose hash mod 32 as a bucket;
5. increment that bucket.

For teaching purposes, shrink this to four buckets:

"book nook payment"

book    → bucket 1
nook    → bucket 3
payment → bucket 1

vector = [0, 2, 0, 1]

This captures repeated token overlap. It is deterministic, free, local, and always available. It is not a neural understanding of paraphrases. Two unrelated words can collide in the same bucket, while “purchase” and “bought” remain different unless their hashes happen to collide.

The fallback keeps retrieval functional. A dedicated learned embedder can later replace passage and query vectors, provided its model identity and dimension are tracked and compatible.

It can feel wasteful to store both contextual chunk_text and source passage_text. The separation prevents the retrieval helper from being mistaken for a quote.

indexed chunk_text:
Subject: Debit Card transaction...
From: Aster Bank Alerts...
Passage 2: The final amount is 805...

displayed passage_text:
The final amount is 805 and the reference is TXN-HC-100005.

The repeated header may improve the vector and lexical match, but those added words do not occur beside the passage in the original body. Quoting the entire contextual string as one source excerpt would synthesize a sentence the sender did not write. The evidence card may show header metadata and body passage in separate labelled fields.

Deduplication by parent avoids a long relevant message occupying every result slot. Suppose passages 2, 3, and 4 from M1 rank first, second, and fourth. Returning three cards for M1 may hide M2, which provides the final approval. A parent-aware result can retain M1’s best passage, attach secondary passage matches as details, and leave room for other messages.

Parent expansion also consumes budget. A 100-character match does not justify attaching a 50,000-character thread. The system can include a bounded body window, selected headers, and nearby passages. “Retrieve small, expand carefully” is the practical form of parent–child retrieval.

These two messages can be almost semantically identical:

A: INR 805 at Book Nook, reference TXN-HC-100005
B: INR 1,805 at Book Nook, reference TXN-HC-100055

They share sender, merchant, transaction vocabulary, date style, and sentence structure. A dense encoder may place them extremely close. Yet substituting B for A is unacceptable. Embeddings answer “what resembles this?” They do not guarantee equality of every digit.

This is why a mature retriever keeps lexical and structured lanes instead of treating embeddings as a replacement for all search.

A beginner-friendly mistake is to picture scope as one more relevance score. It is not. Imagine a librarian who is allowed into room A but not room B. Ranking books from both rooms and hiding the forbidden titles at the end is already too late: the librarian entered room B, read its catalogue, and may have cached what was there. Permission must decide which room can be entered before relevance begins.

The database version of that rule is set membership. Let (U) be every indexed record and let (S\subseteq U) be the authorized scope. Retrieval is not

followed by filtering. It is

That small change affects more than the visible results. An approximate index must not expand to neighbours outside (S). A thread lookup must not cross into a different account merely because two messages share a subject. A graph traversal must carry the same scope predicate at every hop. A cache key must include the scope, or a result calculated for “all mail” might be reused inside “this folder.”

Dry-run the two possible orders with the private record ranked first:

unsafe order                         safe order

rank all records                     authorize work records
1. REF-SECRET-901                    REF-PUBLIC-204 remains
2. REF-PUBLIC-204                           ↓
       ↓                             rank permitted records
remove private row                   1. REF-PUBLIC-204

Both screens may finally show the same public row. Only the safe order prevents the private row from entering candidate logs, latency measurements, reranker requests, and model context. Security properties live in intermediate states too.

Think of a hospital triage desk. One specialist reads names, one reads lab values, and one understands free-form symptoms. Asking the symptom specialist to verify a patient number wastes its strengths; asking an exact identifier lookup to understand “I felt dizzy after lunch” is impossible. Hybrid retrieval gives each signal the job it can actually perform.

Here is a four-message dry run:

For an exact-ID question, A should be locked before fusion. For “the money I spent at the bookstore,” lexical search may promote A and B while dense search promotes A and C. Agreement makes A robust. For “mail from the bank,” the sender lane is more authoritative than either textual lane.

Candidate limits prevent the union from growing without bound. If eight channels each returned a thousand records, the “small” candidate stage would already contain most of a mailbox. A practical system gives every lane a budget, merges by stable record identity, and records which lanes nominated each row. That provenance makes a surprising result explainable:

message A: exact + lexical + dense + sender
message B: lexical only
message C: dense + sender

Hybrid retrieval is therefore not just an accuracy trick. It is an architecture for expressing different kinds of evidence without pretending their raw numbers mean the same thing.

The local fallback preserves several interpretable signals:

For a candidate with all terms, coverage 1, learned score .82, lexical .70, semantic .88, but no exact identifier:

S = 0 + 4 + 2(1) + 6(.82) + .75(.70) + .5(.88)
  = 4 + 2 + 4.92 + .525 + .44
  = 11.885

An exact identifier adds 100, making the intended precedence unmistakable. This is not a universal ranking formula; it is the implementation’s current policy, valuable precisely because it can be inspected and regression-tested.

Models consume tokens, while users reason about messages and passages. A context builder must translate between those levels. A rough English estimate might treat four characters as one token, but identifiers, URLs, emoji, and non-Latin scripts can break that approximation. The endpoint tokenizer is authoritative when it is available; a conservative estimate is safer when it is not.

Suppose the remaining input budget is 1,200 tokens and the ranked candidates cost:

Greedily taking rank order might spend 250 tokens on the passage and its duplicate summary, then include the newsletter. A diversity-aware packer can prefer the source passage, its useful parent context, and the related status notice. The objective resembles a constrained selection problem:

subject to

The formula is an intuition, not a claim that the current code solves an exact combinatorial optimizer. It shows why top N and “best evidence pack” are not always the same operation.

Reserve space for the system instructions, user question, conversation state, and answer. If every input token is filled with mail, the model may have no room to produce the requested table. Budgeting is end-to-end: evidence competes with other necessary context, not with an imaginary empty window.

For the query REF-NOT-FOUND-777, exact search can establish that no record in the locked scope contains the identifier. A local exact route can therefore return no match deterministically.

For an open semantic question—“Who first proposed the undocumented migration?”—absence is harder. Failure to retrieve supporting evidence might mean:

• the mailbox genuinely lacks the fact;
• the relevant message was never analysed;
• chunking hid the passage;
• an embedding or lexical query missed it;
• the current scope excludes it.

“Not found in retrieved evidence” is safer and more precise than “this never happened.” RAG operates under an open-world assumption unless the search domain and predicate permit an exhaustive negative.

A tool is not a paragraph saying “please add these numbers.” It has a name, an input schema, a result schema, and an implementation controlled by the application. A simplified transaction tool might accept:

{
  "name": "sum_transactions",
  "arguments": {
    "messageIds": ["M1", "M4", "M9"],
    "currency": "INR"
  }
}

The application validates that every ID is inside the frozen scope, reads typed minor-unit values, rejects mixed currencies, and returns:

{
  "currency": "INR",
  "totalMinor": 284900,
  "sources": ["M1", "M4", "M9"]
}

Using integer minor units avoids floating-point surprises. For example, binary floating point cannot represent many decimal fractions exactly; money addition should not depend on whether 0.1 + 0.2 displays as 0.30000000000000004.

Tool budgets constrain both cost and exposure. Four planning rounds and six calls mean there is a known maximum amount of iterative work in one turn. The seventh call fails because of policy, not because the model finally chose to stop. Results are appended to a visible trace so a reviewer can reconstruct:

question → proposed call → validated arguments → local result → final answer

MCP is a protocol for exposing tools and resources to an external client. An internal JavaScript function and an MCP tool can implement similar operations, but they have different callers and permission surfaces. Treating “the code exists” as “every connected model may invoke it” would collapse an important trust boundary.

Local GraphRAG is preference-gated. Endpoint-enriched edges and community summaries remain distinct, experimental hints. They are not default authoritative mail evidence, and they do not replace hierarchy for mailbox-wide counts.

RAG research names can sound like upgrades in a single ladder. They are better understood as responses to different failures. “RAG type” is also slightly misleading because the patterns live on different axes:

RETRIEVAL SIGNAL       dense · lexical · learned sparse · hybrid
RETRIEVAL UNIT         child chunk · token vectors · hierarchy · graph node
CONTROL FLOW           one pass · routed · corrective · self-reflective
SOURCE MODALITY        text · table · image · layout · audio

The choices are composable. A production system can be hybrid in its signals, hierarchical in its units, corrective in its control flow, and multimodal in its sources at the same time. Asking “hybrid RAG or GraphRAG?” is therefore like asking “diesel engine or four-wheel drive?” They describe different parts of the vehicle.

The first interactive diagram chooses an architecture from the question. The next one opens each family and shows its intent, machinery, cost, and safer operating pattern. Several are not used by Thunderbird; they are included so that a beginner can recognize when a paper solves a real problem and when it merely adds boxes.

Naive dense RAG: the baseline, not an insult

Intent. Encode the question, find its nearest passage vectors, put the top few passages into a prompt, and generate an answer. This is the smallest architecture that demonstrates semantic retrieval.

Use it when. The corpus is mostly natural-language prose, questions are usually paraphrases, identifiers are rare, and the dataset is small enough to inspect. It is an excellent baseline because every later component must beat it on a named failure slice.

Where it fails. One pooled vector can blur exact numbers, negation, tiny clauses, and multiple topics. Top-$k$ cannot prove collection-wide counts. A nearest passage may be semantically related but factually unable to answer.

Better use. Keep the source ID and exact text with every vector, establish an exact-search control, measure Recall@$k$, and add complexity only for observed misses. “Naive” should mean inspectable, not careless.

Hybrid RAG: let literal and semantic specialists disagree

Intent. Run lexical retrieval and dense retrieval in parallel, then fuse their rankings. Lexical search protects names, reference numbers, quoted phrases, and rare terms. Dense search contributes paraphrases and conceptual similarity.

Use it when. Enterprise documents, email, source code, support tickets, and legal text mix natural language with exact tokens. This is often the strongest general-purpose starting point after the dense baseline.

Where it fails. Fusion cannot recover a document missed by every channel. Uncalibrated raw scores should not be added directly. Duplicate candidates can waste the context window, and semantic near-misses still require reranking.

Better use. Normalize identifiers before search, preserve independent top-$k$ lists, fuse ranks with a method such as RRF, deduplicate by source, rerank a bounded set, and evaluate lexical-only, dense-only, and hybrid ablations.

Contextual parent/child RAG: search small, answer with enough context

Intent. Split a document into small searchable children while attaching a deterministic parent header or retaining a route back to the larger source. The child improves localization; the parent restores identity and surrounding facts.

Use it when. Long emails, manuals, policies, transcripts, or reports contain small answer-bearing regions that become ambiguous when detached.

Where it fails. Copying too much parent text into every child increases index size and can make siblings nearly identical. Returning the entire parent can erase the token-budget benefit of small chunks.

Better use. Index compact contextual children, store stable parent and offset links, retrieve by child, deduplicate siblings, and expand only the winning region needed for evidence.

Multi-query and decomposition RAG: one question, bounded viewpoints

Intent. Rewrite an ambiguous query several ways or decompose a compound question into independent subquestions. Retrieve for each version, then merge and deduplicate candidates.

“Compare Atlas and Beacon launch risks”
             │
             ├── Atlas launch risks
             ├── Beacon launch risks
             └── final dates and owners

Use it when. A question contains several entities, time periods, or facets, or when user vocabulary differs sharply from source vocabulary.

Where it fails. Every rewrite multiplies retrieval cost. A generated rewrite can silently change an identifier, invent a constraint, or omit a facet. Unlimited decomposition becomes an unbounded agent loop.

Better use. Always retain the original query, cap the number of rewrites, separate exact tokens that must be copied unchanged, execute branches in parallel, deduplicate results, and expose which branch retrieved each source. HyDE—a generated hypothetical answer embedded as a query—is one variant, but the hypothetical text is a search probe, never evidence.

Learned sparse RAG: learned expansion with inverted-index behaviour

Intent. Produce a high-dimensional, mostly zero vector over vocabulary terms. Unlike BM25, a model learns both term importance and expansions. A query about purchase may activate payment or debit dimensions even when those words are absent from the query.

Use it when. You already benefit from sparse indexes but need domain-aware term expansion, interpretable active terms, or efficient CPU retrieval over a large collection.

Where it fails. Inference and index construction are heavier than BM25. Learned expansions can introduce surprising terms, and a vocabulary-shaped index can be large. Exact equality still needs an exact lane.

Better use. Compare against tuned BM25 and hybrid dense retrieval, inspect the largest activated terms, version the sparse model, prune carefully, and retain the original lexical query as a control.

Late-interaction RAG: delay pooling until query time

Intent. Keep several token-level vectors for each passage. For each query token, find the strongest document-token match and aggregate those maxima. Fine details survive because the passage was not reduced to one point before seeing the query.

query token “805”       → best match: passage token “805”
query token “reference” → best match: passage token “Reference”
query token “Book”      → best match: passage token “Book”
                                      │
                                      ▼
                               aggregate MaxSim

Use it when. Names, numbers, phrases, and local token alignments determine relevance, while single-vector retrieval repeatedly hides them.

Where it fails. Many document vectors require more storage, indexing, and scoring work. Token maxima can also reward coincidental matches across a long document.

Better use. Use a compressed late-interaction index or apply it to a candidate set retrieved by cheaper channels. Measure the recall gain per byte and per millisecond rather than assuming token-level scoring is universally superior.

Hierarchical RAG: ask at the level where the answer exists

Intent. Represent leaves, groups, and corpus summaries. A global question first retrieves or computes at a higher level, then drills down to representative leaves for verification.

Use it when. Questions ask for themes, changes across a thread, summaries of a folder, or coverage over more documents than a flat top-$k$ window can represent.

Where it fails. Generated summaries can omit minority facts or become stale. Hierarchy does not make an approximate summary authoritative, and top-level nodes cannot support exact leaf claims without a trace back down.

Better use. Prefer typed rollups for counts, keep coverage metadata and child IDs, invalidate ancestors when leaves change, and cite leaf evidence for factual claims. Use summaries to navigate, not to erase provenance.

GraphRAG: retrieve connections, then return to sources

Intent. Convert source-backed entities and relations into a graph so bounded paths or communities can find evidence distributed across documents.

Use it when. The query itself contains a relationship: who approved what, how two projects connect, or which messages establish a multi-hop chain.

Where it fails. Entity resolution can merge two people or split one person. Generated edges can hallucinate relationships. High-degree nodes create irrelevant paths, and graph extraction adds a significant invalidation burden.

Better use. Store source IDs and offsets on edges, restrict edge types, hops, scope, and branching, prefer deterministic relations, and feed the graph-selected source passages back through the ordinary evidence pipeline.

Routed and tool-using RAG: choose an operation, not only a passage

Intent. Classify the question and choose exact lookup, lexical/dense search, SQL aggregation, graph traversal, or another bounded tool. Some questions should not enter a generator at all.

Use it when. One interface receives mixed requests such as “find this email,” “how many unread alerts?”, “sum these transactions,” and “explain this thread.”

Where it fails. A wrong route can be worse than weak ranking. Tool calls create authority, privacy, latency, and retry concerns. A free-running model can repeatedly call tools without improving the answer.

Better use. Start with deterministic patterns for exact IDs and typed aggregates, use a small explicit route schema, freeze authorization before routing, cap calls, validate arguments, and keep a safe fallback that returns inspectable results.

Corrective RAG: make weak evidence trigger a different policy

Intent. Evaluate initial retrieval, then accept it, retry with a changed query or source, or abstain. The defining feature is not “thinking again”; it is a conditional retrieval policy.

retrieve
   │
   ▼
quality gate ── sufficient ──→ pack evidence
   │
   ├── lexical miss ─────────→ try dense/expansion
   ├── scope has no evidence → abstain
   └── source conflict ──────→ retrieve chronology

Use it when. Retrieval quality varies by source or query type and there is a measurable signal that predicts failure.

Where it fails. If the same model that made the error freely judges its own evidence, correction can repeat the mistake with greater confidence. Retries add latency and can drift away from the original question.

Better use. Define observable gates—empty exact results, score margins, missing required fields, contradictory dates, or unsupported claims—map each gate to one bounded repair, retain the original query, and terminate with abstention.

Self-reflective RAG: model-guided decisions need external rails

Intent. Train or prompt a model to emit decisions such as “retrieve,” “evidence relevant,” “claim supported,” or “revise.” This can adapt retrieval and generation to the request rather than applying every stage every time.

Use it when. The task is exploratory, the model’s control decisions have been evaluated, and extra model calls are affordable.

Where it fails. Reflection tokens are predictions, not independent truth. A model can approve its own unsupported answer. Loops can consume latency and money, and a reflection step does not enforce account scope or tool authority.

Better use. Treat reflection as one scored signal, enforce deterministic scope and exact constraints outside the model, cap revisions, log each decision, and evaluate false approvals—not merely average answer quality.

Multimodal RAG: preserve evidence that flattening destroys

Intent. Retrieve across OCR text, page images, layout regions, tables, charts, audio, or video. The goal is not to add vision decoratively; it is to preserve meaning that plain text extraction loses.

Use it when. Scanned receipts, forms, slides, diagrams, handwritten notes, or tables encode facts spatially. “805” alone is ambiguous; its row label, currency column, and page region may establish what it means.

Where it fails. OCR errors, page-image cost, layout-model errors, and weak regional citations compound. A visually similar page is not necessarily evidence for the requested field.

Better use. Keep separate derivatives for OCR, layout, regions, and structured fields; fuse them only after scope filtering; cite page and bounding region; expose OCR confidence; and retain the original attachment as the final source of truth.

A system can combine all four. The beginner-safe design question is not “which one is modern?” It is “which state changes, which facts remain inspectable, and which failure can this component actually repair?”

If a query has two relevant messages and the top three results contain one:

DCG discounts later relevant results:

Thunderbird also measures failure-specific rates:

• abstention false positives: an absent-answer query returned records;
• exact substitutions: an exact query admitted a wrong near-match;
• scope violations: forbidden records crossed the locked boundary;
• latency and memory: the system is correct only if it remains usable;
• update amplification: one changed row should not rewrite the world.

Retrieval metrics must be reported separately from answer metrics. A perfect retriever can feed evidence to a weak writer; a polished writer can hide a broken retriever.

Precision and recall require a labelled set, often called relevance judgments or qrels. Binary judgments say relevant/not relevant. Graded judgments can say that one passage fully answers the question while another only provides useful background. Metrics are summaries of those labels, not measurements that discover truth by themselves.

Slice results by mechanism: exact identifiers, rare lexical terms, paraphrases, long-message boundaries, multilingual text, global aggregates, graph relations, absent answers, and hostile scope cases. Report macro averages so one large query class does not drown the others. For small fixtures, show raw counts alongside decimals; 1.0 over three questions is not the same evidence as 1.0 over three thousand.

Answer evaluation happens after retrieval evaluation. Measure factual support, requested-field completeness, citation mapping, abstention, privacy, latency, and format validity. An LLM judge can provide a noisy scalable signal, but freeze its prompt and version, compare a sample with human labels, and never allow a judge’s high average to waive an exact-substitution or scope-violation gate.

Recall@1 is calculated over the answerable queries. A value of 0.9545 corresponds to 21 of 22 labelled answerable queries recovering relevant evidence at the first cutoff. Recall@3 reaching 1.0 says the remaining answer appeared by rank three. MRR being 1.0 can coexist with Recall@1 below one when a query has multiple relevant records: the first relevant record can be first while another relevant record is missing at cutoff one.

That distinction is easy to miss:

gold evidence = {M1, M2}
ranked result = [M1, X, M2]

reciprocal rank = 1/1 = 1.0
Recall@1       = 1/2 = 0.5
Recall@3       = 2/2 = 1.0

nDCG adds ordering information across the list. Its 0.9964 result means the observed ordering was extremely close to the ideal ordering on this binary- relevance fixture, not that the system is “99.64% correct” on arbitrary mail.

Zero scope violations is a hard gate in this run. Yet three absent-answer queries and the fixture’s cross-account traps are still small samples. More languages, malformed MIME, large attachments, OCR errors, and adversarial senders need their own cases. Regression evidence grows by collecting failure classes, not by adding decimal places to one score.

The fresh-run badge means the focused test was executed against the current source while writing this chapter. The evidence JSON records the source hashes and exact command context. That makes the claim reproducible and prevents a later article edit from quietly turning a historical result into a current one.

An answer can sound excellent while violating the requested structure. For a source-fact case, an evaluation row might require:

must contain:       TXN-HC-100005
must cite:          mailbox://eval/Banking#1
must not contain:   TXN-HC-100055
maximum latency:    product-specific gate
maximum output:     1,024 tokens

The 3B model’s omitted URI is a real failure even if its sentence states the correct amount. The 27B model’s output-budget failure is also real even if hidden reasoning was sophisticated. Users receive visible tokens, not an abstract measure of model intelligence.

Small models can win constrained extraction because the evidence and answer shape are simple. Larger models may help with long comparisons, ambiguity, or synthesis, but they consume more memory and often more time. A router could choose direct RAG for exact facts, a smaller endpoint for short summaries, and a stronger endpoint for complex comparison—provided every route has its own evaluation and fallback.

Temperature zero reduces sampling variability but does not make decoding a formal proof. Provider implementations, quantization, prompt templates, and model digests can still change outputs. Recording only a friendly model name such as qwen3 loses the information needed to reproduce the result.

A promotion gate should contain positive, negative, adversarial, and recovery cases. Test correct facts, absent evidence, wrong near-matches, injection text, endpoint timeouts, truncated output, and citation mapping. Average answer quality cannot compensate for a privacy or scope failure.

Change one layer, keep the corpus and query labels fixed, then compare per-query outcomes. If the embedder, chunk size, reranker, and fixture all change together, you learn only that two large systems differ.

A useful ablation table contains mechanism, prediction, and observation:

Write the prediction before running the test. Otherwise it is easy to invent a story for any result. If disabling dense retrieval changes exact-ID queries, that may reveal accidental coupling. If removing a component changes nothing, it may be redundant, incorrectly wired, or untested by the corpus.

Measure cost as well as quality:

Δ Recall@3
Δ scope violations
Δ p95 query latency
Δ index bytes
Δ rebuild duration

A feature that adds no recall but doubles index size should not survive merely because its algorithm is fashionable. Conversely, a feature that protects one rare cross-account case may be essential even if the average metric barely moves.

Kill switches turn ablation logic into operational safety. If a learned reranker starts timing out after an endpoint update, the product can fall back to dense or exact stages while preserving explicit diagnostics. Graceful degradation means a smaller stated capability, not pretending the failed layer ran successfully.

Our source is ten fictional messages. Each has a stable ID, account, folder, subject, sender, date, canonical body, and a few source-derived typed fields. Those fields are not substitutes for the body. They point back to it.

The first query asks for:

Find transaction TXN-HC-100005
for INR 805 on 24 Aug 2026.

M9 repeats every query term and contains the same reference. It would be an excellent relevance match, but it belongs to server-private. The user selected server1, so M9 must disappear before lexical or vector scoring begins.

Set notation makes the order precise. If $D$ is every record and $S(q)$ is the scope permitted for query $q$, candidate generation receives:

It does not receive all of $D$ plus a polite suggestion to ignore some rows.

The parser then extracts three typed constraints:

reference     TXN-HC-100005
amountMinor   80500
date          2026-08-24

The amount is stored as integer minor units: 80500 paise, not binary floating-point 805.00. A record is authoritative only when all requested exact fields agree in the same row:

The executed core is intentionally direct:

const scoped = records.filter(record => record.account === query.account);
const exact = scoped.filter(record =>
  record.structured.reference === parsed.reference &&
  record.structured.amount_minor === parsed.amountMinor &&
  record.date === parsed.date
);

The live result should say:

10 source rows → 9 permitted rows → 1 exact record

accepted       M1
near miss      M2: wrong amount and reference
forbidden      M9: wrong account

The invariant is stronger than “M1 came first”: no row outside the frozen scope was scored at all. That property survives even if M9’s text becomes a perfect match.

M4 is 1,658 characters long. Its answer begins at character 286 and ends at 383:

The final approved reimbursement was INR 640 for the community library order, reference LIB-2048.

With a 300-character window and no overlap, the sentence is cut between children:

child 0     [0, 300)       contains only the beginning
child 1     [300, 600)     contains only the ending

Neither child can independently support the whole answer. If the window length is $L$ and overlap is $O$, successive starts are:

Switch the laboratory from 300 / 0 to 600 / 60. The complete green answer span now fits in one child. The engine records exact source offsets, so the child can always be checked against the canonical body.

Each searchable child also receives deterministic parent context:

Subject: Library committee reimbursement decision
From: committee@library.test
Date: 2026-08-19

[canonical body slice begins here]

The header helps retrieval, but it is not included in the quoted source offsets. That distinction prevents useful context from masquerading as original prose.

The actual loop is:

for (let start = 0; start < body.length; start += size - overlap) {
  const end = Math.min(body.length, start + size);
  children.push({ parentId, start, end, sourceText: body.slice(start, end) });
}

The visible trade is real: overlap repairs coverage but creates more stored children and more chances for one parent to occupy several result slots.

BM25 and dense search use incomparable score scales. Adding 8.3 BM25 to 0.67 cosine would make the unit choice decide the winner. Reciprocal Rank Fusion uses positions instead:

If M10 is first in dense search and third in BM25:

The live ledger performs that arithmetic for every candidate. Missing from one channel means no contribution from that channel, not rank infinity accidentally added to JavaScript.

The fused pool is then reranked with a transparent teaching rule:

where $E$ is exact-field coverage, $L$ is literal query-term coverage, and $T$ is one only when typed status/category fields satisfy a requested posted-book purchase. This is a deterministic feature reranker, not a hidden cross-encoder.

That honesty matters. A production learned reranker could replace this function, but it would still receive only the bounded fused pool and would still be unable to override an exact lock.

Watch the two ranked lanes converge, then compare the fused and final order. The trace preserves all three so “the order changed” is a debuggable event.

Finally choose Typed aggregate:

How much did I spend at bookstores in August 2026 altogether?

Top-$k$ retrieval cannot prove a collection-wide total. It may omit the third, twentieth, or thousandth matching purchase. The router recognizes altogether as an aggregate intent and calls a typed operation over every scoped record.

The operation requires:

category = bookstore
status   = posted
month    = 2026-08

M2 is excluded because its INR 1,805 attempt was declined. M9 is excluded because scope was frozen at milestone 1. Two integer rows remain:

The final answer cites M1 and M10. The model does not estimate the total and the retriever does not pretend that top-$k$ means “all.”

We did not add a graph because none of these questions requires relationship traversal. A minimal system becomes trustworthy partly by declining architecture that does not repair a measured failure.

Inspect ingestion status, MIME selection, source generation, and truncation. Confirm the canonical body contains the expected characters. No retrieval tuning can recover a body that was skipped or decoded into nonsense.

Run each channel separately. Literal terms absent from FTS suggest tokenization or query normalization problems. A flat dense baseline can distinguish embedding quality from approximate-index recall. An exact constraint mismatch may reveal a normalization or extraction error.

Read the context-budget ledger. It may have been removed as a duplicate, truncated, redacted, or displaced by a larger card. Confirm the answer-bearing passage, not only the parent subject, survived packing.

Inspect source-URI parsing and the mapping from visible marker to evidence card. Then inspect semantic support separately. A valid link can point to a retrieved message that does not entail the sentence.

This sequence localizes failure:

source → parse → chunk → retrieve → fuse → rerank → pack → generate → cite

Changing the language model before locating the broken arrow often hides the bug without fixing it.