RAG Failure Modes: 3 Stages, Not One Root Cause
A wrong RAG answer can originate in retrieval, representation, or generation - and each stage needs a completely different fix. Fixing the wrong stage isn't a minor inefficiency, it's wasted work.
Part of: The RAG Pipeline: 5 Decisions That Control Answer Quality
When a RAG system gives a wrong answer, the instinct is almost always to fix the model - rewrite the prompt, swap in a bigger model, add more instructions. That instinct is often aimed at the wrong stage. A wrong answer can originate in retrieval, in how context gets represented to the model, or in generation itself, and each of those three root causes needs a completely different fix.
The 3-Stage Failure Map of a RAG Pipeline
Why "The Model Got It Wrong" Isn't a Diagnosis
Barnett et al.'s foundational paper on RAG failure points names seven distinct ways a RAG system can fail: Missing Content, Missed Top-Ranked Document, Not in Context, Not Extracted, Wrong Format, Incorrect Specificity, and Incomplete. Wenqi Glantz's widely-cited follow-up extends this to twelve pain points covering the same territory in more operational detail. The value of naming these separately isn't academic - a wrong answer looks the same to a user regardless of which of the seven caused it, but the fix for each one lives in a different part of the pipeline.
Three Stages, Three Different Failure Families
Snorkel AI's practical framing groups these failure points into three pipeline stages, and that grouping is the fastest way to triage a bad answer. Retrieval failures mean the right document never made it into the candidate set at all - either it doesn't exist in the corpus (Missing Content) or it exists but didn't rank high enough (Missed Top-Ranked Document). Representation failures mean retrieval worked, but the information didn't survive the trip from retrieved chunk to usable context - separated from necessary surrounding detail (Not in Context) or present but overlooked by the model (Not Extracted). Generation failures mean the model had everything it needed and still produced a wrong-shaped answer (Wrong Format, Incorrect Specificity, Incomplete).
Three Failures That Look Identical, Different Root Stage
What's the refund policy for the Enterprise plan?
The corpus only has refund policy docs for the Free and Pro plans - nothing about Enterprise exists anywhere. The retriever still returns the 'closest' chunk (the Pro plan policy) because similarity scores are never exactly zero.
The Enterprise plan has a 30-day refund window. (wrong - this is the Pro plan's policy; no Enterprise document exists at all)
This is Barnett et al.'s 'Missing Content' failure - the needed information doesn't exist in the corpus at all. It's not that the retriever picked the wrong chunk; the right chunk was never written. The problem is upstream in the data, not in embeddings or prompting.
→ Add the missing document to the corpus - don't touch the retriever or the prompt
The Same Symptom, Three Different Root Causes
The inspector above shows why this taxonomy matters in practice: a vague or incomplete-looking answer can come from a document that was never written, a fact buried in a long context the model skimmed past, or a model that had the right number and chose not to state it. All three look like "the answer wasn't good enough" from the outside. Misdiagnosing which one you're facing means fixing the wrong layer - rewriting a prompt does nothing for a document that doesn't exist, and adding more documents to the corpus does nothing for a model that already had the answer and phrased it vaguely.
Which Stage Actually Fails Most Often
RAG Failures by Pipeline Stage
Pipeline stage → observed frequency → how easily it gets misdiagnosed
The needed document doesn't exist, or exists but never lands in the top-k - the failure category cited most often across practical postmortems.
The right chunk is retrieved, but the needed information is separated from its context, or gets missed when the model processes a long context.
The model has the right data but presents it wrong - this is the most visible failure category (it's right there in the final answer), so it gets suspected first even though it isn't the most frequent root cause.
Because generation is the last stage - where a failure actually 'shows up' in the answer - teams often prompt-engineer generation first, when most failures actuallyoriginate upstream in retrieval/representation
This is a qualitative ranking synthesized from industry observation (Barnett et al., the Snorkel AI blog, Wenqi Glantz) about which failure category gets cited or encountered most often in practical postmortems - not a frequency figure measured from a specific benchmark dataset. No study publishes an exact percentage split by stage; use this ordering to decide where to debug first, not as a precise statistic for your own system.
Why the Ranking Isn't a Guarantee for Your System
The ordering above reflects what gets reported most often across RAG postmortems in aggregate - it isn't a law that applies identically to every pipeline. A team running RAG over a narrow, well-curated internal corpus may have already eliminated most retrieval failures simply because there's little room for a document to be missing, which shifts their real bottleneck toward representation or generation instead. Use the ranking to decide where to look first when you don't yet have your own error data, not as a reason to skip checking a stage just because it's statistically less common industry-wide.
Building Error Analysis Around the Three Stages
The practical use of this taxonomy is in how you triage errors, not just how you name them. When reviewing a batch of bad answers, sort each one into retrieval, representation, or generation before deciding on a fix - this is a five-minute classification step that prevents the much more common mistake of tuning the same layer (usually generation, since it's the most visible) repeatedly while the actual bottleneck sits untouched upstream. A spike in one stage's failures across many queries is also a signal worth tracking on its own, since it usually points at a systemic gap (a whole category of documents missing, or context windows consistently too long) rather than a one-off.
Common Pitfalls in Failure Mode Diagnosis
The recurring mistakes: treating every wrong answer as a generation problem and reaching for prompt tweaks first, when the fix belongs upstream; not distinguishing "the document doesn't exist" from "the document exists but wasn't ranked highly enough," which point at completely different retrieval fixes (adding content versus tuning the retriever); assuming a correctly-retrieved chunk guarantees a correct answer, when representation failures can still swallow the right information before it reaches generation; and skipping the stage-classification step entirely, which makes it impossible to tell whether a fix actually addressed the root cause or just changed the symptom.