
A retrieval-augmented system gives a wrong answer, and the instinct is to blame generation. Change the prompt. Try a bigger model. Turn the temperature down. Most of the time the model never saw the right document, and none of that helps.
Before touching anything, measure retrieval and generation independently. For each failing question, check whether the correct chunk was in the retrieved set at all. That single check partitions your failures into two piles with completely different fixes, and it takes an afternoon.
Usually the retrieval pile is the larger one, by a wide margin.
A chunk boundary through the middle of a table, or between a claim and its qualifier, produces a chunk that is retrievable and wrong. Fixed-size chunking with no respect for document structure is the default in most tutorials and is the source of a startling share of production errors.
Embeddings are good at meaning and bad at identifiers. Part numbers, error codes, proper nouns and version strings are exactly what users search for and exactly what dense retrieval blurs. Hybrid search (dense plus lexical) is not a refinement, it is usually the fix.
Retrieve broadly, then rerank precisely. A cross-encoder over the top fifty candidates costs tens of milliseconds and routinely does more for answer quality than a model upgrade costing ten times as much per token.
Teams that follow that order tend to stop after step three.

Loss flat, loss spiking, training down and validation up. Each shape has a short list of causes, and guessing is not one of the steps.

Reaching for a neural network on tabular data is a common way to look junior in a system design interview.

Not keyword stuffing, and not a new resume per job. Reordering the truth, with the numbers attached.