~/papers/retrieval-augmented-generation
paper notes · NeurIPS 2020
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
The origin of the acronym. Useful mostly as a baseline to measure how far production RAG has moved: hybrid retrieval, reranking, citations and evaluation sets.
- authors
- Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela
- year
- 2020
- my rating
- read
- 2025-06-02
- views
On this page
key takeaways
- 01Parametric memory (the model) plus non-parametric memory (an index) beats either alone on knowledge-heavy tasks, and the index can be updated without retraining.
- 02Retrieval quality bounds generation quality. Most production RAG failures are retrieval failures wearing a generation costume.
- 03The paper's dense retriever is a starting point; in production, hybrid retrieval with reranking wins on exact terms such as citations and section numbers.
The claim
Combine a pre-trained seq2seq model with a dense retriever over Wikipedia; condition generation on retrieved passages; fine-tune end to end. Better factuality and the ability to update knowledge by swapping the index.
What survived contact with production
- The core architecture: retrieve, then generate, with the retrieved passages visible so you can cite them. That visibility is the whole trust story in CaseCloud: multi-tenant legal SaaS with RAG over 80M+ judgments.
- The insight that the index is a deployable artifact: re-crawl court portals, re-embed, ship, no retraining.
What changed
- Dense-only retrieval misses exact tokens lawyers care about. The hybrid dense + sparse + RRF design in Advanced RAG Architecture: Graph, Vector, Redis, and PostgreSQL exists because of that gap.
- The paper evaluates on benchmarks; production needs per-domain golden sets and separate retrieval vs generation metrics, which is the first thing I'd do differently on every RAG project. More in The Evolution of AI: From RAGs to Autonomous Agents.
Linked from
Related notes
Jun 10, 2026The Evolution of AI: From RAGs to Autonomous AgentsExploring the transition from Retrieval-Augmented Generation (RAG) to fully autonomous AI agents capable of complex reasoning.Read Dec 15, 2025CaseCloud: multi-tenant legal SaaS with RAG over 80M+ judgmentsRead Jun 10, 2026Advanced RAG Architecture: Graph, Vector, Redis, and PostgreSQLCombining Graph databases, Vector databases with Reciprocal Rank Fusion (RRF), Redis caching, and PostgreSQL to build a state-of-the-art RAG system.Read
Discussion (0)
name + comment · no login