Back to ProjectsUI.
03 / AI — RAG SystemFeaturedCase Study

PDF-RAG — Retrieval-Augmented Generation

An end-to-end Retrieval-Augmented Generation system that combines local BGE embeddings, vector search, BM25 keyword retrieval, and Gemini generation/evaluation to turn PDF documents into a searchable knowledge base and generate grounded, evaluated, source-cited answers.

RAGHybrid SearchBM25RRFGenerative AILLM EvaluationSelf-CorrectionTypeScriptNode.jsFastEmbedBGE BaseGoogle GeminiMongoDB Atlas Vector SearchRabbitMQDockerStorj

2026 · AI/RAG project · Full-stack ownership

Live Demo Source Code

At a Glance

Chunk strategy

800 / 100

token-aware chunks with overlap

Embeddings

768D

Local BGE embeddings for document and query retrieval

Retrieval

Hybrid

Vector Search + BM25

Evaluation

Structured

faithfulness + relevance + completeness + context sufficiency

Architecture-focused project demonstrating asynchronous ingestion, local embeddings, hybrid retrieval, rank fusion, grounded generation, evaluation, and bounded self-correction.

The Problem

Large PDF documents contain valuable information but are difficult to query efficiently through traditional search. Sending entire documents directly to an LLM is constrained by context size, increases token usage, and makes it difficult to reliably ground answers in the source material.

A vector-only retrieval approach can identify semantically similar content, but semantic similarity is not always enough. Exact terminology, names, identifiers, technical phrases, and domain-specific keywords can be better captured through lexical search. The system therefore combines semantic and keyword retrieval while keeping document processing asynchronous.

Generation quality also needs to be measured independently from retrieval. A retrieved context can be relevant while the generated answer is incomplete or unsupported, so the system evaluates answers against the supplied query and context and can optionally revise weak answers.

Constraints

Documents needed to be processed asynchronously so uploads could return immediately
Chunks needed to respect model token limits rather than relying only on character or word counts
Embedding generation needed to run locally without depending on an external embedding API, while Gemini remained the generation and evaluation model
Retrieval needed to capture both semantic meaning and exact keyword matches
Results from different retrieval strategies needed to be combined without relying on incompatible raw score scales
Retrieved context needed to remain traceable back to the original document chunks
Generated answers needed to be grounded in retrieved context and evaluated for faithfulness and relevance

Architecture Overview

Ingestion

Express API + S3-compatible object storage

Processing

RabbitMQ + asynchronous document consumer

Chunking

BPE tokenization + token-aware chunks

Semantic

Local BGE embeddings + MongoDB Atlas Vector Search

Keyword

BM25 lexical retrieval

Generation

RRF → Context Builder → Gemini Generator + Evaluator

Key Engineering Decisions

Using token-aware chunking instead of character-based splitting

Embedding and generation models operate within token-based context constraints. I implemented BPE tokenization and used an 800-token chunk size with 100-token overlap so chunks could be controlled according to actual model tokenization rather than arbitrary character counts.

Created predictable chunks that respect model context constraints while preserving contextual overlap between adjacent sections

Running embeddings locally

Embedding generation is part of the retrieval pipeline, so I moved it from a hosted embedding API to a local BGE model through FastEmbed. This removes the embedding API dependency while keeping Gemini for answer generation and evaluation.

Semantic retrieval runs locally with 768-dimensional BGE embeddings while the retrieval architecture remains unchanged

Combining semantic and lexical retrieval

Vector search is effective at understanding conceptual similarity, but it can underperform when a query depends on exact terminology, names, identifiers, or technical phrases. I introduced BM25 keyword retrieval alongside vector search so the system could retrieve both semantically relevant and exact-term matches.

Retrieval can capture both conceptual relevance and exact database-style terminology

Using Reciprocal Rank Fusion instead of merging raw scores

Vector similarity and BM25 produce scores with different meanings and scales, making direct score comparison unreliable. Rather than attempting arbitrary score normalization, I used Reciprocal Rank Fusion to combine the ranked results from both retrieval strategies based on their positions in each result set.

Produced a unified ranking without requiring incompatible vector and keyword scores to be directly compared

Moving document processing to an asynchronous pipeline

PDF extraction, tokenization, embedding generation, and indexing can be expensive operations. Performing them inside the upload request would unnecessarily block the API. RabbitMQ was introduced so the API could persist the document and publish an event while a background consumer handled the ingestion pipeline.

Upload requests remain responsive while document processing runs independently in the background

Adding structured LLM evaluation and bounded self-correction

A generated answer can be well-formed while still being unsupported, incomplete, or poorly grounded. The system evaluates answers against the retrieved query context using structured criteria and, when enabled, gives the generator bounded feedback to revise the answer before evaluating it again.

Answer quality is measured explicitly and the optional feedback loop can revise weak answers without changing the retrieved context

Grounding generation through hybrid retrieval

The LLM should answer questions using the uploaded documents rather than relying on unrelated model knowledge. At query time, semantic and lexical retrieval rankings are fused through RRF, the resulting context is passed to the generation step, and source references remain attached to the retrieved chunks.

Gemini-generated answers are grounded in retrieved document context and returned alongside source chunk references

Outcomes

  • Built a complete PDF-to-RAG pipeline from document upload through grounded answer generation
  • Implemented token-aware chunking using BPE tokenization
  • Integrated local BGE embeddings through FastEmbed for offline document and query retrieval
  • Implemented semantic retrieval using MongoDB Atlas Vector Search
  • Added BM25 keyword retrieval for exact-term matching
  • Combined semantic and keyword rankings using Reciprocal Rank Fusion (RRF)
  • Introduced RabbitMQ-based asynchronous document processing
  • Built context construction and grounded answer generation with source citations
  • Added structured LLM evaluation for faithfulness, relevance, completeness, and context sufficiency
  • Added an optional bounded feedback loop for answer revision and re-evaluation
  • Designed document lifecycle tracking with uploaded, processing, completed, and failed states
  • Containerized the development environment using Docker Compose

Takeaway

“Building a useful RAG system requires more than connecting an LLM to a vector database. Retrieval quality depends on document processing, token-aware chunking, embedding strategy, lexical retrieval, semantic retrieval, rank fusion, and context construction. Generation quality then needs separate evaluation for grounding and completeness, with bounded self-correction available when revision is needed.”
PreviousMitigatelyNext Schema-Mind — Schema-Aware RAG for Text-to-SQL