Multi-Stage Legal RAG with Azure AI Search
Azure AI Search · Hybrid BM25 · Reciprocal Rank Fusion · Cohere Rerank
The Challenge
The Approach
OData filters for known statutory questions. Broader top-k retrieval was rejected because it increased context size without consistently restoring missing facts.System Architecture
Azure Infrastructure
Azure Infrastructure
Architecture for the Multi-Stage Legal RAG Pipeline. Click a node to view its configuration details.
Interactive Demo
Walk through the system design and data flow for this project step by step.
How the Legal RAG Pipeline Works
Step 1Document Ingestion and PDF Extraction
The system processes 3,145 Hong Kong and Singapore legal ordinances as PDF files. Each PDF is opened with pdfplumber and every page is extracted independently. Pages returning empty text are flagged as image-only content and logged for manual review since they are invisible to the retrieval system. When a new PDF is uploaded to Azure Blob Storage, an Event Grid trigger automatically kicks off the ingestion pipeline without any manual intervention.
Image-only pages are classified as a severity-high issue in the corpus audit system. These pages often contain scanned schedules or fee tables that are critical for answering queries but completely invisible to text-based retrieval. The audit identified and flagged these gaps across all 3,145 documents.
Overview
Business Impact
Technical Highlights
- Structure-aware page classification into schedule, definition, preamble, and operative types
- 10 deterministic injection triggers that bypass semantic search for known query patterns
- BM25 keyword retrieval with Azure AI Search semantic reranking on 8,709+ schedule chunks
- Jurisdiction detection with weighted regex scoring for Hong Kong and Singapore legal terms
- Pre-agent interceptors block source disclosure probes and verify deep subsection references
- Idempotent indexing with MD5-based chunk IDs and mergeOrUpload for safe re-indexing
- Cosmos DB conversation memory with GPT-4o summarization every 4 turns
- Corpus audit system detecting 5 classes of retrieval gaps across 3,145 PDFs
- SSE streaming for real-time response delivery to the Next.js frontend
- Azure Blob trigger auto-indexes new documents without manual intervention
Frequently Asked Questions
Q: What makes this Legal RAG system different from standard RAG implementations?
It uses a multi-stage approach featuring structure-aware page classification (preamble, schedule, operative, definitions) to chunk documents by semantic boundaries rather than token counts. This ensures large schedules and rate tables are kept intact, reducing retrieval failures caused by merged context.
Q: How do the deterministic injection triggers work?
I implemented ten deterministic query triggers that detect search patterns mapping to known legal structures (such as stamp duty rates). These bypass semantic vector search entirely, using OData filters to fetch and inject verified legal tables as context, keeping the selected table available for citation and validation.
Q: How was retrieval accuracy validated for the 3,145 Hong Kong and Singapore ordinances?
I created a 200-question automated validation suite covering edge cases (such as Reciprocating Countries lists and non-existent sections). The suite completed three consecutive clean runs with zero errors and zero hallucinations before the system went to production.
Want results like this for your infrastructure?
I specialize in taking complex AI pipelines and cloud setups from concept to high-availability production. Let's discuss how to optimize your workloads, secure your environment, and reduce cloud costs.