The Problem
A French legal tech company needed to process and search 50,000+ legal documents intelligently. Traditional keyword search was failing. Lawyers needed semantic understanding, not just keyword matches.
Architecture I Designed
The system needed three core capabilities: intelligent document ingestion, semantic search, and AI powered answer generation with source citations.
Document Pipeline
Every legal document goes through a multi stage pipeline. First, text extraction handles PDFs and Word documents. Then a chunking strategy splits documents into 512 token chunks with 50 token overlap to preserve context across boundaries. The embedding stage prepares vectors for semantic search. Claude is used separately for generation and anonymization; the original notes do not identify the embedding model.
Triple Index Elasticsearch
I built three separate indices for different search patterns. The main semantic index handles natural language queries. A metadata index enables filtering by court, date, and jurisdiction. An appeals index tracks decision chains so lawyers can traverse the full history of a case.
Claude Powered PII Anonymization
French legal documents contain sensitive personal data. I built an anonymization module using Claude that achieves 98% PII detection accuracy, identifying names, addresses, ID numbers, and financial data before documents are indexed.
Key Technical Decisions
Why Pinecone over pure Elasticsearch for vectors? Pinecone handles billion scale vector search with sub second latency. For 50K documents with 512 token chunks, we generate roughly 200K vectors. Pinecone handles this effortlessly.
Why Claude Sonnet specifically? Legal French is complex. Claude Sonnet outperformed GPT 4 on French legal comprehension benchmarks in my testing, particularly for OHADA compliance language.
Hybrid search over pure semantic. Combining BM25 keyword search with vector similarity gave 15% better retrieval accuracy than either alone.
Results recorded in the project notes
- 50,000+ documents processed and indexed
- Sub 2 second response times end to end
- 95%+ confidence scores on retrieval. Confidence scores are not a measure of retrieval accuracy.
- 98% PII detection accuracy
- 80% reduction in document creation time via AI generation module
Lessons Learned
Chunk size matters more than model choice. I tested 256, 512, and 1024 token chunks. A size of 512 with overlap was optimal for legal documents which have dense, structured paragraphs.
Always build an eval set before optimizing. I created 200 legal Q&A pairs with a lawyer and ran every architecture change against it. This prevented several regressions.