Distributed Systems · MLOps · 2025

Lumen: Low-Latency Enterprise Vector Search & RAG

Sub-30ms hybrid lexical-vector retrieval architecture over 40M enterprise documents with semantic caching and cross-encoder reranking.

Role
Machine Learning Architect
Timeline
8 weeks
Year
2025
Disciplines
Distributed Systems, MLOps
Lumen enterprise retrieval architecture and semantic vector indexing dashboard

Hybrid Retrieval Architecture

Pure vector retrieval frequently fails on exact keyword matches, code symbols, and alphanumeric model codes. Lumen deploys a hybrid retrieval architecture combining BM25 inverted indexes with dense vector embeddings.

Reciprocal Rank Fusion (RRF) harmonizes sparse and dense results, passing top candidates to a lightweight cross-encoder reranker compiled with ONNX Runtime.

Performance & Semantic Caching

By implementing a semantic vector cache that matches incoming queries within a tight cosine similarity threshold, the architecture eliminates redundant retrieval queries for frequent topics.

Ray Serve manages request batching and model concurrency, maintaining stable sub-30ms latencies even during high peak query spikes.

Next project

Nomad: Supply Chain Spatio-Temporal Demand Forecasting

Hierarchical Bayesian forecasting model coupled with distributed PySpark aggregations, predicting inventory requirements across 450 regional hubs.

View project