Faster Retrieval for Better, More Efficient RAG

Apple researchers presented a paper at ICML 2026, "Amortizing Maximum Inner Product Search with Learned Support Functions," introducing a method that trains small neural networks offline to route queries toward their best matches faster in maximum inner product search systems. The approach targets retrieval-augmented generation and other production search applications, where queries from short user questions differ substantially from long, curated key documents, undermining conventional clustering and quantization methods that assume query and key distributions resemble each other. By learning how typical queries tend to match against a fixed database of keys, the technique aims to deliver faster, more accurate retrieval at inference time in exchange for an upfront amortization cost. The work matters because MIPS underpins recommendation systems, search, and RAG, where exhaustive comparison grows costly as key counts reach millions and vector dimensions reach thousands. What to watch next is whether the learned routing approach translates into production-scale gains over standard cluster-based indexes and how it handles shifts in query distribution.