In the fast-paced ecosystem of modern artificial intelligence, recommendation systems, semantic search engines, and retrieval-augmented generation (RAG) frameworks rely heavily on a foundational mathematical operation known as Maximum Inner Product Search, or MIPS. Operating at the core of these technologies, MIPS systems are tasked with evaluating a query vector against a massive database comprising millions of candidate key vectors to identify the single key that achieves the highest inner product match. Once located, the original data point associated with that optimal key is pulled to handle the incoming request. Within RAG workflows, for instance, the document deemed best suited to answer a user’s question is routinely retrieved and prefixed to the query, providing downstream language models with the necessary context to generate accurate responses.

However, as databases continue to scale into the millions of items and vector dimensions expand into the thousands, the computational burden of conducting exhaustive, brute-force comparisons between every query and all available keys grows exponentially. To mitigate this heavy computational cost, researchers and engineers have historically turned to various approximate methods. These baseline techniques typically attempt to trade off retrieval accuracy for processing speed by employing spatial clustering and vector quantization strategies. While these conventional approaches utilize clever indexing structures built upon the geometric properties of the key set, they often rely on loose assumptions regarding the incoming query distribution. Specifically, they frequently assume that the set of incoming queries will naturally mirror the structural distribution of the keys, or they bypass considerations of query characteristics altogether.

This mismatch creates notable inefficiencies in production environments. In crucial RAG deployments, user-facing queries typically manifest as short, conversational, and unstructured questions. Conversely, the database keys represent long, informative, highly curated blocks of text, rendering queries and keys substantially different in nature. Furthermore, the vast majority of production-level MIPS use cases target stable, well-characterized user bases with predictable and consistent query distributions. Standard, query-agnostic indexing methods leave this valuable signal completely untapped. Modern machine learning has long embraced the principle that tailoring algorithms to a likely source of inputs—rather than striving for generalized performance across all theoretically possible inputs—directs compute resources much more efficiently.

Addressing this limitation, a team of researchers from Apple presented a novel methodology at ICML 2026 titled Amortizing Maximum Inner Product Search with Learned Support Functions. The research introduces an approach where small neural networks are trained once, offline, for a fixed database of keys. These networks learn to rapidly route incoming queries toward their optimal matches with unprecedented efficiency. By trading off an upfront amortization cost to thoroughly understand how typical queries tend to align with database keys during matching, the system achieves significantly faster and more accurate retrieval at inference time through lightweight neural components.

Faster Retrieval for Better, More Efficient RAG

Rethinking Geometry and the Hidden Convex Structure of MIPS

To understand the innovation behind amortized MIPS, it is necessary to examine how traditional geometric indexing operates. Conventional MIPS systems generally partition the set of keys into a predetermined number of clusters, denoted as $k$, which is kept vastly smaller than the total number of database keys, $n$. The system tracks a representative centroid for each cluster. Upon receiving a new query, the system evaluates its distance against these $k$ centroids, routes the query toward the cluster associated with the closest centroid, and then executes an exhaustive comparison exclusively against the keys stored within that targeted partition.

If a database containing $n$ keys is split evenly across $k$ clusters, the effective pool of keys evaluated during the second phase drops to roughly $n/k$, yielding a theoretical factor-of-$k$ speedup compared to a naive, exhaustive search across the entire database. Despite its widespread adoption, this routing mechanism frequently introduces errors. Centroid routing is fundamentally an approximation, and a cluster can easily harbor a valid key that sits geographically far from its own centroid yet happens to be remarkably close to an incoming query. When a query targets such an outlier, traditional centroid routing scores the cluster poorly, bypasses it entirely, and inadvertently misses the true nearest neighbor.

What makes MIPS particularly compelling from an advanced machine learning perspective is its foundation in a mathematical structure largely overlooked prior to this research: the function that maps an arbitrary query to the largest inner product with any key in a set corresponds directly to the support function of that set, a well-established concept in convex analysis. This mathematical object possesses two critical properties. First, it is inherently convex, meaning it can be effectively approximated by specialized neural network architectures designed specifically for convex functions, known as input convex neural networks (ICNNs). Second, computing the gradient of that support function at any given query yields the best-matching key itself.

Building upon this theoretical insight, the core philosophy of the Apple research team’s work is to directly learn a neural network capable of mimicking the ground-truth support function—or its direct gradient—at a fraction of the computational cost. This objective is achieved through supervised regression, a classical machine learning technique. Researchers precompute a large repository of training examples by sampling input vectors representing typical user queries and pairing them with the correct output signals generated by MIPS, such as optimal scores or optimal key vectors. A suitably parameterized neural network is then trained to replicate those outputs given the corresponding inputs.

Faster Retrieval for Better, More Efficient RAG

Specialized Architectures: SupportNet and KeyNet

While solving the regression problem perfectly would theoretically allow a single neural network to both evaluate the support function and output the optimal key via automatic differentiation, empirical observations indicate that training separate, specialized networks for each distinct task yields superior performance in practice.

The first architecture, designated as SupportNet, learns the support function directly by employing a loosely constrained input convex neural network. This approach maintains mathematical alignment with the underlying convex structure of the problem and excels when deployed to compute individual support functions for data clusters, thereby elevating routing quality.

The second architecture, KeyNet, bypasses explicit convexity enforcement and instead models how to directly output the best-matching key vector when presented with a query. To maintain geometric grounding in the underlying support function, KeyNets incorporate a specialized consistency loss derived from Euler’s theorem for homogeneous functions.

Both model families are trained using examples drawn directly from the target query distribution. For every sampled query, the true optimal match is computed once via exhaustive search and archived, creating a supervised dataset that accurately captures the solution landscape for the specific query patterns encountered in real-world use.

Faster Retrieval for Better, More Efficient RAG

Advancing Cluster Routing with SupportNets

While traditional inverted file index systems rely on coarse centroid comparisons that are vulnerable to missing outliers, replacing or augmenting these mechanisms with learned support functions transforms how routing decisions are made. If a system has access to the exact support function of each cluster, routing becomes exact because each support function quantifies precisely how well any key within a given cluster can match an incoming query with a high score.

The research team proposes learning these $k$ support functions jointly using a trained SupportNet capable of outputting $k$ scores simultaneously. The system then routes incoming queries based on the largest of these scores. Rather than asking how close a query is to a static cluster centroid, a SupportNet answers a more functionally relevant question: how good is the absolute best key inside this specific cluster for this particular query?

To evaluate this approach, researchers trained SupportNets across varying model scales—categorized as extra-small, small, medium, and large, corresponding to 10%, 20%, and other fractional sizing parameters relative to the database—on the Natural Questions benchmark dataset partitioned into 128 distinct clusters. The Natural Questions benchmark pairs general-knowledge user inquiries with passages drawn from a vast document database consisting of approximately 2.6 million unique passages.

Experimental evaluations measuring routing recall against latency and database scanning fractions demonstrate clear performance advantages. When evaluating routing recall—defined as the fraction of queries whose true partition lands among those chosen by the router—the learned routing models achieve target performance metrics at substantially lower latency than standard centroid routing. For instance, the small SupportNet model achieves 95% routing recall roughly 1.9 times faster than traditional centroid routing, dropping latency from 3.36 milliseconds down to 1.74 milliseconds. Even greater efficiency gains emerge when analyzing routing quality independently of neural network overhead; when scanning just 5% of the database, the learned routers recover the correct partition roughly 90% to 93% of the time, compared to 87.5% for the centroid baseline.

Faster Retrieval for Better, More Efficient RAG

Query Rewriting with KeyNets for Optimized Approximate Search

While SupportNet routing substitutes traditional centroid-based routing found in existing inverted file index systems, the KeyNet approach offers an alternative integration path by leaving the underlying indexing infrastructure entirely untouched. Instead of altering the router, KeyNet functions as an intelligent query rewriter positioned directly in front of a standard approximate search index.

When a query is received, KeyNet predicts where the optimal match is most likely located within the multidimensional embedding space. The system then queries the established approximate search index using this transformed query rather than the raw user input. Because established indexing frameworks like FAISS IVF are highly optimized, they locate the target result significantly faster when assisted by a pre-processed query that sits closer to its intended target in vector space.

Evaluating KeyNets of increasing sizes on the Natural Questions dataset reveals striking efficiency gains across both latency and scan-budget metrics. Measured against raw queries, mapping queries through KeyNet prior to searching reduces the time required to reach 99% recall by up to eight times. When isolating search quality against the fraction of the database scanned by the index, KeyNet-mapped queries achieve up to 96.7% recall while scanning a mere fraction of the database, compared to 86% for raw queries. Reaching 99% recall using KeyNet-mapped inputs requires scanning up to 11 times fewer keys than raw queries, proving that the lightweight overhead of neural pre-processing is heavily offset by accelerated downstream searching.

Practical implementation choices between the two paradigms depend on system bottlenecks. Engineers can deploy KeyNet when inference speed is the primary bottleneck and approximating the best key is sufficient, or utilize SupportNet when calibrated scoring is required for downstream re-ranking and thresholding, or when compute constraints favor smaller model footprints. Ultimately, large-scale MIPS deployments managing massive key repositories may benefit from combining both strategies: employing a SupportNet router across all overarching clusters, paired with dedicated KeyNets inside individual clusters to enable regular indices to zero in on optimal keys with maximum velocity.

Faster Retrieval for Better, More Efficient RAG

By encoding MIPS solutions directly into neural network weights, amortized MIPS establishes a practical framework for retrieval systems explicitly tailored to predictable query distributions in production machine learning environments.