Gemini Tool Retrieval preview defers schemas until agents need them
Google’s Tool Retrieval preview keeps large Gemini agent catalogs on standby, but repeated searches, retained schemas, and extra model calls can outweigh token savings for smaller setups.

Why it matters
- Google announced Tool Retrieval in preview on November 19, 2026.
- In Google’s illustrative two-request test, 200 tools required 164,000 static input tokens versus about 23,650 in input and tool-use tokens with retrieval.
- Early testing produced about 26 tokens per deferred tool for each compact name-and-description entry.
- Google recommends considering retrieval once a tool library passes roughly 20 to 30 tools.
- The preview supports Gemini 4 Argon, Gemini 3.8 Flash, and Gemini 3.6 Flash.
By Elena Vasquez
Google’s illustrative two-request example records 164,000 static input tokens versus about 23,650 in input and tool-use tokens for a 200-tool retrieval setup. Google announced Tool Retrieval on November 19, 2026. The feature entered preview through the Gemini Interactions API.
The feature lets agents keep large tool libraries on standby. They load full parameter schemas only when the current task requires them.
Google says the conventional approach becomes unwieldy as agents connect to dozens or hundreds of tools. Sending every full schema inflates input-token counts, adds latency, and reduces tool-selection accuracy as the candidate list grows.
Tool Retrieval initially keeps only each deferred tool’s name and description in context. Gemini withholds the schema that defines the function’s arguments until the model asks for it.
The stakes sit in agent economics. Tool schemas consume input tokens on each request, so a large integration catalog adds cost across turns. Google’s feature moves part of that payload behind an extra model call, turning tool selection into a speed-and-cost decision.
What changes for Gemini agents?
The Gemini Interactions API became generally available in June 2026. Google recommends that interface for new projects, and Tool Retrieval works exclusively through it. The older generateContent method does not support the feature.
Retrieval changes when Gemini sees a schema, not what the underlying function can do. Developers still choose which function tools load immediately and which ones enter deferred mode.
Google’s broader function-calling guide recommends considering retrieval once a library passes roughly 20 to 30 tools. The launch gives agents a way to keep much larger catalogs on standby, rather than forcing every complete schema into the opening context.
How does server-side retrieval work?
In the default server-side mode, developers add a tool_search tool and set defer_loading: true on selected function tools. Frequently used functions can stay fully loaded, while specialized ones initially contribute only their names and descriptions.
When Gemini needs a deferred tool, one API call can contain a tool_search_call, a tool_search_result carrying the full schema, and the resulting function_call. Those execution steps remain visible for debugging.
For a custom function, the developer’s code still executes it and returns a function_result. The application retains control of the actual function while Gemini handles discovery of the schema.
Google’s setup guide says: “There is no retrieval index for developers to build or host.” Gemini performs the server-side search itself.
Setting defer_loading on a remote MCP server defers every tool that server exposes. A large server therefore does not require each tool to be listed individually.
Google inserts retrieved schemas where they load, rather than at the start of the prompt. That placement leaves the earlier prefix intact for implicit caching.
How does client-side retrieval differ?
Client-side retrieval transfers search, ranking, and filtering to the developer’s application. That approach adds one round trip per search, but it allows tools to enter mid-conversation without being declared upfront.
Google’s setup guide cites a user connecting a new integration as one such case. Client mode gives the application control over discovery, while server mode leaves that work with Gemini.
Server-side and client-side retrieval cannot be combined within one request. Each request may contain exactly one tool_search definition.
What does Tool Retrieval cost?
Tool Retrieval has no separate charge, though it changes how tokens are billed rather than eliminating tool overhead. Server-side search instructions add a small, fixed input block to every request.
Google’s setup guide reports that early testing produced about 26 tokens per deferred tool for its compact name-and-description entry. Every schema Gemini loads remains billable as input on later calls and turns.
Each server-side search also launches another model call with the full context and newly loaded schemas. Google reports that prompt as tool-use tokens and charges it at the input rate.
The guide explicitly labels its two-request example illustrative. It uses round numbers and excludes output tokens and caching.
With 200 tools of 400 tokens each and two tools loaded, static loading totals 164,000 input tokens. Retrieval produces about 23,650 input and tool-use tokens across those two requests.
A second scenario uses 10 tools of the same size and a 30,000-token context. Static loading totals 68,000 tokens, while retrieval reaches about 92,830.
That reversal shows how repeating a large context for search can outweigh the savings from a small tool library. The examples show why: with 200 tools, avoided schemas dominate; with 10 tools and a long context, the repeated search prompt does.
Which models and limits define the preview?
Google’s guide lists Gemini 4 Argon, Gemini 3.8 Flash, and Gemini 3.6 Flash as supported models. A request that sets defer_loading on an unsupported model returns a 400 error.
Operational constraints remain important across those supported models. Retrieved tools cannot be unloaded during a conversation, so every schema loaded through search remains part of the session’s tool set.
Developers must also resend the complete tools array on every request. This requirement includes turns chained through previous_interaction_id, because Google does not store tool declarations between requests.
Google cautions that “the preview API surface may change before general availability.” Developers evaluating the feature must account for an API that can still change before its wider release.
What should developers measure next?
For teams with many tools, Tool Retrieval offers a path to reduce the context carried on every initial request. For teams with fewer tools, Google’s second example shows the opposite result when search repeats a large context.
Google says search frequency and retained schemas decide whether the extra capability reduces or increases token use. The practical test is each application’s own request pattern, rather than the headline reduction in Google’s large-tool example.
The near-term adoption decision is therefore measurable. Teams can compare static loading with retrieval across their real tool libraries, context sizes, and expected search rates. Before general availability, Google may also change the preview API surface, adding one more variable for anyone building around the preview.
Source: unite.ai
More from Elena Vasquez
Show full bio
Market editor covering technology and product news in artificial intelligence and machine learning at AI In Context.
795 articles
Related articles
- Google Ships Gemini 3.5 Flash, Promises Pro Model Next Month
- Google's Gemini Decisions API Answers 1,000 Questions in One Call
- Google launches Gemini 4 Argon globally at $2 per million input tokens
- Google Launches Gemini 3 Pro at $2/Million Input Tokens
- Google Launches Gemini 4 Hydrogen at $1.50 Per Million Input Tokens