Gemini API Tool Retrieval: Load Schemas Only When Agents Need Them
Gemini API Tool Retrieval, in preview on the Interactions API, loads tool schemas on demand. v1beta only, not generateContent; a search can cost another prompt.
Google just added tool retrieval to the Gemini API: you can hang a large library of tools on standby and load parameter schemas only when they are needed, instead of stuffing every manual into every prompt. The official Google for Developers account announced Tool Retrieval at 03:06 Beijing time on November 20. The company blog is dated November 19. The feature is in preview in the Interactions API, on v1beta, and Google says the API surface may change before general availability.
This is not a new model, and it is not available on every Gemini API entry point. The docs are explicit: generateContent does not support it. You add {"type": "tool_search"} to the request's tools array and set defer_loading: true on the tools you want to hold back. The model still sees the name and description of every deferred tool, but the parameter schemas stay out of context. When it needs one, it emits a tool_search_call, the API returns a tool_search_result in the same request, and then it emits the actual function_call. From there you still run the function and send back a function_result. Default server-side retrieval does not require you to build or host an index. Frequently used tools can stay loaded up front; the long tail can wait.
Saving tokens is not automatic, because a search is billed too
Officially there is no separate feature fee. Tokens are billed at the standard input, cached, and output rates for the model you call. A deferred tool first counts as a single line: its name plus short_description, or the first sentence of the full description if no short description is set. Parameter schemas are not billed until they are loaded. Google says that in early testing, that line came to about 26 tokens. Enabling server-side tool_search also adds a small, fixed block of instruction tokens to every request.
The easy misread is the search itself. When the model kicks off a server-side search, the API runs another model call in the same request, sending the full context again plus the newly loaded schemas, and reports that pass in total_tool_use_tokens. Officially, a search costs roughly as much input as the current prompt again, minus any cache hits; each extra search in the same request adds another pass. Once a schema is loaded, it stays in the conversation, is billed like a tool declared up front, and cannot be unloaded.
Whether this saves money depends on how large the library is, whether a request searches, and how many times. Google published an illustrative comparison that it flags as round numbers, not measurements: with 200 tools and only a few used per request, input can fall from about 164,000 tokens to about 23,650; with 10 tools, a 30,000-token context, and a search on almost every request, the cost can rise from about 68,000 to about 92,830. The docs put the conclusion plainly: measure it against your own traffic before rolling it out broadly. A small library, or a long conversation that searches on nearly every turn, is not automatically cheaper.
Remote MCP servers can be deferred the same way. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand, without listing them one by one. The API can complete a remote MCP call in the same response; your own functions still run locally. There is also a client-side path: set tool_search to execution: "client" and let your app query its own registry, embedding index, or per-user permission filter, then return the matching full schemas. That path adds one round trip per search and does not require tools to be declared up front, which is useful when a user connects an integration mid-conversation. It cannot be mixed with server-side defer_loading in the same request, and a request may include only one tool_search.
It also lines up with implicit caching. Retrieved schemas are appended later in the conversation, so system instructions, the tool list, and earlier turns can remain an unchanged prefix. Google also warns that if a prompt was only above the minimum cacheable size because it carried full schemas, deferring those tools can drop it below the threshold, and the cache may not apply.
The docs list is narrower than the blog, and hot tools should stay loaded
The official blog says Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models. The official docs currently list a shorter set: gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, and gemini-3.6-flash, and they state that setting defer_loading on an unsupported model returns a 400. Until that list expands, do not assume Neon, Carbon, or any unlisted model already works.
The docs also spell out a few sharp boundaries. Leave the most-used tools loaded. Do not defer a tool whose name appears in your system instructions or in another tool's output, or the model may call it without searching first and the next turn will fail. Deferred functions still need their complete parameters included; a tool sent with only a name and short description can come back from a search with an empty schema the model cannot call. In stateless conversations, send tool_search_call, tool_search_result, and their signatures back unchanged. Send the full tools array on every request; declarations are not stored between calls. The docs put static loading in the range of fewer than about 20 stable tools, server-side retrieval on large but relatively fixed libraries, and client-side retrieval on proprietary registries and per-user tool sets.
Tool Retrieval is aimed at a familiar failure mode: once an agent has too many tools, every request is taxed by the manuals, and selection accuracy falls as the candidate list grows. What Google shipped is not a new model name. It is a retrieval layer that lets the library wait on standby and loads schemas only when they are needed. Whether it is worth turning on still depends on how long that library is, how often the model searches, and the shorter model list in the docs.