Article

When to use Gemini API tool retrieval

For teams building Gemini agents, compare static tools and on-demand retrieval, including extra model passes, caching and supported models.

Editorial illustration for When to use Gemini API tool retrieval: documents enter a shared index with two query paths. Not documentary evidence.

Google launched Gemini API tool retrieval in preview on November 19, 2026, for teams building assistants connected to many functions and services. Keep a small, stable tool set loaded upfront; consider retrieval when most of a large library goes unused on each task. The deciding cost is the whole conversation, because searching for tools adds model passes that can outweigh the input saved.

A tool schema describes what a function does and the inputs it accepts. Retrieval lets the model load detailed tool definitions only when needed. This is a developer feature for applications, not a setting in the consumer Gemini app.

Choose static loading, server-side search or your own registry

The practical choice, based on Google’s documented retrieval modes, is:

  • Static loading: use it for a few stable functions needed on most tasks. Every schema enters the prompt, but there is no discovery pass or search system to maintain. Long conversations with frequent tool use can also favour this approach.

  • Server-side retrieval: consider it for a large, mostly fixed library when each task needs only a few functions. Google handles discovery inside the API request, so you do not build a search index. However, discovery still adds model processing and charges.

  • Client-side retrieval: choose it when your application needs its own ranking, user-specific catalogue or newly connected integrations. Your code searches a registry and returns matching schemas. Each search requires another application/API round trip, plus any registry costs; it is not automatically cheaper.

A useful middle ground is to keep common functions loaded and defer specialist ones. Google also advises keeping explicitly named functions loaded if they appear in instructions or other tool outputs: otherwise the model can call one without first retrieving the schema, breaking the follow-up request.

Server-side deferral does not mean withholding the library from Google. You must still send full schemas in the complete tools array on every request, including function-result follow-ups. It changes what initially enters model context, not what your application sends to the API.

Check the API and model before changing your app

The feature guide requires the v1beta Interactions API; generateContent does not support tool retrieval. Its explicit model list is gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash.

The launch announcement describes broader support across Gemini 3.6 and later, including all Gemini 4 models. That wording is wider than the feature guide’s list. Use the four listed models as the documented starting point; the omission of Carbon or Neon does not establish that either has been tested and failed.

Remote Model Context Protocol (MCP) servers—connections exposing external tools—need a separate compatibility check. The Interactions overview says Gemini 3 does not support remote MCP, despite two Gemini 3 models appearing in the function-retrieval list.

No separate retrieval fee does not mean free searches

Google’s tool pricing lists no separate retrieval charge. On paid usage, the short deferred-tool list, search instructions and loaded schemas count as input. Each server-side search starts another model pass over the current context, with the newly loaded schemas added.

That extra pass happens inside the same external API request. The usage fields report its prompt under total_tool_use_tokens, billed at the input rate; total_input_tokens covers the first pass. Several searches can therefore mean several charged passes, even though your application made one request.

Once retrieved, a schema remains in the conversation and contributes input on later calls and turns, with cached pricing where applicable. There is no supported way to unload it mid-conversation. An assistant that gradually uses most of its library can lose much of the initial benefit.

A longer conversation can reverse the saving

Hypothetical input-cost comparison, not a benchmark. Assume two external requests: one selects a function and one supplies its result. The retrieval path makes exactly one extra internal model pass on the first request and no further searches.

Assume full static schemas occupy 12,000 tokens; the deferred list occupies 1,000; search instructions add 200; and the retrieved schemas occupy 1,200. All input is uncached. Tokens are the units used to count model usage; these sizes are assumptions, not measured overheads.

At Gemini 4 Hydrogen’s Standard paid rate of US$1.50 per million input tokens on November 19, the comparison is:

Non-tool context per pass

Static input and cost

Retrieval input and cost

8,000 tokens

40,000 tokens · US$0.060

30,000 tokens · US$0.045

30,000 tokens

84,000 tokens · US$0.126

96,000 tokens · US$0.144

For the first row, static loading uses 20,000 tokens twice. Retrieval uses 9,200 initially, then 10,400 after discovery, then 10,400 on the result follow-up. Multiplying each total by $1.50 ÷ 1,000,000 gives the input charge.

This simplified comparison holds non-tool context fixed. It omits new conversation content, function results, generated search metadata, output and thinking charges, retries, external tool or registry fees and taxes. It is not a total-bill forecast.

With the same assumed library, retrieval saves 1.5 US cents in the shorter case but adds 1.8 cents in the longer one. The extra pass processes more conversation than the deferred schemas save. Neither result is a universal break-even rule.

For the separate choice of free versus paid access and its data conditions, see Gemini 4 Hydrogen access and pricing.

Caching can change the comparison again

The uncached example deliberately leaves out an important variable. Hydrogen’s Standard cached-input rate is US$0.075 per million tokens. Reused input can therefore cost much less, but a repeated prompt does not guarantee a cache hit.

Google’s caching guide lists a 4,096-token minimum for all four explicitly supported retrieval models. Deferring schemas can take a short prompt below that threshold. Conversely, an unchanged long prefix can qualify for caching on additional search passes. Compare actual cached usage on both paths, not just their initial token counts.

Interactions supports implicit caching, but not explicit cache objects or the Batch API. Do not apply a model’s advertised Batch discount to this Interactions-only preview.

For document-heavy assistants, tool definitions are only one input component. Selecting Gemini API document pages and spreadsheet cells addresses the file content itself—a different source of input cost.

What to check before rollout

Use the integration requirements to preserve correctness as well as control cost:

  • Keep common tools loaded; defer only those worth discovering. Choose one retrieval mode per request, since server-side and client-side retrieval cannot be combined.

  • With server-side retrieval, resend the full tools array every turn. If you manage conversation history yourself, preserve retrieval steps and signatures unchanged and use an SDK that retains them.

  • With client-side retrieval, return complete function-declaration objects. Inspect actual function_call steps rather than relying only on the response status.

Compare representative short and long tasks for completion quality, end-to-end response time, search count and total spend. Count input, tool-use, cached and generated-token usage across all requests, applying the cached rate to cached tokens rather than charging them twice.

The decision is whether the schemas you avoid loading save more than discovery and later turns cost, while still completing the task correctly. A smaller first prompt alone cannot answer that.