Pydantic AI, a Python library for building AI applications, added provider batch processing in its October 27, 2026 release. Teams sorting records, extracting document details or running recurring checks on AI answers can send work to discounted processing queues instead of requesting immediate answers.
The workflow is simple in outline: group requests, submit them and collect results later. The change in version 2.62.0 is integration into agent runs, not a new provider price cut. Anthropic, for example, introduced discounted batching in 2024.
Start with the deadline
A backlog of independent records with a flexible deadline is a reasonable candidate. A live customer reply, or a report needed for an imminent meeting, is not. The practical distinction is whether the business can use the result later—not merely whether the job runs in the background.
OpenAI and Anthropic give batches a 24-hour processing window, after which unfinished work can expire. Google targets 24 hours and expires jobs that remain pending or running for more than 48 hours. These are provider terms, not a guarantee that every business task will finish successfully by a deadline.
Multi-step agents add another delay. Pydantic’s batch guide says each model call after a tool action or an output-validation retry needs another batch round trip. A single batch window therefore does not describe the duration of a whole agent workflow.
A half-price model call is not a half-price workflow
The current guides from OpenAI, Anthropic and Google describe batch rates at 50% of corresponding standard processing prices. Check the chosen model and features: those rates do not establish the total cost of an application, and Google notes that cache hits use standard context-caching rates.
For an illustrative calculation, assume the same workload uses $100 of model tokens eligible for that discount and $40 of unaffected operating costs. The original total is $140. Batching reduces the model component to $50, making the total $90—a saving of $50, or 35.7%, not 50%.
This is arithmetic, not a measured customer result. It assumes unchanged token use and excludes additional integration work, storage, retries, caching differences, taxes and tool charges. Use the workload’s own bill to decide whether the saving justifies the operational work.
Check the connection, not just the model name
The supported routes in Pydantic AI 2.62.0 are narrower than the providers’ full range of services. The table describes this library integration, not every way to buy batch processing.
Provider | Supported Pydantic route | Not supported by this integration |
|---|---|---|
OpenAIResponsesModel through the OpenAI API | OpenAIChatModel and Azure OpenAI | |
AnthropicModel through the direct Claude API | Bedrock, Vertex AI and Microsoft Foundry | |
GoogleModel through the Gemini API | Vertex AI |
The linked provider guides also exclude Pydantic AI Gateway for these batch routes. An unsupported route here does not mean the underlying cloud has no batch service: Pydantic’s Gemini guide, for example, distinguishes Vertex AI’s separate batch-prediction workflow. OpenAI’s own Batch API supports Chat Completions even though Pydantic’s integration uses its Responses model.
Before changing a working setup, confirm the exact provider connection and model eligibility. If provider-native batching already meets the need, the discount alone is not a reason to migrate. Claude teams should also check request compatibility when changing dependencies; RohitAI’s Claude API upgrade guide covers that separate decision.
Plan for interrupted jobs and results collection
Pydantic lets a run return a pending response after submission. The application can save its message history and resume later to collect the same request. Saving that history and arranging collection remain application responsibilities; simply submitting work does not complete the business task.
Recovery also affects cost. Pydantic’s durability guidance warns that a crash after a provider accepts a batch but before the submission step is recorded can lead to a second billable submission. Recovery behavior also varies by workflow engine. Check the chosen workflow engine’s batch-specific behavior, and investigate an uncertain submission before sending the work again.
The library can resubmit eligible provider-side failures in up to two new batches by default. Invalid requests, exhausted quota and explicitly cancelled batches do not get those automatic retries. Recovery can therefore add waiting time without fixing every failure.
Download results promptly. OpenAI’s guide says output files are deleted 30 days after completion; Anthropic makes results available for 29 days from batch creation, not completion; Google retains them for six weeks by default. Match results to the original records and inspect individual errors rather than treating a finished batch as proof that every record succeeded.
There is a separate upgrade caveat: the 2.62.0 release notes warn that a change to GraphBuilder broadcast-branch start order can break replay of durable workflows paused across the upgrade. Teams using those workflows should review paused runs before upgrading; this is not a claim that all durable workflows break.
When to choose it
Choose batching when independent work can wait, the provider route is supported, and someone owns result collection and recovery. Begin with a small, non-urgent workload and compare output quality, completion time, failures and total charges before expanding it. Keep deadline-sensitive work on the interactive path, or separate out an offline stage that can genuinely wait.