NVIDIA’s November 5, 2026 guide to Cosmos simulation describes two ways for robotics teams to build virtual trials: predict video from proposed robot actions, or add varied imagery to an existing physics simulator. The Bimanual and Transfer model cards date their release to October 20, so these are already available tools, not a launch today.
For automation teams and managers funding a pilot, the choice is between predicting the view after an action and changing how an existing simulated scene looks. Each needs different inputs—and different evidence before its results can inform physical deployment.
Choose the missing part of your trial
Your project needs | Candidate | What your team supplies |
|---|---|---|
Video predicting what follows a proposed action | An initial image and correctly formatted robot or camera actions; adaptation data where needed | |
Varied visual observations from an existing simulator | A physics simulator, spatial controls describing its scene, and a text prompt |
With Transfer, the simulator still calculates motion and contact between objects. Depth controls describe distance from the camera; segmentation controls identify regions and object boundaries. Cosmos uses those controls to generate the images seen by the robot’s decision-making software. NVIDIA assigns physics and scene state to the simulator. The practical implication: changing the imagery cannot fix an incorrect friction or contact model.
Bimanual instead predicts observations from supplied actions. It does not choose those actions: a person or a separate policy—the software deciding the robot’s next move—must do that. Current inference examples cover AgiBot and YAM robots, not arbitrary hardware. For background on the separate control layer, see our guide to Gemini Robotics 2.
For a different robot, first check whether its recordings can meet the Bimanual data requirements: synchronized images and actions, consistent coordinate conventions and calibrated gripper measurements. Adapting the model requires suitable data, not simply renaming the robot in a configuration file.
A saved video is not a feedback test
The standard Bimanual example replays recorded actions without feeding the generated images back to a policy. That can help assess observation prediction, but it does not show how a controller reacts to what the model generates.
A separate session interface accepts fresh controls and returns images before the next decision. Your team must supply the policy or human-control adapter. The documented interface runs serially in one process without the compiled batch path, so optimized generation benchmarks cannot be assumed to describe its performance.
For a live trial, measure the full delay from choosing an action to receiving an image and choosing again. A video’s playback rate is not its generation speed, and generation speed alone is not the response time of that complete loop.
Budget for more than the download
Both models have public weights and use OpenMDW 1.1, which grants use without a licence charge under its terms. Redistribution requires the licence and applicable origin notices to be retained. That permission does not pay for the computers or engineering needed to run a trial.
The published container setup requires Linux, an NVIDIA GPU, Docker and NVIDIA Container Toolkit. NVIDIA says it tested that setup on one GB200; it explicitly does not present this as a minimum-memory specification. Do not treat the download’s availability as proof that a particular workstation can run the intended workload.
A project budget should include GPU time, storage, simulator and controller compute, data preparation, integration and any further training. The cited setup does not establish a per-trial price or an adaptation budget. Ask for a cost estimate tied to the intended resolution, trial length and number of simultaneous environments.
What the published results can tell you
NVIDIA’s Bimanual robot benchmarks compare generated images with reference images and assess video quality. Those measures can reveal prediction errors. They do not directly establish whether a physical robot will grasp an object successfully or operate reliably in a workplace.
NVIDIA’s technical report, Table 9 also describes an apple-manipulation comparison in Isaac Lab, keeping the GR00T N1.7 policy and simulator physics fixed while changing the images supplied to the policy. It reports 27 successes from 32 starting states with Transfer, versus 22 with the conventional renderer. In a red-plate variant, however, Transfer recorded 19 successes versus 22.
The report says those starting states were reused to calibrate the prompts. These are therefore calibrated simulation results, not an independent held-out test or real-robot success rates. They show that changing appearance can change a controller’s behavior; they do not establish a universal improvement.
For a project decision, making a virtual task easier and making it a better predictor of physical performance are different goals. Decide which one the pilot is meant to demonstrate.
Define a pilot that can answer your question
Name the missing capability. Choose action-conditioned prediction for a Bimanual pilot, or appearance variation around an existing physics simulator for a Transfer pilot. Neither choice by itself establishes that the physical task is modeled accurately.
Reserve unseen trials. For Bimanual, keep complete recorded episodes out of training and align predicted images with their targets. NVIDIA’s evaluation guidance calls for fixed settings and accounting for failed and skipped samples as well as successful ones.
Check what the controller actually sees. Transfer’s model card says simulator annotations must be checked against the generated imagery. When comparing renderers, keep the controller and starting conditions matched, and reserve separate starts for evaluation after prompt tuning.
Set task-level acceptance criteria. Measure completion and the full feedback delay, not just visual appeal. Before using virtual results to justify deployment, require separate evidence from the actual robot, task and operating conditions.
The useful first investment is a bounded pilot with the right inputs and a clear pass condition. Choose Transfer when the missing piece is simulator imagery; choose Bimanual when it is action-conditioned observation prediction. Expand only when the resulting evidence answers the project’s real question.