The new model, part of the Gemini family, aims to accelerate the development of helpful robots by enabling AI to run locally on the device, a significant step forward for robotics applications in areas with limited or no connectivity.
Google has unveiled a new robotics AI model, Gemini Robotics On-Device, that can run directly on a robot’s local hardware without the need for an internet connection. This development, announced by Google DeepMind, promises to boost the performance and versatility of robots by bringing powerful AI capabilities to the “edge,” enabling faster, more responsive, and more reliable operation in real-world scenarios.
The on-device model is a specialized version of Google’s powerful Gemini models, optimized for the computational constraints of robotic systems. By processing data and making decisions locally, the model significantly reduces latency and overcomes the challenges of unreliable network access, which is often a limiting factor for robots operating in dynamic and complex environments such as warehouses, manufacturing plants, or even homes.
According to Google, the Gemini Robotics On-Device model is designed to be highly adaptable, capable of learning new tasks quickly with minimal training data. This is achieved through a combination of a general-purpose dexterity model and the ability to fine-tune the model for specific tasks with as few as 50 to 100 demonstrations. To further empower developers, Google is also releasing a Gemini Robotics SDK, providing tools to evaluate and adapt the model for a wide range of applications. The initial release is being made available to a select group of trusted testers to gather feedback and ensure safety.
In a demo, Google showed robotics arms responding to text prompts, such as “pick up the fruit and put it in the yellow basket”, or “unzip the lunchbag”. The two robot arms, powered by Google’s model, then coordinated the complete the task.
Robotics AI landscape
Google’s announcement comes at a time of intense competition and rapid innovation in the robotics AI space. Several technology giants and specialized startups are vying to provide the brains for the next generation of intelligent robots. NVIDIA recently announced updates to its “Isaac” platform, including the Isaac GR00T N1.5 foundation model for humanoid robot reasoning and skills. NVIDIA’s Director of AI has proposed a physical Turing Test for robots, and indicated the company is looking to pass it. Then there are a bevy of humanoid robotics firms including 1X and Figure, which are looking to build entire robots that can perform everyday tasks. But if companies like Google can build an AI platform for robotics — and then make it generally available — it could really spur the development of robotics like generally-available LLMs have done for digital AI.