Punyo The Soft Gripper For Robots

The Soft Gripper for Robots

Developed by the Toyota Research Institute, the Punyo technology transforms rigid robots into soft and adaptable machines. Their current focus is on creating robots designed to assist individuals with disabilities and age-related challenges, with a particular emphasis on enhancing the "helping hands" of these machines.

In robotics, a robot's hands are often referred to as grippers or manipulators. These components must excel in three key areas: stable grasping, precise object placement, and safe, gentle interaction during contact with humans. Additionally, to make household robots a practical reality, these grippers need to be cost-effective. Toyota's innovative solution to this challenge is the Soft Bubble Gripper, a cutting-edge approach to robotic hand design.

Instructions for building the Punyo soft gripper for robots can be found at: Punyo Tech

Previously, the robots used two-fingered grippers guided by external cameras to direct their movements. However, this approach had a critical limitation: a lack of tactile feedback. While cameras can estimate an object’s distance, they cannot measure the force applied by the robot’s arm or fingers during movement or grasping.

Sense of Touch

To address this, TRI's tactile team developed an innovative solution using air-filled, elastic bubbles. These compliant and durable bubbles serve as the grippers, incorporating cameras placed inside the bubbles themselves. This internal perspective allows the system to sense forces that are otherwise invisible, providing an unprecedented view inside the grasp.

These bubble grippers can "feel" the shapes of objects, recognize what they are gripping, and measure the force between the object and the gripper’s fingers, enabling more precise and gentle interactions.

The latex in the bubble gripper is inflated to a level of softness that optimizes its ability to conform to the shapes of held objects, ensuring maximum grasp stability. Thanks to its air-filled bubbles and the gummy, high-friction texture of latex, the gripper excels at securely grabbing and holding onto a variety of objects. This concept of passive compliance is central to soft robotics and manipulation—it relies on the inherent physical properties of materials rather than motors, allowing the gripper to adapt naturally to the shape of the objects it interacts with.

How It Works

Embedded within the bubble gripper is a readily available Time-of-Flight depth sensor and IR camera, which uses visual data to "feel" the object being gripped. Within seconds, the system can recognize an object's shape and other physical characteristics, determining how best to handle it. Much like human fingers can "see" through touch, these sensors provide tactile feedback that enables the creation of a detailed model of the object in real time.

The Soft Bubble Gripper can also detect external forces attempting to take, twist, or pull an object from its grasp. It achieves this by utilizing the internal camera to monitor a dense dot pattern printed on the latex membrane. By analyzing how this pattern moves and distorts, the system can infer the magnitude and direction of the forces at play. This capability enables the gripper to quickly recognize when an object it is lowering has landed on a surface, if it has accidentally bumped into something, or when an item it is handing over has been successfully received. Essentially, the gripper provides the robot with tactile awareness—a sense of touch—based on real-time feedback from its environment.

Working Blind

One remarkable test exercise demonstrated the gripper’s abilities by having the robot stack transparent wine glasses. For traditional vision-based robots with hard grippers, this task is particularly challenging, as cameras struggle to detect transparent objects and determine when to release them. In contrast, the Soft Bubble Gripper robot can perform this task "blindly," with no prior knowledge of the object’s height or the table’s position. Changes in the dot pattern inside the latex membrane signal when one glass has been successfully placed atop another, instructing the robot to release it, showcasing its precise tactile feedback system.

The robot can pick up objects without needing to see them first, relying instead on the sense of touch to determine how to handle them. It doesn’t need to identify an object beforehand; instead, it decides what to do based on the object’s shape. This adaptability is crucial for tasks outside of controlled factory environments, where objects and conditions can vary widely—such as in a person’s home. Attempting to program every possible scenario into a robot would be a very expensive challenge.

The Future

Traditional robots primarily rely on cameras for vision, which comes with limitations. Adequate lighting is essential for accurate perception, and a clear line of sight is required for effective operation. Transparent, shiny, or dark objects often pose significant recognition challenges. In contrast, the Bubble Gripper uses touch as its primary guide, making it immune to issues caused by lighting or object texture.

This technology isn’t designed to replace traditional vision systems but rather to complement them. By adding tactile feedback to existing visual inputs, the Bubble Gripper enhances a robot's sensory capabilities, increasing its precision and adaptability in a wider range of tasks.

Traditionally, robots guided by computer vision rely on supervised training, where they are repeatedly shown images to classify objects into predefined categories. In contrast, the Punyo Soft Gripper takes a groundbreaking approach by using tactile feedback. Shear forces are detected through directional changes in the dot pattern printed inside the latex membrane, providing the robot with real-time information—such as when a glass has reached its destination. This shifts the paradigm of robotic control: instead of pre-programming solutions for every possible scenario, the robot can adapt its actions dynamically based on touch-sensitive data.

Outside The Factory

Outside of factory settings, where conditions are controlled, variables in a home environment can change unpredictably. A robot operating in such an environment must adjust to these changes without creating risks that hard grippers might introduce. The Punyo Soft Gripper enables the robot to self-annotate, learning by repeatedly grasping and releasing objects. This allows the robot to refine its behavior while performing tasks, eliminating the need for exhaustive pre-training for every potential situation.

The Toyota Research Institute's robotics team is focused on developing methods to enable robots to perform real-time course corrections. Their approach emphasizes distributed robotic sensing, where data is processed locally rather than relying on a central "brain" to make every decision. By bringing processing closer to the action, they can significantly reduce latency. Their goal is to create soft robots that enhance human-robot interactions, particularly in home environments, to assist individuals with physical challenges.

Working with advanced materials presents unique challenges. Unlike traditional methods such as machining or welding, inflatable materials introduce constantly changing shapes, disrupting conventional calculations. In the world of robotics, describing these dynamics mathematically is no simple task—when it comes to engineering soft robots, "soft is hard."

Final Thoughts

Incorporating soft robotics into everyday life represents a significant leap forward in creating machines that are not only functional but also safe and intuitive to interact with. By shifting data processing closer to the point of action and embracing the complexities of advanced materials, the Toyota Research Institute is paving the way for robots that can adapt to dynamic environments and respond in real-time.

These innovations hold the potential to transform human-robot interactions, making robots more accessible and useful in settings like homes, where unpredictability and personal needs are common. While the challenges of working with inflatable, shape-shifting materials are substantial, the rewards—a future where robots can seamlessly assist people in their daily lives—are well worth the effort.