GelSight is a technology that has been eight years in the making. A research group lead by Ted Adelson out of MIT’s Computer Science and Artificial Intelligence Laboratory is responsible for GelSight. GelSight works by using physical contact with a surface or object which then sends back a detailed 3-D map of it. Now, the team from MIT has mounted this technology onto robot arms which give them a greater sense of sensitivity and dexterity. The applications for such a technology are broad from manufacturing to surgical robotics. It could be implemented just about anywhere.
The GelSight sensor is, in some ways, a low-tech solution to a difficult problem. It consists of a block of transparent rubber — the “gel” of its name — one face of which is coated with metallic paint. When the paint-coated face is pressed against an object, it conforms to the object’s shape.
The metallic paint makes the object’s surface reflective, so its geometry becomes much easier for computer vision algorithms to infer. Mounted on the sensor opposite the paint-coated face of the rubber block are three colored lights and a single camera.
“[The system] has colored lights at different angles, and then it has this reflective material, and by looking at the colors, the computer … can figure out the 3-D shape of what that thing is,” explains Adelson, the John and Dorothy Wilson Professor of Vision Science in the Department of Brain and Cognitive Sciences.
In both sets of experiments, a GelSight sensor was mounted on one side of a robotic gripper, a device somewhat like the head of a pincer, but with flat gripping surfaces rather than pointed tips.
MIT News has a great comprehensive write-up on the whole project and you can read that by clicking the link below.