Press a vision-based tactile sensor

A GelSight-type sensor turns touch into an image. A soft silicone pad with an opaque, reflective coating on its contact side sits on a clear window, and a camera underneath sees only how that surface bends, never the object's own color or texture. Red, green and blue LEDs light the coating from three directions at a shallow angle.

1. Color → slope. A patch tilted toward an LED looks brighter in that LED's color, so each pixel's (R, G, B) encodes the local surface normal: photometric stereo with three lights in a single frame. Real sensors calibrate this once by pressing a ball of known radius and storing a color-to-gradient lookup table.
2. Slope → shape. Integrating the gradient field, i.e. solving a Poisson equation, gives the depth map. The Depth view does this on the simulated camera image (with a DCT-based Poisson solver) and plots it against the true profile. Errors gather at steep walls, where part of the surface turns away from a light.
3. Markers → force. Dots printed on the gel move with its surface. Their mean displacement follows the shear force, their rotation the torque, and their outward spread the normal force; when outer markers lag behind inner ones, slip is starting. In practice these mappings are calibrated against a force/torque sensor or learned by a neural network.