Cable Manipulation with a Tactile-Reactive Gripper

Yu She*   Shaoxiong Wang*   Siyuan Dong*   Neha Sunil

Alberto Rodriguez   Edward H. Adelson


Manipulation of flexible cables is relevant to both industrial and household environments. In this work, we develop a perception and control framework to enable robots to accomplish the task of following a cable. We rely on a vision-based tactile sensor, GelSight, to estimate the pose of the cable in the grip as well as the friction forces during cable sliding. We decompose the behavior of cable following into two tactile-based controllers: 1) Cable grip controller, where a PD controller combined with a leaky integrator are responsible for regulating the gripping force to maintain the frictional sliding forces close to a suitable value; and 2) Cable pose controller, where an LQR controller based on a learned linear model of the cable sliding dynamics is in charge of keeping the cable centered and aligned on the fingertips to prevent the cable from falling.

This behavior is enabled by a designed reactive gripper with force and position control capabilities fitted with GelSight-based high resolution tactile sensors. With the proposed framework, we show that the robot can follow one meter of cable in a random configuration from beginning to end within 2-3 hand regrasps. We further demonstrate that the closed-loop system adapts to cables with different materials and thicknesses, moving at different target velocities.

Introductory Video

(1 minute)

RSS'20 Presentation

(5 minutes)

Supplementary Materials

(3 minutes)


Cable Manipulation with a Tactile-Reactive Gripper
Yu She*, Shaoxiong Wang*, Siyuan Dong*, Neha Sunil, Alberto Rodriguez, and Edward H. Adelson
Robotics: Science and Systems (RSS) 2020
Best Paper Award Finalist
[arXiv][Poster][MIT News][BibTeX](* indicates equal contributions)

Gripper Deisgn

We designed a tactile-reactive gripper. The gripper mechanism uses compliant joints, allowing simplified 3D printing. The controller operates at 60Hz.

Tactile-Reactive Gripper

Design Concept
Mechanism Design

Mechanism Design (a) A motor drives the slider-crank mechanism via the slider-string-spring system, further actuate the parallelogram mechanism via the crank linkage and finally yields the lateral displacement of the finger. (b) The rigid parallelogram mechanism includes 28 assembly parts. (c) The compliant parallel-guiding mechanisms design replaces the rigid parallelogram mechanism reducing the assembly parts from 28 pieces to a single piece.

Tactile Sensing

The gripper is equipped with GelSight tactile sensors. From tactile imprints, we can estimate the pose and force of the cable.

Camera View
Cable Pose Estimation
Cable Force Estimation

Cable Pose Estimation: white ellipse shows the estimation of the contact region. Red and green line show the first and second principal axes of the contact region, with lengths scaled by their eigenvalues.

Cable Force Estimation: the marker displacements indicates the magnitude and direction of the frictional forces.

Cable Grip Control

Since a small grasping force gives a poor pose signal, and a large grasping force results in too much friction, controlling grip force is essential. We use a PD controller to keep constant friction force and a leaky integrator to ensure good signal quality.

Cable Pose Control

We model the dynamics of the cable between the gripper, as a planar pulling problem. We use a data-driven approach to build a linear model and a LQR controller to regulate the cable around the center of the gripper.

Different Controllers

We compare the performance of different controllers.

Open-loop w/o Regrasp
Open-loop w/ Regrasp
#Regrasps: 7
LQR (v=0.045 m/s)
#Regrasps: 4

With open loop controller, the robot loses the cable easily. With open loop controller and emergency regrasp, the robot finishes the task, but need many regrasps. With LQR controller, the robot can control the cable in hand for longer distances, so that it needs fewer regrasps.

Different Velocities

We collect data with 0.025 m/s. The system still performs well with 0.045, and 0.065m/s.

LQR (v=0.025 m/s)
LQR (v=0.045 m/s)
LQR (v=0.065 m/s)

Different Cables

We collected the data with one cable, and our system can be adapted to different cables with different physical properties.


The controller adjusts pose to keep the cable centered, overcoming outside disturbance.


The tactile sensor can also estimate the pose of the plug to perform insertion. Given the socket's position, the robot can adjust based on plug's pose and successfully insert it in.


Here we combine the cable following with the insertion task. The robot follows a headphone's cable to the end, localizes the plug, adjusts accordingly, and inserts it into the cellphone.