eYSIP 2026, co-mentor
VLA-GRASP: Imitation Learning on a UR5
A teleoperation-to-policy pipeline that brings an industrial UR5 into the LeRobot workflow.
- Learning
- Manipulation
- Education & Outreach
My contributions
- Planned the project with my co-mentor and guided one intern through it.
- Supervised a UR5 teleoperation and data pipeline with Pika Sense: joint states and camera images recorded to ROS bags and converted into LeRobot format.
- Enabled imitation learning on a platform LeRobot does not support natively.
Most VLA work targets low-cost research arms like the SO-101. This eYSIP 2026 project adapts the workflow to a UR5 industrial arm with an OnRobot RG2 gripper. I co-mentored it with Jaison Jose; the intern was Kavish V.
Collecting demonstrations
A handheld Agilex Pika Sense tracks the operator's hand in 6-DoF, and the motion is mapped to the arm in real time. The Pika SDK was ported from ROS 1 to ROS 2, and a custom recorder writes episodes straight into LeRobot format, with an overhead and a wrist camera.
Training and deployment
SmolVLA was fine-tuned on these demonstrations and run on the real arm through a custom inference node. The arm locates and picks the target cube; larger datasets and higher-contrast objects are the next step to make it more robust.
Behind the project
e-Yantra's eYSIP 2026 video on the project.
Gallery
