eYSIP 2026, mentor
VLA-Based Bimanual Collaborative Stacking
Teaching two SO-101 arms to pick, transfer and stack cubes together, from teleoperated demonstrations.
- Learning
- Manipulation
- Education & Outreach
My contributions
- Defined the project, its plan and timeline, and what to build at each stage; guided three interns through it.
- Guided dataset collection and the training and benchmarking of ACT and SmolVLA across single-arm and bimanual tasks.
- Oversaw an ablation replacing the RGB wrist camera with a RealSense D405, which needed changes to the ACT codebase.
An eYSIP 2026 project I planned and supervised: training vision-language-action (VLA) policies so that a bimanual SO-101 setup can pick, transfer and stack cubes from language instructions and camera feedback. I set the plan, timeline and milestones; three interns built it.
From teleoperation to policy
The team calibrated a leader-follower SO-101 pair for each arm and recorded teleoperated demonstrations in LeRobot: 75+ single-arm episodes, 50 bimanual pick-and-place episodes and 100 bimanual stacking episodes. ACT and SmolVLA were trained and compared on these, then run on the real arms.
Tasks
The policies were evaluated on single-arm pick and place, bimanual pick and place into a box, collaborative stacking (one arm holds a cube while the other places a cube on top), and placing pens in a stand. A further ablation swapped the RGB wrist camera for a RealSense D405 depth camera.
What we learned
SmolVLA coordinated the two arms more smoothly than ACT on stacking. Calibration mismatch, noise in the demonstrations, small timing delays between the arms and changes in lighting were the main sources of failure.
Task videos
Behind the project
e-Yantra's eYSIP 2026 video on the project, with the interns talking through what they built and learned.
Gallery
