Retrieving sub-trajectories boosts robustness of few-shot imitation learning by leveraging offline datasets!

w/ @jacob-berg.bsky.social, Bingqing Chen, @abhishekunique7.bsky.social, Jonathan Francis

Paper: arxiv.org/abs/2412.15182
Website: weirdlabuw.github.io/strap

Thanks to Bosch 🔧

STRAP retrieves semantically meaningful trajectories on large datasets like DROID making it a powerful general-purpose algorithm for robot trajectory retrieval!

🧵 5/6

STRAP uses features from vision foundation mode, e.g., DINOv2 as a metric space for the S-DTW matching.

This results in a non-parametric retrieval algorithm that can be used out-of-the-box and naturally scales to larger datasets.

🧵 4/6

Key insight: while entire trajectories contain multiple tasks and don’t match our demos, sub-trajectories are much more likely to be shared while still capturing temporal dynamics!

STRAP uses Subsequence Dynamic Time Warping (S-DTW) to find such relevant matches.

🧵 3/6

While behavior cloning memorizes the demos, STRAP learns a much wider distribution through its augmented dataset.

This leads to performance gains of 25% over prior retrieval methods (sim) and 40% over behavior cloning (real)!

🧵 2/6

Have some offline data lying around? Use it to robustify few-shot imitation learning! 🤖

STRAP 🎒 is a retrieval-based method that leverages semantic sub-trajectories in offline datasets to augment the training data.

🧵 1/6