We demonstrate a strong trend toward LAMS improving based on the number of mode switches, and also find that our method is the most preferred mode switching method compared to two baselines for automatic mode switching.

(4/4)

LAMS leverages the strong zero-shot and few-shot prompting capabilities of LLMs. We give the LLM a single-sentence description of the task. By incorporating user-generated mode-switch examples, we show vast improvements between trials.
(3/4)

The use of LLMs for mode switching in teleoperation eliminates the need for task-specific demonstrations or predefined heuristics.

This approach is valuable when controlling a high-DoF robot arm with a joystick while doing new tasks–a common modality for users in assistive settings. (2/4)

We are actively recruiting participants with cervical spinal cord injury in the Pittsburgh area to further this research. Please reach out if you are interested in participating

See my lab's previous works at www.meche.engineering.cmu.edu/faculty/neur... and rchi-lab.github.io/publications/
(4/4)

In this work, we demonstrate one of the most exciting applications, which is controlling mobile manipulator robots for completing tasks of daily life.

We are excited about the potential of our system to enhance daily living for individuals with motor impairments. (3/4)

We developed a wearable sleeve and algorithm that enable real-time classification of 10 distinct gestures for controlling a robot.

The system works in just one session for each of 13 different users, paving the way for seamless integration across devices like smart devices, VR, and robotics. (2/4)