Chen Liang
LightNews LightNews
chenliangcs.bsky.social
Chen Liang
@chenliangcs.bsky.social
8 followers 47 following 8 posts
PhD @ UMich https://liangchenlc.com
Posts Replies Media Videos
chenliangcs.bsky.social
HandProxy introduces a virtual hand as an interaction proxy. It expands the affordance of speech interfaces to support expressive hand interactions. Users can initiate hand interactions through natural speech, and the system interprets and decomposes it to hand movements (3/n)
A figure showing a comparison between the existing approach and the proposed HandProxy approach. The existing approach usually relies on a list of discrete, predefined commands such as "Activate the lever," "Select the cube," "Open the box," "Close the box," "Resize the window," and "Resize the object". The proposed HandProxy approach supports more natural, expressive interactions using varied and context-rich phrases like "{punch, push, cut} the cube," "grab the {lever, fruit}," "{move, twist} the {hand, object}," "put apple into the basket," "pinch up the window corner," and general control commands like "stop, continue, undo." These inputs are decomposed into virtual proxy hand movements, which then interact directly within a virtual environment.
August 1, 2025 at 3:31 AM
chenliangcs.bsky.social
Hand interactions are widely used in XR but are not always feasible, e.g., due to (situational) impairments, environmental constraints. Speech can be used as an alternative, but is often limited to initiating basic gestures and system controls (2/n)
a figure showing possible factors causing hand interactions to be unavailable, including situational impairments (e.g., occupied hands, fatigue after long term use), environmental constraints (e.g., confined physical space, virtual object out of reach), and upper-body limitations (spasm or tremor, rapid fatigue, poor coordination, difficulty controlling distance/direction)
August 1, 2025 at 3:28 AM
chenliangcs.bsky.social
Introducing HandProxy, a speech-controlled virtual proxy hand that interprets the user’s natural speech input and performs hand interactions in XR on the user's behalf. Accepted at #IMWUT 2025 (1/n)
Paper: bit.ly/4fjdEJN
Demo Video: bit.ly/40KXtPy
August 1, 2025 at 3:26 AM