IMWUT 2022
DRG-Keyboard: Enabling Subtle Gesture Typing on the Fingertip with Dual IMU Rings
Chen Liang, Chi Hsia, Chun Yu, Yukang Yan, Yuntao Wang, and Yuanchun Shi

We present DRG-Keyboard, a gesture keyboard enabled by dual IMU rings, allowing the user to swipe the thumb on the index fingertip to perform word gesture typing as if typing on a miniature QWERTY keyboard. With dual IMUs attached to the user’s thumb and index finger, DRG-Keyboard can 1) measure the relative attitude while mapping it to the 2D fingertip coordinates and 2) detect the thumb’s touch-down and touch-up events combining the relative attitude data and the synchronous frequency domain data, based on which a fingertip gesture keyboard can be implemented ...
[PDF]

VR 2022 Workshop
From 2D to 3D: Facilitating Single-Finger Mid-Air Typing on Virtual Keyboards with Probabilistic Touch Modeling
Xin Yi, Chen Liang, Haozhan Chen, Jiuxu Song, Chun Yu, and Yuanchun Shi

Mid-air text entry on virtual keyboards suffers from the lack of tactile feedback, bringing challenges to both tap detection and input prediction. In this poster, we demonstrated the feasibility of efficient single-finger typing in mid-air through probabilistic touch modeling. We first collected users' typing data on different sizes of virtual keyboards. Based on analyzing the data, we derived an input prediction algorithm that incorporated probabilistic touch detection and elastic probabilistic decoding. In the evaluation study where the participants performed real text entry tasks with this technique, they reached a pick-up single-finger typing speed of 24.0 WPM with 2.8% word-level error rate.
[PDF] [Video]

IMWUT 2021
DualRing: Enabling Subtle and Expressive Hand Interaction with Dual IMU Rings
Chen Liang, Chun Yu, Yue Qin, Yuntao Wang, and Yuanchun Shi

We present DualRing, a novel ring-form input device that can capture the state and movement of the user's hand and fingers. With two IMU rings attached to the user's thumb and index finger, DualRing can sense not only the absolute hand gesture relative to the ground but also the relative pose and movement among hand segments. To enable natural thumb-to-finger interaction, we develop a high-frequency AC circuit for on-body contact detection ...
[PDF] [Video]

CHI 2021
Auth+Track: Enabling Authentication Free Interaction on Smartphone by Continuous User Tracking
Chen Liang, Chun Yu, Xiaoying Wei, Xuhai Xu, Yongquan Hu, Yuntao Wang, Yuanchun Shi

We propose Auth+Track, a novel authentication model that reduces redundant authentication in everyday smartphone usage. By sparse authentication and continuous tracking of the user’s status, Auth+Track eliminates the “gap” authentication between fragmented sessions and enables “Authentication Free when User is Around”. To instantiate the Auth+Track model, we present PanoTrack, a prototype that integrates body and near field hand information for user tracking. We install a fisheye camera on the top of the phone to achieve a panoramic vision that can capture both user’s body and on-screen hands ...
[PDF] [Video]

CHI 2019
HandSee: Enabling Full Hand Interaction on Smartphones with Front Camera-based Stereo Vision
Yu Chun, Xiaoying Wei, Shubh Vachher, Yue Qin, Chen Liang, Yueting Weng, Yizheng Gu, Yuanchun Shi

AAAI 2019
Deepchannel: Salience estimation by contrastive learning for extractive document summarization
Jiaxin Shi*, Chen Liang*, Lei Hou, Juanzi Li, Zhiyuan Liu, and Hanwang Zhang
* indicates equal contribution.

We propose DeepChannel, a robust, data-efficient, and interpretable neural model for extractive document summarization. Given any document-summary pair, we estimate a salience score, which is modeled using an attention-based deep neural network, to represent the salience degree of the summary for yielding the document. We devise a contrastive training strategy to learn the salience estimation network, and then use the learned salience score as a guide and iteratively extract the most salient sentences from the document as our generated summary ...
[PDF]