Mobile Computing: Human-Machine Interactions

Virtual Writing Tablet for Laptops Leveraging Audio Signals. Human-computer interaction based on touch screens plays an increasing role in our daily lives. Besides smartphones and tablets, laptops are the most popular mobile devices used in both work and leisure. To operating on many emerging applications, it becomes desirable to equip both writing and drawing functions directly on laptop screens. In this project, we design a virtual writing tablet system, VPad, for traditional laptops without touch screens. VPad leverages two speakers and one microphone, which are available in most commodity laptops, for trajectory tracking without requiring additional hardware. It employs audio signals to accurately track hand movement trajectories and recognize characters user writes in the air.


Sensing Silent Scrolling Human-Machine Interactions for Recommendations on Smartphones. Recommendation services based on users’ interests to contents are widely used in popular websites and APPs. Usually, users' interests are reflected with the explicit ratings given by users to contents, but numerous users do not give their ratings. Scrolling interactions are the main human-machine interactions on smartphones, which can be utilized to extract users' browsing speed. In this project, we exploit two features underlying the scrolling interactions which are related to users’ interests, i.e., the browsing speed stability and browsing speed sequence. Based on these two features, we propose a recommendation system, S2Recom (Silent Scrolling for Recommendations), which first extracts features from the sensed scrolling interactions during the browsing, and then infers users' interests to contents for recommendations.


Sensing Ambient Light for User Experience-Oriented Color Scheme Adaptation on Smartphone Displays. With the rapid development of information technology, mobile devices have exhibited increasing popularity.To support the anytime-anywhere service model of mobile devices, one special problem is surfaced when using these devices (e.g., smartphone and tablet) under various lighting conditions. This problem related to the mobile device display can significantly degrade user experience and undermine the successful deployment of the anytime-anywhere mobile service model. In this project, we take a approach by investigating automatic color scheme adjustment to improve user experience. We find that Readability, Comfort level and Similarity are major factors that contribute to user experience. In recognizing these problems, we propose a system, ColorVert, which utilizes the DKL color space to adaptively transform color schemes by sensing ambient light to improve user experience under various lighting scenarios.

Relative Papers:
Jiadi Yu, Yingying Chen, and Jianda Li, "Color Scheme Adaptation to Enhance User Experience on Smartphone Displays Leveraging Ambient Light," IEEE Transactions on Mobile Computing (IEEE TMC), 16(3):688~701 (2017). [pdf]
Jiadi Yu, Jiaming Zhao, Yingying Chen, Jie Yang, " Sensing Ambient Light for User Experience-Oriented Color Scheme Adaptation on Smartphone Displays," in Proceedings of the The 13th ACM Conference on Embedded Networked Sensor (SenSys 2015), Seoul, South Korea, November 2015. [pdf] [slides]


Energy-Efficient Engine for Frame Rate Adaptation on Smartphones. Touch-screen technique has gained the large popularity in human-phone interaction with modern smartphones. Due to the limited size of equipped screens, scrolling operationsare indispensable in order to display the content of intereston screen. While power consumption caused by hardware and software installed within smartphones is well studied, the energy cost made by human-phone interaction such as scrolling remains unknown. Our solution seeks to address the energy issue caused by interaction while keeping a disireable user expierence at the same time.

Relative Papers:
Jiadi Yu, Haofu Han, Hongzi Zhu, Yingying Chen, Jie Yang, Yanmin Zhu, Guangtao Xue, and Minglu Li, "Sensing Human-Screen Interaction for Energy-Efficient Frame Rate Adaptation on Smartphones," IEEE Transactions on Mobile Computing (IEEE TMC), 14(8): 1698-1711 (2015) [pdf]
Haofu Han, Jiadi Yu, Hongzi Zhu, Yingying Chen, Jie Yang, Guangtao Xue, Yanmin Zhu, and Minglu Li, "E3: Energy-Efficient Engine for Frame Rate Adaptation on Smartphones," in Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems (ACM SenSys 2013), Rome, Italy, November 2013. [pdf] [slides]