Abstract—Multi-touch has become the dominant interaction technique on shared displays, such as interactive tabletop surfaces. Alternative techniques include in-air gestures, interactive pens, and conventional pointing devices such as a mouse. A theoretically appealing but less explored approach is tangible interaction. Tangibles are physical objects that can act as both control and representation for the underlying system, allowing users to create, access, and manipulate digital information. Tangibles can offer a comparatively richer interaction experience by providing additional sensory information, such as pressure and friction, and by extending the interaction and display space, for instance through off-screen content control or feedback. One limitation, which often discourages researchers and designers from using such techniques, is the need for additional hardware or devices (as tangibles). Since touchscreen smartphones are gradually becoming ubiquitous, the possibility of using such devices as tangibles may encourage researchers to explore the matter further. In this hands-on tutorial, we will discuss and explore how touchscreen-based smartphones can be used as tangibles to interact with shared displays, and participants will be guided through the process of designing and prototyping their own mobile-based tangible interactions on an interactive tabletop surface.
Ali Mazalek, Ahmed Sabbir Arif. 2014. Mobile-based tangible interaction techniques for shared displays. In Proceedings of the 16th international Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '14). ACM, New York, NY, USA, 561-562. DOI: http://dx.doi.org/10.1145/2628363.2645668
Date: Tuesday, September 23, 2014
Tutorials MobileHCI 2014
- Ali Mazalek, Ryerson University, Toronto, Ontario, Canada
- Ahmed Sabbir Arif, Ryerson University, Toronto, Ontario, Canada
Audience & Background
- 9:00—9:10 AM: Introduction
- 9:10—9:25 AM: Tangible Tabletops and Mobile Devices
- 9:25—9:40 AM: Interaction Scenarios and Demonstration
- 9:40—10:30 AM: Setup and First Exercise (download a simple solution to the first exercise )
- 10:30—10:50 AM: Coffee break in hallway
- 10:50—11:45 AM: Design and Development Activity
- 11:45—12:00 PM: Presentation and Discussion
- Kaltenbrunner, M. reacTIVision and TUIO: A tangible tabletop toolkit. Proc. ITS 2009. ACM (2009), 9-16.
- Kurtenbach, G. and Buxton, W. User learning and performance with marking menus. Proc. CHI 1994. ACM (1994), 258-264.
- Steimle, J., Khalilbeigi, M., Mühlhäuser, M., and Hollan, J. D. Physical and digital media usage patterns on interactive tabletop surfaces. Proc. ITS 2010. ACM (2010), 167-176.
- Ullmer, B. and Ishii, H. Emerging frameworks for tangible user interfaces. IBM Systems Journal 39, 3-4 (2000), 915–931.
- Wobbrock, J. O., Wilson, A. D., and Li, Y. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. Proc. UIST 2007. ACM (2007), 159-168.
- Wu, A., Mendenhall, S., Jog, J., Hoag, L.S., and Mazalek, A. A nested API structure to simplify cross-device communication. Proc. TEI 2012. ACM (2012), 225-232.
Prepare Your Laptop for the Tutorial
- Set up local web server
Install node.js and npm. node.js is a platform for building fast and scalable network applications, and npm is the default package manager for node.js
- How to install node.js and npm
- Make sure to intall all the dependencies using command
$ npm install
$ npm install tuio
$ npm install socket.io
- Download reacTIVision fiducial markers