MobileHCI 2014

Invited Interactive Tutorial
Mobile-based Tangible Interaction Design for Shared Displays

MobileHCI 2014, Toronto, Ontario, Canada


Multi-touch has become the dominant interaction technique on shared displays, such as interactive tabletop surfaces. Alternative techniques include in-air gestures, interactive pens, and conventional pointing devices such as a mouse. A theoretically appealing but less explored approach is tangible interaction. Tangibles are physical objects that can act as both control and representation for the underlying system, allowing users to create, access, and manipulate digital information. Tangibles can offer a comparatively richer interaction experience by providing additional sensory information, such as pressure and friction, and by extending the interaction and display space, for instance through off-screen content control or feedback. One limitation, which often discourages researchers and designers from using such techniques, is the need for additional hardware or devices (as tangibles). Since touchscreen smartphones are gradually becoming ubiquitous, the possibility of using such devices as tangibles may encourage researchers to explore the matter further. In this hands-on tutorial, we will discuss and explore how touchscreen-based smartphones can be used as tangibles to interact with shared displays, and participants will be guided through the process of designing and prototyping their own mobile-based tangible interactions on an interactive tabletop surface.

Date: Tuesday, September 23
Time: 9:00-12:00
Tutorials @MobileHCI 2014


Ali Mazalek, Ryerson University, Toronto, Ontario, Canada
Ahmed Sabbir Arif, Ryerson University, Toronto, Ontario, Canada

Audience and Background

This tutorial is intended for mobile HCI enthusiasts who are interested in exploring mobile-based tangible interactions. Participants must have experience developing with HTML5 and JavaScript. In addition, a basic understanding of gesture recognition and touch-based interaction approaches is useful. Participants are expected to bring their own laptops for development.


The tutorial begins with an overview of the concept of tangible interaction based on recommended reading materials, provided by the organizers ahead of time. Open discussion addresses design possibilities and practices for mobile-based tangible interactions. Participants then divide into groups of four, and each group is provided with two Android mobile devices, a development toolkit containing a gesture recognizer, JavaScript libraries, and sample source code. They are also provided with several example scenarios, where the use of mobile devices as active tangibles may benefit users. Refer to the tutorial paper for exmaples.


Reading Materials

Prepare Your Laptop for the Tutorial

Sample Source Code