Stephen Brewster - Multimodal Interaction for Mobile Devices

View slides of this talk here.

Stephen Brewster.JPG Mobile user interfaces are commonly based on techniques developed for desktop computers in the 1970s, often including buttons, sliders, windows and progress bars. These can be hard to use on the move which then limits the interactions, applications and services that can be provided on mobile devices. This seminar will look at the possibility of moving away from these kinds of interactions to ones more suited to mobile devices and their dynamic contexts of use where users need to be able to look where they are going, carry shopping bags and hold on to children. I will present a range of multimodal (audio, haptic and gestural) interactions that we have developed which can be used eyes and hands free, and allow users to interact in a 'head up' way.

I will present some of the work we have done on input using pressure, and gestures done with fingers, wrist and head, along with work on output using non-speech audio, 3D sound and tactile displays in applications such as text entry, camera phone interfaces and navigation. I will talk about how we designed these for mobile use and the evaluation techniques we have developed to assess whether they are effective or not for users on the move.