Episode 51 of The Informed Life podcast features an interview with Cheryl Platz. Cheryl is an accomplished interaction designer who has worked on multimodal systems like Alexa and Cortana. Our conversation focused on multimodality, which is the subject her new book, Designing Beyond Devices.
This is how Cheryl explained the concept of multimodality:
For the purposes of my book, the definition we’re working with for multimodality is [that] multimodality is an exchange between a device and a human where multiple input or output modalities can be used simultaneously or sequentially, depending on context and preference. So, if we think about the traditional desktop-to- human relationship or laptop-to-human relationship, you have your keyboard and mouse and your monitors. There was one output, for the most part, which was the dominant output is visual. And the dominant input is haptic, where you’re using your hands to manipulate physical input devices. It’s not really super multimodal. And it’s certainly not optimized for multimodality.
You could argue that occasionally there’s a secondary output in audio. And some designers are doing a little bit of kinetic input when they use like a Wacom tablet or something like that. But it’s not the default way of working. And there’s so much more potential there. And we think about what’s happened in the last few years with the arrival of smart speakers, with the arrival of voice search on Google, with the fact that most of our customers are deeply comfortable speaking to their devices now, with the arrival of Kinect back in like 2010-2011 timeframe, and the fact that some customers are even comfortable, like waving to their devices and gesturing at them now. There’s so much more potential than just moving a mouse and keyboard around.
We’re moving to a multimodal world, and Cheryl explains why even designers who are working primarily on screen-based systems would benefit from knowing about multimodality. The book offers a good overview of the concept and provides practical frameworks that can help us design more efficacious multimodal systems. Our conversation is a good introduction to the subject; it’s likely to be of value to designers of all sorts of digital systems — and their users.
By the way, there are two other few features this week on The Informed Life: the show is now available on Spotify and features an all-new website. I hope both help more people enjoy the show.