Voice User Interface Design VUI

Error recovery strategies and the verbiage around them has always been a hot topic of debate. We’ve all heard the classical “I’m sorry I didn’t hear you.” and “I’m sorry I didn’t understand you.” messages that are normally implemented as global prefixes to further attempts to help users get back on track. Some other designers prefer to eliminate this generic approach and opt instead for a more context-sensitive alternative, where based on the possible cause of error, you could very well eliminate them completely and simply attempt to reprompt the user in a more natural way, with maybe a slight change in intonation to convey the meaning of “Hello, are you listening to me?” in a subtle way.

In regards to the content of the error messages themselves, we’ve all heard that they should not simply be repetitions of what the user has already heard, but rather slightly different variations based on the context and possible cause of the problem in the first place, so as to try to help them recover: is it due to a noisy environment? is the user providing me more information than I’m requesting? are they struggling to find it? do they need more time? are they getting confused by what I’m asking?, etc.

Of course, errors are nothing new and are particularly prevalent in the software and web world, where the value of the message and its ability to help users recover is very often dubious (or flat out ridiculous), resulting in bad user experiences. Some examples:

“Unknown Error -1″

“Keyboard error (press F1 to resume)”

“Wrong parameter”

“An unexpected error occurred, because an error of type – 110 occurred.”

“It is not necessary to dial 0 after the country code for this country.” (If they know that, why not simply recognize it, remove/ignore the 0 and move on?)

Some others here and here.

With that in mind, I have to say I found it very refreshing when my Firefox browser recently crashed and I was presented with the following message:

I found a few interesting things about it that made me think about my own error prompts:

1. It’s unexpected - talk about user expectations. You know having an error (or crashing in this case) is not fun. Yet the unexpected style distracts you and in my case made me feel a little better about the situation (ok, ok, I’ll admit it, it made me smile)
2. Even though it had a funny side, it was still useful. It clearly states what the problem was in terms I understand (my windows and tabs), plus it gives me a possible reason for the problem which might help me avoid the problem in the future (a recent web page)
3. It provides solutions on how to fix it

Reduce the negative impact of an error + clear description of error + clear explanation of the possible cause + alternatives to solve it. When was the last time your error messages achieved all these goals?

As you know, I enjoy looking at other fields that might have design elements that could be leveraged in a speech and multimodal world.

My latest discovery was the film “Objectified” by Gary Hustwit. Even though the documentary is centered around the topic of Industrial Design and the process by which well known products are designed, created and injected into the marketplace, there are some great quotes by various designers that I couldn’t help but feel compelled to share with you and analyze in an attempt to find a way to apply them to our field. With so many quotes, I though this might be better off divided in parts so people can add comments and share their own insights and experiences. Let’s get started:

“What we really need to do to design is look at the extremes – the weakest, with arthritis, the athlete, the strongest, the fastest – because if we understand what the extremes are, the middle will take care of itself.” — Dan Formosa, Design and Research, Smart Design

Wow, what a way to start this topic! After readings this one, I couldn’t help but feel a little guilty about perpetuating the common design approach of the 80-20 rule. We try to capture what 20% of the population which will use 80% of the features might do, add support for a few other common “corner cases”, and ignore the really obscure and unlikely scenarios altogether. This point definitively made me wonder what if… what if we were to do it backwards, design by looking at those extremes - the distracted caller, the multitasking mom, the user that requires extra time to process the information or respond - and letting the middle take care of itself.

Case in point, the creation of the Oxo kitchenware, a peeler originally designed for people with arthritis that turned out to be more comfortable and easier to use for everyone!

“What we’re really always looking for whenever we design are ways we can improve the way people do things or improve their daily life… without them really even knowing, ever thinking about it.” — Davin Stowell, CEO & Founder, Smart Design

Another quote from Smart Design, but this time addressing the reasons behind our designs. How often are we really looking for ways in which we can improve how people do things or improve their lives? How often can we articulate this need and help evaluate it in the context of other seemingly more important needs such as completion rates, retention and automation? Can we really tell we designed something that not only solved someone’s issue or allowed them to complete their task but that in fact had a positive impact on them without them even knowing? Quite a challenge (and intrinsic motivator for me)!

“Good design should be innovative. Good design should make a product useful. Good design is aesthetic design. Good design will make a product understandable. Good design is honest. Good design is unobtrusive. Good design is long-lived. Good design is consistent in every detail. Good design is environmentally friendly. Good design is as little design as possible.” — Dieter Rams, Former Design Director, Braun

I think Mr. Rams said it perfectly. Seeing what goes on inside the minds of product creators behind brands like Braun and their philosophy definitively makes me appreciate the responsibility of a designer.

That’s it for part 1. Stay tuned for more quotes and nuggets of wisdom. And if you get a chance, watch the movie, you won’t regret it (and your users will appreciate it)!

Oh yeah, it’s that time of the year again. If you’re planning to attend this year’s SpeechTEK in New York, please stop by and say hi.

Also, you can now look at the final version of the program. In particular, I would like to invite you to the following sessions:

1. Introduction to Voice User Interface Design (STKU-2)

   Sunday August 23rd, from 1:30 PM to 5:00 PM. This workshop is designed to quickly get those new to VUI design up-to-speed so they can make the most of the Principles of VUI Design track at the conference

2. Efficient Design (B102)

   Monday August 24rd, from 11:15 AM to 12:00 PM. Here we’ll talk about “Truths and Myths About Reusable Designs”. How can you design for reuse? Can user requirements be captured the standard way?

3. Bilingual Spanish/English Design (B301)

   Wednesday August 26th, from 10:45 AM to 11:30 AM. Here we’ll talk about “How to Present Names of
   Geographical Locations in Spanish Systems”. Yes, listening for and capturing names of places seems like a trivial task, but what factors should be considered when making translation/pronunciation decision? What do those decisions say about you and your company?

Safe travels, and see you there.

Someone recently brought to my attention the fantastic keynote presentation by Bill Buxton (the author of Sketching User Experiences) from this year’s Mix 09 event.

The concepts and ideas mentioned by Bill — particularly the notion of ROE or Return On Experience — resonated so much with me, that I think his vision should help anyone in the Design profession feel awesome about what they do (even though most people still don’t really understand what is it we do) and feel energized about the potential and future of any User Experience profession.

One of the points I completely agree with is the notion of learning from the past (both successes and failures) and figuring out how to exploit that past, not in the sense of simply copying what has been done before, but to figure out how relevant the core concepts might be, and figure out how to bring them over to our time, age and circumstance. As he points out, that’s the real definition of Creativity, Design, and of course, ROE.
The second point I loved, had to do with Experiences. He makes the point of how in the past everyone focused on the products and the services, but now we need to refocus and be aware that the real differentiation now comes from what a product, image, or sound might trigger in us! And figuring out the origin of the feeling we’re trying to provoke in our users is the real art of what we do.

“How can we tailor what we’re making to generate those feelings?”

The third point I want to mention is his assessment of how nowadays the Interface is just as important as the Object, yet it is really hard to sketch/prototype interfaces as fast as we do products in rapid iterations. He also added that it is not about a device/product/service, it is about the whole ecosystem (think iPod + iTunes). And along this idea of prototyping, he points out that going through multiple iterations is the essence of Design, in fact, that is the only way to explore a more broad design space compared to the typical process of choosing a single direction and spending time and effort refining it.

I think he summarized his concepts in a beautiful way:

“Our job is not to answer questions. It’s to ask the right questions to get us to the right question that would get us to the right answer.”

What do you think? How many different variants have you done lately for each of your designs?

You can watch the full presentation here, or download it from here (Windows Media Audio/Video file, 748 MB).

Very interesting debate was triggered by the recent tests (Part 1 and Part 2) on Google Voice performed by readers of the Gadgetwise blog.

The overall premise was for readers to call the article′s writer phone number and leave a creative voicemail to gauge the effectiveness of Google′s voicemail transcription system.

Even though at first glance this seems to be another “why speech recognition isn’t ready for prime time” type of article (yes, I know the author claimed they wanted to test the boundaries of the technologies, but as many readers pointed out, individual items such as the president′s name aren′t that far fetched and should′ve worked), I think it also brings up some interesting issues often faced when testing speech recognition systems:

1) What are we testing? - This one very often depends on who you′re talking to, particularly on the business side of things. Some team members look care about containment rates (how many individuals stay in a system without having to talk to an agent), some others care about transfer rates (the increase or decrease in the volume of calls going into the call center), while some others care about customer experience (how long does it take for someone to solve accomplish their goal), and even a few (sorry to say upper management included) call systems to see how well they work when given odd statements or commands, or even worse, how close the system matches their particular expectations (without taking into consideration how the system was designed in the first place). So for me, this is one of the most important aspects of any system, which should be captured as part of the requirements phase – knowing what project owners want to test allows you to stir your design in the right direction (and push back when necessary as early as possible). For example, in the case of these Google Voice tests, there were some very interesting comments from readers because some felt they were testing the accuracy of the transcriptions, while others thought the test should only involve how well is the overall intent being captured, while some others (sadly) though they were testing how does speech recognition work nowadays.

2) How are we testing it? - This one depends a lot on what the answer to #1 might be. For example, in the case of Google Voice, I felt the test would′ve been much more valid if readers were asked to forward samples of their own voicemails into the writer′s voicemail (meaning real world examples) instead of having them come up with messages that seemed to have turned into a challenge to see who came up with the one that broke the system the most. Going back to some of the things business owners normally want to test, some of the methods in which we might need to test those items might vary significantly: for example, to test containment or transfer rates, one should not only look at raw numbers but at reasons behind those numbers – it′s very different if the numbers are driven by users exceeding a failure threshold than if they are due to users pressing 0 or if they are truly due to business requirements whose proper behavior is to retain/transfer the user.

3) What do these results mean? Particularly when dealing with numbers and percentages, the interpretation of results if very often tricky. For example, would you modify a menu if 50% of your users end up making the wrong selection? (I′m sure your gut reaction is “yes”, “of course”)… but what if that number is based on 2 out of 4 users that someone listened to during a morning′s test? Similarly, we sometimes run into situations where decisions are based solely on someone′s like or dislike (often C-level individuals) about how the system is performing (subjective analysis) without any consideration for the reasons behind the choice, the data of a much larger sample, or the fact that the system might still be on a pre-pilot phase that will eventually get tuned. I felt this was probably one of the main things lacking from the article.

The examples are definitively interesting (and funny sometimes), but I think it would′ve been worth doing some sort of analysis about the possible reasons behind some of those misrecognitions (line quality, odd pausing, user′s accents, etc.) as well as a more detailed explanation of what the transcription process really is. Some readers might think the results reflect the accuracy of an advanced speech recognition engine when in reality most transcription processes out there in the market involve a hybrid environment where the recognition engine might perform the first pass, and then human beings perform a second pass, reviewing what the machine recognized and/or interpreting those segments the machine might not have been able to recognize in the first place.

Have you tried it yet?

Outbound calling (meaning automated phone calls that go out to specific individuals) is a very profitable business that thrives at times such as this one when companies need to reach more consumers yet want to reduce the costs of making those calls since most of the time they are nothing more than the equivalent of “phone spam”.

Therefore, I’ve never been a big fan of these types of services, except for those situations where I know we’re adding value to the conversation. Those situations where we’re providing a benefit to consumers, particularly in win-win scenarios where both parties benefit from the interaction.

One product/service I recently found out about that does exactly that is GlowCaps Connect. GlowCaps are electronic pill caps that use some very clever means to ensure patients take their medicine at the times and frequency that they should.

So picture this. If you know someone that needs to manage a chronic disease like diabetes or depression, daily medications are essential for their well being. What this device does is that every day, at the prescribed time, the GlowCap uses a myriad of modalities to remind users and attract their attention. For example, it may flash a visual reminder which is followed by sound if the bottle is not opened within the first hour. If the patient still doesn’t open the bottle, then the cap triggers a phone call to remind them and can even send weekly updates to friends and family as well as send reports to the patient’s doctor with a monthly summary of the bottle’s activity.

So, to summarize, better prescription handling which can be rewarded with coupons and incentives, better healthcare management with the doctor, and an opportunity for pharmacies to handle automatic refills. Those are the types of calls I wouldn’t mind at dinner time.

As a follow-up to my previous post, it was interesting to read David Pogue’s review of Google’s First Phone,  particularly in regards to some of the UI Compromises designers had to make on this first iteration of the Android-based phone:

1. The Menu Button – This feature provides context-relevant options based on the current task.  David compares it to the functionality of a mouse right-button that offers commands like Hold, Mute and Speaker when you’re on a call.  It also offers next-step related commands such as Archive and Delete once you’ve read an email.  This is a great strategy I always like to implement, particularly on Voice User Interfaces where callers can only be presented with a limited set of choices, and there’s a clear set of task-related options that callers would be looking for without having to ‘go back’ to a so-called Main Menu.  In my mind, this should be renamed as the “Common-Sense Button”
2. Two different programs for e-mail – Ouch, this one really hurts.  Granted Gmail has a different mental model and framework than other e-mail programs, I think this one shows a lack of understanding of what users look for: simplicity and efficiency We know complexity exists everywhere, but that complexity should be hidden, whenever possible, from the UI and the user interaction.  And to add insult to injury, it seems that replying to an email in the non-Gmail program puts your cursor in the To box…  I’m just glad they have an open architecture that allows anyone to improve these interfaces
3. (Useless?) Tilt sensor – This has to be the weirdest one of them all.  If the phone contains a sensor similar to the one powering the iPhone, why did they not hook it to the screen?  The fact that someone is turning the phone 90 degrees should be enough indication of intent, so why put users through the extra step of making a menu selection or pressing a key?  This one feels like those menu prompts that first ask you to press 1 for “Arrivals or Departures Information” – which gives intent information, albeit not in an ideal way – followed up by an absurd follow-up menu asking you to “press 1 for Arrivals or 2 for Departures”.

I’ve always been fascinated by the cross-pollination that can be achieved when various fields and disciplines work together. As you might have noticed from my various posts, I also like to keep an eye out for opportunities in the UI field – things happening in other fields that may not be working together in ours yet, but whose potential definitively makes them worth a second thought and an evaluation to see what properties and ideas can be applied to UI design.

Therefore, you can imagine my excitement after seeing this particular video from TED: Yves Behar demonstrates in a very engaging presentation how business requirements, resource limitations and the status-quo (read “fear of change”) tend to yield “us-too” type of designs and solutions that don’t really add any value to those companies or the customers of those companies.

I loved the phrase he presented: “…Advertising is the price companies pay for being un-original.”, which makes me wonder what other things aside from Advertising may be “prices” companies are paying for not trying to push the envelope, for not questioning “best practices”, and as he puts it, from not “designing from the inside out”.

For me, the biggest UI lesson contained in his words have to do with how we all do our best to deliver VALUE to our customers and to their users, but how many times have you brought YOUR VALUES to the table? When was the last time you, as he puts it, “fought like an animal” to make sure the VALUES that drove the project in the first place are maintained until the end, without compromises?

I recently ran across a series of posts and a SpeechTEK magazine article about a new service currently in Private Beta called Fonolo.

The premise is definitively very interesting. What this Canadian start-up is attempting to do is to replicate the concept of bookmarking and deep linking (process of linking pages in the lower levels of a Web site from a home page – or other pages – to help search engines index them) so common these days on the web. What this means for a phone user is that they don’t need to get to the application’s “home page” (aka Main Menu) every time to then have to traverse the phone tree to reach a specific destination…

According to their founder’s pitch, the way it works is that you “bookmark” a spot deep inside a telephone’s system tree. To do this, you go to the Fonolo’s website, find the Company you need, review a transcription of their menu structure, find the spot you need and click on it. By doing that, they call the company for you, navigate the menu up to that specific spot and call you back so you can continue your transaction from that point on.

How do they know what systems look like? Well, as noted in the VoIP weblog, they seem to be using a combination of voice recognition and human editors to generate “maps” of the interactive voice response system.

Aside from the obvious privacy concerns a service like this might raise, the hottest one in my opinion is the service they refer to as “Intelligent Call History”. Since in reality all your calls start from their home page, they are attempting to become something like a Google of sorts for “phone sites”. What I mean by that is that they would keep track of all the interactions you’ve had with a certain company (regardless of the phone you used), along with the actual recordings of those conversations!

In the web world, we’re all familiar with how certain companies keep logs that track your web habits which could then be linked to your IP address. The biggest difference to me is that they mostly keep track of where you’ve been and where you’re going, but not of what’s happening when you are there… and in this case, since you’re using them as a bridge to connect to a Company, how can you be sure those recordings (which may contain account numbers, PINs, etc.) are kept safe and out of anybody’s reach?

Presumably the advantage of something like this is that in case of a dispute, you could play back a recording from the actual conversation and prove a certain transaction happened. But is this benefit really worth the risk? Particularly when it is known that once it becomes available to the public, it will be ad-supported.

On the other hand, they have also expressed that their ultimate goal is to craft partnerships with those companies Fonolo has mapped so that those companies can notify them “when they change or update their IVR” to the point where they hope companies will indeed send transcripts of their IVRs so they don’t have to be mapped anymore

Again, I definitively like the idea of empowering users and allowing them to accomplish their task in the most efficient way, but I think a system like this would be a much better fit for an actual device feature (similar to the GOOG411 dedicated button now present in some phones) – which you could turn on at the beginning of a call and stop once you reach the spot you want to “bookmark” so that in the future your phone would simply repeat the steps you followed and get you to that same spot. And of course, rather than finding more ways to patch user-unfriendly architectures, companies should be looking at fixing the root problem, which in the short term can be somewhat addressed via the deployment of more SayAnything/ SpeakFreely-type menus so callers can say what it is they need right at the beginning of an application…

How good is automation when it only works for part of the process? That’s what I often wondered while traveling since in many occasions I’ve found myself trying to use technology only to find out it didn’t save me from having to get in line at the airport. For example, sometimes you try to check-in online but get a message about having to stop by the counter to get your final seat assignment (aka someone overbooked the flight and will scramble at the gate to get you in). In other cases, you get an electronic version of the boarding pass yet have to go to a counter to check-in baggage.

Therefore you can imagine how happy I was when I read about Continental offering a real electronic check-in option, without the paper trail. Basically you get the equivalent of a boarding pass in the form of a two-dimensional encrypted bar code that gate agents and security personnel can simply scan.

Now, if they could just link that same code with an RFID tag attached to my luggage, I could also skip that lost luggage line…