Conversational design


Featured speakers


Erika Hall: Hello. Oh, yeah. This thing’s definitely on. It is on. Hi. I am so excited to be here back at Confab and back in Minneapolis. I just want to take a moment to say I’m still deeply upset about Prince, so. Right. Still hurts, but now I’ll talk about me. I’ve been a design consultant for a mighty long time. Most of that is talking, so that makes it fun. I’ve always been interested in the relationship of language to design. My very, very first conference talk way back in 2007 was called Copy As Interface. I’ve been fighting against Lorem Ipsum ever since. I thought, “I should write about this,” but then I put it off and I procrastinated for a very long time, because writing is really hard and really terrible. And I wanted to tell designers, “You all need to be much more conversational.” Then technology came along, and all of a sudden there were talking microwaves. I said, “No, that’s not what I meant.” I said, “It’s really important to look at the deeper principles of conversation.”

Finally, last year, and I will say with the help and guidance and a little bit of booty-kicking of Lisa Maria Martin, who’s here. Thank you so much. Book not possible without her. Buy her book, probably also available in the lobby during intermission. I really wanted to get people to think about the deeper principles of what makes conversation work and why it’s so powerful, because here in 2019, the world is pretty weird. One of the weirdest things about the world now is, yeah, I talk to my microwave much more than I talk to my friends. I type at my friends. I have whole relationships, we’re just typing back and forth. I think today was the first time Kristina and I actually heard each other’s voices. It’s like, “Type-y, type, type, type, type.” I still feel like I have really solid relationships with my friends. This isn’t where we thought we’d be. We thought that we would eliminate words from interface, right? Designers used to think, “Oh, if I can just come up with the perfect icon, we can eliminate all those messy words.”

That was the dream. That was the dream, but somehow, somehow words have become even more important, especially as interfaces start becoming multimodal. And language is the one thing that can persist from device to channel to mode. Words are more important than ever. I’ll tell you, I really started to think about this. I’ll tell you a sad story, because when you’re a design consultant, you collect sad stories. You’re like, “Mm,” and then you get together, and you share them. You share your sad stories, because they’re great opportunities to learn. Actually, I have a lot of compassion for the people I’m going to tell you about in this story. There’s a very large traditional publishing company. Very, very successful. Because publishing is changing a lot because of technology, they wanted to do something more entrepreneurial. So they held a contest and gave a certain amount of internal venture funding to an internal project. This internal project was pretty cool. It was a semantic search engine for business news.

The people who’d received the funding in this business unit went off, and they invested in the technology to really create a system that made intelligent connections among business stories to help leaders make better decisions. They created the technology, they created an interface that they deemed worthy of this very complex underlying system. Then they went around to all of their friends in the media, and they demonstrated it and received great praise. They launched it with great confidence into the world, and all of their partners drove traffic to it. The best way to describe what happened on launch day was ... Forgive this, it’s going to get a little gross. It was like a flock of birds hitting a plate glass window. Boom. All of this traffic went to the front end of this amazing, sophisticated tool to help business decision-makers make better decisions, and bounced right off, because everybody saw all these widgets and knobs and dials and thought, “I don’t know what to do with this. I don’t know what it’s for.” Sounded like, “Whatever,” went away. Then they came to us.

They said, “Okay, after launch, we did some usability testing.” I was like, “Cool.” Yeah, yeah. That’s my other fight. They said, “What we learned is that everybody expects a search engine to work like Google with just one box you type whatever into. Ugh.” I said, “Well, why didn’t you make it do that?” They were like, “No, it’s so powerful. We had to have all of the widgets and knobs and dials on the front end.” I said, “Okay.” I said, “We can help you with this. We’re design professionals, but hey, in the meantime, because it’s going to take us a while, why don’t you put a couple sentences of instructions above that interface?” They said, “Oh. Well, we talked about that, but we can’t decide. We’ve been going around, and we can’t decide what it should say.”

I was like, “You know what? It’s the internet. On the internet, put anything. Really. Then tomorrow, we can just think of something better and change it, and then keep going like that for a while. Iterate, iterate, iterate.” They said, “No, no, no. Those are words. Words have to go through our publishing process. That will take six weeks and we have to be absolutely sure. The vice president has to sign off. Then we can’t change it, because it’s been published.” I said, “But it’s the internet and words are part of the design.” They said, “No. The pictures are part of the design. The underlying technology’s the design. Words are writing.” Sadly, we tried to help them, but they ran out of runway and the project got yanked and defunded and sold for parts. That’s why it was sad, but I learned something really important from that project. I learned that words freak people out a little bit. For whatever reason, there’s this idea of, “Oh, if only we can think of an icon to get us off the hook to committing to verbal meaning,” because we think anything with words is capital W Writing. We’re like, “Oh, that’s terrible. That’s terrible.”

So I wanted to look at why words freak us out and perhaps how to create a more comfortable relationship with language, so we can do things like maybe provide a little verbal instruction to our interfaces when it’s called for. To do this, I’m going to take you on a very brief tour of the entire history of human communication. Yeah. Accompany me in my time machine. Writing is so hard that we’ve been putting it off for a very, very long time. Archeologists and linguists estimate that humans have been communicating in speech for 150,000 to 200,000 years, around the campfire. Talking, talking, and talking. We don’t know exactly how long, because of course, no written record. It wasn’t until 30,000 years ago that we got to the point of having figurative paintings. If you’re keeping score, yes, the graphic designers got there first. 30,000 years ago, still no writing. Pictures of buffalo. Then it wasn’t until maybe 6,000 years ago, that’s some procrastination for writing. We’re like, “I don’t want to write, can we just keep talking? Look, I drew a buffalo.”

Ancient Sumerian commodities traders got tired of getting ripped off, so they thought, “Okay. Maybe we can come up with a way of keeping accounts.” They hired some scribes to make wedges in mud, so that’s cuneiform wedge-shaped writing. A conservative estimate, for 4% of the time we’ve been human, we’ve been writing at all by any description. Yeah, that. So next time you’re putting off writing an email or report, yeah. You’re probably not putting it off as long as all of humanity put off the creation of writing. Don’t feel bad, because it’s hard. Once we started making marks to capture the meaning in speech, things really started to accelerate. So 2,800 years ago, alphabets. Then we had individual glyphs to capture the individual phonemes, and that made everything more economical and flexible, and we could express much, much more, but we were still at the point of having to create by hand every individual instance of writing.

The hand of a human was present on every manuscript. Even copying things was an act of creation. It was still very, very tied to individual humans. Then everything changed. So less than 600 years ago, which is nothing, which is the blink of an eye, there was the Gutenberg revolution in 1450. Movable type, which meant you could just typeset pages really fast, print them off, print them off, print them off. This was the beginning of mass communication. This changed everything. This led to actual revolutions, because information could cross borders. You didn’t have to be elite to possess a book anymore or to create a book. This completely upended the power structure. The sharp increase in literacy broke the power of the elite to control education, and it contributed to the rising middle class. It really changed everything about human society. Widespread literacy led to the creation of more, and more, and more books. Also more, and more, and more technology, because being able to very quickly reproduce complex ideas and study them and reflect upon them accelerated absolutely every other type of technology. So things really started to get interesting after that.

It’s important to take a moment to really think about the vast difference between an oral culture and a literate culture. It’s not just that we act exactly the same way, but then we write things down. It really changed what it meant to be human, because without the ability to record your thoughts in a separate object, there’s no such thing as the concept of privacy. You can’t have a diary, you can’t have private thoughts. In order to really think about anything of any complexity, you have to have a conversation partner. In order to exchange knowledge or share knowledge with anybody, you have to do it using spoken language or gestures in front of other people. There was no such thing as authorship or individual attribution. Everything was shared out in the open. There was no such thing as being done. You were always continually communicating, because to communicate, to share knowledge was to have relationships with your community. Before literacy, words were ephemeral. They were events, they weren’t objects anymore. There was no saving. The only way to record an idea was in the mind of another living human person.

Humans had to develop strategies to help preserve and transmit knowledge. The Homeric poets came up with formula, like the wine-dark sea, or the famous town of Troy to kind of create hooks, because you couldn’t pass something along totally verbatim. You had to repeat patterns. The audience for these epics and odes read the works in terms of hearing those chunks of meaning with those metaphorical hooks, so that they could retain them in memory, because that was the only way to retain knowledge. Biblical proverbs used very concrete language, because there’s no such thing as abstraction, because in order to create an abstraction, you have to be able to lay out all the items in a particular category and organize them essentially in some sort of card sort. But if you can’t write things down, you can’t step back and identify all the members of a category. Concrete metaphors were the only way to share general principles. We even see this today. Sharing quotes is very, very compelling. We really enjoy sharing bits of stories or things we’ve learned in terms of quoted utterances. It’s so powerful. I mean, you just have to look on Etsy to see how much merchandise involves using quotes.

So before literacy, all knowledge was communal and social and contextual. Then we were able to separate the thoughts from the thinker, put them in an object, and share these objects around and multiply their effect. Because of this, because we could separate out the thoughts into a separate object and step back and contemplate them, our entire way of expressing ourselves, our entire way of thinking and being changed. We became verbose. We could go on at length. We could afford the paper. Benjamin Franklin was considered a pretty witty dude. Even within a text, even when we were telling a story like, “This is an excerpt from Moby Dick.” Within the story, the author is commenting on the story, thinking about their thoughts. Hm, ah. We get semicolons. We get em dashes. We get a lot of internal reflection that was not possible before we were able to write things down. Once we moved from an oral culture to a literate culture, yes, the invention of boredom, because it didn’t matter.

You didn’t have to be interesting for your ideas to persist. You just had to use high-quality paper, because the writer could be separate in space and in time from the reader. The writer could be long dead by the time their audience finds their work. There are completely different values in an oral culture and a literate culture. An oral culture, the relationships among people and the knowledge they share are immediate and ephemeral and direct. You’re always sharing knowledge with an audience in front of you. It’s social. It’s a participatory experience. And very importantly, it involves context awareness, because when you’re communicating with somebody in an oral culture, you’re in a shared context. You can tell, “Oh, it’s raining. You’re afraid. You look hungry. Lions are coming.” You know when you’re communicating with them what’s going on around them. You can adjust accordingly. The language you’re using is very concrete and very related to your daily sensory experiences—but once you can write things down, once you separate the ideas from the human mind into an object, those ideas are mediated through that object and become part of that object.

You can have passive voice and complex ideas. You can have authority. You can have an author. You can have ownership over an idea. The acts of reading and writing are often solitary. A literate culture separates you from your community and you have a relationship between ideas in abstract contained in a non-living, inanimate object that can be finished like a work, a written work is complete and done and you have closure. We see this even today, even online. The old Slack slogan, “Be less busy.” It’s a great example of oral cultural values, because it’s direct, it’s concrete. You have a sense that someone is speaking to you clearly and succinctly. Then you get something like, “Leverage innovative market specific solutions,” which would be impossible without a process in which people could pass a document from one person to another, making it progressively worse and more abstract. Good job, literacy. Even when systems are trying to help you use the system, how often do you go online hoping for an interaction, right?

It’s supposed to be, “Oh, the web, online. It’s interactive,” but then you’re handed a document and you don’t know like, “Oh, cool. Thanks. Thanks.” This is often when communicating online goes wrong. You expect an interactive experience and you get handed a PDF, which feels like a real violation. This is the fundamental difference. Communication is alive and lively, or it’s through an inanimate object where you feel separate and distant from whoever originally thought those thoughts. This is why talking is so much easier than writing. Writing is not only involving a very specific technology that takes a lot of effort to learn. I don’t know if you remember, if you have small children, you can really see, it’s really hard to learn to read and write. Then when you go through the act of writing, you think like, “Wow. My words have to stand apart from me. I won’t be there to defend them. I won’t be there to say, ‘Oh, wait, wait, wait. Let me take that back and say that again.’” You really commit.

This is why writing feels so difficult and so often, frankly, terrifying when you’re confronting that blank piece of paper. Even professional writers talk about this a lot. Yeah. This is a photo of me asking a visual designer for some sample copy. It’s really tough. I’m making light of this, but this is why. So if you have that feeling, that’s a legitimate feeling, because we humans talking to other people in conversation is one of the things that makes us human, but writing is taking all of our skills and using them for a much different purpose, for something we’ve been doing for a much shorter period of time as humans. Then all of this writing got us to a really interesting place. Because we were able to create and share ideas of greater and greater complexity, and our technology advanced faster and faster and faster, we started to develop electronic means of communication across time and space. There was a revolution at least as meaningful and powerful as the Gutenberg revolution. By communicating by telegraph, telephone, radio, and television, all of a sudden we were brought back into community with each other.

It accelerated at such a pace. I was like, “Okay, 200,000 years ago we were talking. 30,000 years ago, cave paintings.” It was only 150 years from the telegraph to the World Wide Web. It’s like no time, no time. Because we were able to create these technologies based on literacy, we’ve come to a point where we’ve been able to come back around and be in more of a shared, electronically mediated, continuous present together. The Reverend Walter Ong is a Jesuit scholar who studied with Marshall McLuhan. He studied and wrote about this a lot. He articulated this return to our oral culture mediated through electronic communications media as secondary orality. Okay, it’s an awkward phrase. I have not been able to think of a better phrase. What he said is, “Orality knits persons, individuals into community.” The more that we return to that continuous present, that exchange of ideas like an oral culture, the closer we are in community. We see this in how we communicate today. We love texting each other and we’re typing. Physically, the acts of having these conversations is indistinguishable from writing.

Let’s sit this for a while, yeah. Darth Badger don’t care. What we’re doing is physically indistinguishable from writing, but it doesn’t feel like writing. Texting, it’s tapping into something much deeper. People will text with people much more than talking on the phone or visiting in person. It feels immediate and fun. We’re having these conversations that can also be saved as documents. That’s secondary orality, right? You’re having a conversation, but it’s a conversation that also has all of these qualities of technology embedded into it. They’re even new art forms. This is a text from dog. It’s an imagined series of conversations between a man and his dog who can somehow use an iPhone. See, this is a really delightful way of telling a story. It doesn’t feel like you’re reading a story that’s been written. It feels like you’re eavesdropping on an intimate conversation between members of this family, or butler and dog. It’s gotten so strange. It’s not uncommon to turn on our televisions, our electronic hearth and see somebody on the news reading Twitter to us.

We don’t even have words for what this is, because we’re still thinking in this bifurcated, “Oh, are we talking or are we writing? One or the other. I’m in one mode or the other,” but secondary orality is this mode of having everything combined into something really, really super weird and different every day that we’re somehow totally used to. We have these relationships with our friends that are indistinguishable from texting with a chat bot, but are still meaningful and still matter in our minds. We still feel like we have this continuous relationship with somebody out there on the other side that feels really, really good. We just roll with it, because that’s the thing about humans and conversation is, in conversation, we don’t have to be really specific and accurate. Okay, unless you’re a pedant and not a fun person to talk to. You could have a conversational exchange that goes from spoken words to texting to answering a question with a gif, and we don’t even question it. We’re like, “Sure. I understand what you mean.” We as people have these cross-channel, multimodal conversations all the time.

It’s really nice when you’re expecting something from literate culture and a little bit of humanity peeks through. For example, this is from the Guardian. You expect to just see a regular caption, but what you get is a human voice coming through. It doesn’t matter that this is text on a screen. You still sense this as conversation. You don’t need technology to interpret this or generate this from machine learning. This is very conversational, but there are ways to get it wrong. It’s very important when you’re being conversational to not be creepy. Don’t be creepy. Don’t be creepy. It can go too far. Conversation isn’t about just uttering words aloud. There are deeper principles. It starts with a shared goal. Just like good collaboration, you can’t have conversation without starting from a shared goal. That’s fundamental. You see this. If you can be lost in any city and as long as you can make yourself understood to anybody on the street, you can ask for directions. They’ll give you directions. That is a miracle. You don’t have to establish something. You don’t have to negotiate something with a total stranger.

You could just spontaneously enter into functional conversation with any other human as long as you share enough of the same language instantly, and you don’t even think about it. That is really miraculous. That happens because we cooperate with each other. The linguist Paul Grice, who worked in the field of pragmatics, which is the branch of linguistics that deals with how context contributes meaning. He came up with his cooperative principle. As you can see, ironically, it was written in a very literate form. I couldn’t remember this verbatim and repeat it back to you. So you can think of the cooperative principle as, “Read the room. See how people are, and pitch in. Do your part to help.” That is fundamentally the base of cooperation that makes conversation go. He came up with four maxims to articulate the conversational principle. Those were quantity, quality, relation, and manner. A functional conversation will adhere to these maxims. You can think of them as giving just enough info, being truthful, being relevant, and brief and unambiguous.

You can tell if a conversation doesn’t feel right or is starting to go off the rails, it’s probably because one of the people involved is violating one of these maxims. Like cornering you at a party and just boring you. You’re like, “You are violating Grice’s conversational maxims.” Because of in Minnesota, can’t leave this one out, politeness. Politeness was not one of Grice’s original maxims. It was contributed later by Robin Lakoff, a scholar at UC Berkeley who worked with gender in communication. She came up with the politeness principle. Don’t impose, give people options, make the listener feel good. Very, very important part of a functional conversation. You can probably tell that these are things that when we’re talking about system design, interactive system design, these are really important principles that also apply to good software. What makes conversation work? It isn’t about speaking loud. It’s about being cooperative, goal-oriented, fast, truthful, turn-based, polite, and error tolerant, right? You don’t have to get exactly the right input to respond.

This investor, very smart guy at Bloomberg Beta said, “Well, if we believe machine intelligence will make our applications smarter, they might as well start talking to us.” That’s what makes conversation the new interface. Well, conversation is the oldest interface, but just because our systems are getting more sophisticated under the hood doesn’t mean having them talk to us is necessarily the best way. Because conversation as a set of deep principles, underlying interaction can make systems more humane, but as an interface, if something looks like a door, if you can’t walk through the door, it’s not better. What’s happening is people are saying, “Oh, we’ll invent these new interfaces that will make things easier to use,” but they aren’t actually functioning like conversation. If they’re not functioning and fulfilling the principles of conversation, then we want to go back to using websites, because websites are great.

I’m worried that we’re in a place where we’re thinking, “Oh, the easiest thing to do is make our microwave talk to us. Websites, old-timey. Writing things down, old-timey. Those aren’t conversational.” But oftentimes, a website will be more conversational than something that’s trying to talk to you, but isn’t really well-equipped, because it’s not about the mode. It’s not about something speaking aloud. It’s about following the principles. Cooperative, goal-oriented, fast, turn-based, truthful, and polite. A system is conversational to the extent it embodies these principles. Error tolerant, very important. One of the most conversational systems has been around for 20 years. Google Search, and you probably wouldn’t think about it like that. We all take it for granted, but if you look at the way it works, you enter some input, simple search box. You get something back. If it isn’t quite what you wanted, you revise it. It’s really fast. It tells you how fast it is. “You got over four million results in half a second.” It is error tolerant. It’s polite. It gives you options.

This is why so many people treat Google like the homepage to the internet, because it feels like, “Okay, it’s cooperating with me. It’s being truthful.” Google is very upfront about the source of the information it’s giving you, whether something’s sponsored or not. Google Search, even the text part, even without the talking part that you can now do, is extremely, extremely conversational. Now that software is on a path to replacing people in our relationships, or mediating our relationships between two people, it needs to behave like a person, alive and present. But from those fundamental principles, not pretending to talk and then not understanding us, because it’s about being more humane. In order to really share labor and divide the work up between humans and computers, it’s important to know what humans are good at and what computers are good at, because even though we invented computers, we actually aren’t very much like them. It’s a bad metaphor for the brain, or the brain’s a computer. Not really. Humans can be good at empathy, and persuasion, and unstructured solving—you know, solving problems where you don’t quite have all the information or don’t know what you need in advance, or assessing relevance.

You could just say to your friend, “Oh, help me find shoes that go with this outfit,” and your friend wouldn’t have to have been trained on every outfit in your wardrobe in order to make a recommendation. They could just say, “Oh, that goes with that.” Super easy for a person. Humans are bad at remembering, right? We’ve got to memorize things off of written sheets of paper. It’s really hard to remember verbatim just in conversation. Bad at calculating. You can learn. Really bad at logic. Really bad at precise repetition. You can hear something and then maybe paraphrase. We’re really, really bad at these things. Computers are great at remembering, calculating, logic, kicking back exactly the thing you typed in. Computers are awesome at that. Terrible: empathy, persuasion, unstructured solving, assessing relevance. Very, very bad. Have to be trained on all of that information. Still bad. Still just incorporate the biases of whatever information they were fed.

So computers and humans, better together if you take advantage of the flexibility and feelings and all the squishy stuff about humans in society, and support that with the ability to manage rules, and retain information, and find patterns, and make predictions based on patterns. But this only works if humans put humans first and don’t just declare some new technology conversational and say, “Oh, learn how to use that technology. Once you do that, it’ll make your life so much easier,” because life is really complex now, right? It’s all this amalgamation of systems, and we’re scaling all the things, and we’re ops-ing all the things, but Bill Gates had a very astute observation that any technology that’s used, like in a business, if you automate it, if it’s already an efficient process, automate it more efficient. If it’s inefficient, it’ll just magnify the inefficiency. This is true in every context and for any attribute.

If you have bias in your processes and then you automate your processes, they’ll just ge,—it’ll be like more bias on rails. Just automating things doesn’t make them better, because fundamentally, that’s what a zombie apocalypse is. It’s just a bad process, eating brains, automated. Just automating and scaling things doesn’t make them better. We have to figure out the things, I know, a lot of uncomfortable conversations before we introduce all the technology. This is what’s happening. This is why things are so hard to use now. We have all these different ways to get things wrong, because we’re just throwing technology at our human interactions before we’re sorting out the human interactions. We’re getting more and more systems that are designed around the technology, and not designed around the people. Which system is best? There’s no one best. The fanciest one is not the best. The one that takes the most engineering to develop is not the best. It’s the one that takes the least effort to use. That completely depends on the perspective of the user. It depends on which one feels the most cooperative.

Does it feel like, “Oh, this system is helping me out, this system and I have shared goals”? Or do you feel like, “Oh, I’m fighting this thing, and I have to keep repeating the right phrase to see if it will give me the response I need”? Not cooperative. We have to ditch this idea of literacy. We love literacy, especially those of us who are writers or readers or have any sort of authority and expertise that was granted to us in institutions based on literacy. We want to hang onto literacy a little bit, but in the words of Alan Cooper, “Literacy is a euphemism for making it easy on yourself, so you’re happy making it harder on the user.” We just need, in order to have a conversational system, need to completely get rid of the idea that anybody needs to be literate in any system we’re designing, right?

We know that we want to make systems that are intuitive, but we’re sort of tricking ourselves to say, “Oh, they still need to learn how to use it, because it’s cool and they should learn. We shouldn’t put a line of instruction, because that would be cheating.” We need to have our products stretch to meet the ways of thinking of people out in the world without any need for any learning or study, because there’s no one right channel or mode. It’s what matters for a given goal and context, right? If it’s urgency, privacy, accessibility, cost, status. Whatever matters. Whatever matters, whatever helps make that system feel familiar, then that’s what’s important to the system. In order to do this, you need to know, “What’s your role? What your organization’s role in somebody’s life? What value are you bringing?” Because a system is only conversational to the extent that it’s context aware, that it really takes into account the lived experience of each individual person interacting with that system.

That will make it feel to people out in the world like real, caring humans were involved. They didn’t just put their ideas into an object and walk away and stop caring, and create a document, and publish the document, and go on to the next thing. It’s about existing with people in that shared community with shared values and shared respect. No matter what systems you go forth and design, I just have one request. Do not tell people to tap on Erika. Thank you.

Join the mailing list

Sign up and be the first to know about all things Confab. We promise to email you only if it’s important.

Thanks! You have been added to the Confab mailing list.
Oops! Something went wrong while submitting the form.