Chatbot Pitfalls and Solid Advice on Getting it Right
According to Gartner research, it is estimated that 60.5 million Americans use a virtual assistant at least once a month. If this trend continues, chatbots will likely power 85 percent of all customer service interactions by the year 2020.
It is said that chatbots are the new FAQ. Every second, 40,000 search queries are made online worldwide. That adds up to 3.5 billion per day or 1.2 trillion each year. That’s a lot of people looking for a lot of answers. If chatbots are built right they definitely enhance the customer experience and make it easier for a customer to find the content they need. Technical support is the most common type of chatbot content. Chatbots provide a bridge between technical documentation teams and customer support call centers. Salesforce’s Chief Digital Evangelist, @ValaAfshar predicts that “The line-of-business that is most likely to embrace AI first will be the customer service – typically the most process-oriented and technology savvy organization within most companies.” AI here refers to NLP and NLU enabled content structures.
From my vantage point at SDL, it has become increasingly clear that content really matters. The modern digital customer journey is a journey that is marked and defined by interaction with different content. The conversational interaction is needed everywhere, not just with Alexa and other voice-based virtual assistants. Imagine searching for technical support content and finding 20 large PDFs that may contain the answer to your question. Wouldn't it be useful to have the ability to zero in on the right content within these PDFs through a series of clarifying questions? The real question, then, is not where does the content for chatbots come from, but rather how do we prepare, organize and structure that content so the chatbots can use it? This superior delivery of the right content to many customer questions can only happen if you have a proper content organization model. At SDL we call this GCOM and it involves content architecture, organization and process to make it happen.
 | ||
| Source:5 ways chatbots are revolutionizing knowledge management |
However, it is very easy to build chatbots that are frustrating, useless and inefficient. We have all experienced websites where the only question the IVA (interactive virtual assistant or chatbot) is able to ask is "Hey dude, do you wanna buy my s*&t?" Chatbots are purely a reflection of the capability, fastidiousness and patience of the person who created them; and how many user needs and inputs they were able to anticipate. When done badly, which is very often at this stage of evolution, chatbots will kill customer service.
This is not so different from DIY machine translation 10 years ago. Many really bad systems were built and forced upon poor unsuspecting translators. Like MT, this too will take competence and skill and it is wise to work with experts.
Digit’s Ethan Bloch sums up the general consensus: “I’m not even sure if we can say ‘chatbots are dead,’ because I don’t even know if they were ever alive.”
Our guest post by Ultan O'Broin is a look at how to do the right things the right way to make chatbots that are indeed useful in providing the right content to the right people and provides insight from the design and UX perspective in particular. While the content organization issues are discussed often, it is worth a review and is well summarized in this article. Ultan has several other posts on Medium on the subject of chatbots and is worth a closer look.
What excites me the most is that once you have the IVA working properly and actually useful, it then makes sense to make it multilingual with optimized machine translation. But first things first.
=====
Ultan O’Broin looks at how the Jobs To Be Done framework and simple build approaches for the minimum viable product can counter chatbot product management failure and fatigue.
 |
| Guerrilla testing of Snapchat Spectacles at Hook Head in Wexford, Ireland. We can try out products in the wild, but are we as focused on the job we’re hiring our product or service to do? |
One of the compelling hopes for chatbots was that utter power of an agency to increase digital participation in a natural way, partly through reducing app fatigue. Ironically, I now believe we have reached the stage of chatbot fatigue.
Just like everyone was “doing” a smartwatch solution two years ago, suddenly chatbots are the tech “game changer” du jour.
And botification is being done just as badly as the smartwatch overkill.
Chatbot Game Chancers
There’s bot rot out there. The problem is often there is no design thinking about whether a new or existing job is worth botifying or not.
I have seen some criminal chatbots out there. From a barista training chatbot aimed at millennials that delivered a 50 page PDF manual in response to the utterance “how can I be a barista?” to that fitness chatbot telling me that my nearest Parkrun was 8,000 kilometres away, the assumption being that anyone asking was within a 300 kilometres radius of an Irish location. I was in San Francisco at the time, where there is a local run.
None of the arrivistes responsible for such rotbots have asked: “Why is this person hiring this chatbot to do this job?”
The problem is the wrong chatbots are being built, wrong. Fundamentally, begin by asking, “Why would anybody hire my chatbot to do this job?”
This article is about designing the right chatbot, right. But the profound principles to do so can be applied to any app or service.
The “User” and the Damage Done
First, the term “user” must go from the design conversation! User? WTF!
I hate that term “user”; only the usability community and illegal drug business uses it. Let’s reclaim “user” as a role in real life with a real job (unpaid or otherwise) to do. So, our “user” becomes a sales rep, a technician, a concert-goer, a mother, a parent, a patient, and so on.
But how do we find such people, especially if you’re a small startup or innovator? The answer is to think smart, get out on the street, watch real people, and ask them about what they’re doing and encourage them to articulate reasons they would use a chatbot, app, or service instead.
Job Profile Persecution
Forget about the job profile approach to identify your “users”.
Those unwieldy tl;dr profile documents of job titles, descriptions, qualifications, experience, org chart position, motivations, IT expertise, and so on. These tomes owe more to recruitment agencies and HR departments than to design thinking and product management.
Profiles list many tasks that the role performs, not just the focussed critical 80/20 reason why your conversational UI might really be a game changer and around which a killer solution can be built.
A job profile is not a real person. In the user experience business, it’s a statement for the prosecution of the mediocrity of design.
Don’t Take It Persona-ally, But…
Personas are next to go onto that “user” bonfire of the inanities.
Personas are stereotypical people, dumbed down versions of “a day in the life”, complete with stock photographs, typical responsibilities and tasks, personal characteristics, motivations, and tools.
Again, a persona is not a real digital adopter, but a distant idealisation, performing a non-prioritized list of tasks. These persona peeps may use many types of technology too; not good for innovation disruption by understanding why they might switch or integrate with your innovation.
There’s a reason the word “stereotype” usually comes with a negative connotation.
Get Close and Personal. Swipe Right for The Right Design
Instead of “user” personas and profiles, move closer to the customer, and discover what they do.
How? Great chatbot design begins with a contextual conversation.
Take a walk in their shoes. Try a little fun UX ethnography, guerrilla research, and the “Wizard of Oz” technique to identify that killer question or job that might inspire someone to continually turn to a conversational UI to actually do something.
 |
| Research and design a chatbot that resonates emotionally and culturally, as well as functionally for your region. Check out Hofstede’s country comparison dimensions to help you shape that experience. |
For example, if we were building a chatbot for fitness enthusiasts we might hit the local gym session schedule, post to a Facebook fitness group or fitness app discussion section, or engage on Twitter or Instagram with an appropriate hashtag.
Use a deep-listening, “tell me more about that”, approach.
 |
| Original “The Wizard of Oz” movie poster. From Wikipedia; image in the public domain. |
Amazon, in the run-up to the French launch of its Echo, for example, introduced Alexa to employees in its French fulfillment centres who interacted with the emergent voice assistant to help the voice chatbot learn French, cultural nuances and behaviours, and how to respond.
 | ||||||
| The launch of the French version of the Amazon Echo was preceded by real people to learn the language, cultural norms, and how they actually behaved! |
Fundamentally, focus on the person’s behaviour, and not on what they say or think they want. Watch their actions. Think of this as a Lean product management approach, a way to quickly design and build a solution to determine the certainty about its value. As Eric Ries says:
“The minimum viable product (MVP) is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.”
I encourage you to read Eric’s book, “The Lean Startup” and apply the thinking to chatbot product management and design.
What Features to Design?
Why do drivers of cars buy milkshakes at drive-in takeaways? There could be many reasons. By way of Harvard professor Clay Christensen’s (and others) Jobs To Be Done framework, it was found that people don’t buy and use things because they fit a persona or user profile.
Instead, people hire a service or product to do a job for them. In our milkshake example, the drink was hired because the drivers were bored and in danger of nodding off on a long journey.
So, use the Jobs To Be Done (JTBD) approach for designing the right thing, right!
Again, read (or listen to the podcast) about Clay’s theory of disruptive innovation and Google JTBD for more.
Think of it this way, as mentioned on the startup Intercom’s blog: JTBD allows you to focus on the essential product feature, and to generate a user story about why it’s needed. The story plot line structure for a product story, using the JTBD method, would be something like:
“When X happens, I want to Y, so I can Z.”
An example from the sales rep job world might be:
“When new sales collateral comes online, I want an SMS on my phone, so I can take it on the road with me.”
Designing using JTBD enables a template approach, listing the job to be done, the role, needs, and expected job outcome. We can add outcome performance goals in there too, as a way of testing and proving ROI.
Here’s a one-page enterprise JTBD example you can turn into a template and use yourself:
 |
| A JTBD template completed for a mobile sales rep. You can use this template approach to frame the job to be done and as a way to shape a story about the job itself. |
In our example, a sales rep on the road might hire a chatbot to enter new sales prospect details rapidly without using a special device, easily capture meeting locations and images of the opportunity, and share that data in SaaS for work later. She might want to do in 5 seconds. So, a voice-driven chatbot using smartphone out-of-the-box features such as the camera and GPS capability seems like a good opportunity for a chatbot solution.
Using the JTBD approach also helps shape and communicate the job story. A story untold is not a story. Here’s your storytelling formula: SUCCESS — Simplicity, Unexpectedness, Concreteness, Credibility, Emotion, Story.
Phrase your job story well; using bullet points is fine. As John Dewey put it:
“A problem well put is half solved.”
You can tell you JTBD story to customers, product managers, developers, investors, designers, or any stakeholder (although C-level management will probably just want an executive picture on the return on investment for their budget spend).
Competition is Fierce, from Unexpected Combatants
Remember: There is competition with other tools for hiring your product or service.
In the case of our sales rep, she will still carry a notebook and pen — it helps recall — sticky/Post-it notes, an Apple iPad, a Windows laptop, and a Samsung smartphone, maybe.
This hire competition is at the job level, it’s not about the category the tool fits into. As Intercom points out, Microsoft Skype, for example, addresses the same purpose as an airline seat for a business trip— communicating with colleagues.
Chatbots compete against other tools and methodologies on the job hire level in many arenas — communicating, scheduling engagements, ordering, onboarding of employees, managing things to do, marketing, educating, entertaining, finding simple solutions and fixes — and must do so without special training, equipment, and so on.
But fundamentally, the JTDB approach really means the end of design as many people have come to think they know it. Design now becomes about the job the chatbot is being hired to perform by a human.
tl;dr? OK, Summary
So, to design and build the right product right, remember these six simple points.
Build the Right Thing
1. Use the JTBD framework: Why is this chatbot (or other product or service) being hired to do the job by this person? Watch their actual job behaviour, ask, and listen.Build It Right
2. Get to the MVP stage fast. Apply proven UI principles for chatbots such as creating prompts and messages with a conversational style, shape a personality, pick use cases based on speed, simplicity, pain alleviation, and that requires no special device or training.
3. Have a clear primary job to be done. What is the 80/20 effort of the job? Avoid corner cases, nice-to-haves, and “what ifs?”.
Consider Microsoft Excel; a super-popular desktop spreadsheet and now service-offering, packed with features. But how much of that functionality do you use and how often? Probably 20% of the features or less to do 80% of the jobs you need Microsoft Excel for. There are other use case examples from the enterprise to get you thinking along those lines for chatbots.
4. Sketch and wireframe your solution first. Balsamiq, for example, offers great digital solutions, and there are now conversational UI specific options. But all you really need to start is a pen, paper, and ideas. You do not need to be an artist.
Using wireframes means you can also apply familiar UX design patterns, making for productive development. Document any open questions, use an Agile backlog and try collaborative, integrated cloud tools like Slack, or whatever suits your context of work, to agree your sketch with stakeholders.
 |
| Balsamiq wireframing stencils. There are awesome tools for designers, developers, product managers to sketch and agree on that chatbot JTBD |
5. Use an accelerator design kit or platform with suitable backend AI and NLP capability to innovate fast. Leverage usability heuristics, and use real text and voice scripts in your designs — in the language of the hirer— and iterate with the key stakeholders to agree on a chatbot solution before you code anything. This agreement eliminates surprises later!
 |
| Smartly.ai, a platform for creating conversational UI chatbots for SaaS, across different devices. |
6. Don’t dribblise your design. With chatbots and conversational interactions stay true to the idea that no UI is the best UI of all. The UX design toolkit is now API calls, AI, NLP, ML, integrations with services and regular device features (GPS, SMS, camera, and the microphone, for example). Design platforms and kits often provide any UI widgets if and when needed (for maps, attachments, avatars, and so on).
No UI is the best UI to create that killer new user experience.
Job Satisfaction? Test It!
Finally, do remember to test your innovation.
UI heuristics and baked-in platform usability can do some heavy lifting to prove your idea’s value, but for JTBD the most reliable testing of having solved a design problem are tests that focus on real tasks by real people doing real jobs.
Plan to test your innovation before going live and after, and iterate if needed. Keep focused on the JTBD. If that is wrong, then no amount of fancy visual UI design is going to fix it and improve switching or adoption from another app. As Seymour Chwast puts it:
“If you dig a hole and it’s in the wrong place, digging it deeper isn’t going to help.”
Remember, people are hiring your chatbot to do a job. Hiring. In the age of the cloud, it’s easy to switch chatbots, apps, and services because of a subscription model.
There’s competition for that apps, services, and chatbot job hire. So, be competitive!
=======
Ultan O’Broin (@ultan) is a digital transformation and user experience consultant working with startups and the STEAM community, specialising in conversational UI and PaaS4SaaS. With over 20 years of product development and pre-sales design thinking outreach with major IT companies in the U.S. and Europe, he also speaks and blogs about UX and technology.
Watch out for ConverCon 2018 in Dublin, Ireland in September 2018 (previously reviewed), an event with chatbots at its heart. I might see you there to chat in person about the JTBD of your chatbot!
All images and screen captures in this article are by Ultan O’Broin. Copyright or trademark ownership vested in any screen capture is acknowledged, where applicable.
Comments
Post a Comment