Product Design

Let's chat: conversational UI training

Talking to computers

Whether you’re typing a question into a search engine or asking a virtual assistant, the answers are often readily available through our devices. With all of these answers so easily accessible, what is the benefit of a conversational UI? The answer is in its name- the conversation.

Imagine you want to see a movie with some friends in the evening, but you are unsure what movies are out, what they’re about, what times they’re showing, or what theaters are around you. It would be easy to simply ask these questions to someone who knew all the answers and was tailored specifically to cater to that need. This is the type of need that’s perfect for a conversational user interface (CUI), or chatbot, to solve.

The conversation facilitates problem solving, discovery, and planning. Using a messaging platform such as Facebook Messenger, or another messenger nestled within a companion mobile application, you could start a conversation to plan your evening. At the end, you could have your tickets sent directly to you by email, text, or natively in the app.

This is just one example of the possibilities and helpful interactions that a CUI can bring to a user’s experience. But how do we gather data for the conversational “script” of a chatbot?

It can seem like a large task. However, by setting up a workshop, you can conduct many conversations at once and gather the data in an organized fashion.

What does the workshop look like?

Hosting a workshop is a useful way to gather highly relevant training data for a conversational UI in order to make its interactions more user-friendly. At WillowTree, we conducted this workshop and found that it produced usable data for training a CUI. The workshop itself consists of :

  1. The participants (an even number is preferable, as the participants will be split into subgroups, the “users” and the “bots;” the participants are then put into 2-person user-bot pairs).
  2. Laptops for each participant.
  3. A consistent messaging platform for the participants to chat on (Slack, Facebook Messenger, Hipchat, etc.).
  4. Content cards (to guide conversations into usable data points).

Continuing the theme of a night out at the movies, the output of this specific workshop example will be records of conversations between workshop attendees playing the roles of “users” and “bots.” This workshop is limited to basic functionality to provide a sample dataset that matches the current capabilities of CUIs.

Two people acting in different roles with given content cards while chatting in a messenger app encourages the conversations to follow the natural convention of a person-to-person conversation over an online messenger platform. It is important to analyze these kinds of cyber-interactions as they are often timed differently than a conversation in person. For example, a person has more time to think of a response over Messenger, as their response is not expected instantly. Therefore, more complex responses may be formulated over Messenger than they are in person.

Setting up the workshop

Conversational UI img 2 In preparation for the workshop, create info cards for your “chatbot” role participants and prompt cards for your “user persona” participants. These cards will keep the data within a range that you control.

The “bot” will have a series of cards that represent all of the information that the bot has at hand about the movie showings (location, movie, date, and time).

The “user” will have a deck of cards sorted by suit (location, movie, date, time). Some of the cards have complete information, while some have unknowns. These cards may also be a bit more vague than the chatbot cards, to mimic how a real person might ask a bot a question. For example, instead of “7:00 PM” as a time, the card might say “in the evening.” This mimics a real-life example in which a person might ask a bot, “What’s playing on Thursday in the evening?”

In addition, the criteria may not exist at all. A “lack of preference” card can be denoted with a question mark. Including ambiguity in the conversation allows the bot participant to respond to more natural requests that a user may have.

Each person should have a computer with 1:1 an instant message service like Slack or HipChat. Be sure that all participants are using the same service.

Conducting the workshop:

Conversational UI img 3 The workshop is broken into several “scenarios.” At the beginning of each scenario, the user will shuffle each stack and choose one card from each. The conversation will proceed with the user asking the bot questions based on the cards they’ve drawn, until all four pieces of info are agreed upon by the pair to reach a satisfactory outcome. Here’s how the scenarios unfold.

As we mentioned above, your conversational UI workshop should have an even group of participants. These participants should be split into pairs and each given one of two roles: the user and the chatbot.

Give your participants the corresponding information cards for their role and have the user initiate a conversation with their chatbot partner by selecting one card from each category (date, time, location, movie title). The bot will use this information to request a movie.

Encourage banter and playful conversation. In the messaging application, have the subjects change their screen names to NAME-BOT or NAME-USER in order to denote who is in which role.

Have each pair work through a scenario three times, varying the language of their exchange before moving onto a new scenario. Once the pair has worked three scenarios, they can switch roles (and change their usernames) and complete the same process as the opposite player. Once pairs have completed three scenarios with the opposite role, they can trade partners with an adjacent pair. This process can continue for as long as time allows. Throughout the process, ask pairs to read through their interactions for the group so you can identify snags or give pointers. Also, ask for successful interactions, funny interactions, and awkward or weird interactions.

Once a few rounds have been completed in this way, add “plot twists” to the mix. Have the user pick up a “plot twist” card after communicating their first choices. These cards add another dimension to the conversation by either making the user change their mind about a location, movie, date, or time, or add a new conversation topic.

Collecting data:

After gathering a sufficient number of conversations, save each to a .txt file. If you are training a conversational UI in software that tags intent or categorizes responses, tag (or have the users tag) the conversation lines with the proper label. For example: “Hey! What’s up!” Can be tagged as: #greeting #friendly. Conversational UI img 4 By using this data as input for your bot training system, you can train your conversational UI into a knowledgeable and fluent chatbot, able to assist users in their endeavors while making talking to computers a little more personable.

For more information about chatbots and CUI capabilities:

Part One: Shortcuts to ChatBot emotional intelligence

Part Two: Shortcuts to ChatBot emotional intelligence

Quiet innovations and a push for mindfulness: Takeaways from Apple’s WWDC 2021 Keynote

From iOS 15, iPadOS 15, and macOS Monterey getting announced to a clear push for...

Read the article