Chat Logs : how to train your customer service bot by monitoring its chat logs, using keywords and topics

Bot Libre supports a diverse set of features for creating and training your bots. Today we will discuss the Training & Chat Logs feature, and how to create and train a customer service bot using the Training & Chat Logs page. The Training & Chat Logs page provides the best way to monitor and train your bot, by letting you see its conversations and let you correct its responses. No scripting or programming is required.

To create a customer service bot, create your bot using the 'service_template'. This template has learning disabled. This is important, as normally for a customer service bot, you do not want it to learn responses from your users. It also does not include the generic AI scripts WhatIs, WhereIs, Understanding, as these scripts can normally lead to the bot responding to questions outside of its domain. If you created your bot through other means, you can disable Learning, and Comprehension from the Learning & Settings page in its Admin Console. You can also remove the WhatIs, WhereIs, and Understanding scripts from the Scripts page in its Admin Console.

Once you have created your bot, you can enter or upload some common questions and answers for your domain. If you already have a FAQ on your website, uploading this would be a good way to start. Once you have your basic bot built, you can deploy it to your website or beta site, and let others interact with it. Letting users talk with your bot is the best way to see what questions you should train it for.

To view your bot's conversations or trained responses go to the Training & Chat Logs page. To get to your bot's Training & Chat Logs page, first select your bot from Browse (click on My Bots), then click the Admin Console button (gear icon), and finally click the Training & Chat Logs link from the top toolbar or menu.

The Training & Chat Logs page initially shows nothing, you need to first select what your want to see from the Search drop-down.
The choices are:

• conversations : the log of each conversation your bot has had
• responses : all responses your bot has been trained with
• greetings : all greetings your bot will use to start conversations
• default responses : all default responses your bot will use when it does not know a better response
• phrases : all phrases (sentences) your bot has encountered
• words : all words your bot has encountered
• flagged responses : all responses that have been flagged by users or administrators

Next you need to select a Duration from the next drop-down. The duration is just the time period to view, from the last 24 hours, to week, month, or all. You can also filter the results that contain certain words, or filter the type of conversation (chat, email, tweet, Facebook, Telegram, Slack, etc).

Conversations

To see how your bot is doing, search for all conversations since you last checked it. This will give you a list of conversations. The top toolbar gives a set of buttons for administrating your bot's responses.



From the left to right the buttons are:

• menu : choose icons from menu
• home : return to the training home screen
• enter : enter a new default response.
• correction : enter a better response for each selected response in the conversations
• inspect : browse the selected responses or phrases.
• select all : select all listed conversations
• invalidate : mark the selected responses as incorrect, or decrease their % correctness (the bot will never use these responses to the question again)
• validate : mark the selected responses as good responses, or increase their % correctness (the bot will use this response to similar questions)
• flag : flag the selected phrases as offensive, the bot will never repeat them, and will not let users enter them
• unflag : unflag the selected phrase as not offensive phrases
• delete : remove the response from the question, or delete the conversation
• download : export and download the listed conversations as a chat log, response list, or AIML set
• upload : upload and import a chat log, response list, or AIML file from your computer
• import : import a chat log, response list, or AIML script from the shared script library

When you review your bot's chat logs, search for relevant questions that it answered poorly. Click the checkbox beside its response to the question, and click the correction button. This will take you to a screen to enter a better response to the question. Enter the response, and optionally a keyword or required words from the question, or a topic to categories the response. Click Save. Your bot will now answer this question, and similar questions, or questions with the keywords, using your new response.

If the user's question was specific to the bot's previous response, then click the checkbox beside the bot's previous response. This will add the previous response to the new response, so that it will use this response in the correct context.

Responses

To see all of your bot's trained responses, search for responses for the all duration. This will list all of your bots responses, ordered by when they were added. A response will not only be used for the exact question it was learned for, but also any similar questions. When your bot encounters a question it does not know a response to, it will find the best matching question that it knows a response to, and use that response. If the bot does not know any similar questions, then it will use its default response.

The response browser lets you enter a new response, or edit, delete, invalidate, validate, or flag any existing responses.

The response editor lets you enter the question and response and optional response properties and metadata. You can choose to edit your response using a rich 'WYSIWYG' (what you see is what you get) editor, or a plain text editor. Responses are in HTML, and can contains most HTML tags such as <b> for bold, <a> for links, <img> for images, and <button> for buttons.

Response Properties and Metadata

Response can contain several different metadata properties that can be set to influence when the response is used, or how it functions. To show the different response properties click on the checkbox for the property under the toolbar. For example to show topics, click on show 'Topic' checkbox, and then the response topics and topic input will be available.

To edit a response and its properties, click on the checkbox before the response in the list of responses, then click on the 'Edit' toolbar button or menu. You do not need to enter any of the response properties, but they can be used to improve when you bot uses its responses, and for more complex conversational dialogs.

Keywords

Keywords are important words in the question that should influence when the response is used.

// Keywords are used to allow this response to be used for any questions on hockey or spinach.
question: do you like hockey
response: Hockey is my favorite sport.
keywords: hockey

question: i love spinach
response: Spinach is is my favorite food.
keywords: spinach

A keyword will be used by the bot to select the best response. Keywords let some words count for more than other words. In the example, the bot knows two responses, how will it respond to the phrase "I love hockey"? Without the keywords the bot would probably match this question with "I love spinach", but since hockey is a keyword, it will instead match it with "Do you like hockey?".

If the response has no defined required words, then the keywords will also be required for the response match. So "I love hockey" would not be matched with "I love spinach." because it is missing the keyword.

Be careful using too many keywords. If every word is a keyword, then their value is diminished. Never make common words like "the", "a", "he", "what" keywords, if you require one of these words for a match use a required word instead.

You can also review all of your bot's keywords by searching for 'words' and restrict to 'keywords'. This lets you remove a word from being a keyword if you added it by mistake.

Required

A required word is a word that is required to be in the question for a specific response.

// This example requires the words 'like' and 'hockey'.
question: do you like hockey
response: Yes, I like hockey.
required: like hockey

A required word can be used to ensure the bot does not choose a response if the required word is missing from the question. For example the question "Do you play hockey?" would normally be matched to this response, but because it is missing the required word "like", it will not be considered as a match.

Previous

A previous responses is the bot's preceding response. Previous responses can be used to give a response a context.

// This example uses previous to give different answers to 'yes' depending on the context.
question: yes
response: Great, wanna go out sometime?
require previous: Are you single?

question: yes
response: I am please to hear that.
require previous: Are you happy?

The question "Yes" has multiple responses. The bot can use the previous response to choose the best response. You can also invalidate a previous response to ensure the bot does not use a response if the previous response was preceding.

A previous response can either be 'required' or optional. If required, the response will never be used unless one of the previous response matches. If optional (default) the response is given precedence if the previous response matches, but can still be used if the previous does not match.

Topic

A topic is the general topic category for the response. For example a response on a hockey related question would belong to the "hockey" topic. You can use topics to categories your responses and give the bot additional context. If the bot has multiple responses to a question, it will use the one that matches the current topic.

Topics can be either optional or required. The response topic is optional by default, it you click on 'Require Topic', then the response will only be used if that topic is active in the conversation. If optional the topic will be set as active for the conversation, until a new topic is set.

Topics can also be defined as exclusive. An exclusive topic defines an isolated conversational space. For a normal topic that is not exclusive the bot can still use responses that do not have the same topic, but responses with the active topic will be given precedence over responses without a topic. For exclusive topics only response that have the same topic as the active topic will be used. No other responses will be used (other than possibly the default response if there is no topic default).

Topics can also be set on greetings to start the conversation with a topic. Topics can be set on default responses to provide a different response when that topic is active.

// Categorizes the product response.
question: what are your products
response: Our products include ACME CRM and ACME ERP.
topic: products

// Use an exclusive topic to define a joke conversational dialog.
question: joke
response: Knock knock.
exclusive topic: joke

question who is there
response: Boo.
require topic: joke

question: boo who
response: Don't cry. lol
require topic: joke
think: conversation.topic = null;

default: Say 'Who is there?'
require topic: joke

default: Say 'Boo who?'
require topic: joke
previous: boo

Intent Labels

A response can be given an intent label to let you reuse the same response in other questions. You can also use labels in a response's previous.

// Use a label to reuse the common products response.
question: products
response: Our products include ACME CRM and ACME ERP.
label: #products

question: what are your products
response: #products

question: what software do you sell
response: #products

question: are they good
response: Our products are the best in the world.
previous: #products

On Repeats

If the user asks the same question, or similar questions multiple times, you may want your bot to give a different response the second time. On repeat lets you set one or many responses to use the second time that response is triggered in the same conversation. You can also set 'No Repeat' to never repeat the response in the same conversation.

// Offer to escalate the issue if the user repeatedly asks for help.
question: help
response: How can I help you?
on repeat: What is the issue that you are having?
on repeat: Would you like me to have a support staff contact you?

question: goodbye
response: Before you go, would you like to take a survey?
no repeat:

question: goodbye
response: Goodbye.

Conditions

Conditions let you enter Self code that must evaluate to true for the response to be used. This is an advanced properties as it requires you understand Self (a dialect of JavaScript), but can be used to do very powerful things.

// Response to goodbye differently based on the time of day.
question: goodbye
response: Goodnight.
condition: Date.get(#hour, Date.time()) > 18

question: goodbye
response: Goodbye.

Think

Think let you enter Self code that is executed when the response is used. This is an advanced properties as it requires you understand Self (a dialect of JavaScript), but can be used to do very powerful things.

// Track the user's name.
pattern: my name is *
template: Pleased to meet you {star}.
think: speaker.name = star;

question: what is my name
template: Your name is {speaker.name}.

question: can we change the subject
response: Sure, what would you like to talk about?
think: conversation.topic = null;

Confidence

You can enter multiple responses for the same question. The bot will use the response that it is the most confident in for the current conversational context. By default a response is given a confidence level of 90%, but you can customize the value. Note that in a group discussion such as when the bot is added to a chatroom, or on Twitter, the bot will only use responses that have >=90% confidence.

Sentiment

You can associate a user question or phrase, or individual words with sentiment (good vs bad). Sentiment can be used to track how the user is feeling about the current conversation. The sentiment statistic is tracked by your bot daily and can be graphed from its Analytics page.

question: thank you
response: You are most welcome.
keywords: thank
sentiment: good

question: you suck
response: Sorry, I am doing my best. Would you like me have someone contact you via email?
keywords: suck
sentiment: bad

word: suck
sentiment: bad

word: great
sentiment: great

Next and Conversation Flows

You can add a next question to a response to start a conversational dialog. The next is a follow-up question that the user might ask after the bot gives the response. After you enter the next question and click save, the next question will appear below the response in browse. Click on the question to edit it and define its response (and optionally another next for a nested dialog). You can add multiple next question to a single response. The response's next question form an isolated conversational context.

If a response has a next question, the bot will only choose a response from the response's next, it will not consider any other responses. The bot will find the best matching next question, otherwise use the default response. To enter a default next response, enter the question as #default. You can nest next questions and response to easily form complex conversations.

question: help
response: Do you want help with <button>web</button> or <button>mobile</button>
keywords: help

    question: web
    response: Is you issue with <button>HTML</button> or <button>JavaScript</button>?

        question: html
        response: See https://www.w3schools.com/html

        question: javascript
        response: See https://www.w3schools.com/js

    question: mobile
    response: Are you using <button>Android</button> or <button>iOS</button>?

        question: android
        response: See https://www.android.com

        question: ios
        response: See https://developer.apple.com/ios

    question: quit
    response: Okay. Let me know if you need further help.

    default: Please specify <button>web</button> or <button>mobile</button>, or type <button>quit</button> if you do not need help.

The next question are shown in the response browser as a set of nested tables. You can edit or delete next questions and response from the browser.

Importing

You can import shared, or upload response lists, chat logs, or AIML sets into your bot. Your bot will learn all of the imported responses. This is a good way to kick-start your bot. Be careful of importing too much, or importing responses that are not relevant to your domain, as this will make your bot less able to give responses relevant to its domain.

Response Lists

A response list is a list of questions and responses. Each question and its responses must be separated by a new line. When you import a response list, the bot will learn all of the new responses.

Hello
Hi

Hey
Hello

Hi
Hi

Bye
Goodbye

Good night
Good night, have a good sleep.

You can also include keywords, required, previous, topics, and other meta data in a response list. You can also define default responses, greetings, or execute scripts. A default response is used when no other response is known, and a greeting is used as the bot's first response in a new conversation. A script can include Self code to be executed, which can add knowledge directly to your bot's object database.

greeting: Hello, how may I be of service?
greeting: Hi, how can I help?

default: Sorry, I do not understand.
default: I do not understand, please rephrase your question.

what are your services
Our services include chat bot hosting and development.
keywords: services
topic: services

error
Are you getting an error?
keywords: error exception

yes
Please email the error to support at support@botlibre.com
previous:are you getting an error

script: #self.botmaster = "Admin";

Note all of the examples in the previous section are in the response list format as well.

Chat Logs

A Chat Log is the log of one or more conversations. Each phrase must be prefixed by the speaker, and each conversation must be separated by a new line.

Example Chat Log

User: Hello
Bot: Hello
User: How are you?
Bot: I am fine, yourself?
User: I'm good.

User: Hey
Bot: Hey whats up?

This log contains two conversations. When you import the chat log, the bot will learn the responses in context, in that it will both record the responses to the questions, and the previous response. This will allow the bot to choose the best response for questions it knows multiple responses to, based on the context. For the example above if you say "Hello" the bot may answer "Hello" or "How are you?", however if its previous response was "Hello" then it will answer "How are you?" because of the context.

You can also add emotions to responses in a chat log. This will tag the word or phrase with the emotion. Emotions will affect the bot's mood and avatar.
The available emotions are:

• love, like, dislike, hate
• serene, calm, anger, rage
• ecstatic, happy, sad, crying
• courageous, confident, afraid, panic
• surprise, bored
• laughter, serious

Example Chat Log with emotions

User: Do you like me?
Bot:<love> I love you.

User:<anger> Your are stupid.
Bot: That is not a nice thing to say.

Bot:<laughter> lol

There is an import option to Process learning and comprehension. Instead of just importing the responses, this will have the bot listen to the conversation and learn from it. If you have comprehension enabled in your bot, it will process each response through comprehension.

Comprehension will analyze the questions and responses and try to generalize the responses into scripted formula responses. For example is may take the response "Bot: My name is Bot" to be generalized to Formula:"My name is {get #name from :target}". Or could learn how to count from the response "Bot: 2" to "User: 1".

It will also lookup each word for its meaning from Wiktionary, and process questions and answer through its state machines. For example the question "User: What is the Earth" will have the bot lookup in Freebase all information related to the planet Earth, and import the information into its brain. Comprehension will take significantly longer to import the chat log, so only use this for small chat log files that you want the bot to learn from.

AIML

AIML stands for the Artificial Intelligence Markup Language. It is an XML standard for defining chat bot responses. You can import any AIML file as a chat log. The bot will learn all of the patterns and templates as questions and responses. It will convert any AIML tags in the template to Template responses using Self code, and use wildcard Patterns for any AIML patterns.

AIML is normally executed as a script, where each pattern either matches or does not match a question. When importing AIML as a chat log, it will not be executed as a script. Each AIML pattern will be matched based on how well it matches the question. For example the pattern "what is meaning of life" could match the question "what's the purpose of life", if it were the best match. Standard AIML would not perform such as match, as only exact or wildcard matches are allowed.

The chat logs method of importing AIML allows the AIML to be integrated with the bot's responses, knowledgebase, and learning. Importing AIML as a chat log is the best way to import large AIML files. You can also import AIML from the Scripts page, if you want the AIML to be executed precisely.

<aiml>
  <category>
    <pattern>*</pattern>
    <template>Sorry, I do not understand.</template>
  </category>
  <category>
    <pattern>*</pattern>
    <template>I do not understand, please rephrase your question.</template>
  </category>
  <category>
    <pattern>what are your services</pattern>
    <template>Our services include chat bot hosting and development.</template>
    <topic>service</topic>
  </category>
  <category>
    <pattern>^ error ^</pattern>
    <template>Are you getting an error?</template>
  </category>
  <category>
    <pattern>yes</pattern>
    <template>Please email the error to support at support@botlibre.com</template>
    <that>are you getting an error</that>
  </category>
</aiml>

Summary

You now should better understand how your bot responses to questions, and how to best train your bot for your domain using Training & Chat Logs, Responses, Keywords, Required, Previous, Topics, etc. If you are interested in learning scripting, you can program more advanced responses using Self or AIML from the Scripts page.