Unlike the metrics that are obtained In other channels, when we talk about a chatbot we have to take into account those users who use it simply for being a “new” channel and those who use it simply to see what will respond to the “nonsense” I tell them. Yes, in the metrics of your chatbot the importance of spontaneous or haters (which is what we have decided to call them) is total. But what do we mean by this terminology? We will tell you in the next lines.
When we talk about a web page or a mobile app we are aware that there are users who are going to see its design, its usability and the functions it has in the environment. However, these types of practices do not directly influence your metrics and KPIs (such as number of users or page hot spots), they simply remain in residual experiences that do not directly affect the final results of the channel . However, when we talk about a chatbot, things change. The absurd inputs or that go 100% of the scope of your chatbot affect the data that you can offer from it. From how many users have used it to what is the percentage of effectiveness of the bot the proactive messages of these users directly affect the results of your assistant.
Although we do not know why this type of practice, we think that the novelty of the channel and, above all, how interesting it is to see what “the machine” is answering invites users to play with the conversational assistant, testing its capabilities. Therefore, how can you prevent this data from influencing your metrics? What do you have to pay attention to? Keep reading.
What types of spontaneous profiles do we find?
Yes, do not fool us, if working within this sector we are sure that in some occasion you have tried to find the “ bugs ” of the chatbots that appear in the market (whether they are your competition or not). This type of tests triggers that you suddenly receive a series of input but if you detect this type of practice in your chatbot, remember not to count the final data.
Those who get bored and those who insult
Without a doubt, it is about of the majority group. Throughout our experience we have seen how chatbots become the “ punching ball ” of many users. Insults, bad-sounding words, inconsistent conversations, or repeated use of misspelled words are some of the most common cases. In the case of insults (which are more predictable), it is possible to create an intention that allows you to answer something that you have already set out for these cases or, directly, to program that in those cases nothing is answered.
If it is a chatbot in a messaging app that enables groups, you will find users who decide to include your chatbot as another contact of the group . This causes your chabtot to start receiving receiving hundreds of messages that your chatbot is not going to know how to answer and, consequently, they will end up in your message tray not understood, completely deconfiguring the final metric.
How do they affect the metrics of my chatbot?
As you already know, a chatbot is not intended to be a virtual assistant. Its objective is NOT to answer all kinds of queries as if it could be from Alexa or Google Assistant. In fact, on many occasions, a chatbot launched by a company is not even raised to answer ALL the queries and procedures that are carried out within it. They are usually focused on a specific objective, whether for business, ATC or specific campaigns. What does this mean? Very simply, we don’t have to answer questions about the weather, current news or all the questions.
For example, a health agency launches a bot to attend only to questions related to COVID-19. If a user opens a conversation and begins to ask for his electronic card, the copayment of the medicines or his health folder, the chatbot would NOT have to answer him. Understood?
Therefore, it is important that when doing our analyzes and metrics in which we evaluate the effectiveness of the chatbot we eliminate all these inputs. Taking this into account, it is essential to c have a platform that allows you not to count as valid messages all those inputs that your chatbot has not understood due to the simple fact that these are outside (or very strong) of scope for which it has been designed. Only in this way will you be able to know what is the real percentage (%) of inputs correctly understood based on the target of your chatbot and, consequently, know if you have met your KPI’s.