Introduction

In Smartly.AI, the task of predicting which of intent corresponds best to a user requests is made possible using a statistical classification algorithm, also called the Classifier.

What is Statistical Classification?

Statistical Classification is the task of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known.

Classification has many applications such as computer vision, medical image analysis, speech analysis, handwriting recognition and of course natural language processing.

In the example below, the classifier gets many various user requests as inputs, and we can see that the input is either a match with an existing intent or no intent at all.

How it works?

Each sentence is seen as a vector with an indication of how often distinct words of your training data appear in a message. In the illustration below you can see how the count vectors would differ for the sentences My bot is the best bot and My bot is great, e.g. bot appears twice in My bot is the best bot.

Training vs Inference

During the training phase, the classifier learns separate embeddings for feature and intent vectors.
During the inference phase, the classifier measure the vector distance between the embedded features and the embedded intent labels using cosine similarity to define which is intent is the best candidate.

Below is a simplified representation of a word vector space (it should have much more dimensions in reality). Anyway, what you can see is that by measuring the distance between vectors we can see how similar a user request is from the pre-trained intents, and by doing so, transforming a semantic problem to a mathematical one.

Confidence Threshold

With each inference, the classifier will give you score that is a good indication of the probability of confidence of the result. This score ranges between 0% and 100%.
In order to let you filter the most certain inferences, our platform allows you to define a confidence threshold.

What happens if the threshold is too low?

If the threshold is too low, you will almost always have a result, but these result can be false. This kind of results is called False positives.

What happens if the threshold is too high?

If the confidence threshold of your bot is too high, you will have less results, but they will be almost always true. This kind of results is called False negatives.