Smartly.AI

Intent Detection

In Smartly.AI, the task of predicting which of intent corresponds best to a user requests is made possible using a statistical classification algorithm, also called the Classifier.

What is Statistical Classification?

Statistical Classification is the task of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known.

Classification has many applications such as computer vision, medical image analysis, speech analysis, handwriting recognition and of course natural language processing.

In the example below, the classifier gets many various user requests as inputs, and we can see that the input is either a match with an existing intent or no intent at all.

A simple four classes classifier

A simple four classes classifier

How it works?

Each sentence is seen as a vector with an indication of how often distinct words of your training data appear in a message. In the illustration below you can see how the count vectors would differ for the sentences My bot is the best bot and My bot is great, e.g. bot appears twice in My bot is the best bot.

Count vector featurizer (Credits Rasa)

Count vector featurizer (Credits Rasa)

Our Classifier needs more training data

As this classifier trains word embeddings from scratch, it needs a little more training data and more training time but will generally perform better than pretrained models.

Training vs Inference

During the training phase, the classifier learns separate embeddings for feature and intent vectors.
During the inference phase, the classifier measure the vector distance between the embedded features and the embedded intent labels using cosine similarity to define which is intent is the best candidate.

Cosine similarity between trained intents and a user request

Cosine similarity between trained intents and a user request

Below is a simplified representation of a word vector space (it should have much more dimensions in reality). Anyway, what you can see is that by measuring the distance between vectors we can see how similar a user request is from the pre-trained intents, and by doing so, transforming a semantic problem to a mathematical one.

Confidence Threshold

With each inference, the classifier will give you score that is a good indication of the probability of confidence of the result. This score ranges between 0% and 100%.
In order to let you filter the most certain inferences, our platform allows you to define a confidence threshold.

What happens if the threshold is too low?

If the threshold is too low, you will almost always have a result, but these result can be false. This kind of results is called False positives.

What happens if the threshold is too high?

If the threshold is too high, you will have less results, but they will be almost always true. This kind of results is called False negatives.

Setting the right confidence threshold

By default, all new intents come with a confidence threshold of 70% which is a quite good compromise in our experience and a good value to start with. Once you have more data and better results, you can put the bar higher between 80 and 90%.

For more tips on how to design your intents, please check out the section below.


Intent Detection


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.