Orchestrators & Microbots

The problem with large monolithic bots

Imagine you have built a corporate bot with a limited scope which is working really nicely.
The model of this bot has 9 intents and some custom entities.

Then you may want another use case: that's now 21 intents in our model and many custom entities.

Then you add some small talk... More intents...

And it keeps growing.
Before you know it you have built what we call a large monolithic model!

Large models have many drawbacks compared to small ones.

• they often creates intents conflicts
• they may enter in an unstable equilibrium
• they take longer to train
• they are harder to debug
• they do not allow granular access creating friction on collaborative work
• a change to a small part of the model requires to rebuild it fully from scratch

Dealing with large monolithic bots, can quickly become a nightmare.
So we decided to find a solution.

Introducing Orchestrators (and micro-bots too!)

An orchestrator is a bot that can collaborate with other bots in order to solve a user request.

In the orchestrator bot paradigm, bots are seen as Microservices.

With our previous example, the orchestrator approach leads to the following architecture.

Notice that while the orchestrator has access to his own local intents, he can also ask other bots how they interpret the user request and therefore pick the best model.

📘

Good to know

According to many tests we ran internally, we have found that cutting a large monolithic bot into smaller bots leads to better results.

Some advantages of the orchestrator approach:

• Each microbot has his own model and logic, naturally enforces a modular structure.
• Lends itself to a continuous delivery software development process. A change to a small part of the application only requires rebuilding and redeploying only one or a small number of services.
• Adheres to principles such as fine-grained interfaces (to independently deployable services), business-driven development (e.g. domain-driven design).

📘

Orchestrator vs Masterbot

We prefer to talk about orchestrators, super bots, super-bots, superbots rather than masterbots or master bots because of the bad connotation of this appellation.
As it is really important for us, we are currently updating our documentation and any reference to masterbots in favor of the term "orchestrator".

Building your first Orchestrator

Interested in orchestrators? We have made easy for you!
Let’s assume that you have some functional bots that you want to group under the management of your orchestrator.

1. First, create prepare your microbots, and create a new bot.

📘

Common languages

Be sure that your the microbots you want to enrole are sharing a common language.
Example:

• Orchestrator (fr-fr)
• Microbot 1 (fr-fr)
• Microbot 2 (en-gb)

Microbot 1 can be enrolled with the Orchestrator but the bot 2 wont because of the language incompatibility.

2. Then open the settings of the Mainbot

3. Pick one or several microbots you want to attach to your orchestrator then hit Save.

🚧

Don't break the internet! 🙋

Because we want to avoid any sort of recursivity, orchestrators can't enrole other orchestrators.
Lets keep it simple and stable folks

Once done saved, your orchestrator, you will have a specific icon!
You can see several robot icons, depending on the number of bots selected to work with the Main.

Here is an example below:

also you can now filter your bots by suer bot using the dropdown menu below.

4. Congratulations!
   From there, all the complexity will be managed in the background.

How does it work?

Once our dialog engine detects that a bot has been promoted to a orchestrator, it will behave differently.

Suppose you have a orchestrator connected to 3 bots.
We will have 4 independently trained NLU models:

• Orchestrator
• Bot 1
• Bot 2
• Bot 3

When the orchestrator receive a user request for the first time, here is the steps he will go through:

1. Check the NLU result for the user sentence with each of the available 4 models
2. Compare the results, pick the best bot
3. Get an answer from the best bot and send it to the user
4. Remember the best bot

When the orchestrator receive additional requests from the same user, here is the steps he will go through:

1. Check the NLU result for the last best bot, if the sentence is understood, no need to check with the other bots, the current bot will keep the priority
2. If the current bot is not able to answer the question, we will check with the other bots

When dealing with a orchestrator, the API will return you the following additional information.

"current_bot": {
		"_id": "5c179336f3b66446b69b9297",
		"name": "Small Talk"
	}

Also, you can share data across your bots using the following value in your code:

conversation.shared_memory.favorite_color = 'blue';