A breakdown of chatbot architecture and how it works

July 10, 2019

Share this post

So how do they work and what does the typical chatbot architecture look like?

In a simple summary, chatbots are usually made up of a combination of platforms and software, usually, a messaging platform, a natural language processing (NLP) engine and a database.

The user writes a message, which is sent to the NLU engine, which uses machine learning and natural language processing to understand the words used in a sentence and identify the user’s intent (their need/requirement). The engine then decides which answer to send back by looking into a database full of candidate responses and picking the one that best fits the user’s intent.

This is the most basic process of one type of chatbot of which there are many, and you want to make sure that you use the right chatbot for the job.

Let’s dive a bit deeper with some example architecture diagrams of the different levels and run through each of them:

What does the most basic level of chatbot architecture look like?

A lot of businesses have demonstrated huge value using basic bots like the one we’re about to cover. These bots typically respond to short tail questions. For example, “What are your opening hours?”.

We call this a level 1 chatbot. These sort of chatbots are usually great for small businesses or as part of a marketing campaign. They typically can be built on just one platform or sometimes expand to 2 or 3 tools, but definitely not more.

For context, a short tail question is something like: “Turn on my headlights”.

While a long tail would be something like: “My exhaust is making a rattling sound, how do I troubleshoot this issue?”

Let’s take a peek inside one:

Natural Language Understanding Engine

When someone writes a message, it’s here where the “magic” happens. The message goes in, the NLU engine figures out the intent from the sentence given.

We cover more in our “What is natural lanWhen someone writes a message, it’s here where the “magic” happens. The message goes in, the NLU engine figures out the intent from the sentence given and the rules engine works out how best to respond given the intent, any entities and the context of the conversation (what was said before and what we know about the user)

Knowledge base

Once the sentence is turned into ones and zeros and the computer decides what the intent is, it pulls on its knowledge base to figure out what to respond with.

A knowledge base is a library of information about a product, service, department, or topic. The data in your knowledge base can be stored anywhere. The subjects range from the ins and outs of your HR department to an FAQ guide to your products.

The knowledge base can include FAQs, troubleshooting guides, and any other details you may want or need to know. It usually takes a bit of work to make your knowledge base usable by the chatbot.

Data storage

This is where all the conversation logs and analytics go, which in basic bots will be built in the platform. As you increase in capability, you will likely want more advanced analytics for actionable insights.

A popular example of a one-stop-shop for all of the above is Chatfuel & Manychat.

Anyone without coding experience can get started and build an effective marketing campaign or simple frequently asked questions (FAQ) automation.

Typically, these will be heavily reliant on buttons and flow in a scripted path, so they’re limited in capability. We cover more about basic bots in our 7 levels of chatbots article.

What does the most common type of architecture look like?

These are level 2 chatbots, or as we call them: semi-scripted.

You’ll find things start getting a lot more complex here. Which means the capability of the chatbot can really start to take off.

There should be some recognisable things from the last diagram:

Knowledge Store (Discovery is IBM Watson’s name for it)
Facebook Messenger (Webhook is the method by which the two platforms are connected)
Datastore (where all the conversation history and analytics is stored)

Let’s highlight some of the new key features:

(Note: Want help with figuring out your own chatbot architecture? We give 30 minutes of free consulting to talk it through. No catch. No pestering follow up calls. Just free knowledge.)

HTTP & Chat Interfaces

As you’ll notice, most websites today have a live chat widget on them. It will typically look something like this:

This is where you can talk directly to the customer support team directly from the front page. WordPress is a popular platform where websites are built on. Because of this, chatbots will need a way to play along with the website and the live chat widget.

Message broker

As more and more digital devices and software platforms come onto the market, software developers face the problem of having to exchange data via multiple different formats, programming languages and methods.

Using message brokers is simply a way to allow interoperability between different platforms. To give a simplified example of how the message broker works:

The publisher, such as the chat interface adds a message to the queue. While the subscriber, such as Filament EBM, constantly polls the queue to see if new messages were added.

As customers become more familiar with digital channels, the case is building up to become omnichannel. That way, your business is accessible to your customers favourite messaging platforms such as Messenger, Slack & Livechat. It’s here the message broker shines.

Live agent platform

Chatbots in 2019 are recommended to be integrated into your customer support team. So when the bot fails to identify the intent correctly, the human agent can seamlessly take over. Occasionally, the agent may solve the problem and have back over to the bot.

Customer Relationship Management (CRM)

In more complex use cases, you may want the chatbot to be able to recall your customer’s details from the CRM, such as to change a password or recall their bank balance.

Enterprise Bot Manager (EBM)

As you’ll see in the next diagram below, enterprise chatbots can get extremely complex as they need to integrate with so many systems to work effectively. Secondly, getting all these platforms to play along, push and pull data correctly can be tricky,

EBM acts like a central nervous system, to make managing things much simpler.

What does enterprise-level architecture look like?

If you want to take your chatbot to the next level and have contextual understanding, you’ll need to use bleeding-edge technology and techniques to enable complex conversations.

Add on top this enterprises requirement for data security and the whole system quickly becomes complex and convoluted.

Below is an example of a bot built entirely with the Microsoft Azure cloud. Building your whole chatbot on one platform has its pros and cons. It is something that will entirely depend on your individual circumstances.

So you’re not here all day. I’ll cover two important parts and leave the rest for you stare at for your own “amusement”.

Security & Governance

Security, governance and data protection should always be a high priority, even for small businesses. However, it’s particularly important to enterprises where they can have datastores on millions of peoples details. In some situations, highly confidential and personal.

It’s here where we need a layer of security governance to check that the user:

Needs to stay anonymous, then any revealing details get filtered out
Wants to access personal information, it’s done so securely.
Is trying to hack or access sensitive systems without authority, they’re stopped immediately.

Quality Assurance

Enterprises have another challenge: small mistakes can create massive ripples. For example, when your chatbot has 100,000 users month, you really don’t want all of them to see those typos or hyperlinks.

You get the idea. Any changes you make need to be tested with multiple layers and people involved. It’s why the swiss cheese model is so effective.

This one of the reasons we built EBM: because all the current AI platforms like LUIS & Watson don’t have simple ways of making amendments without deploying to a live environment.

Natural Language Generation Bots

Lastly, we have end-to-end machine learning chatbots. This is where we use machine learning to create models based on historical conversation logs that aim to generate response messages from scratch, rather than via intent detection and looking up the relevant response in a database.

These are usually single instances where a bunch of data is turned into “computer speak” (pre-processed), sorted and organised (feature engineering) and then trained to see if the answers it generates are similar to those in the real human transcriptions. These types of bots aren’t often used in companies and large scale applications yet as, frankly, they don’t perform as well vs NLU-and-flow-based chatbots like the ones shown above.

More to explore

What is natural language processing (NLP) and how it helps

July 17, 2019

How to Promote Your Chatbot

April 27, 2022