Now it’s 2019 those that have successful chatbots programs and proof of concepts are asking themselves:
How do you use natural language processing to upgrade your chatbot?
Basic chatbots can operate just fine without natural language processing. However, their capability is limited. NLP allows computers to understand and act on human language and have far more flexible and natural-feeling conversations. This usually means the overall capability of a chatbot increases and so does user satisfaction.
Secondly, as we briefly mention, you can build chatbots without NLP, but they’re for limited use cases (which can bring a higher return of investment!) so it’s well worth going through the differences to understand when it’s worth implementing it.
In this article we’ll cover:
- The three types of chatbots
- What is NLP
- When you DO need to use NLP/NLP
- When do you NOT need to use NLP/NLU
- Other related questions
Let’s get started.
What is Natural Language Processing?
Simply, natural language processing (NLP) is concerned with how technology can meaningfully interpret and act on human language inputs. NLP allows technology such as Amazon’s Alexa to understand what you’re saying and how to react to it. We go into NLP in much greater detail here.
Chatbots can work without NLP. But their functions are limited. NLP for the majority of use cases is a necessity. When I first got into chatbots, I grossly underestimated how many different components are needed to start making chatbots capable of even remotely fluid conversation.
NLP is a very broad subject, often in chatbots, you will hear the term natural language understanding:
Natural language understanding
Natural language understanding (NLU) is a subset of NLP.
This sub-field focuses specifically on the understanding of intents, figuring out context and ambiguity in sentences.
We cover what is NLU, why it’s so important and our favourite examples here.
We cover later the third type of chatbot: “machine learning”. Which is technically referred to as Natual language generation:
Natural language generation
We cover more specific chatbot examples of NLG chatbots, later on, for now though, the most famous example of NLG is GPT-2 created by Open AI. GPT-2 can create an entire article, just from a small sentence prompt. You can give it a go yourself here!
How does NLP break up a sentence and help the chatbot interrupt language?
Understanding language to us humans is second nature, so it’s easy to underestimate the complexity involved.
For computers, it’s one of the biggest challenges to tackle. When you send a sentence such as “What’s the weather like today?” That message goes through a process of:
- Tokenisation: Which turns a sentence into data a computer can understand.
- Parsing: which understands the underlying structure of the data we are given.
Which from there the NLP engine will try to find:
- Intents: to understand what the user wants (their intent).
- Entities: to identify categories of data within the sentence.
Again, we cover this in far more depth in our “What is NLP article” if you’d like to learn more.
The three types of chatbots:
1. Logic only
These are the chatbots that many consider the root cause of the recent chatbot hype cycle. However, without NLP, all these new bots didn’t understand any message sent and only worked when you pressed the buttons given.
In 2019 onward, these logic and button only bots still have their place. They’re cheap and fast to build.
Types of logic only chatbots
Messenger Marketing
Email is known as a great marketing method for converting early leads. Messenger marketing is touted as “the next email”. Messenger does a great job of making a more personal and interactive funnel.
Lead capturing on your website: Typeform
Many spout a great use case for chatbots as a great alternative to having long forms. And they definitely are. But it’s only worth investing all that time and effort when you plan to scale your chatbot into other areas of your business. Otherwise, there are tools such as Typeform which give a great fluid and conversation-esque experience, for a fraction of the build time and cost.
Website chatbot
It’s often not possible for many companies to invest the manhours and cost into a chatbot and that’s where we come in to help.
2. Logic & NLP hybrid
To go beyond basic buttons, you need NLP. Amazon Alexa is a popular example of using NLP technology. There are multiple tools such as Dialogflow & Watson to achieve this, but they need a lot of work, cost and integrations to make it worthwhile. Sometimes it is easier and brings a higher return of investment to keep it simple and use buttons.
Google Assistant
What’s quickly becoming the most common as chatbot NLP technology becomes more accessible to the wider market.
By using a hybrid of conversation design and logic, you get to roughly dictate how the conversation flows. However, by implementing NLP, you have some room for user error when they type poorly grammatical sentences or want to deviate from the current conversation flow into a different one.
These are currently the most successful chatbots and what we commonly build for clients.
3. Machine learning
Lastly, there are machine learning chatbots that usually need supervised training. But once trained, don’t need any further human input, nor do they need any conversation design. Most chatbots built this way are either for very specific use cases like Babylon, for academic research like GPT2 or for gimmicky purposes like Xiaoice.
In terms of Applied AI and business applications, hybrid chatbots are still very much the way to go in terms of cost and effectiveness.
When DO you need to use NLP?
Basically, any time you’re looking for your chatbot to understand language.
There are many use cases where it makes for much better user experience by allowing flexible conversation flows.
We’ll iterate, however, while the free text significantly better experiences, it also takes a lot more resources to build.
