NLP Engine and Language comprehension using bot language

A conversation is relevant when the message is clearly understood by the receiver of the conversation. There are a few maxims of conversation or conversational implicature that a person should follow in order to have a complete meaningful conversation. Not following the maximums would lead to ambiguity, which results in a conversation breakdown, as the primary goal of intent communication is not met.

Eliminating Ambiguity

What is ambiguity in a conversation?

Ambiguity is when an utterance or sentence may have more than one meaning or interpretation.

It is further divided into lexical and structural ambiguity.

Lexical

Lexical is mostly due to the spelling or pronunciation of the word.

For example, these four words, right, rite, write and wright, are all pronounced as /rʌɪt/, but they are quite different from each other.

Structural or Syntactic Ambiguity

Structural ambiguity is caused due to grammatical ambiguities when a sentence can be interpreted in different ways.

For Example – old men and women can mean –

1. Old (men) and women

1. Both men and women are old

That takes us to the importance of context in text and communication.

What is context in language or communication.

Communication cannot take place without context; there are various types of contexts that need to be taken into consideration in order to have a meaningful conversation.

This includes cultural context, social context and physical context –

Cultural Context

Refers to the culture, background etc. of a social group, considering language is deeply tied to the social norms and cultures and hence language gets affected by social status, roles etc.

Social Context

Refers to the environment, time, place and relationship between the people communicating, for example – the way a speaker speaks to his boss is different from the way he speaks to his friends.

Physical Context

Physical context involves location, time of the day, setting etc. For example – a certain language used for delivering a public speech to arouse the audience may not work in an office setting.

Context in general

Take the word “gross” as an example. Gross in general English has a different meaning than gross in financial terminology. We can’t understand the exact connotation of “gross” in a sentence without the linguistic context to make clear the exact meaning of this word.

Conversational interpretation in an NLP Engine has certain differences than the way a human being perceives and interprets conversations due to the reasons stated above. So how do you get an NLP Engine to understand human language and respond to take the conversation forward?

Language being a subjective entity, there are n- ways in which a message can be conveyed. There are prescribed ways of conversing, though not everyone follows it. People believe that as long as the message is conveyed, conversation has taken place. Example –

COMBINATIONS	NORMAL SENTENCE
I want to have tea. I want to drink tea. Can I have a cup of tea? I want tea. Tea please. I need tea	I want to drink tea.

Considering there can be a million combinations or utterances of a particular sentence, it becomes very difficult to feed each and every combination into the training system. Also, if I key in a particular sentence or phrase, say for example – “I want coffee” and the user ends up typing something else which is not present in the training system, then there are chances that the NLP Engine could generate a default response instead.

In order for the Engine to respond like a human, we need a BOT ontology (domain specific) on similar lines of any human language

What is Ontology?

Ontology is naming, defining properties, relations and interrelations between entities in a particular domain. Majorly used in computer science and linguistics, domain ontology helps the engine to understand and group similar words and form relationships between them which further aids in the process of communication.
Instead of training the NLP engine for sentences or phrases, we train it for identifying parts of the speech, e.g. – nouns, verbs, adjectives etc. breaking a sentence into its constituents and then forming combinations for generating responses.

For example –
“I want to apply for a loan”
verb + verb + noun

Steps of creating domain ontology (LOAN)

Creating dictionaries/grouping synonyms together aids in the process.

We start by defining classes –
Class 1 – applying for a loan
Class 2 – information about loan

Then we move onto adding synonyms in each class. For example – “want”, “apply”, “need”, “interested” all of this words indicate towards the fact that a person wants a loan. Also, check if all the synonyms of the word fit the bill, (you may not want to use all the synonyms of the word “want”).

Map each of these classes to their respective identified intents along with sentiments. Sentiments play a major role in responses.
Example – if someone keys in “horrible service”, I want to know my loan status, automatically suggests that the customer is unhappy and needs to be catered to immediately as opposed to someone keying in “I want to know my loan status”.

The combination of class + intent + sentiment would elicit the required response from the NLP Engine.

In conclusion, teaching language to an NLP Engine is similar to teaching language to a child; we start by laying down the semantic rules and giving examples to explain those rules in common speak. The child learns by associative means of mapping the examples with the underlying ontology. Once the underlying ontology is comprehensible to a child, they soon pick up the language constructs which are built on top of these rulesets and the more data you feed it, the better the child perceives, associates, learns and remembers. Same goes for an NLP engine.