AI unmasked: Why long-term success of your business depends on conversational AI

Crowdsourced data can often be inconsistent, messy or downright wrong 

We all like something for nothing, that’s why open source software is so popular. (It’s also why the Pirate  Bay exists). But sometimes things that seem too good to be true are just that. 

Repustate is in the text analytics game which means we needs lots and lots of data to model certain  characteristics of written text. We need common words, grammar constructs, human-annotated corpora  of text etc. to make our various language models work as quickly and as well as they do. 

We recently embarked on the next phase of our text analytics adventure: semantic analysis. Semantic  analysis the process of taking arbitrary text and assigning meaning to the individual, relevant components.  For example, being able to identify “apple” as a fruit in the sentence “I went apple picking yesterday” but to  identify “Apple’ the company when saying “I can’t wait for the new Apple product announcement” (note:  even though I used title case for the latter example, casing should not matter)

AI training is a critical part of conversational AI solutions, a part that makes AI software different from any kind of software previously created.
AI training is not coding.
Unlike all other existing software which is fully coded.

Let us consider a simple example:
We create chatbots for two companies, one company is selling shoes, another is selling cars. From the software standpoint it is one chatbot solution running as an online service accessed remotely or a program available locally. In both cases they are two identical instances of the same software (one instance for the shoes company, another for the cars company).
Yet, for the first company the chatbot is supposed to talk about flip-flops, summer shoes, high heels and so on. For the second company, however, the chatbot is not expected to know any of that. Instead, the chatbot should be able to support conversations about car brands, car models, should know how to tell Toyota Camry from Toyota Corolla, etc. This shoes and cars knowledge is not programmable. It is trainable. It is not coded, instead it is a part of language processing capability that AI solutions like chatbots have. And herein lies the major differentiation and advantage of the AI solutions compared to traditional software.

How to train AI?
There are several ways to do it. Sometimes AI system can train itself, improve its linguistic ability over time. It also can be trained by professional linguists. And in some cases, by the users. The latter is the desirable scenario because businesses know better than anybody else what they want their chatbot to talk about.
It is not easy, given the existing state of AI technology, and usually requires a high level of technical knowledge. You may have heard mentions of intents and entities in chatbot discussions. These are examples of linguistic elements AI training is currently based on.
Without proper understanding of what these linguistic elements are and how language acquisition process works in existing AI systems it is better to leave AI training to professional linguists.