Beware the lure of crowdsourced data

sasha uritsky

Jun 18, 2014

Crowdsourced data can often be inconsistent, messy or downright wrong

We all like something for nothing, that’s why open source software is so popular. (It’s also why the Pirate Bay exists). But sometimes things that seem too good to be true are just that.

Repustate is in the text analytics game which means we needs lots and lots of data to model certain characteristics of written text. We need common words, grammar constructs, human-annotated corpora of text etc. to make our various language models work as quickly and as well as they do.

We recently embarked on the next phase of our text analytics adventure: semantic analysis. Semantic analysis the process of taking arbitrary text and assigning meaning to the individual, relevant components. For example, being able to identify “apple” as a fruit in the sentence “I went apple picking yesterday” but to identify “Apple’ the company when saying “I can’t wait for the new Apple product announcement” (note: even though I used title case for the latter example, casing should not matter)

Interested in reading more? Check out our other blogs:

How nmodes technology is unique

nmodes AI is based on semantic algorithms. They require significantly less computational capacity compared to standard machine learning algorithms used by a majority of conversational AI systems today.

As a result, the infrastructure requirements are drastically reduced. In simple terms, what Google Home or Amazon Alexa do with the help of supercomputers or advanced computer farms, nmodes AI can do on a basic server.

And it gives nmodes ability to delegate conversational capacity to the users. With the help of nmodes AI every business (and individual) can create their own AI to handle the details of the business (products, customers, etc) not accessible from the outside.

WHAT IS CONVERSATIONAL AInmodes.com

nmodes Technology - Overview

nmodes ability to accurately deliver relevant messages and conversations to businesses is based on its ability to understand these messages and conversations. Once a system understands a sentence or text, it can easily perform a necessary action, i.e. bring a sentence about buying a car to the car dealership, or a complaint about purchased furniture to the customer service department of the furniture company.

Understanding sentences is called semantics. nmodes has developed a strong semantic technology that stand out in a number of ways.

Here is how nmodes technology is different:

1. Low computational power. We don’t use methods and algorithms deployed by almost everyone else in this space. The algorithms we are using allow us to achieve high level of accuracy while significantly reducing the computational power. Most accurate semantic systems, e.g. Google’s, or IBM’s, rely on supercomputers. By comparison our computational requirements are modest to the extreme, yet we successfully compete with these powerhouses in terms accuracy and quality of results.

2. Private data sources. We work extensively with Twitter and other social networks, yet at the same time we process enterprise data. Working with private data sources means system should know details specific only to this particular data source. For example, when if a system handles web self-service solution for online electronics store it learns the names, prices, and other details of all products available at this store.

3. User driven solution. Our system learns from user’s input. Which makes it extremely flexible and as granular as needed. It supports both generic topics, for example car purchasing, and conversations concentrating on specific type of car, or a model.