When Big Data is not so big anymore

sasha uritsky

Oct 11, 2014

We are inundated with information. There is so much information around us they coined a special term - Big Data. To emphasize the sheer size of it.

It is, of course, a problem - to deal with a large amount of data. Various solutions have been created to address it efficiently.

At nmodes we developed a semantic technology that accurately filters relevant conversations. We applied it to social networks, particularly Twitter. Twitter is a poster child of Big Data. They have 500 million conversations every day. A staggering number. And yet, we found that for many topics, when they are narrowed down and accurately filtered, there are not that many relevant conversations after all.

No more than 5 people are looking for CRM solutions on an average day on Twitter. Even less - two per day on average - are asking for new web hosting providers explicitly, although many more are complaining about their existing providers (which might or might not suggest they are ready to switch or looking for a new option).

We often have businesses coming to us asking to find relevant conversations and expecting a large number of results. This is what Big Data is supposed to deliver, they assume. Such expectation is likely a product of our ‘keyword search dependency’. Indeed, when we run a keyword search on Twitter, or search engines, or anywhere we get a long list of results. The fact that most of them (up to 98% in many cases) are irrelevant is often lost in the visual illusion of having this long, seemingly endless, list in front of our eyes.

With the quality solutions that accurately deliver only relevant results we experience, for the first time, a situation when there are no longer big lists of random results. Only several relevant ones.

This is so much more efficient. It saves time, increases productivity, clarifies the picture, and makes Big Data manageable.

Time for businesses to embrace the new approach.

Interested in reading more? Check out our other blogs:

The End of Digital Monitoring Paradigm

Digital industry is changing rapidly.

For the last decade analysis of social chatter and capture of consumer sentiment was considered the cutting edge of the marketing strategy. In these early days of the new era of digital information businesses were told to listen to what market is saying about them. They were educated on the importance of media monitoring and the advantages it creates for strategic growth.

This picture has become outdated.

Listening to Big Data, in all its aspects and forms, is no longer enough. After you successfully listened and understood what customer said the next natural step would be to act, or respond. And so the digital domain is now spreading to include responses, with a host of innovative technological solutions reshaping the field rapidly. Advances in artificial intelligence in particular create disruptive scalable opportunities in the space traditionally known for its slow manual progression.

Facebook was among the firstto enter the market, introducing bots into the process of connecting users with brands. Then there was Microsoft's turn.

Following these developments bots became the hottest trend in Silicon Valley in 2016.

nmodes fits seamlessly into this new world order. We deliver AI solutions that power business sales process. Our listening solution accurately monitors and captures real-time needs and interests of individual customers within the defined audience. And our Intelligent Assistant solution brings scalability to responses without compromising on quality.

Building Facebook Messenger chatbot: what they forgot to tell you.

There are lots of written tutorials and online videos on this subject.

Yet many of them omit important details of the bot building process. These details may vary from one user to another and are difficult to describe in a unilateral fashion. Consequently it is easier for tutorial writers not to mention them at all. We try here to fill the gap and provide some additional clarity.

1. Creating Facebook app.

One of the first steps in building a Facebook Messenger bot is creating a Facebook App. It requires a business Facebook page. This might seem obvious to avid social users yet worth mentioning: a business Facebook page can only be created from a personal Facebook page. If you already have a business Facebook page move on to the next step. If you have a personal Facebook page go on and create a business page. If you are among the lucky ones that live without Facebook presence now is your chance to become like everybody else.

2. Getting SSL certificate.

Next you need to setup a webhook. Your web application is hosted on a web server and the webhook’s role is to establish connection between Facebook and your web application via your web server. In order for the webhook to work you need SSL certificate because Facebook supports only secure connections (HTTPS) to external web servers. So first, you need to purchase it. The costs change from one company to another but it is important to buy a reliable certificate otherwise Facebook might reject it. All major ISP companies offer SSL products. Second, you need to install it on your web server. The installation process can be tricky. Sometimes you can get technical help from the ISP company that sold you the certificate (as a rule of thumb, the bigger the brand the better their technical support is supposed to be. But the cost may be higher too). You can also rely on popular tools, such as keytool command utility, assuming you know how to use them. In any case, it might be a good idea to allocate several days, up to a week, for this step when planning your project.

3. Choosing the server environment.

Your options are (almost) unlimited. Many online tutorials use Heroku which is a cloud-based web application platform, but a simple Tomcat web server would suffice too. Your decisions should be based on your business requirements. A lightweight server such as Tomcat is a good fit when it comes to web centric, user facing applications. If backend integration comes into play, a web application server should be considered.

Your choice of programming languages is also broad. PHP is one popular option, Java is another but the list by no means ends here. Your chatbot app communicates with Facebook using POST requests, so any language that supports web protocols will work. Again, make decisions having your business goals in mind.