Abundance of Information Often is a Liability

There is a huge amount of information out there. And it is growing. To make it efficient and increase our competitive advantage we need to evolve and start using information in a smart way, by concentrating on data that drives business value because it is accurate, actionable, and agile. Accuracy is an important measure that determines the quality of data processing solutions.

How accuracy is calculated?

It is easy to do with structured data, because the requirements are formalizable. It is less obvious with unstructured data, e.g. a stream of social feeds, or any data set that involves natural language. Indeed, the sentences of natural language are subject to multiple interpretations, and therefore allow a degree of subjectivity. For example, should a sentence ‘I haven’t been on a sea cruise for a long time’ be qualified for a data set of people interested in going on a cruise? Both answers, yes and no, seem valid.

In these cases an argument was put forward endorsing a consensus approach which polls data providers is the best way to judge data accuracy. This approach essentially claims that attributes with the highest consensus across data providers is the most accurate.

At nmodes we deal with unstructured data all the time because we process natural language messages, primarily from social networks. We do not favor this simplistic approach, as it is considered biased, inviting people to make assumptions based on what they already believe to be true, and making no distinction between precision and accuracy. Obviously the difference is that precision measures what you got right, and accuracy measures both what you got right and what you got wrong. Accuracy is a more inclusive and therefore more valuable characteristic.

Our approach is

a) to validate data against third party independent sources (typically of academic origin) that contain trusted sets and reliable demography. Validating nmodes data against third party sources allows us to verify that our data achieves the greatest possible balance of scale and accuracy.

b) to enrich upon the existing test sets by purposefully including examples ambiguous in meaning and intent, and providing additional levels of categorization to cover these examples.

Accuracy is becoming important when businesses move from rudimentary data use, typical of the first Big Data years, to a more measured and careful approach of today. Understanding how it is calculated and the value it brings helps in achieving long-term sustainability and success.

Right now we're in a world that sees  transparency as the new form of integrity. Right now we're in a world that understands that reputation is everything. Loyalty is somewhat fleeting as consumers, armoured with this incessant flow of knowledge from the web, have the ability to make swift  judgements and decisions about individuals, companies and governments, often times to the detriment of the target.

The emergence of social media has forced companies to stop hiding from behind that veil of corporate spin and address the very things that the web has thrown at them. Nothing is secret any longer. Even secrets that were once held secure behind invulnerable fortresses now have a strong probability of materializing today.

Is transparency as a norm working? Or, are the results of transparency surfacing a new order that will create yet another tier of acceptance from the masses?

"Anonymity is Authenticity"

Following the death of Rahteah Parsons, who, after being assaulted by 4 boys, was tormented relentlessly by classmates and other kids on social networks; and also following the suicide of Hannah Smith, who experienced the same torment, it's clear the internet has evolved to an era that has given free reign to voice an opinion and use like-minded affiliations to express and further spread that opinion. In these cases, anonymous profiles proliferated the incessant stream of hateful attacks that eventually wore down both girls' defences.

And while I originally argue that anonymity was a cowardice state that allowed people to be and feel comfortable being the anti-self that runs away from accountability, my stance has seen another side of this coin.

Anonymity is Safe

It becomes clear that humans, while inherently social, are discriminating of the things we disclose and to those to whom we share. 

If transparency breeds contempt, then anonymity should build acceptance

The freedom to express opinion and judgement without feeling guarded, or without fearing others linking you to a statement is indeed liberating. And while this free reign may take the form of a soapbox soliloquy or criticisms (and perhaps bullying attacks) against opposing views, there is a large segment of users who want the ability to share a secret, or have a place to vent their frustrations or challenges -- without the fear of reprisal.

Despite revelations from Snowden and the NSA that nothing on the net is private, this does not stop the wave of user adoption for applications like SnapChat, Whisper or Secret.

Here are some recent stats for Snapchat from Mashable

;

I've recently downloaded Whisper and my experience has been more than liberating. It has allowed me an outlet to record my hopes, desires and more importantly, my anger and not-for-public emotions. Being judged in real life or on social takes its toll. If my reputation precedes me, then I will be discriminating about what I say in places where my content and identity are linked.

Popular opinion just doesn't matter. It's irrelevant. But I want to track progress in my life: my emotions, my dark moments, my personal observations, my milestones -- all in my own digital diary.

Why shouldn't users have the option to keep part of their identities secret and separate?

It's up to the next generation

This new medium has created is an endless volatile loop of positive and negative reinforcement. While transparency has extreme benefits, there are just as many negative consequences that have come as a result of creating this honesty within social channels. Society continues to send the wrong message to Millennials and GenZers, warning them to be more discerning and to suppress who they really are as individuals... warning them of the potential consequences should they venture down the wrong path.

How we communicate today poses tremendous issues for this younger generation. Their experiences are grounded in the fear of being vulnerable... fear of being misjudged... fear of not being accepted... fear of being punished. When the next generation grows up, it'll be up to them to shape the landscape and determine how to balance the impacts of transparency and anonymity.