Subscribe to our Newsletter | To Post On IoT Central, Click here

Transform Big data to Smart Data

We are entering into the age of IOT, lot of connected devices will talk to each other using sensors/signals and it is expected that those devices will generate an enormous amount of data.

Handling that much of data [Big data] and generating actionable signals from that would be a challenge.

Companies are investing a lot of resources to build the platforms that can process the big data and retrieve some intelligence out of it. I am wondering that IOT will generate limitless data. Would these platforms will still be able to handle that situation and would it still remain cost effective to process the data of thousands of devices and then retrieving the signals that could be understand able  to other devices.

For example.

Today SkyServer had data on 200 million galaxies in the database. The SDSS data exceeds 150 terabytes, covering more than 220 million galaxies and 260 million stars. The images alone include 2.5 trillion pixels of original raw data.  

As the accuracy of devices will improve with the passage of time, their discovery capabilities will also improve resulting into more petabytes of data. Would we continue adding Servers, RAM, Hard drives and processors to manage all this? 

Why we cannot invest all of these resources in making the devices intelligent? Every device do necessary intelligence at its own end its output interface expose only the actionable data to the outside world?

For example the sky telescope can compare all the images itself and only send the images to sky server which has some new insights. Some kind of machine learning built-in to sky telescope which could help it to learn it from the past and generate actionable insights based on that…in simple words, fit a human brain inside telescope that will transform big data to smart data. Storing, managing and processing garbage [Big data] at SkyServer do not make any sense to me. Why we should bear the data transfer cost of garbage? May be hardware device keep it in its unconsciousness and can use it if needed to generate the smart data. 

In the world of IOT if output interface of all devices could expose smart data, it will tremendously decrease the data size that needs to be traveled on the IOT wire and software world would be more comfortable with handling those megabytes of data. 

We also need to rethink the way we use the internet today, the data being generated from the internet is tremendously increasing with the passage of time. Software and hardware technology cannot handle the situation at all in future.

May be we think of a new language for the internet which can help to convey the message in a few signals instead of writing a story of 1000 words. If you see conversations between deaf people, they use few signals and tell a full a story in few actions. Consider internet users as deaf people and think of a language which could tell a story in few signals.

We think of a platform that could help to reduce the size of the data that we are generating today on the internet. 

Transform the world towards the smart data instead of wasting money to handle the garbage [Big Data]

Originally posted on Data Science Central

Follow us @IoTCtrl | Join our Community

E-mail me when people leave their comments –

You need to be a member of IoT Central to add comments!

Join IoT Central

Charter Sponsors

Upcoming IoT Events

More IoT News

Arcadia makes supporting clean energy easier

Nowadays, it’s easier than ever to power your home with clean energy, and yet, many Americans don’t know how to make the switch. Luckily, you don’t have to install expensive solar panels or switch utility companies…


4 industries to watch for AI disruption

Consumer-centric applications for artificial intelligence (AI) and automation are helping to stamp out the public perception that these technologies will only benefit businesses and negatively impact jobs and hiring. The conversation from human…


Answering your Huawei ban questions

A lot has happened since we uploaded our most recent video about the Huawei ban last month. Another reprieve has been issued, licenses have been granted and the FCC has officially barred Huawei equipment from U.S. networks. Our viewers had some… Continue

IoT Career Opportunities