Subscribe to our Newsletter | To Post On IoT Central, Click here


Are you drowning in Data Lake?

Today more than even, every business is focusing on collecting the data and applying analytics to be competitive. Big Data Analytics has passed the hype stage and has become the essential part of business plans.

Data Lake is the latest buzzword for dumping every element of data you can find internally or externally. If you Google the term data lake, you will get more than 14 million results. With entry of  Hadoop, everyone wants to dump their siloes of data warehouses, data marts and create data lake.
The idea behind a data lake is to have one central platform to store and analyze every kind of data relevant to the enterprise. With the  digital transformation, the data generated every day has multiplied by several times and business are collecting this  consumer data, Internet of Things data and other data for further analysis. 
As the storage has become cheaper, more data is being stored in its raw format in the hopes of finding nuggets of information but eventually it becomes difficult. It is like using your  smartphone to click photographs left, right and center, but when you want to show some specific photograph to someone it’s very difficult.
Data Lakes, if not maintained properly, have the potential to grow aimlessly consuming all the budget. Some companies have their data lakes overflowing on premise systems into the  cloud.
Most data lakes lack governance, lack the tools and skills to handle large volumes of disparate data, and many lack a compelling business case. But, this water (the data) from your data lake has to be crystal clear and drinkable, else it will become a swamp.
Before getting into bandwagon of creating the data lake that may cost thousands of dollars and months to implement, you should start asking these questions.
·        What data we want to store in Data Lake?
·        How much data to be stored?
·        How will we access this massive amounts of data and get value from it easily?
Here are some guidelines to avoid drowning into data lakes.
·        First and foremost - create one or more business use cases that lay out exactly what will be done with the data that gets collected. With that exercise you will avoid dumping data, which is meaningless.
·        Determine the Returns you want to get out of Data Lake. Developing a data lake is not a casual thing. You need good business benefits coming out of it.
·        Make sure your overall big data and  analytics initiatives are designed to exploit the data lake fully & help achieve business goals
·        Instead of getting into vendor traps and their buzzwords, focus on your needs, and determine the best way to get there.
·        Deliver the data to wide audience to check and revert with feedback while creating value
There are many cloud vendors to help you out building data lakes – Microsoft Azure, Amazon S3 etc.
By making data available to  Data Scientists & anyone who needs it, for as long as they need it, data lakes are a powerful lever for innovation and disruption across industries.
E-mail me when people leave their comments –

Sandeep Raut - Digital Transformation, Data Science, BigData Analytics IoT Evangelist

You need to be a member of IoT Central to add comments!

Join IoT Central

Comments

  • Sandeep,

    Very wise advice: I absolutely agree that the specification, design and implementation of IoT infrastructure such as a "Data Lake" must be business driven. My takeaway quote from your article is: "Determine the Returns you want to get out of [your] Data Lake." In the end, the deliverable of an IoT project must be value measurable in $$$, and the system designed must be necessary and sufficient to support that value. Thank you.

This reply was deleted.

Upcoming IoT Events

More IoT News

How wireless charging works

Wireless charging technology has been around for over 100 years, but it has only recently found mainstream practical use for powering electronic devices like smartphones. Learn how this technology works and what advancements we may see in the future.

How wearables can improve healthcare | TECH(talk)

Wearable tech can help users track their fitness goals, but these devices can also give wearers ownership of their electronic health records. TECH(talk)'s Juliet Beauchamp and Computerworld's Lucas Mearian take a look at how wearable health tech can… Continue

IoT Career Opportunities