Best Practices for Creating an Optimal Data Lake
Big data can only get bigger. Data lakes help you manage data in all forms, shapes and sizes. But how do you get more value from all this data?
About the Best Practices for Creating an Optimal Data Lake Ebook
Data today is quickly growing in volume, variability, and complexity. This has left organizations with the challenge of harnessing all this data – however broad in a variety or large in volume – to derive more value and insights from it.
Traditional enterprise data warehouses have problems dealing with the complexity of the data and the flexibility required for today’s range of analytic questions. To meet these demands, data lakes were conceived.
Your data lake is quickly becoming the answer to effectively managing a large volume and variety of data. The question now is: How do you provide the flexibility, speed, and accessibility to fuel analytics that truly drives business results?
Data Lake Adoption and Role
Curated, Purposeful, Consumable.
Three Key Practices
Vital practices that help create an optimal data lake that’s architected for success.
Deliver Strong Security and Governance
Enterprise Security & Governance Integration, Encryption & Obfuscation, Full Lineage.
Facilitate Consumption from the Data Lake
The Data Lake User Personas, Drinking from Your Data Lake, Method, Pros & Cons.
TRANSITIONING TO THE DATA RESERVOIR
As enterprises build up knowledge and experience in using data lakes, they come to the realization that they’re not just forming a data lake. Instead, they are gearing up for a much more robust and agile data and analytics architecture.
In version 2.0 of the data lake, a data reservoir can effectively generate more value because the data is:
Curated: Raw datasets from disparate sources are processed into organized and readable information that can be used by various business organizations.
Purposeful: Data can be analyzed from different perspectives so as to gain insights for very specific business purposes.
Consumable: Data can be presented in a manner and structure that is easily available and much more consumable by people in the organization.
DEMOCRATIZE BY CURATING AND GOVERNING DATASETS
One of the things that set the data lake apart from the traditional data warehouse is its ability to support all data types, not just the structured data. Data does not need to become a waste in the data lake for organizations because ingestion takes place continuously. In essence, the data lake becomes the catch-all repository for your data.
Adding context adds value – In today’s data-rich environment, a lot of information comes from non-traditional data sources –devices, mobile application log files, web server logs, sensors, social media activity, and so many more. But most of this data is simply events – items that happened and were detected at a point in time.
Data governance is not single-threaded – When enterprises only had relational data stores and enterprise applications, managing individual data teams could do data. But this governance was typically stringent and single-threaded, creating bottlenecks in getting access to data.
Data curation is more than preparation – Vital role in helping data lakes deliver on their promise. And it is important to recognize that curation is much larger than simply data preparation. If applied properly, data curation utilizes a bottom-up approach to turn any raw data into information that produces useful analytics and can easily be consumed.