Is big data getting too complex?
Are the skills you need to succeed in your big data project in short supply?
What do you need to succeed?
These are all questions the big data market continually asks. A recent TechRepublic article painted a daunting picture when examining them. The article cited a Gartner report that states:
“Through 2018, 70% of Hadoop deployments will fail to meet cost savings and revenue generation objectives due to skills and integration challenges.”
It also states “you need to know 10 to 30 different technologies, just to create a big data solution,” an issue Andrew Brust discussed in his blog post, “The Big Data Ecosystem Is Too Damn Big.”
The TechRepublic article does offer a ray of hope at the end, with former Wall Street analyst Peter Goldmacher declaring one major category of winners being “the Apps and Analytics vendors that abstract the complexity of working with very complicated underlying technologies into a user friendly front end.”
Through our time working with big data analytics seekers, we have found two very important success factors:
The current conversion about complexity of big data analytics boils down to a classic decision that has occurred in many other growing software sectors – the buy some, build some argument, and especially, buy versus build:.
Do I buy a big data analytic platform that offers a simpler metaphor so I can create and deploy analytics more quickly and effectively, or do I need the deep level of control of building my own analytic services directly on Hadoop?
The crux of the buy versus build argument is the tradeoff between reducing complexity to increase analyst productivity, and having the power and control needed to scale your big data analytics solution.
But what if you don’t have to make this tradeoff?
Let’s look at four ways a modern BI platform that’s native on Hadoop can reduce the complexity of your big data journey. We’ll also look at how it can give you needed power and control, while also letting you focus on delivering more value to the business.
Creating analytics directly on Hadoop requires specialized program and languages to ingest data into Hadoop, create analytics and access data.
A modern BI platform running natively on Hadoop gives you the best of both worlds — capturing the power of Hadoop while eliminating the need to find scarce Hadoop programming skills. The end-to-end capabilities of modern BI on Hadoop make it easy to piece together every step in the analytic process — integration, preparation, analysis and visualization — in an easy-to-use user interface.
The common notion about big data on Hadoop is that you’ll need to find new analysts with specialized Hadoop skills, or data scientists who know R or SparkML. If you build a raw data lake on Hadoop, this is true.
With modern BI on Hadoop you can use your existing analysts, without the need to add or replace your current staff. The analyst-friendly UI with drag-and-drop advanced analytics allows your existing team to apply their knowledge of the business and answer new questions on your big data.
Governance and security require you to find additional modules for your Hadoop stack that require specialized skills to operate. This further increases the need to find these new skills, adding to the cost of your big data deployment.
Modern BI platforms have built-in governance and security, and additional features such as lineage and encryption. These capabilities give you the governance and control you require, and are easily applied to the analytic platform through an administrative user interface requiring no specialized skills.
Big data analytics are unique in the need to operationalize processes to deliver fresh new results to businesses on a consistent basis. Analytics built using a custom data lake on Hadoop require a great deal of custom programming and scripting to operationalize, which makes for brittle, easily broken processes.
Modern BI platforms have built-in job execution, policy management, logging and monitoring that is easily managed via a graphical administrative UI. This simplifies the definition of operationalization processes that are easy to monitor and manage, reducing the deployment costs of your big data analytics.
With big data, it’s important to keep in mind the end game. An organization does not embark on a big data journey simply to store and manage large amounts data. That’s the 1990s equivalent of building a data warehouse as a place to simply put all your data in one place. We all know how badly those projects turned out.
The end game is to answer a new range of analytic questions your business teams need to improve their effectiveness, increase efficiency and mitigate risk. Scale up from descriptive historical analytics to diagnostic, predictive and prescriptive ones that deliver new actionable insights.
Modern BI platforms eliminate the complexity of big data technology, removing the dependency on acquiring new skills and leveraging existing analysts that know your business inside and out. This makes it easy to put your insights to work across the organization.
Want to learn more about how a modern BI platform reduces the complexity of your big data analytics? Download your ebook, “Build, Bridge or Buy Analytics on Hadoop.”