Datameer Blog post
My Big (Fat) Data Governance Rant
by Datameer on Feb 23, 2018
I recently came across a blog post from Cindi Howson, a Research Vice President focused on BI and analytics at Gartner, “Using the BI and Analytics MQ to Modernize and Select the Right Tool.” In it, she shared some insights and viewer questions from her recent webinar on the same topic. It’s a post worth reading on its own merit. However, one of those viewer questions struck a chord with me:
Do you think that the governance capabilities in these BI tools are powerful enough for a large enterprise?
It’s been a few weeks since my last rant, so buckle up.
How BI Relates to Big Data
A pet peeve I’ve had about the big data industry is how it has both mirrored, and yet rebelled against, the products and technologies offered by the Business Intelligence (BI) industry over the last 20 years.
Where big data has mirrored BI, of course, is in its central premise: moving beyond mere capture and maintenance of data, to analyzing all of the data together. Because analysis of the data enables finding patterns in it and revealing information that can help managers, directors and executives make better decisions.
Think the expression “getting insights from your data” is new? It may be. But “turning data into information” is nearly two decades old. What about the concept of data-driven decisions? It’s a relatively new term; but in the early days, BI was called “decision support.” Data visualizations in today’s fancy dashboards were there before too. But we simply called them “charts” and they were part of executive information systems (EIS).
So are BI and big data two peas in a pod? Maybe. But then why have all of the enterprise niceties in BI seemed so unimportant, to so many big data vendors, for so long? Enterprise BI got beyond mere “slicing and dicing” many years ago, and it moved on to adding Corporate Performance Measurement (CPM), planning and consolidation, Master Data Management (MDM), Data Quality (DQ), audit, lineage and impact analysis quite some time ago.
You can almost imagine the personification of a BI vendor talking to a big data vendor as if it were a father speaking to his son: “The folks at the meetups may like your heat maps today, son, but the enterprise customers won’t buy you without lineage tools.” And you can imagine the response: “Aw dad, no one at the demos cares, and the VCs don’t care either. They say I’m doing great.”
It’s Time For Big Data Governance Functionality to Grow up
Our personified big data vendor has a point. Master Data Management doesn’t demo well. Lineage and impact analysis isn’t sexy. Role-based access controls may be a bit of a snooze. Data Quality can be tedious. And audit is an operational feature that few users ever touch.
Perhaps that’s why these features, taken collectively as “data governance,” have been stubbornly avoided by many big data vendors for quite some time. Be that as it may, these vendors now need to show real revenue and can’t just exist as machines built to burn through venture capital money. And lo and behold, they’re starting to pay attention to data governance. The question is whether merely paying attention is good enough.
Data Governance is Big Data’s Law and Order
Why is big data governance important? Because data isn’t just “stuff” that you make visualizations from. Data is a series of point-in-time recorded facts about your business, your non-profit organization, the disease that you’re researching, the medication you’re testing or the vehicle that you’re managing and protecting. And those facts have to be accurate, they have to protected, they have to be understood and whoever has looked at, or updated, them has to be identifiable.
While demoing big data governance may not make for a big spectacle, showing what happenswithout it might be very compelling indeed. We’re talking:
- invading customers’ privacy
- marketing campaigns failing
- stores having excess inventory of unpopular items while the popular ones run out
- time being wasted
- employees getting dismissed or laid off
- analytics projects showing overall poor ROI.
No laughing matters listed there. And there are countless other ramifications I could list.
Ask the Analysts About Data Governance
The fact is, BI and big data are converging. Gartner’s BI and Analytics Critical Capabilities and Magic Quadrant (MQ) reports are changing too, in order to reflect this sea change in the market. In that same blog post I referenced earlier, Howson explains that in this year’s MQ, Gartner “considered only those vendors and products who meet the definition of modern.”
So back to that viewer question, then:
Do you think that the governance capabilities in these [modern] BI tools are powerful enough for a large enterprise?
Cindi replied quite diplomatically, “Governance and metadata integration, as well as security and user administration are two of the capabilities that [Gartner evaluates] in the Critical Capabilities for BI and Analytic Platforms. Some of the products get high marks and others do not, showing room for improvement.”
BI’s Room For Improvement, Indeed
Traditional BI tools have had these facilities for many years, so there’s little allowance for modern BI tools not to have them. Fine-grained access, lineage and audit capabilities, accessible through user interfaces (UIs) and application programming interfaces (APIs) are must-haves, and will ultimately separate the eternal startups from the profitable, established players.
That’s a key insight for vendors in this space, and therefore, for you potential buyers and consumers of modern BI tools, too. It’s time to ask tough questions about big data governance capabilities in the tools you’re considering. It’s a critical part of your due diligence that can no longer be skipped.