First, lets talk about two truths. Based on 30 years of observations, I believe that relevant and competent data integration is critical in driving good analytics. Without good and relevant integration, it is impossible to get relevant insights across all your data and you’re flying half blind as a result. That has been proven historically, its true today and will be true in the future.
Second, the only constant in analytics is that data continues to increase in volume and complexity. Analytics are changing, in fact Big Data is changing everything. And while data volumes are certainly a factor, new data types and the velocity of new data sources are revealing new customer behaviors and preferences, operational intelligence, security threats, etc.
Typically, technology doesn’t drive change. Rather, it’s an enabler to address change. In the case of big data, technology is responding to the user’s need to integrate and analyze all of the new data that is being generated. Data integration has to evolve as well, both in capabilities and approaches, if it is to keep up with the need for new analytics.
Data integration tools developed historically to meet the need to integrate multiple, structured data sources so users could analyze data around transactions and products. Developed during the time of limited hardware resources, these tools forced software engineers to require pre-built models of the data so user queries would not overwhelm the available hardware.
Of course, the requirements of big data changes all of that. New types of data, rapidly evolving data sources and high volumes of data make traditional data integration obsolete. Big data discovery differs significantly from traditional BI in that it looks to iteratively reveal unknown patterns, relationships and insights across all available data rather than focus on a simple question and answer paradigm.
Big data integration must access and integrate all types of data, otherwise only partial views are possible. Today’s fast pace of business means that new data sources must be be added instantly, without having to model or remodel the data, so that analytics can be delivered quickly and iteratively. And often, the volume of today’s data quickly overwhelms traditional hardware architectures so big data must leverage inexpensive hardware resources that can be quickly and cost effectively scaled to meet data demands.
So, a new data integration ecosystem and approach is required to address big data. Leveraging the memory (and analytics) of an elephant, Datameer is leading that ecosystem with comprehensive wizard-led big data integration built on top of Hadoop. This includes dozens of connectors to enterprise data that covers all relevant structured and unstructured data sources as well as the ability to read and understand those sources so business folks don’t have to wait for scarce IT resources so sort it all out for them. Second, Datameer leverages the linear scalability of Hadoop to avoid the need to pre-model the data for analytics. Users simply ingest their relevant data into Datameer then perform unlimited analytics in a “model upon read” approach that enables users to iteratively discover insights, patterns and relationships without constraints. And if they find they need additional data sources or more data, they simply add those sources and continue on.
Datameer thrives in Fortune 100 companies and data-driven startups, often with Hadoop acting as the central data hub for all data sources. Users simply point Datameer at the data to quickly and easily perform analytics, discovery and visualizations that reveal insights across all of the data. The results can also feed additional analytics solutions or recommendation engines. Datameer’s big data integration ecosystem approach closely parallels how business is evolving and is critical to maintaining insights in today’s competitive markets and industries.
To find out more about Datameer’s Data Integration capabilities, check out this short video.