A comment last week by Frank Seldin on the Wall Street Journal article, Oracle’s Little Issue with Big Data by Rolfe Winkler got me thinking.
“There are 4 fundamentally different problems in the world of “Big Data”. First is collecting and storing data. The second is finding data when you need it. The third is turning that data into useful information. The fourth is differentiating what does and doesn’t need to be secure, and keeping it safe…
…I don’t think anyone is even close to cracking #3. Whoever comes up with that solution will be the business software winner. As to security, all that can be done is the best than can be done.”
As my colleague Alex discussed Hadoop security (number 4) previously, I thought I would respond to 1, 2 and 3 with some examples of how our customers accomplish these things using Datameer.
Collecting and Storing Data
A department store chain wanted to better understand their customer’s needs in order to maintain and increase wallet share. The department store uses Datameer to import all types of data (transaction data, competitor pricing data, geolocation data and more) into Hadoop in the raw data format from where the data is generated. Some of this data is then cleansed and transformed for analysis and some of it is exported into existing data warehouse technology for different use cases. This department store now has access to more data, is able to make correlations between the different data sets, and is able to store current data sources for longer time periods.
Finding Data when Needed
A financial services company has millions of transactions per day. Due to the high cost of storing data, the company only kept a few months of transaction data in an MPP database and then transferred the data to tape which limited their data analysis capabilities. The cost effectiveness of Hadoop allowed the company to keep one years worth of data and is planning on expanding the cluster to include more transactions. Their challenge was the time it took to access and analyze the data writing one-off map reduce jobs for Hadoop. With Datameer, the technical and business users now can easily access and analyze the data as needed with Datameer’s spreadsheet interface.
Turning Data into Useful Information
Customers using Datameer are doing an amazing job turning data into useful information. An example is an internet security company that takes data from malware and other threats, uses ad hoc analysis and then builds solutions to solve threats to networks. The data is in several formats (structured, semi-structured and unstructured). Finding answers to questions requires ad hoc data access and exploration and Datameer grants access to all users direct access to the data.
To learn more about how people are using Datameer to solve problems, read our Big Data Analytics solution briefs on Retail, Telco, Internet Security, Financial Services and more.