By
Post from Dianna Doan, Senior Manager, Marketing Programs
The world is changing, whether we’re ready for it or not. I spent a number of years in the world of traditional BI with Actuate so I understand the BI space. But few could and would predict the changes that have taken place in the last few years.
What’s causing this change? Its really driven from the likes of Facebook, Twitter and smart phones, the rise unstructured data. This is redefining the world in which we live in. As Gen Y’s begin to take over the social blogosphere, we will see the changes multiply. Retail chains, banks, government, all are plugging into the world we know as ‘big data’. I joined Datameer recently to dive into this new big data world and the gold mine created for researchers, business analysts and marketeers wanting to make sense of it all.
As I begin to understand the intricacies of what this big data ecosystem is really all about, from Hadoop, to the Hadoop distributors, to Datameer, I realize big data has opened us up to a world of very interesting and often mind blowing analysis and insight.
For example, I started reading my friend Jonathan Chang’s thesis “Understanding, Uncovering and Predicting Links” to understand more about machine learning to prepare me for the job here at Datameer. He’s a Data Scientist at Facebook, so naturally, his thesis seemed like an interesting extension to what Datameer does, (BI platform on top of Hadoop for big data).
While reading, I stumbled upon some really cool insights that got me excited about what the future holds for the players in this space. Big data is a beast. It’s wide, it’s expansive and will intersect all of us at some point in our lives over the next few months, if not days.
Allow me to share snippets of Jonathan’s thesis. Below is a simple, but insightful pictogram of how people are related, made possible by Hadoop.
Figure 1.1 shows what a subset of an online social network might look like. The nodes in the graph represent people and the edges represent self-reported friendship between members. Even in this simple example, a rich structure emerges with some individuals belonging to tightly connected clusters while others exist on the periphery. Characterizing this structure has been one major thrust of network research (Newmanet al. 2006b).
Going beyond the ‘social networks’, and without getting too technical, companies can now see how one person is related to another by building predictive and probabilistic models leveraging software companies like Zementis and their predictive modeling plugins to Datameer to understand and predict consumer buying behavior.
I then started reading earlier posts from my colleagues here at Datameer like “Predicting the stock market with Datameer or “Fishing the Clickstream…” and my mind started racing. What other insights can companies gain? What are companies doing if they aren’t already tapping into big data? It seems that the way companies look at customer and prospect information and interaction has changed forever. Without giving away any secrets, I’d love to hear about what insights companies are now capturing in their big data analytics….
