For those unfamiliar with big data technology, getting started can seem complex and intimidating.
However, with recent technological advances, setting up a user-friendly big data platform can be relatively easy and straightforward. To maximize the results, you’ll need a good idea of what problems you want to solve – and what information you want to know.
There are many essential questions that anyone starting their big data journey should ask. Over the years, I’ve fielded questions from customers and prospects, so I thought I’d take a moment to go over some of the most common and important ones with you.
Let’s get started.
Whether you’re considering a big data solution or already have one, this question needs to be at the top of your list. Don’t embark on a big data project simply because everyone else is. First, ask yourself what you want to know, and what value will big data add?
Big data can add value to just about every company. But before implementing a solution, you should have a good idea of what questions you want to answer and where big data can add the most value.
Do you want to understand more about customer behavior on your website, or are you trying to discover signals for customers who are looking to drop your services? Do you want to discover product deficiencies, or analyze problems in your logistics operations? Big data can provide answers, but first you need to make sure that you know the right questions to ask.
As an example, researchers at the United States Fish and Wildlife Association (USFWS) must rely on data for much of their research, reporting and analysis. It’s essential to pull together data across regions to uncover potential relations among countless variables and provide fact-based reporting and analysis. For the USFWS, the questions weren’t necessarily always defined; often, the questions were exploratory and iterative. An answer to one question could open up an entirely new branch of inquiry requiring different sets of data and analysis methods.
What was essential, however, was empowering researchers to define their own questions and most importantly, to be able to quickly find answers to those questions.
Just about every company has data. Operate a website? Have a Facebook page or Twitter account? How about a point-of-sales system, or inventory and accounting software? Assets and activities like these generate data. Even seemingly small businesses and processes can generate surprisingly large amounts of data. But how do you know whether it’s big data?
You shouldn’t think about big data only in terms of volume. Yes, the right business intelligence (BI) platform can effortlessly crunch even huge data loads. But beyond volume, big data is also useful for tying together and analyzing data from a huge range of sources. In many ways, this ability to digest and analyze data from a broad range of sources is one of the most valuable aspects of a big data-empowered, business-intelligence platform.
Volume-wise, the USFWS’ needs weren’t that high. To be sure, with data coming in from research efforts and reporting agencies across the country, there was a lot of data. But ultimately it was the diversity of sources that presented the biggest challenge.
How could they quickly tie together a sizable amount of data from a wide range of sources? Even if they wanted to use common tools like Excel, simply compiling the data was a massive effort in itself. With a user-friendly, big data-enabled BI platform, however, such analysis can be accomplished quickly.
Do you have the people to implement a big data solution? There are two factors you have to consider: your company and its IT staff, and perhaps more importantly, what type of big data solutions you want to implement. The type of solution you seek to implement will determine the size and scope of the IT staff you’ll need to get started.
If you’re looking to build your own 100-percent, in-house big data solution, you’re going to have to assemble an all-star team. When the big data field was emerging, only the most well-funded companies used it regularly because of its once-extensive staffing requirements.
Big data experts are always in short supply, especially outside of industry hubs like Silicon Valley. And if you’re located in an IT hub, you probably already know that retaining top talent is an expensive proposition. On the other hand, if you happen to be in a smaller metropolitan area, you’ll have a smaller talent pool to draw from.
That’s why the second part of the question is so important. What type of big data solution do you want to implement? By choosing a business intelligence platform that works out of the box with big data, you’ll be able to reduce IT needs and can rely on business analysts to run your big data efforts. Ultimately, it’s analysts, marketers and executives who must make decisions based upon the data so it makes sense to put them in charge of your big data tools. In the past, this simply wasn’t possible because big data was so technically complex, but new business intelligence platforms make big data easy to work with.
The USFWS found this question to be perhaps the most important one. They have a small team, especially in regards to their IT staff. Yet they support countless researchers and scientists with high demands for data across the country.
Self-service was essential. There was no way that they could personally prepare the data crunching and analysis needed for every ongoing project and research effort. Yet, if scientists could have the tools to do their own data analysis, research could be conducted far more quickly.
Often, companies already have tools for gathering data in place. Implementing a new big data solution can create conflicts with existing reference architectures and data sources. However, when done right, a new big data solution won’t necessarily replace or create conflicts with existing assets. Instead, a new solution can be used to fill gaps and to tie divergent sources and tools together to create a more holistic picture.
Think of it as filling the holes in Swiss cheese. With the right big data platform, you’re augmenting your current efforts and creating something that’s more whole. Big data makes it easy for data to “talk to each other” and to be governed under the same reference architecture.
For the USFWS, rather than replacing data and tools, Datameer was used to tie everything together and create a more cohesive, functional reference system. With a BI platform in hand, they could gather and compare more data and most importantly, increase data utilization.
This last question is in some ways, the most important question to ask. It’s closely related to that first question of “What do you want to know?” Some companies make the mistake of engaging in big data simply because everyone else is doing it. Make no mistake, big data adds a lot of value and if your competitors are already using it, that should come as no surprise. At the same time, however, when implementing your own big data solution, you should dig a bit deeper.
Any big data efforts need to be grounded in tangible, measurable outcomes. Determining the ROI of a big data solution is essential both for justifying the decision to invest in a new solution, and to hold you accountable.
As for the USFWS, their ultimate end goal was to increase self-service and to empower scientists to quickly and easily use big data. With a smart, easy-to-use business intelligence platform, they could accomplish exactly that. As a result, scientists can now gather and analyze data far more quickly, thus increasing the speed at which research is conducted and ultimately published. They can now provide higher quality research in a faster manner, which is a goal that should apply to everyone who decides to embark on a big data project.