While each use case may be unique, several common design patterns can be used for data preparation in data pipelines that curate data for analysis. Let’s explore these and see how you can apply them.
Each business case is a unique opportunity to harness data for analysis. These analyses take many different forms depending on the matter at hand. For example, if your function is marketing, you might be interested in analytics to understand customer sentiment, or a retail company might be interested in customer churn. Although these analytics vary, there are some common patterns and best practices that we can use across any analytics project and industry.
Data preparation is often used to merge different data sources with different structures and different levels of data quality into a consistent, reusable format. This canonical format is what makes all that data preparation, that is, the cleansing, the transformation, the blending, the enrichment, and the exploration, so powerful. This is how to find meaningful business insights in data.
See the four stages of data preparation, including identifying the right data, cleansing it, enriching it, and finally delivering a curated dataset.
Explore some common patterns to prepare data to analyze the various aspects of customer churn.
Explore some common design patterns for marketing analytics, including social media and sentiment analysis.
See ways to use built-in advanced algorithms in Datameer (Smart Analytics) to find consumer and transactional data patterns.
A common analysis in a retail environment is determining customer churn. We can add new features to the purchasing data we have just created to determine the customers at risk of leaving our business – all with existing functions in Datameer. In addition, once we have finalized our dataset, it can be scheduled and operationalized.
Using a simple model, we will look for patterns that are outside a customer’s purchase behavior. Again, this example uses retail data, but the pattern can be applied to similar projects; essentially, wherever behavior is analyzed using historical transactions.
We accomplish all this in two Datameer workbooks. The first workbook contains the two imported datasets, and then the joined set we will use for modeling. The second workbook is the workbook used for grouping, filtering, adding features, and holding the final data that we can operationalize for future analytics.
This is a ubiquitous and iterative process in an analysis project to find and add value to the data. In Datameer, we will add three worksheets to a new workbook. Each worksheet represents a primary stage in the preparation process – this is simple to do and provides clear steps as we create the features for our models and analysis.
We begin by adding the previously prepared dataset, in this case, customer purchases, to the first worksheet in our new workbook. This is the source data for grouping, filtering, and enriching to understand customer churn. For churn, we really want to know the period between customer activity and purchase, which will take some preparation to parse out of the data.
Grouping and Sorting – We want to structure the data for analysis. The goal at this stage in the churn analysis is to determine the number of days between purchases for each customer.
Modeling – We have the data needed to create our analytical model to determine customer shopping behavior. We begin by adding a final worksheet to our existing workbook.