Skip to end of metadata
Go to start of metadata

When importing data via an import job or a file upload, you can partition your data using date parameters. When this data is loaded into a workbook, you can choose to run your calculations on all or on just a part of your data. Also if you decide to export data, you can choose to export all or just a part of your data.

 

Please note that if you are using Microsoft Internet Explorer 7 or 8, these do not support Scalable Vector Graphics (SVG). When using partitions in a workbook or when exporting a partition, the simple partition filter will not be available - only the Time Window and Advanced Partition Filter. Please see Supported Browsers for alternate web browsers.

 

Partitioning Data During Import Jobs and File Uploads

To use partitions you must first configure your data within an import job or a file upload.

Simple Partition Filter

On the define fields step when importing your data, you can choose to partition your data. Currently, Datameer only supports time-based partitions.

Here you can choose a column that contains a date. You can also choose which scale to use. If you want to concentrate on only periods of years, months, weeks, etc.

Advanced Partition Filter

 

With the advanced tab, you can enter a formula to represent the partitions. This especially useful when you do not have one column containing dates, rather the date is split into more than one column.

ASDATE(CONCAT(#Year;"-";#Month;"-";#Day;" ";#Hour;":";#Minute;":";#Second);"yyyy-MM-dd HH:mm:ss")

 

Using Partitioned Data in Workbooks

After partitioning your data during an import or file upload. Open this data in a workbook (see Working with Workbooks for more details).

Simple Partition Filter

When you first load your partitioned data into a workbook, you will see a sunburst graph representation of the partitions in your data. You must first select which partition or partitions to use before you can work with your data.

This sunburst graph shows the different periods of time in your data. In this example the outside ring represents hours, the next ring represents days and so on, until you reach the center which represents all of the partitions. You can choose a ring by clicking. By double cliking you can zoom in. You can also selcet a range of partitions by holding down 'shift' and clicking, or you can select more than one individual partition by holding 'control' and clicking.

Time Window Partition Filter

Another way to select which partioin to use is with a calendar.

Simply choose the start and end date for the time period desired.

Advanced Partition Filter

There is also an advanced partition filter. Here just enter a formula to signify which partition should be returned.

Here is an example of a formula returning the partitions representing the past seven days along with the partition for today:

 $partition > TODAY()-7d && $partition <= TODAY()

Exporting Partitioned Data

It is also possible to export only the desired partition or partitions during export.

Simple Partition Filter

During export you can choose to export only certain partitions.

This sunburst graph shows the different periods of time in your data. In this example the outside ring represents hours, the next ring represents days and so on, until you reach the center which represents all of the partitions. You can choose a ring by clicking. By double cliking you can zoom in. You can also selcet a range of partitions by holding down 'shift' and clicking, or you can select more than one individual partition by holding 'control' and clicking.

Time Window Partition Filter

It is also possible to use a calendar to partition your data.

Simply choose the startn and end date to partition your data.

Advanced Partition Filter

There is also an advanced partition filter. Here just enter a formula to signify which partition should be expoerted.

Below is an example formula showing how to export data from 15 Sep 2010 between 5:00 AM and 8:00 AM.

$partition > ASDATE (2010-09-15 05:00;yyyy-MM-dd hh:mm) && $partition <= ASDATE (2010-09-15 08:00;yyyy-MM-dd hh:mm)

Repartitioning

After a partition has been set up it may become necessary to change the partition's resolution. It is possible to change the resolution for import jobs and file uploads. Changing the resolution requires a few simple steps.

  1. Select the radio button beside the partitioned import job or file upload and click Edit.
  2. Click next until you reach define fields.
  3. Under time-based partitions you can then change the resolution, like you would upon initial import.
  4. After choosing the desired resolution, click next until you reach the final screen and save the repartitioning under the same name or under a new name.

The repartitioning process runs automatically and as long as the resolution has become finer, all workbooks are updated automatically, also. If the resolution has become less fine, then you will be prompted to choose partitions again when opening an existing workbook.

If due to connectivity problems, a running repartitioning is aborted, just process the import job or file upload again. As long as the job configuration has been saved, then you just need to re-start the repartitioning job.

Labels
  • None