7.4.x Release Notes

Owned by Juliane Wetzel
Last updated: Jan 28, 2020
18 min read

7.4.13 Release Notes

Bug Fixes

1

Property API: Keyup event for property info input fields of the Property API is not workingDAP-39015

Keyup event now triggers the same actions as for example a change event.

(tick) Bug fixed.

2

Cannot export Runtime Analytics sheets to HiveDAP-39472

Runtime Analytics sheets can be exported to Hive again.

(tick) Bug fixed.

7.4.12 Release Notes

Improvements

1

Add Support for CDH 6.3.2DAP-39657

Datameer now supports Cloudera CDH 6.3.2.


Bug Fixes

1

REST API to download results doesn't return all records by setting 'rest.download-data.records-max=0'DAP-39386

Datameer respects the setting 'rest.download-data.records-max=0' and provides all records for the download.

(tick) Bug fixed.

2

'Workbook' tab doesn't close after 'Close & Save' DAP-39417

After closing and saving a workbook, the 'Workbook' tab closes and the user is redirected to the 'File Browser' tab.

(tick) Bug fixed.

3

Hive string queryTemplate for SHOW PARTITIONS is incorrect DAP-39438

The string for queryTemplate now shows the required syntax using back-ticks for every single part of the Hive query.

(tick) Bug fixed.

4

Workbook config: false errors on correct CRON patternDAP-39458

Datameer now recognizes a valid CRON pattern and does not throw an error on a valid pattern erroneously.

(tick) Bug fixed.

5

Scheduled jobs remain indefinitely in the processing queue if the Kerberos password is changedDAP-39474

A scheduled job occurring between a Kerberos password change no longer blocks the processing queue.

(tick) Bug fixed.

6

Workbook: save dialog doesn't open when user selects 'Close & Save'DAP-39509

Closing a new workbook from the file browser now redirects to the workbook save dialog.

(tick) Bug fixed.

7.4.11 Release Notes

Improvements

1

Workbook validity reportDAP-39389

Datameer provides report that identifies workbooks missing data sources or other similar issues.


2

Add Support for CDH 6.2.1DAP-39461

Datameer now supports CDH 6.2.1.

3

Add Support for CDH 6.3.1DAP-39462

Datameer now supports CDH 6.3.1.


Bug Fixes

1

Backup/Restore does not completely store all artifacts
DAP-39297

Even if there are broken artifacts, a backup continues and logs the broken files.


(tick) Bug fixed.

2

Hive-JDBC: Rescan Schema leads to SaslException on NMU environment
DAP-39308

Rescan schema now works without generating an exception for Hive JDBC and Datameer running in NMU mode.


(tick) Bug fixed.

3

Customer Upgrade: ExportJob forcing append
DAP-39365

Hive export jobs are no longer duplicated after upgrading from Datameer 7.1 to 7.5. In case of recreating or overwriting, old data from a Hive managed table is deleted. In case of recreating or overwriting from a Hive external table will not be deleted.


(tick) Bug fixed.

4

Customer Upgrade: Join Columns in Error after Datameer upgrade
DAP-39418

Cascading errors in join sheets don't appear any more. After an upgrade to Datameer version 7.4, columns are migrated successfully. Circular dependencies are detected. Broken workbooks stay broken with the same error.


(tick) Bug fixed.

5

Artifact (IJ, Wbk, FU, DL, EJ) status not updated in dependency view
DAP-37822

The entity status in the dependency view reflects the current status (success / error / canceled) of this entity.


(tick) Bug fixed.

6

Workbook: Inconsistent column names if a sheet has empty columns
DAP-39322

Column naming for SQL sheets now follows the usual convention (A, C, C1, ...)


(tick) Bug fixed.

7.4.10 Release Notes

Improvements

1

Join Operation Should Compute Inputs in Parallel instead of one at a timeDAP-39146

For improved performance, join of two data links now computes inputs in parallel to prevent the creation of too many DAG(s) on the cluster, and the Map-Side join picks the smallest of the two inputs to cache in memory.


2

Add Support for CDH 5.16.2DAP-39131

Datameer now supports CDH 5.16.2.

3

Add Support for CDH 6.3.0DAP-39169

Datameer now supports CDH 6.3.0.


Bug Fixes

1

Upgrade scripts are ignored when using Maria DB
DAP-39293

For a fresh installation, the optional_upgrade table in Datameer's metadata database now gets filled, therefore all included optional upgrades for the current Datameer version are set as expected.


(tick) Bug fixed.

2

BigQuery import job Data Details wizard page UI reload doesn't adhere to selected table
DAP-39230

When editing an existing BigQuery import, the selected table is the same as before the point of creation.


(tick) Bug fixed.

3

Google BigQuery import connector non-US region datasets failure
DAP-39217

BigQuery connector works for datasets residing in regions other than USA.
Note: the temporary dataset and dataset from which you are importing data must reside in the same geographical region.


(tick) Bug fixed.

4

After opening a workbook by path can't switch tabs back to that workbook
DAP-39197

It is possible to switch tabs from file browser to the workbook when opening a workbook by path.


(tick) Bug fixed.

5

TDE/Hyper export with UNICODE symbols over U+D7FF fails job/crashes Datameer (in local ef)
DAP-39164

Characters that caused a crash when exporting a sheet to Tableau are replaced with HTML entities. Such replacements are logged in the application log.


(tick) Bug fixed.

6

Datameer is missing the  "X-XSS-Protection" HTTP response header
DAP-39048

`X-XSS-Protection: 1; mode=block` is set in the header of every response coming from the Datameer backend.


(tick) Bug fixed.

7

Can't read DECIMAL from an Avro file
DAP-39006

Datameer now imports Avro data with primitive type bytes logicaltype decimal as BIG_DECIMAL in Datameer.


(tick) Bug fixed.


7.4.9 Release Notes

Improvements

1

Improve responsiveness of UI for customers with large number of job executionsDAP-39030

Querying very large tables for job executions could take seconds, rendering the file browser UI unusable. Datameer now has an index for the job_status column in the metadata database to speed information queuing and display.

2

Integrate Plug-in Code into Plug-in NMU to notify users by email when jobs fail due to missing/stale keytabDAP-38761

Users get now notified by email when a Datameer job is failing because of a missing or stale keytab file.

3

No plugin with id '<plugin_name>' found for deserializing class '<plugin_internal_class>'DAP-38023

Datameer now handles occurrences of YARN-359 or YARN-6641 gracefully so that errors indicating a specific plug-in isn't available can be mitigated. The workaround can be activated with the das.cluster.plugin.resources.fixer.enabled property.

4

Merge Google Big Query Connection and Import Plugin into Datameer 7.4DAP-39139

Datameer can now import data from Google BigQuery data sources.

Bug Fixes

1

Datameer can't recognise Int32 and Int64 Decimal values on a Parquet File Ingestion
DAP-39110

Datameer now can handle Parquet files containing Binary, Fixed, INT32 and INT64, as well as DECIMAL column types


(tick) Bug fixed.

2

Hive Plug-in: data-driven Hive ExportJobs submitting with random impersonated user
DAP-39046

Datameer now runs the Hive export job with correct user impersonation.

(tick) Bug fixed.

3

Hive Plug-in: Transport Mode HTTP with Table Filter set to "Table and View" leads to an error on Kerberos secured environments
DAP-38979

This setup and configuration combination no longer generates an error.

(tick) Bug fixed.

4

Hive Plug-in: Hive Metastore - Create Delegation Token for Kerberos Secured Hive independent of configured Impersonation Mode
DAP-39034

Datameer now creates the required delegation token for accessing a Kerberos Secured Hive.


(tick) Bug fixed.

5

NPE when configure plug-in where preferences extension are disabled
DAP-39091

Datameer now correctly handles disabling an extension point of plug-in and does not produce a "NullPointerError".

(tick) Bug fixed.

6

Visual Exploration button and menu item isn't disabled if workbook has no permissions to edit
DAP-39047

Visual exploration button and related menu entries are now disabled when a user does not have adequate permission to edit.


(tick) Bug fixed.

7

Upgrade: Many Workbooks failing after upgrading to 7.4.6 because of wrong auto-resource computation
DAP-39055

Datameer auto-resource computation for required vCores no longer results in zero.


(tick) Bug fixed.

8

Workbook exchange data source should respect "kept" attribute of the datasource being replaced
DAP-39001

When exchanging a datasource, the "kept" flag respects the original data sheet "kept" value.

(tick) Bug fixed.

9

Workbook: wrong syntax highlighting on formula sheets
DAP-39130

Datameer now renders with the correct syntax highlighting in workbook formulas.


(tick) Bug fixed.

10

Hive Plug-in: enable column projections for inputformat Parquet
DAP-38959

Datameer now supports column projection for input formats ORC and PARQUET to increase read performance.
Column projections work for Import Jobs and Data Links.


(tick) Bug fixed.

11

Memory-Backed Map-Side Join will never be chosen when input is two Data Links
DAP-39095

The chosen join algorithm will be MEMORY_BACK_MAP_SIDE when two data links are being joined and at least one of the sides is small.


(tick) Bug fixed.

7.4.8 Release Notes

Improvements

1

Re-enable Save button on workbook toolbar

DAP-38960


Save button is available on the workbook toolbar, right next to Add Data icon


Bug Fixes

1

SAML: Upload Metadata File Button is inactive and metadata JSP missing
DAP-38975

Uploading and downloading metadata information works as expected.


(tick) Bug fixed.

2

ExportJob fails with java.io.NotSerializableException: datameer.dap.sdk.entity.v2.workbook.operation.ImmutableOrderingColumn
DAP-38969

Amend serialisation logic in order to avoid NotSerializableException.


(tick) Bug fixed.

3

FixedWidthImport: validate splitting logic should use column length instead of hard coded 1000
DAP-38941

FixedWidthImport supports variable and configurable import width.


(tick) Bug fixed.

4

Creating / Overwriting job configurations with REST v2 does not handle capability for 'Hadoop properties'
DAP-38342

REST v2 API now respects the capability settings for 'Hadoop properties'.


(tick) Bug fixed.

7.4.7 Release Notes

Epics

1

Add Support for HDP 3.1.0*DAP-38207

*Currently in preview mode.


Improvements

1

Export to Tableau in HYPER format

DAP-35887


Datameer now supports data export in Tableau's Hyper format.


2

Native Multiuser: Introduce configurable properties to determine Kerberos salt from LDAP attribute and default realmDAP-38755

Datameer now allows you to specify and configure the Kerberos salt authentication during job execution.
Define salt settings on the LDAP Authenticator and Hadoop Settings configuration pages.

Bug Fixes

1

Fixed Width File Import: Can't ingest more than 1000 characters.
DAP-38316

Datameer can now import a fixed width format file with more than 1000 characters. The maximum number of imported characters can be configured with an advanced option in the job-level property.


(tick) Bug fixed.

2

Unable to create S3 connection to an Internal Endpoint URL
DAP-38788

Datameer now supports the definition of an 'Internal Endpoint URL' property in the connection wizard, allowing you to import from there.


(tick) Bug fixed.

3

Unable to search for files if owner consists of numeric characters
DAP-38801

Users are now able to search for configurations where the owner name consists of numerical characters only.


(tick) Bug fixed.

4

HiveServer2 Plugin: Partitioned Export doesn't respect in Wizard changed column names
DAP-38815

HiveServer2 export configurations allow for column names to be changed in the export job wizard. The changed names are now retained during job execution.


(tick) Bug fixed.

5

Workbook: Creating an Advanced Filter fails with 'absent value'
DAP-38842

Datameer now provides a better error message when an invalid value is used to reference another sheet in the advanced filter expression.


(tick) Error message improved.

6

Workbook Dependency Graph overlaps when window gets resized.
DAP-38871

Dependency graph renders all nodes without overlap and horizontal scrolling opens if needed.


(tick) Bug fixed.

7

Zoomed chart overlay when opening a Filter dialog.
DAP-38921

Zoomed chart popup automatically closes when a filter dialog is opened.


(tick) Bug fixed.

7.4.6 Release Notes

Improvements

1

Add support for Azure Data Lake Store as Datameer private folder storage

DAP-24985


Datameer now supports the Azure Data Lake Store protocol for private folder cluster storage.


2

Introduce Job Timeout Parameter for Hadoop Jobs that do not Make ProgressDAP-24181

If Hadoop jobs have made no progress within the timeout period, the job will fail and alert automatically using the existing failed job alerts.

3

Make warning about Hive Metastore server deprecation configurable

DAP-38710

Allow the Hive Metastore deprecation warning to be disabled for job execution results.

Jobs using a Hive Metastore will have a final Status of Completed rather than Completed with Warnings.


4

Hive plugin: Improve Help Texts and Error Messages for Partition Location HandlingDAP-38646

Help text and error messages for working with Hive partitions have been extended and improved.

Bug Fixes

1

Hiveserver 2 Plugin: "Create new if not exists" - Export Doesn't Fail as Expected if Schema of Existing Table Doesn't Fit
DAP-38738

An export job now fails as expected if the schema for the target table (column number, column names) doesn't match the export job's schema.


(tick) Bug fixed.

2

External authentication: Active Directory / LDAP - can not add new User or Groups
DAP-38734

It is possible to administrate user and group settings again.


(tick) Bug fixed.

3

Display meaningful Error Message for rename Column with more than 255 chars
DAP-38732

The user will now receive a meaningful error message.


(tick) Bug fixed.

4

Log4j Parser Doesn't Read the Last Line of the File
DAP-38692

An import from a Log4J file now includes the last line as well.


(tick) Bug fixed.

5

Not Possible to Delete an Upload Job if the Uploaded File has Spaces in the Name
DAP-38600

Spaces in an upload job's file name will no longer prevent them from being deleted.


(tick) Bug fixed.

6

hadoop.usernameForSalt Property Cannot be Enabled
DAP-38598

It is now possible to control the hadoop.usernameForSalt property.


(tick) Bug fixed.

7

Hive JDBC Plugin: List Tables on a 40k Tables Database Takes very long for Hive Versions less than 2.0.0.0
DAP-38527

A configurable performance workaround for listing large numbers of Hive tables and views is now available.


(tick) Bug fixed.

8

Workbook: "Go to line" functionality won't jump to the desired line in the Workbook
DAP-38518

"Go to line" jumps to the correct line in a workbook sheet.


(tick) Bug fixed.

9

Cannot upload an Avro file
DAP-38350

It is now possible to import an Avro file respecting the current Hadoop configuration for split operations.


(tick) Bug fixed.

7.4.5 Release Notes

Epics

1

Workbook Security UpdatesDAP-38178

Introduce a System Property to Configure the Source Permission Evaluation in Workbooks

DAP- 38188

Introduce a Evaluation Unit That Checks Readability of Workbook SourcesDAP- 38189

Use a Security Property Based Evaluation Within the Workbook Overwrite APIDAP- 38190

Use a Security Property Based Evaluation Within General Workbook ResponsesDAP- 38191

Use a Security Property Based Evaluation Within Browser Listings of WorkbooksDAP- 38192

Use a Security Property Based Evaluation Within the Add Data Dialog of WorkbooksDAP- 38193

Improvements

1

Show the Tooltip If a Column Name Is Longer than the Column SizeDAP-34634

During the mapping step of import and export jobs, the column widths can't be adjusted. Now, if you hover over a column name that is too long to fit, a tooltip message is displayed.


2

Hive Partition Location ModeDAP-38462

A new Hive plug-in configuration setting has been added to improve performance when fetching the partition location. You can now select between fetching only the default partition or every partition within the Hive metastore.

3

Improve the Handling of Mismatches Between the Partitions' Metadata and Their Actual HDFS Location

DAP-38469

Users can now configure their Hive plug-in to:

  • Drop any invalid partition.
  • Abort the entire job if an invalid partition is discovered.

4

Add Support for Apache 2.7.7DAP-38590

Support added for Apache 2.7.7

5

Add Support for CDH 6.1.1DAP-38675

Support added for CDH 6.1.1

6

Add Support for CDH 6.2.0DAP-38674

Support added for CDH 6.2.0

Bug Fixes

1

In Workbooks, the Recommendation Dialog for Column Validation Is Missing
DAP-38112

It is no longer possible to submit changes if there are invalid columns in a workbook.


(tick) Bug fixed.

2

Export Job Failure Using sFTP Connection
DAP-38479

Export jobs using the sFTP connection now work as expected.


(tick) Bug fixed.

3

Slow to generate a Large Table from a HiveServer2 Connection
DAP-38526

When creating a data link to a table in a database with a high count of tables, the performance time to list the tables has been greatly improved.


(tick) Bug fixed.

4

Real Time Analytics on a 7.4.4 Build Using S3 Contains Errors (Invalid Version String)
DAP-38608

The Real Time Analytics (RTA) plug-in has been updated to resolve the errors.


(tick) Bug fixed.

5

The Capabilities for an Analyst User to Access the Admin Area Aren't Handled Properly
DAP-38610

  • If a user has the READ but not EDIT capability for any section, the admin buttons are disabled.
  • Non admin roles can no longer open user/role editing without the EDIT capability.
  • If a user has create/edit/delete capabilities, the corresponding sections are visible and the user can work with them.
  • The Job Scheduler and Event Bus sections work for users that have Admin capabilities.
  • The variables section is visible for users if the READ capability is set.
  • The admin tab can be opened by any user with at least one capability in the admin area.

(tick) Bug fixed.

6

When Datameer Is Deployed on an EMR Cluster Which Uses S3 as Job Storage, It Isn't Possible to Use Visual Explorer 
DAP-38621

Visual Explorer is again available when Datameer is deployed on an EMR cluster using S3 as job storage.


(tick) Bug fixed.

7

Unable to Load UI Directories in the File Browser: 502 Proxy Error
DAP-38643

The file browser UI now continues to be responsive even if the job configuration of a folder has thousands of data entries.


(tick) Bug fixed.

7.4.4 Release Notes

Epics

1

Workbook Security UpdatesDAP-38178

Introduce a New System Property to Configure the Source Permission Evaluation in WorkbooksDAP- 38188

Introduce an Evaluation Unit That Checks Readability of Workbook SourcesDAP- 38189

Use a Security Property Based Evaluation Within the Workbook Overwrite APIDAP- 38190

Use a Security Property Based Evaluation Within General Workbook ResponsesDAP- 38191

Improvements

1

Set a Default "Error Handling Mode" / "How to Handle Invalid Data Strategy" Globally for All ExecutionsDAP-38145

  • New custom properties have been added. In the default properties you can now set the default ErrorHandling mode for all workbooks and export jobs and the default How to handle invalid data strategy for import jobs and data links.
  • Datameer administrators can now set their own default ErrorHandling mode and How to handle invalid data strategy for every artifact type.

2

Add Support for CDH 6.1DAP-38147

Support added for CDH 6.1.

3

Auto-Sort Source Data While Exporting into a Partitioned Hive Table

DAP-38508

Data is now automatically sorted via partitioned columns before an export job.


Bug Fixes

1

Duplicating a Sheet Doesn't Copy the Sheet Description
DAP-38020

When a workbook sheet is duplicated, the description is now copied.


(tick) Bug fixed.

2

Concurrent Edit Warning Is Shown When Deleting a Column from a Referenced Worksheet
DAP-38360

A concurrent edit warning no longer displays when deleting a column from a worksheet which is linked in another workbook.


(tick) Bug fixed.

3

A Null Pointer Exception Occurs When Trying to Configure a Secured MapR Cluster
DAP-38423

Configuring a secured MapR cluster is again possible.


(tick) Bug fixed.

4

Configuring the Auto-Reduce Parallelism for the Tez Execution Engine Doesn't Take the Configured Job Queue into Account for the Max Number of Reduce Tasks
DAP-38473

  • The configured das.job.queue property now scales the max number of reducers for each Tez job.
  • The configured das.job.queue property is now used when generating input splits.
  • If the YARN queue can't return a percentage, the das.job.cluster-resource-percent property is defined if not set to auto.
  • If none of the above, the whole cluster's vcore availability is used.

(tick) Bug fixed.

5

Using Reserved Words in a Hive Partitioned Column Fails
DAP-38502

It is now possible to use Reserved Keywords for Hive partitioned columns.


(tick) Bug fixed.

6

A Duplicated Sheet Is No Longer Kept per Default
DAP-38531

Duplicated worksheets are now kept per default when a workbook is saved.


(tick) Bug fixed.

7

The DESCRIBE EXTENDED Call to Hive Is Not Being Impersonated
DAP-38564

The correct delegation token is now being returned and DESCRIBE EXTENDED is correctly being impersonated.


(tick) Bug fixed.

7.4.3 Release Notes

Improvements

1

Change EMR Cluster SettingsDAP-38357

Users can now choose between EMR Cluster ID, EMR Cluster Name, or YARN Resource Manager to decide which is the host cluster.

Bug Fixes

1

Workbooks Failing with Unkept SQL Sheets
DAP-38431

Workbooks now no longer fail even if running when some sheets in the workbook aren't marked as kept.

(tick) Bug fixed.

2

Infographic Legend Vulnerabilities
DAP-38427

Scripts are no longer runnable if crafted in the prefix/suffix for an infographic widget.


(tick) Bug fixed.

3

Users Can't Properly List Tables via a Hiveserver2 Connection
DAP-38416

An updated Hive plug-in has resolved this issue.

(tick) Bug fixed.

4

Adding a Spark Execution to the Job Graph Causes an Error
DAP-38409

Spark jobs now run successfully when added to the Job Graph.


(tick) Bug fixed.

5

HdfsPermissionSynchronizer Doesn't Respect auth_to_local Rules
DAP-38399

secure_hdfs_tool.sh now translates usernames so that the Datameer conductor reads them correctly. It now sets permissions according to hadoop.security.auth_to_local rules introduced on the Datameer Admin tab.

(tick) Bug fixed.

6

Problem Using the RTA Plug-in 5.1 from 7.5.0 Utilizing the Yarn Backend
DAP-38380

Users can now utilize visual exploration when using Yarn as a backend for the RTA plug-in.


(tick) Bug fixed.

7

The Direct Task Executor Doesn't Close the Entity Manager in an Error Case
DAP-38373

The direct task executor now closes the entity manager in case of an error during task executions.

(tick) Bug fixed.

8

Housekeeping Tries to Purge Export Job Data Files from HDFS Which Don't Exist
DAP-38324

This was an issue due to status logging in Housekeeping. Upon starting Datameer, the non-existent jobs that were set to be deleted are reset.

(tick) Bug fixed.

9

The Browse All Data Page Doesn't Show Data Sometimes
DAP-38314

If you Browse All Data and no sheets are kept in the workbook an error message is now displayed explaining the issue.


(tick) Bug fixed.

10

In the Remote File Browser, the Path to File Is Invalid When Selected Through the Datameer Browser with a Space in the Name
DAP-38303

The paths generated by the Remote File Browser are now valid.


(tick) Bug fixed.

11

Workbooks are Failing with an Error - Parquet Is Not a Parquet File (Too Small)
DAP-38224

The DatameerFileWriter opening an output file for write in a task now fails that task immediately to prevent creating a corrupted file.
  • Error logs are no longer written with VerifyExceptions for every record stating the writer has already been closed.
  • These tasks now fail with an UnrecoverableException.

(tick) Bug fixed.

12

 The PushDown Filter to Partition Fails after Changing the Data Type When Creating a Data Link
DAP-38082

It is again possible to change data types during the Define Fields section of creating or editing a data link.


(tick) Bug fixed.

13

The CAST Function is Returning Errors
DAP-38081

The SQL worksheet function CAST has now been documented so users can understand what to expect.


(tick) Bug fixed.

14

Problems Exporting to Partitioned Hive Tables in Parquet Format
DAP-37015

All partitions are now readable after an export to Hive has completed successfully.


(tick) Bug fixed.

15

The Preview Record View Changes While Moving a Column
DAP-33298

The preview record view is now kept and doesn't change while moving a column.

(tick) Bug fixed.

7.4.2 Release Notes

Epics

1

Hive Performance ImprovementsDAP-30302

Hive Data Link Wizard Time Reduction DAP-30226

Don't Cache All Partitions If Filtered Partitions Are Selected DAP-30017

 Improve Logging of Hive Cache for Data Links DAP-30010

Improve Wizard Step Performance DAP-29162

Disable the Logging to Improve Performance for the Hive Cache DAP-30317

Reduce Partition Parsing for Local Hive Partitions DAP-28760

Fetching Partitions Processed with Orc Table Have Very Poor Performance Due to Fetching All the Partitions Even When Filtered DAP-29902

Removed HiveImportJobModel.fileMatcher(Matcher) DAP-30397

Partitions from a Hive Partitioned Table  Are Being Read over and Over DAP-28971

Data Link on Partitioned Hive Performance Tweaking DAP-29977

Remove Info Logging on Caching from Hive Data Links DAP-30834

2

Adding Data to WorkbooksDAP-37598

Created a New Data Warehouse From Top Level "+" Plus Menu DAP-38022

Duplicate / Copy-paste / Backup-restore Wasn't Working for Data Warehouse Connections DAP-38200

JDBC Passwords No Longer Saved in Clear Text in the Meta Database DAP-38153

On Logout, the Push Down Filter Usage is Recorded in the Job.log DAP-38074

Created Workbook from DWH File Browser Context Menu DAP-37599

Now Possible to Create a Workbook from the Data Warehouse in the File Browser DAP-38149

Complete Coverage of Filter Push Down DAP-38000

Deleted Data Warehouse JDBC  Dialects That Don't Have Official Support DAP-38146

Use the Data Warehouse Driver on Open Connections DAP-38003

Verified Consistency Between the Driver in DataWarehouseEntity and DbDriverImpl DAP-38001

Now Possible to Configure a Data Warehouse Connection in File Browser Inspector. DAP-37447

Data Warehouse FileBrowser Integration DAP-37461

Corrected the "Driver Not Found Issues" When Providing the Driver via the Ui DAP-37976

The Data Warehouse Sheet Doesn't Handle Unused Columns DAP-38123

Fixed "Operation Not Allowed after Resultset Closed" Error in the Workbook DAP-38072

Null Pointer Exception When Duplicate a Data Warehouse Sheet DAP-38154

Redshift/MySQL - Reserved Word as Column Name Leads to an Error DAP-38121

Workbook UNDO Now Working Correctly With a Data Warehouse Sheet DAP-38180

Insert Data Warehouse Sheet after Creating a Data Warehouse sheet from the Top Level "+" Plus Menu DAP-38259

Dialogs Now Open Correctly in the Workbook for IE and Edge Browsers DAP-37598

A "Select Table" Dialog Has Been Added to the Workbook When Adding Data From a DatabaseDAP-37462

Created a New Data Warehouse Option from the Top Level "+" Plus MenuDAP-38022

Improvements

1

Add Support for MapR 6.1 (Along with MEP 6.0)DAP-38148

Support added for MapR 6.1 and MEP 6.0.

2

Restrict Variables to Specific Roles / GroupsDAP-38219

It is now possible to limit which users are able to use / edit / delete variables within Datameer.

3

Simple Theta-joins Are Now Supported Where Equi-joins and Filters Are Possible on SQL Sheets

DAP-37703

  • Datameer supports inner, left outer, and right outer joins with multiple criteria.
  • When running full outer joins, an error message is now displayed stating that it isn't supported.

4

Support OpenJDK 8DAP-38162

OpenJDK is now supported.


5

Rollbacks are Allowed Within a Couple of Days of Upgrading Without Risking Data LossDAP-38184

To allow safe rollbacks for a longer time, Datameer no longer deletes data from HDFS for the first 2 days after an upgrade. After this Datameer switches back to the configurable time of 30 minutes.


6

Add Support for CDH 5.15.2DAP-38270

Support added for CDH 5.15.2


Bug Fixes

1

Hive Data Link Wizard Time Reduction
DAP-30226

The wizard for data link creation run a long time when the cache is created. This process was optimized to take less time.


(tick) Bug fixed.

2

Command conductor.sh start|stop|check Was Reporting False Results
DAP-32789

Incorrect reports have been eliminated.


(tick) Bug fixed.

3

Max Log Errors Aren't Mandatory but Expected as Such in Immutable Generation
DAP-34795

"Max number of errors" field can no longer be left blank.


(tick) Bug fixed.

4

After Changing the Owner in NMU Mode, the Old Job Folder Isn't Being Deleted
DAP-35088

Housekeeping now deletes orphaned job folders.


(tick) Bug fixed.

5

The Smart Execution Link Leads to a Page Could Not Be Found Error in Import Job Run Details
DAP-37606

The Smart Execution link has been updated.


(tick) Bug fixed.

6

Can't Change Workbook-Undo Settings on /Dev Page
DAP-37612

Undo setting on the /Dev page works correctly after updating CSRF token for POST actions.


(tick) Bug fixed.

7

In Visual Explorer, It Isn't Possible to Drill into a Value That Includes "*" Character
DAP-37696

It is now possible to drill into a value with length > 16 that contains "*" and other characters.


(tick) Bug fixed.

8

Custom Properties Missing from job-definition.Json If User Is Restricted from Settings
DAP-37697

The job-definition.json file is now properly storing the custom properties when created.


(tick) Bug fixed.

9

Stabilize Date-Time Functions for SQL Worksheets


DAP-37854

The following functions have been updated:

  • CEILINGDATE
  • FLOORDATE
  • TIMESTAMPDIFF
  • DAYOFYEAR
  • QUARTER
  • WEEKOFYEAR

(tick) Bug fixed.

10

Single Group Sharing for Secure Impersonation Isn't Being Handled Properly in the Ui
DAP-37904

  • An error message is displayed in UI if multi group sharing isn't allowed.
  • An error message is displayed in UI, if the CSRF token is invalid.

(tick) Bug fixed.

11

Fuzzy Simple Select Elements in IE11
DAP-37936

The Text now looks sharper.


(tick) UI updated.

12

Publicly Shared Infographic Doesn't Show Data
DAP-37945

Publicly shared infographics now show data.


(tick) Bug fixed.

13

A Workbook Was Stuck Processing an SQL Sheet
DAP-38015

Long processing time due to cross-products when the join attribute differs between worksheets.

Workaround:

  • If a job is sticking, use Datameer traditional instruments to re-create the logic that the MySQL query is supposed to introduce.

The job logs now show when joins will be a cross-product so services can quickly diagnose the problem for users.


(tick) Workaround.

14

The Cell Context Menu in Workbooks Should Only Include Input and Nothing Else
DAP-38033

A cell's context menu now only displays the current cell input.


(tick) Bug fixed.

15

Inconsistent Behavior of Nested Lists in Workbooks
DAP-38085

List with different nesting levels aren't be valid and displays an error for the user.


(tick) Bug fixed.

16

Can't Delete an SQL Sheet When Having a Circular Reference to Itself
DAP-38113

It is now possible to delete a sheet which is referenced by itself.


(tick) Bug fixed.

17

 Error Accessing File with "." Prefix
DAP-38116

  • Datameer now only shows files that the user is able to actually pull.
    • Files that start with "." are now excluded (filtered out) from the remote data browser.
    • The global filter in the Remote Data Browser is now set to filter out files (NOT folders) that start with a "."

(tick) Bug fixed.

18

Worksheet Error after Applying SQL Query with "Sub"
DAP-38131

The query option now works without error.


(tick) Bug fixed.

19

It Isn't Possible to Resize the Multiline Editor after Zoom In/Out


DAP-38172

It is now possible to resize the multiline editor.


(tick) Bug fixed.

20

Attempting to Set tez.task.launch.cmd-opts to Prevent the Container Running Beyond Memory Limits Always Fails
DAP-38225

Setting the tez.task.launch.cmd-opts property correctly now prevents the possibility of a container running beyond its limits.


(tick) Bug fixed.

21

Workbooks Using a Hive Data Link with the Small Job Framework Isn't Working in a Kerberos Secured Environment
DAP-38226

Now when using a Kerberos secured cluster, a workbook using a Hive data link in Small Job works correctly.


(tick) Bug fixed.

22

Data Links Against Partitioned Tables Fail on Secured Environments Without Impersonation


DAP-38257

Data links against partitioned Hive tables now run successfully on a Kerberos secured environment without Impersonation.


(tick) Bug fixed.

23

Can't Rename SQL Sheet When Using FLOOR with Dateunit Statement
DAP-38263

It is now possible to rename SQL worksheets.

(tick) Bug fixed.

24

Inconsistent Implementation of HTTP Methods for Creating and Updating Variables
DAP-38265

Updates to the REST API:

  • Creating a variable:
    • curl -X POST -u <username>:<password> -d name=anotherVariable&value=testvalue&type=STRING <DatameerIP>:<port>/api/variables
  • Updating a variable:
    • curl -X PUT -u<username>:<password> -d name=anotherVariable&value=testvalue&type=STRING <DatameerIP>:<port>/api/variables

(tick) Bug fixed.

25

It Isn't Possible to Export to Hive a Sheet from a Workbook Using a Transformed Sheet
DAP-38266

An export job that exports its data to Hive can now be run successfully.


(tick) Bug fixed.

26

 Sunburst Dialog Is Incorrect When Adding More than One Partitioned Source
DAP-38269

The sunburst dialog is now shown for all added data sources.


(tick) Bug fixed.

27

Users Are Unknown to the User Authentication Service after Upgrading Datameer
DAP-38275

Users from authentication services are now recognized correctly after upgrading.


(tick) Bug fixed.

28

The Support Engineer Report Won't Deploy
DAP-38276

The SER (Support Engineer Report) package now works as expected.


(tick) Bug fixed.

29

TIMESTAMPDIFF Function Returns Negative Value for Milliseconds Interval
DAP-38283

TIMESTAMPDIFF now returns the properly signed value for "MILLISECOND" intervals.


(tick) Bug fixed.

30

Backing Up of Only Specific Files Includes Unwanted Folder Definitions
DAP-38296

Only the specified files are now included in the zip w hen backing up specific files from a folder.


(tick) Bug fixed.

31

It Isn't Possible to Nest the GROUP_MOVING_AVERAGE Function
DAP-38298

It is now possible to nest data using the GROUP_MOVING_AVERAGE function.

(tick) Bug fixed.

32

An Implicit Join on Certain Data Types Result in a Cross Join
DAP-38310

Implicit joins with a "=" join condition on string keys no longer result in a cross product being calculated.

(tick) Bug fixed.

33

The Workbook Shortcut cmd+c Doesn't Work in the Formula Editor When Using the Firefox Browser
DAP-38320

This shotcut (cmd+c) for copying data now works in the formula editor when using Firefox.


(tick) Bug fixed.