Eliminate rigid analytical models! Datameer Spotlight frees you from the restrictions of OLAP and gives you true self-service modeling, cataloging, and access to ANY data from across your enterprise for agile analytics. Your analytics community can quickly discover, model, consume, and govern data for analytics on an automated SaaS-based service that delivers faster, trusted analytics, and immediate time to value.
AtScale is a modern cloud-based OLAP platform designed to accelerate complex multidimensional queries and reduce compute load on data sources, specifically cloud data warehouses. It provides a semantic layer that maps source data into multidimensional models (cubes) on which users can perform OLAP-style queries.
Under the covers, AtScale uses data virtualization to reduce the need to ETL data into a central data warehouse and leaving data managed securely in the source. It provides automated performance optimization that pre-aggregates OLAP cubes’ results to increase performance, provide faster response time, and lower compute resource requirements.
AtScale has three major components:
Data engineers work in the toolset to create and structure multidimensional OLAP models. In a very typical OLAP manner, engineers define dimensions, hierarchies, and measures that make up the cube structure, with the result looking like a star or snowflake schema.
AtScale can use data from two major areas: data lakes and databases. Data lake sources include files, cloud object stores (AWS S3, Azure ADLS, etc.), and big data SQL data stores (Hive, Impala, Spark, etc.). It can also connect to and query from typical databases (Oracle, Teradata, Microsoft SQL Server, MySQL, etc.) and cloud data warehouses (Snowflake, Redshift, BigQuery, etc.)
The semantic layer is the repository of multidimensional models that analysts can use for their analytics. Users can see physical metadata about the models: dimensions, measures, etc. Users can find models to use by browsing through a basic catalog. Queries can be run on models via BI and analytical tools via OLAP interfaces (MDX/XMLA) or SQL ones (ODBC, JDBC) or AtScale’s REST API.
The OLAP engine executes user queries on the data, based on the models registered in the semantic layer. It uses data virtualization to query the source data and then its engine to index and generate the results. Models can be pre-aggregated, pre-indexed, materialized, and cached. This provides better performance and compute cost savings versus directly querying the data sources.
Datameer Spotlight is a virtual data management platform with a distributed query engine and optimizer, self-service tools, and a collaborative data catalog that gives analytics teams easy access to all enterprise data assets—regardless of type or location. Spotlight flips the analytics data model on its head, eliminating the need for costly ETL and data replication for analytics and wasted time waiting for data.
Spotlight lets analysts quickly discover, create, share, and collaborate on data assets, building knowledge and trust along the way. It provides a single place where analytics teams can quickly discover all these analytics assets and understand which best solve their problem to produce actionable results promptly. It provides an environment where teams can share and reuse assets, collaborate to form new assets, and increase knowledge using familiar social media-like features and AI-augmented information about asset utilization.
Under the covers, Spotlight provides a scalable, performant virtual data query and access environment that brings together all the data analysts need without the need to ETL or replicate data. Spotlight is a SaaS managed service that does not require IT administration and uses patent-pending optimization techniques and an elastic compute architecture to maintain performance and scale.
Spotlight increases the ROI on your data, BI, and analytics investments. It works with any data source you may have – databases, data warehouses, data lakes, files, and applications – and any BI, analytics, and data science tool used. Best of all, the virtual query engine eliminates the need for ETL, allowing you to lower your data integration costs.
At its core, AtScale is an OLAP engine with multi-dimensional modeling tools and semantic layer that uses data virtualization to eliminate the need to ETL data into the OLAP models. Spotlight is purpose-built to accelerate any analytics (not just structured OLAP) with a highly optimized virtual data management server, a broad suite of connectivity to any data, and a collaborative catalog for easy data discovery.
Spotlight and AtScale have a few things in common:
Beyond this, Spotlight offers several key differentiated capabilities that allow it to facilitate faster analytics of any kind:
AtScale has a minimal set of data connectors focused on databases, data warehouses (on-premises and cloud), and data lakes. The product has 12 data connectors plus file connectors in total. This forces companies to ETL and consolidate data from other sources. The primary use cases for AtScale are not for virtualizing data but rather as an OLAP query accelerator on a data lake or cloud data warehouse.
Spotlight has over 200 connectors to a wide variety of data sources: databases, data warehouses, cloud data warehouses, analytical data sources, SaaS applications, cloud services, and more. Spotlight’s objective is to facilitate cloud-based virtual access to ANY and ALL of your data, supporting analytics of any form, not just structured OLAP.
While AtScale claims to support self-service analytics, the reality is that data experts must do all the modeling in AtScale with highly specific skills in multidimensional modeling techniques and building star- and snowflake-schemas. AtScale explicitly calls this “data engineering.” Most users of AtScale will only use the models built by the data engineering experts and will not perform any ad-hoc modeling for data discovery.
Spotlight is designed for true self-service data modeling to facilitate ad-hoc data discovery on any data, as opposed to pre-defined and structured OLAP models. It has a codeless, visual approach to modeling through its intuitive point-and-click interface requiring no specialized skills and can be performed by any analyst. Spotlight introspects and catalogs the objects from your sources, lets you discover the right assets for your analysis, and has AI-driven recommendations to guide the modeling process.
AtScale provides a simple catalog view where users can browse and find cubes to use. It provides basic technical metadata information about the cubes – names, dimensions, measures. It offers no additional data cataloging capabilities or the ability to search for specific datasets.
Spotlight has a rich catalog that captures valuable information about datasets and models, both system generated and user added. Beyond the technical metadata, users can view system-generated metadata such as owners, collaborators, source information, timestamps, query usage, and more. Users can also add and view additional information such as descriptions, tags, certification levels, custom properties, and other business level metadata. Spotlight provides a rich faceted search of its catalog, enabling fast, easy discovery of the right datasets for the job.
AtScale does provide shared “projects” that allow data experts to work together and build multidimensional models. Power users can be given editing rights to a model and can publish it when completed.
As previously mentioned, Spotlight contains a very rich data catalog and semantic layer that allows users to share what they know about the data, including tags, descriptions, and comments. They can also certify assets, provide custom properties, and add business-level metadata.
Spotlight also allows users to work together in shared workspaces to collaborate, add knowledge (tags, properties, etc.), and create additional shared assets. It supports social media-like features around assets. The owner can add collaborators, and users can request to follow or fully collaborate on an asset. Once added, followers will receive notifications in their activities inbox. Collaborators can fully exchange comments and notifications on assets.
AtScale provides its own user-, group-, and role-based security and, in general, has very good security features. It requires integration with LDAP, Active Directory, or a cloud-based identity program and supports SSO. At the data level, it supports encryption, masking, and dimension-level security.
AtScale DOES NOT validate user access rights to data objects with the originating data source, instead of having a single set of credentials for each connection. This forces administrators to re-implement data-level access controls in AtScale and creates potential data security holes.
Spotlight provides a deep set of security capabilities, including:
Spotlight is intentionally designed NOT to replicate already-in-place access control mechanisms in place for the data. Metadata visibility controls in Spotlight and data assess controls independent of each other. The data source maintains access control to the data, and the security credentials of the Spotlight user are passed down to the source at query execution time, eliminating potential conflicts and loopholes. Even when it caches datasets, Spotlight always re-authenticates with the originating data sources before permitting access, maintaining consistent security across all data.
Data governance goes beyond security, allowing organizations to understand what data assets are made up of, their meaning, and how they are being used. AtScale provides simple governance capabilities, specifically technical metadata, data lineage, and usage statistics.
Spotlight contains several features to maintain governance, including:
AtScale provides automation under the covers to provide improved performance and response time. It uses machine learning algorithms to monitor user activity and manage “acceleration structures” (aggregate tables) to automatically optimize query performance. Certain acceleration structures are automatically generated and cached, while others are pre-built at intervals when the cubes are built/refreshed. What AtScale caches are the cubes, NOT the raw data, so performance is accelerated on only the structured aggregations defined in the cube.
Similar to AtScale, Spotlight provides automated facilities to accelerate performance. Spotlight is a SaaS managed service that requires no operational administration, and under the covers uses managed Spark clusters that are elastic and can auto-scale to your environment’s needs. Like AtScale, datasets can be materialized and cached, either in base form or aggregated, to further accelerate query performance and response time.
Spotlight lets you simplify and scale-out your data management for any data and form of analytics, not just structured OLAP. It virtually connects directly to over 200 different data sources, offering much broader access to data. Spotlight also provides visual, code-free data modeling and a much richer data catalog and semantic layer, facilitating faster discovery, knowledge-sharing and collaboration, and better data governance. And the auto-scaling elastic service and caching provide a better way to accelerate any query, not just ones pre-defined in the cubes.
|Datameer Spotlight||AtScale Enterprise|
|With connectors to over 200 different sources, Spotlight lets teams work with ANY data for any type of analytics.||AtScale only has connectors to databases, data warehouses, and data lakes, limiting your analysis.|
|Spotlight provides true self-service data modeling for any form of analytics, with a code-free, visual data modeling environment.||AtScale requires data engineering using specialized skills to create multidimensional models and star- and snowflake-schemas.|
|Spotlight contains a rich data catalog and semantic layer with physical metadata, tagging, descriptions, comments, custom properties, business-level metadata, and usage information. It allows users to easily search across names, descriptions, tags, custom properties, and any item in the catalog.||AtScale only provides a very basic catalog and semantic layer based on technical metadata and provides no search facilities.|
|Spotlight allows users to work together in shared workspaces to collaborate, add knowledge (tags, properties, etc.), and create additional shared assets. It also supports social media-like features around assets.||AtScale provides only rudimentary collaboration with shared projects.|
|Spotlight provides complete, end-to-end enterprise security and pushes down data access controls to maintain data source security integrity.||AtScale has user- and role-based security that requires you to replicate security controls and create potential security holes.|
|To maintain good governance, Spotlight maintains multiple forms of metadata about assets, full lineage, and usage auditing. It maintains multiple system-level and user-set properties such as status, which can define an asset's state.||Dremio has limited data governance features (physical metadata, lineage, and usage monitoring).|
|Spotlight is a SaaS managed service that requires no operational administration, particularly for performance and scale. Under the covers are managed, elastic Spark clusters that can auto-scale to your needs and caching for performance optimization.||AtScale provides automated performance optimization via pre-aggregation and materialization, but only on aggregated views, not raw source data.|