Broaden and simplify your data management for Looker to deliver faster, more in-depth, and trusted insights. Spotlight makes your entire data landscape look like a single database, providing virtual access to over 200 data sources – no more ETL. Spotlight offers visual, codeless data modeling and easy discovery, knowledge-sharing, and collaboration to allow your Looker teams to deliver faster results and build data literacy.
Looker is an emerging leader in BI and analytics, with a modern data analytics platform designed for agility, ease of use, and for the cloud. The objective of the platform is to enable broader access to data and enable more data-driven business decisions. The company has over 2,000 customers and was recently acquired by Google Cloud.
The Looker Server provides key data modeling and management services that help model data for consumption for analysis, reports, and dashboards, then serve up that data. The preferred model is for data analysts, who are experts in the data, to perform the data modeling. Then business analysts, who will analyze the data, will use the data models for their analysis.
While Looker supports connections to many different databases, the model recommended by Looker is for data to be ETL’ed into a central database (a data warehouse, data mart, or cloud data warehouse) where it is secured, managed, and accessed. Looker database connections can then access the data. Looker data models, coded in LookML, describe the data both physically (the database schema) and logically – how views map to the schema and appear to the BI users.
When creating a new BI project, Looker provides you with a good head start, so you don’t need to code LookML from scratch. When you declare what database connection to use in a project, Looker will introspect the database and automatically pre-define views and explores (logical views) to use. From there, you can delete the views and explores you don’t need and add additional explores.
The LookML logic is spread across multiple files – a project file, a model file (contains the explores), and view files. The files are held and managed on the server, and each project is stored as a git repository.
Datameer Spotlight is a virtual data management server and catalog that gives analytics teams easy access to all enterprise data assets—regardless of type or location. Spotlight flips the analytics data model on its head, eliminating the need for costly ETL and data replication for analytics and wasted time waiting for data.
Spotlight lets analysts quickly discover, create, share, and collaborate on data assets, building knowledge, and trust along the way. It provides a single place where analytics teams can quickly discover all these analytics assets and understand which best solve their problem to produce actionable results promptly. It provides an environment where teams can share and reuse assets, collaborate to form new assets, and increase knowledge using familiar social media-like features and AI-augmented information about asset utilization.
Under the covers, Spotlight provides a scalable, performant virtual data query and access environment that brings together all the data analysts need without the need to ETL or replicate data. Spotlight is a SaaS managed service that does not require IT administration and uses patent-pending optimization techniques and an elastic compute architecture to maintain performance and scale.
Spotlight increases the ROI on your data, BI, and analytics investments. It works with any data source you may have – databases, data warehouses, data lakes, files, and applications – and any BI, analytics, and data science tool used, Looker included. Best of all, the virtual query engine eliminates the need for ETL, allowing you to lower your data integration costs.
Spotlight appears as just another database to Looker, using a standard JDBC/ODBC connection. Through that connection, Looker projects can consume the data Spotlight has modeled and serves up. Also, from within Spotlight, users can directly open Spotlight views in Looker with a single click for rapid analysis and data discovery.
The three key advantages Spotlight provides are: (a) that it requires no ETL, (b) it can access hundreds more data sources, and (c) data from different sources can be easily combined into a single model without any coding. Spotlight virtually queries data, leaving it securely in place and eliminating the need for complex and costly ETL and data replication.
Let’s explore these eight ways Spotlight can turbocharge your Looker environment:
Looker has a limited set of data connectors, only supporting databases and analytic data sources. To use data not supported by one of their connectors, you need to ETL it into a local database. Spotlight has over 200 connectors to a wide variety of data sources: databases, data warehouses, cloud data warehouses, analytical data sources, SaaS applications, cloud services, and more.
Looker’s recommended mode of operation is for you to ETL data into an analytics data source (database, data mart, or data warehouse) using one of their ETL partners. From there, you connect Looker to an analytical data source and perform your analysis. This is the only mode of operation if you want to analyze data residing outside of databases (applications, SaaS applications, cloud services, etc.).
Spotlight makes your entire data landscape look like a single data source to Looker, eliminating ETL processes’ need to centralize the data. Spotlight provides live, virtual access to any of your data, regardless of where it resides, so your team can analyze in minutes and get instantaneous value.
While Looker does a good job automating the initial modeling process, any Looker data models’ customization requires custom coding in the LookML markup language. Also, your LookML data modeling logic is disjointed over hundreds of files, making it challenging to find the desired logic.
Spotlight has a codeless, visual approach to modeling through its intuitive point-and-click interface. Spotlight introspects and catalogs the objects from your sources, lets you search and discover the right assets for your analysis, and has AI-driven recommendations to guide the modeling process.
Looker can only join and unify data within a single data connection – not across data sources. This is one reason it recommends ETL’ing your data into a centralized source.
Because Spotlight makes your entire landscape look like a single data source, it can unify data from any of your sources, regardless of type and location. You can quickly bring together disparate sources into a single model without moving the data, such as customer data from Salesforce, marketing data from Marketo, and orders data from internal systems.
As mentioned, Looker spreads your LookML data modeling logic across many different files in your Looker Server. There is no catalog of data models, no means to search for data models, and no mechanisms to add and share knowledge to the data models.
Spotlight lets users provide information about your data, allowing users to add tags, descriptions, and comments, certify assets, and provide custom properties and business-level metadata. Spotlight supplements this by capturing information on where an asset is referenced, who is using it, and how often it is used. Users can search for assets across technical metadata and all of the added knowledge.
Spotlight allows users to work together in shared workspaces to collaborate, add knowledge (tags, properties, etc.), and create additional shared assets. It also supports social media-like features around assets. The owner can add collaborators, and users can request to follow or fully collaborate on an asset. Once added, followers will receive notifications in their activities inbox. Collaborators can fully exchange comments and notifications on assets.
Spotlight works with all your BI, analytics, and data science tools, Looker included. Use Spotlight to create a unified analytics data repository across your entire analytics community. Share data assets between your reporting, analytics, and data science teams. Take advantage of Spotlight capabilities to get your community collaborating and sharing their knowledge to improve data literacy.
Looker Server comes in three pre-packaged flavors: a single server and two- and eight-server clusters. Using the clustered versions requires setting up and administering complex load balancing processes. User sessions can be spread across a cluster, but executing queries and data logic within a session, cannot be scaled across a cluster.
Spotlight is a SaaS managed service that requires no operational administration, particularly for performance and scale. Under the covers are managed Spark clusters that are elastic and can auto-scale to your environment’s needs to maintain high performance and fast response time.
Spotlight lets you simplify and scale-out your data management for Looker. It makes your entire data landscape look like a single database to Looker and virtually connects directly to over 200 different data sources, eliminating the need for ETL. Spotlight offers visual, code-free data modeling and the ability to unify data across various sources. Easy search, discovery, knowledge-sharing, and collaboration let teams find and use the right data assets for their analysis to deliver faster, trusted insights.
|Datameer Spotlight||Looker Server|
|Spotlight makes your entire data landscape look like a single database, letting you work with over 200 different data sources.||Looker lets you create individual connections to a limited set of sources: databases and analytic data sources.|
|Spotlight provides live, virtual access to any and all of your data sources, regardless of location, without any ETL, extracts, or data replication.||Looker's recommended mode of operation is to ETL data to a central repository for analysis.|
|Spotlight provides a visual data modeling environment, with AI-driven guidance, that requires no coding, all contained in a single, searchable repository.||Looker data modeling requires coding in LookML, with some initial automation, and logic is spread across many files.|
|Spotlight lets you unify data across different data sources into a single model for more complete analytics views.||Looker can only unify data within a single data source connection.|
|Spotlight provides a full suite of knowledge sharing, including tagging, descriptions, custom properties, a business glossary, and more.||Looker does not offer cataloging, the ability to search for data models, or knowledge-sharing.|
|Spotlight allows users to work together in shared workspaces to collaborate, add knowledge, and create additional shared assets. Social media-like features let collaborators exchange comments and receive notifications on assets.||Looker allows users to share and reuse LookML data models but offers no collaboration features.|
|Data assets in Spotlight can be consumed and used by ANY of your BI, analytics, and data science tool, facilitating collaboration and reuse across your entire analytics community.||Assets modeled in Looker Server can only be used by Looker clients and are not sharable with other tools.|
|Spotlight is a SaaS solution that includes automated, elastic compute clusters for scalability and performance.||Looker offers single node instances and 2- and 8-way clustered nodes. Setting up and administering clusters is complex, and only user sessions can be scaled across a cluster, not data processing.|