What you Need to Understand about Snowflake Data Catalog

In this article, we will discuss the concept of Snowflake’s data catalog, its benefits, and some frequently asked questions.

Disclaimer: This article does not refer to the ‘snowflake data catalog’ as metadata lineage tools used  for discovery and cataloging of datasets .

In this article, the snowflake data catalog is referred to as an object, a place – a data marketplace.

A marketplace where users can instantly browse to discover and securely retrieve data into their snowflake ecosystem.

That said…

Let’s dive in!

Assume you’re a data scientist and need to conduct a hypothesis test on a dataset.

For this, you require last year’s forecast data in order to train your model.

How do you obtain this data ?

Traditionally, you or your team would reach out to data brokers or a trusted provider in this niche to purchase and import this data.

However, things can get very time-consuming and resource intensive, quickly.

This is because you may need to allocate some of your technical bandwidth for pre-processing & importing.

With Cloud data catalogs like Snowflake data marketplace, all this is now in the past.

So, What is a Data Marketplace?

A data marketplace, also known as a data catalog, is an online store where data can be purchased.

Data marketplaces often provide data for various markets and sources, from business intelligence and advertising to demographics, weather, etc.

Three main categories of data catalogs are:

a.) Personal Data Marketplaces

b.) B2B Data Marketplaces

c.) IoT Data Marketplaces

Types of Data Catalogs

1. Centralized Data Marketplaces:

Centralized data marketplaces are central data marketplaces that provide self-service data preparation and provisioning through augmented data catalogs. They allow end users to locate and deliver reliable data to their preferred target location.

Examples include Snowflake data marketplace, AWS data marketplace, Oracle data marketplace, TechsaleratorDataShop, etc.

2. Decentralized Data Marketplaces:

Decentralized data marketplaces rely on the blockchain to ensure security and privacy for data buyers and sellers. They are decentralized so that no single party can access the user data.

In addition, data marketplaces frequently have policies prohibiting the purchase of specific data types when the buyer intends to use the data for “blacklisted” use cases.

Some examples include Advaneo data marketplace, Streamr, ocean protocol, etc.

Our focus today is on Snowflake’s data catalog, a.k.a. The Snowflake data marketplace.

So, What’s The Snowflake Data Marketplace?

The Snowflake Marketplace offers you a single spot to access the data as a Snowflake user, irrespective of your industry.

The marketplace provides a medium for data providers to monetize their data and offer it to consumers.

Snowflake provides accessibility to a wide range of third-party data. Using Snowflake Secure Cloud Data Sharing, users can query the datasets immediately after purchase – without any modification or configurations.

Snowflake marketplace data has over 650 datasets and is utilized by organizations of all sizes – government, corporate, and market intelligence agencies.

What makes the Snowflake data catalog Unique?

Answer: The Snowflake Platform

The Snowflake data catalog is a feature of the Snowflake DW. This means that the Snowflake marketplace inherits the power of the Snowflake data cloud, which makes it unique.

Let’s take a look at some of the features that make the Snowflake market place the first choice for Snowflake users.

1. Snowflake’s multiple cloud deployment options

With Snowflake DW, customers can choose their cloud providers before deployment, i.e., AWS, Azure, or GCP.

2. Snowflake’s unlimited storage and compute power

Since the Snowflake data catalog is cloud-only, it takes advantage of the infinite scalability and elasticity that Snowflake offers, allowing you to store more and scale up or down as needed.

3. Data platform as a service

Snowflake marketplace is a true Saas offering.

This means no hardware (virtual or physical ) to select, install, configure or manage.

With Snowflake, there is no software to install or ETL pipelines to build when importing data from the marketplace.

Data Sharing and consumption within Snowflake’s marketplace takes place within seconds using Snowflake’s secure cloud data-sharing features.

4. Immediate Access to Purchased Listings

Once you purchase a data listing from the marketplace, you can immediately begin your SQL coding and analysis.

You can query your acquired datasets using Snowflake-enriched SQL commands.

FAQs On Snowflake’s Data Marketplace

Question #1: Is Snowflake Data Marketplace “open-source” or “closed-source”?

• Open Source

Question #2: Who are some of the biggest partners Snowflake Data Marketplace works with?

• Snowflake has tons of data providers in different business segments. On the marketplace, providers can earn the status of “featured providers.” Some of Snowflake’s most significant partners are Crunchbase, Safegraph, weathersource, Factset, etc.

Question #3: How do Snowflake Data Marketplace and its partners load their data services onto the web platform (i.e., ETL, CSV, API, etc.)?

• Snowflake Data Marketplace Providers do not necessarily load their data services into the Snowflake marketplace.
• The datasets (possibly accessed through APIs) already exist within the customer’s Snowflake account.
• If a customer and their dataset(s) meet the marketplace provider requirements, their data listings can be shared on the marketplace via Snowflake’s secure data-sharing cloud technology (Direct Access).

Question #4: How exactly do end users receive and extract the data after purchasing a data service? (e.g., does it come in the form of a CSV File)?

Consumers who need to purchase listings from the Snowflake data marketplace can do it in one of two ways.

Via Standard Listings:

With Standard/free listings, providers on the marketplace offer non-proprietary data sources for free on the marketplace.

 

Consumers can directly access and query these data listings in seconds via Snowflake’s secure no-Copy Data Sharing (Direct Access Pattern) cloud technology.

Via Personalized Offerings:

On the other hand, providers can offer personalized data listing as premium services.

 

 

 

 

Although consumers can query the sample versions of a premium data listing, they would be required to fill out a request form and sign an agreement with the provider to use the accurate premium data.

Question #5: Which Cloud Delivery Model does Snowflake Data Marketplace use (IaaS, PaaS, SaaS, or a mix of the 3)?

The Snowflake data cloud, which also functions as a data PaaS, enables you to operate all of your important data tasks on a single platform, including data sharing, data lakes, data warehouses, purchasing on the marketplace, and custom development options.

Snowflake Marketplace leverages the Snowflake data cloud platform and can therefore be considered an IaaS or, in some cases, an Saas.

It can be considered an IaaS.

Datameer, Your Snowflake Aide

Datameer, a modern analytics platform for Snowflake, provides a multi-persona data transformation platform that brings together your entire team — data engineers, analytics engineers, analysts, and data scientists —on a single platform to transform and model data directly in Snowflake collaboratively.

Your data costs may rise due to your over-reliance on your data engineers to manage your data.

Reduce your dependency on data engineering teams and invest in a toolset that works for all code levels.

What better way to derive quick insights from acquired marketplace data – within minutes?

Choose Datameer today!