Low-Code Alternative and Competitor to dbt

Dbt , which is short for data built tool, is a data transformation tool that enables data analysts and engineers to transform, test, and document data in their cloud data warehouse.  With dbt, data teams work directly within the warehouse to produce trusted datasets for reporting, ML modeling, and operational workflows.

The primary language for dbt is their own SQL dialect.  Anyone in the organization – typically data engineers and data analysts – who knows SQL can create SQL-based data models and link them into a pipeline.  Dbt’s SQL dialect is designed specifically for data transformation, replacing boilerplate DDL/DML with simple SQL SELECT statements that infer dependencies, build tables and views, and run models in order.

The dbt tool not only lets users define data models but also offers a workflow that lets teams quickly and collaboratively deploy data transformation code following software engineering best practices like modularity, portability, CI/CD, and documentation.  Under the covers, the product uses Git for version control, sharing, and collaboration.

First and foremost, organizations use dbt to transform raw data loaded into their cloud data warehouse into a consumable analytics form.  This is the “T” in their ELT (Extract, Load, and Transform) process.  All models are stored in the cloud data warehouse, and execution of the models is performed using the cloud data warehouse’s compute capabilities.

Organizations also use dbt to add software development best practices to their data transformation processes.  As mentioned above, dbt also provides facilities to enable a software-development-like workflow.  This includes the typical three phases of the software development lifecycle:

• Development – Write modular SQL models with SELECT statements, a ref() function, and macros for modularity,
• Testing and Documentation – the ability to interactively test data models and to auto-generate “documentation” about the models,
• Version control – Git-enabled version control enables sharing, a return to previous states, and the ability to deploy safely using dev environments.

The use of Git under the covers allows organizations to use existing GitHub repositories for their data transformation models and to keep all their software libraries, including data models, all under one roof.

Datameer is a powerful SaaS data transformation platform that runs in Snowflake – your modern, scalable cloud data warehouse – that combines to provide a highly scalable and flexible environment to transform your data into meaningful analytics.  With Datameer, you can:

• Allow your non-technical analytics team members to work with your complex data without the need to write code using Datameer’s no-code and low-code data transformation interfaces,
• Collaborate amongst technical and non-technical team members to build data models, and the data transformation flows to fulfill these models, each using their skills and knowledge
• Fully enrich analytics datasets to add even more flavor to your analysis using the diverse array of graphical formulas and functions,
• Generate rich documentation and add user-supplied attributes, comments, tags, and more to share searchable knowledge about your data across the entire analytics community,
• Use the catalog-like documentation features to crowd-source your data governance processes for greater data democratization and data literacy,
• Maintain full audit trails of how data is transformed and used by the community to further enable your governance and compliance processes,
• Deploy and execute data transformation models directly in Snowflake to gain the scalability you need over your large volumes of data while keeping compute and storage costs low.

Datameer provides a number of key benefits for your modern data stack and cloud analytics, including:

• Creating a highly efficient data stack that reduces your data and analytics engineering costs,
• Allowing you to share the data transformation workload across your broader data and analytics team,
• Fostering collaboration among the data and analytics team to produce faster, error-free projects,
• Efficiently using your Snowflake analytics engine for cost-effective data transformation processing,
• Enabling you to crowd-source your data governance for more effective and efficient governance processes, and
• Improving data literacy to expand knowledge and effective use of your data.

At the surface level, both Datameer and dbt are data transformation tools – the T in your ELT data stack.  Both allow you to take raw data loaded from your data sources into your cloud data warehouse and transform the data into an analytics-ready form using the compute and storage of your cloud data warehouse.  Both tools also help to add best practices to your data transformation process through capabilities such as data documentation.

However, Datameer offers a number of distinct differences and capabilities that go beyond what dbt offers, including:

• A hybrid code/low-code/no-code user interface
• Multi-persona toolset
• Easier data enrichment
• Collaboration
• Deep data profiling
• Richer, catalog-like data documentation
• Google-like faceted search and discovery

Datameer offers a hybrid user experience that has three different user interfaces: a coding UI in SQL, a low-code, formula-driven spreadsheet-like UI, and a no-code, graphical UI.  Each offers distinct ways to transform your data using different skills.  Dbt only offers a single, SQL-coding and template language (Jinja) programming IDE and articulates that vision in this quote from their website:

At dbt Labs, we have developed strong opinions on how companies should practice analytics.  Specifically, we believe that code, not graphical user interfaces, is the best abstraction to express complex analytic logic.

With Datameer, data transformation models can be mixed and matched within a data flow using the various three interfaces.  Under the covers, models created by any of the three interfaces are translated into SQL views inside of your cloud data warehouse.  And the overall data flow chain is maintained by Datameer by linking the views together.

Because Datameer offers three different UIs, any persona – data engineer, analytics engineer, data analyst, data scientist – can use the Datameer toolset with their existing skill set.  Datameer also fosters collaboration between the various personas.  With dbt, you need to have SQL skills and some simple programming skills, which really only target the data engineer persona.

Datameer’s spreadsheet-like UI and its ability to easily add file-based data make for a much easier path to enrich data, especially for data analysts who may not be highly SQL-savvy.  Adding new, enriched columns is led by an easy, wizard-driven formula builder.  With dbt, any data enrichment must be done with SQL formulas and coded in SQL.

Datameer maintains a rich set of both auto-generated and user-created data documentation that allows teams to easily discover and share knowledge about data models. The solution automatically documents system-level metadata and properties.  Users can further enrich the information with wiki-style descriptions, custom properties and attributes, tags, and comments. Dbt only offers simple, auto-generated documentation that is taken from comments within their SQL code.

Teams can use shared workspaces to share, reuse, and collaborate around models to speed projects, divide up the workload, and ensure models are designed properly the first time. Different model types can be mixed-and-matched into larger dataflows for maximum flexibility and reuse.  Catalog-like features such as comments, custom properties, tags, and others also allow teams to easily share information about the data and transformation.  Dbt only offers limited collaboration via model sharing and reuse via references.

Datameer maintains a deep data profile that is expressed visually to users so they can see the full shape and contents of the data as they transform it.  This easily allows users to identify invalid, missing, or outlying fields and values, as well as the overall shape of the data.  Dbt only offers a snapshot of the data and does not maintain detailed data profiles, with users having to blindly write SQL with limited visibility into the data.

Datameer offers a Google-like faceted search that allows users to discover data models and datasets.  The search covers all the information captured on the data, including system-level metadata and properties, descriptions, custom properties and attributes, tags, and comments.   Dbt offers no search and discovery capabilities having just a simple project browser and IDE metaphor.

At the most basic level, Datameer and dbt share common characteristics around data transformation and helping teams apply engineering best practices for data transformation and engineering.  From there, the two products deviate, with Datameer offering a much more inclusive and easier user experience that supports multiple personas, collaboration among team members, and a much deeper set of searchable, catalog-like data documentation.

Are you interested in seeing Datameer in action?  Contact our team to request a personalized product demonstration .