Unlocking dbt

This book shows how dbt is used to build data transformation pipelines that enable dependency management and allow for version control and automated testing. It explains how dbt is revolutionizing data transformation and the advantages that a command-line tool like dbt provides over and above the use of database stored procedures and other ETL and ELT tools that handle data transformations. You’ll see how to create custom-written transformations through simple SQL SELECT statements, eliminating the need for boilerplate code and making it easy to incorporate dbt as the transformation layer in your data warehouse pipelines. Additionally, you will learn how dbt enables data teams to incorporate software engineering best practices such as code reusability, version control, and automated testing into the data transformation process. Unlocking dbt walks you through using dbt to establish a project, build and modularize SQL models, and execute jobs in away that is easy to maintain and scale as your data ecosystem matures. You’ll begin by establishing and configuring a project, a process covered using both dbt Cloud and dbt Core, so that you can confidently stand up a project using either platform. From there, you’ll move into building transformations with peace of mind that your project will scale appropriately as you continue to develop it. After learning the basics needed to get started, you’ll continue to build on that foundation by looking at the unique ways in which dbt combines SQL with Jinja to take your code beyond what is capable in normal SQL. You will learn about advanced materializations, building lineage in your data flows, the unlimited potential of macros, and so much more. This book also explores supported file types and the building of Python models. Rounding things out, you will learn features of dbt that will assist you in making your transformation layer production ready. These includehow to implement automated testing, using dbt to generate documentation, and running CI/CD pipelines. What You Will Learn Understand what dbt is and how it is used in the modern data stack Set up a project using both dbt Cloud and dbt Core Connect a dbt project to a cloud data warehouse Build SQL and Python models that are scalable and maintainable Configure development, testing, and production environments Capture reusable logic in the form of Jinja macros Incorporate version control with your data transformation code Who This Book Is For Current and aspiring data professionals, including architects, developers, analysts, engineers, data scientists, and consultants who are beginning the journey of using dbt as part of their data pipeline’s transformation layer. Readers should have a foundational knowledge of writing basic SQL statements, development best practices, and working with data in an analytical context such as a data warehouse.