dbt: Your SQL, Promoted to Software
Why analysts and engineers stopped writing throwaway SQL scripts and started building data transformations the way we build code — with version control, tests, and a dependency graph.
Analytics Engineering with dbt
Turn SQL into a real software project — models, tests, and a DAG — with dbt, verified end to end on DuckDB.
-
-
From Nothing to `dbt run` in One Sitting
Install dbt Core, point it at DuckDB, and build your first model — the whole loop, with every command and its real output.
-
ref() Is the Whole Trick
One function turns a folder of loose queries into a dependency graph dbt can order, parallelize, and reason about. Meet the DAG, and the staging-to-marts layering everyone uses.
-
A dbt Project Is a Product, Not a Junk Drawer
How to organize staging, intermediate, and mart models so the project stays readable after the first dozen files.
-
Where Data Comes From: Sources and Seeds
Two ways raw data enters a dbt project — sources that name tables someone else loaded, and seeds that load small CSVs you control — plus freshness checks on the stuff you don't.
-
View, Table, or Something Cleverer
The same SELECT can become a view, a table, or an inlined CTE. Choosing the materialization is choosing your trade between build cost and query cost — here's how to pick.
-
Don't Rebuild What Didn't Change
When a table gets too big to recompute from scratch every run, incremental models process only the new rows. The materialization, the is_incremental() guard, and the escape hatch when logic changes.
-
Incremental Models After the Happy Path
Late-arriving data, composite keys, microbatching, adapter strategies, and the edge cases that decide whether an incremental model stays trustworthy.
-
Tests That Read Like Assertions
dbt's data tests turn a line of YAML into a query that fails the build when your data is wrong — unique keys, non-null columns, referential integrity — plus the config knobs that decide how loudly each one fails.
-
Data Quality Is More Than not_null
How to use thresholds, failure storage, packages, custom generic tests, and project health checks without turning tests into noise.
-
Unit Tests for SQL, Finally
Data tests check the rows flowing through your models. Unit tests check the logic itself — feed a model fixed inputs, assert the exact output, and catch a broken CASE statement before it ever touches real data.
-
SQL That Writes SQL: Jinja and Macros
The templating layer under every dbt model. Jinja expressions, control flow, macros you write once and reuse, and the dbt_utils package that has already written the ones you need.
-
The File Everyone Edits and Nobody Reads
dbt_project.yml key by key, the + config cascade and its merge rules, hooks that fire around every build, and run-operation — the macros that do things instead of returning SQL.
-
Packages Are How dbt Projects Borrow Good Ideas
How package installation, locking, dispatch, private dependencies, and the dbt package ecosystem fit into a production project.
-
dbt build Is a Graph Command
Selectors, graph operators, indirect test selection, artifacts, and the mechanics behind the one command you eventually run everywhere.
-
Remembering What Changed: Snapshots
Source systems overwrite the past — a customer's plan flips from free to plus and yesterday's value is gone. Snapshots capture each change as history, giving you slowly changing dimensions with almost no code.
-
Contracts Are How dbt Models Grow Up
Model contracts, constraints, versions, access levels, groups, and the governance features that keep a popular model from becoming a liability.
-
One dbt Project, Many Places to Run It
Profiles, targets, custom schemas, environment variables, secrets, and the dev/prod split that keeps analytics work from colliding.
-
Docs, Lineage, and Shipping It
The last mile: turn your project into living documentation with a real lineage graph, then run it in production — environments, CI, and building only what changed.
-
Shipping dbt Without Crossing Your Fingers
Slim CI, state artifacts, dbt clone, production jobs, orchestration handoffs, and the monitoring loop around dbt builds.
-
The Semantic Layer Is Where Metrics Stop Wandering
A practical orientation to dbt's semantic layer, MetricFlow, semantic models, entities, measures, dimensions, and when a team actually needs them.