Data Warehouse# Onchain Data Pipelines: Blockbatch Load Ad-hoc Blockbatch Queries Data Loading Jobs Loading by Blockbatch Loading by Date and Chain Load Job Specification Location Naming convention Job Specification Data Readiness, Markers, and Job Idempotency Building Data Pipelines Execution Prototyping and Backfilling Scheduling Monitoring General Purpose Data Pipelines: Transforms Transform Groups Transform Specification Directory Structure and Naming Convention Transform Execution Model Building Data Pipelines Execution Prototyping and Backfilling Scheduling Markers Monitoring Comparison to dbt and SQLMesh What dbt and SQLMesh offer Why we didn’t chose dbt or SQLMesh Lineage Testability SQL First (no indirection) When dbt and SQLMesh might be a better fit
