Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Categories

      List of categories.

    • Tags

      List of tags.

    • Series

      List of series.

    • Archives

  • Projects
  • Slides

/

  • Github Linkedin RSS

  • Font Size
  • Palette
  • Mode
  1. Home
  2. Tags
  3. dbt

Guide to Running DBT in Production

Guide to Running DBT in Production
September 13, 202423 min read Data-Engineering Dbt Guide for ProductionBigQueryContinuous DeliveryContinuous IntegrationDbtGitHub Actions

Deploying a dbt project to dev and prod on BigQuery, covering slim CI, unit tests and a write audit publish step that builds on a cloned dataset.

Read More: Guide to Running DBT in Production

DBT CI/CD Demo with BigQuery and GitHub Actions

DBT CI/CD Demo with BigQuery and GitHub Actions
September 5, 202418 min read Data-Engineering Dbt Guide for ProductionBigQueryContinuous DeliveryContinuous IntegrationDbtGitHub Actions

GitHub Actions gives a dbt project on BigQuery a slim CI run on pull requests and a deploy job that publishes the project as a container image.

Read More: DBT CI/CD Demo with BigQuery and GitHub Actions

Data Build Tool (dbt) Pizza Shop Demo - Part 6 ETL on Amazon Athena via Airflow

Data Build Tool (dbt) Pizza Shop Demo - Part 6 ETL on Amazon Athena via Airflow
March 14, 20248 min read Data-Engineering Dbt Pizza Shop DemoAmazon AthenaApache AirflowAWSDbtDockerPython

Orchestrate the Amazon Athena and Iceberg pizza shop dbt project with Apache Airflow, running the dimension and denormalised fact table builds.

Read More: Data Build Tool (dbt) Pizza Shop Demo - Part 6 ETL on Amazon Athena via Airflow

Data Build Tool (dbt) Pizza Shop Demo - Part 5 Modelling on Amazon Athena

Data Build Tool (dbt) Pizza Shop Demo - Part 5 Modelling on Amazon Athena
March 7, 202416 min read Data-Engineering Dbt Pizza Shop DemoAmazon AthenaAWSDbtDockerPython

Model pizza shop data as Apache Iceberg tables transformed on Amazon Athena with dbt, using array and struct types to denormalise the fact table.

Read More: Data Build Tool (dbt) Pizza Shop Demo - Part 5 Modelling on Amazon Athena

Data Build Tool (dbt) Pizza Shop Demo - Part 4 ETL on BigQuery via Airflow

Data Build Tool (dbt) Pizza Shop Demo - Part 4 ETL on BigQuery via Airflow
February 22, 20249 min read Data-Engineering Dbt Pizza Shop DemoApache AirflowBigQueryDbtDockerGCPPython

Orchestrate the BigQuery pizza shop dbt project with Apache Airflow, running the SCD type 2 dimension tables and the denormalised fact table.

Read More: Data Build Tool (dbt) Pizza Shop Demo - Part 4 ETL on BigQuery via Airflow

Data Build Tool (dbt) Pizza Shop Demo - Part 3 Modelling on BigQuery

Data Build Tool (dbt) Pizza Shop Demo - Part 3 Modelling on BigQuery
February 8, 202416 min read Data-Engineering Dbt Pizza Shop DemoBigQueryDbtDockerGCPPython

Model pizza shop data on Google BigQuery with dbt, keeping SCD type 2 dimensions and denormalising the fact table with nested and repeated fields.

Read More: Data Build Tool (dbt) Pizza Shop Demo - Part 3 Modelling on BigQuery

Data Build Tool (dbt) Pizza Shop Demo - Part 2 ETL on PostgreSQL via Airflow

Data Build Tool (dbt) Pizza Shop Demo - Part 2 ETL on PostgreSQL via Airflow
January 25, 20249 min read Data-Engineering Dbt Pizza Shop DemoApache AirflowDbtDockerPostgreSQLPython

Orchestrate the PostgreSQL pizza shop dbt project with Apache Airflow, scheduling the SCD type 2 dimension and fact table builds as an ETL process.

Read More: Data Build Tool (dbt) Pizza Shop Demo - Part 2 ETL on PostgreSQL via Airflow

Data Build Tool (dbt) Pizza Shop Demo - Part 1 Modelling on PostgreSQL

Data Build Tool (dbt) Pizza Shop Demo - Part 1 Modelling on PostgreSQL
January 18, 202415 min read Data-Engineering Dbt Pizza Shop DemoDbtDockerPostgreSQLPython

Model fictional pizza shop data on PostgreSQL with dbt, building two SCD type 2 dimension tables and one transactional fact table.

Read More: Data Build Tool (dbt) Pizza Shop Demo - Part 1 Modelling on PostgreSQL

Data Build Tool (dbt) for Effective Data Transformation on AWS - Part 5 Athena

Data Build Tool (dbt) for Effective Data Transformation on AWS - Part 5 Athena
December 6, 202216 min read Data-Engineering Dbt for Effective Data Transformation on AWSAmazon AthenaAmazon QuickSightAWSAWS GlueDbt

Transform IMDb data with dbt on Amazon Athena using the dbt-athena adapter, the last part of the dbt on AWS series.

Read More: Data Build Tool (dbt) for Effective Data Transformation on AWS - Part 5 Athena

EMR on EKS - Data Build Tool (dbt) for Effective Data Transformation on AWS Part 4

EMR on EKS - Data Build Tool (dbt) for Effective Data Transformation on AWS Part 4
November 1, 202220 min read Data-Engineering Dbt for Effective Data Transformation on AWSAmazon EKSAmazon EMRApache SparkAWSDbtEMR on EKS

Transform IMDb data with dbt on Amazon EMR on EKS, through a Spark Thrift server kept running on Kubernetes by a small wrapper class.

Read More: EMR on EKS - Data Build Tool (dbt) for Effective Data Transformation on AWS Part 4
  • ««
  • «
  • 1
  • 2
  • »
  • »»
Profile
Jaehyeon Kim
Jaehyeon Kim
Data Engineer | Data Streaming | Powering ML & AI in Real Time
Taxonomies
Data Streaming 70 Data Engineering 45 Development 28 Data Analysis 17 Open Source 15 Data Integration 12 Machine Learning 7 Kubernetes 6 Security 5 Data Architecture 3 Data Processing 3 Big Data 2 System Architecture 2 Web Development 2
Python 79 Apache Kafka 73 AWS 50 Docker 50 Apache Flink 39 Apache Beam 17 Apache Spark 17 Kafka Connect 16 Amazon MSK 14 AWS Lambda 14 Benchtop 13 dbt 13 desflow 13 odctl 12 Amazon EMR 11 PostgreSQL 10 Discrete Event Simulation 8 Kubernetes 8 PyFlink 8 Apache Iceberg 7 Change Data Capture (CDC) 7 Debezium 7 Kotlin 7 SimPy 7 Apache Airflow 6 Kafka UI 6 Amazon DynamoDB 5 PySpark 5 Amazon API Gateway 4 Amazon Athena 4 AWS Glue 4 AWS Glue Schema Registry 4 BigQuery 4 Digital Twin 4 FastAPI 4 Minikube 4 Amazon EKS 3 Amazon QuickSight 3 Amazon S3 3 Apache Hudi 3 ALL 144
Kafka Development with Docker 11 Apache Beam Python Examples 10 Building Real-Time Digital Twins with desflow 8 Real Time Streaming with Kafka and Flink 7 dbt Pizza Shop Demo 6 Apache Beam Local Development with Python 5 dbt for Effective Data Transformation on AWS 5 Getting Started with Real-Time Streaming in Kotlin 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started with PyFlink on AWS 3 Kafka Development on Kubernetes 3 Parallel processing on single machine 3 Realtime Dashboard with FastAPI, Streamlit and Next.js 3 API development with R 2 dbt Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Download Stock Data 2 From Prototype to Production: Real-Time Product Recommendation with Contextual Bandits 2 ALL 25
2026 19 2025 13 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • desflow from the Command Line: One Model, Every Command
    desflow from the Command Line: One Model, Every Command
    October 30, 2026
  • From dynamic-des to desflow: What Changed in Five Releases
    From dynamic-des to desflow: What Changed in Five Releases
    October 28, 2026
  • odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    October 12, 2026
  • Defining Data-Streaming Simulations in YAML, Without Writing Python
    Defining Data-Streaming Simulations in YAML, Without Writing Python
    October 6, 2026
  • Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    October 2, 2026
  • Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    October 1, 2026
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Current London 2026: Building End-to-End Data Lineage
    Current London 2026: Building End-to-End Data Lineage
    May 22, 2026
  • Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • Productionizing an Online Product Recommender using Event Driven Architecture
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
  • desflow from the Command Line: One Model, Every Command
    desflow from the Command Line: One Model, Every Command
    October 30, 2026
  • From dynamic-des to desflow: What Changed in Five Releases
    From dynamic-des to desflow: What Changed in Five Releases
    October 28, 2026
  • odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    October 12, 2026
  • Defining Data-Streaming Simulations in YAML, Without Writing Python
    Defining Data-Streaming Simulations in YAML, Without Writing Python
    October 6, 2026
  • Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    October 2, 2026
  • Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    October 1, 2026
  • Data Streaming and Machine Learning Projects That Run on Your Laptop
    Data Streaming and Machine Learning Projects That Run on Your Laptop
    September 30, 2026
  • Learning MLOps with a Feature Store: A New Series
    Learning MLOps with a Feature Store: A New Series
    September 28, 2026
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • desflow: A Declarative API with Postgres and Redis Connectors
    desflow: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
Actions
Go back Reload Copy URL

Jaehyeon Kim

Data Engineer | Data Streaming | Powering ML & AI in Real Time

Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.