Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Categories

      List of categories.

    • Tags

      List of tags.

    • Series

      List of series.

    • Archives

  • Projects
  • Slides

/

  • Github Linkedin RSS

  • Font Size
  • Palette
  • Mode
  1. Home
  2. Tags
  3. Python

Data Build Tool (dbt) Pizza Shop Demo - Part 5 Modelling on Amazon Athena

featured.png
March 7, 202416 min read Data-Engineering Dbt Pizza Shop DemoAmazon AthenaAWSDbtDockerPython

Model pizza shop data as Apache Iceberg tables transformed on Amazon Athena with dbt, using array and struct types to denormalise the fact table.

Read More

Data Build Tool (dbt) Pizza Shop Demo - Part 4 ETL on BigQuery via Airflow

featured.png
February 22, 20249 min read Data-Engineering Dbt Pizza Shop DemoApache AirflowBigQueryDbtDockerGCPPython

Orchestrate the BigQuery pizza shop dbt project with Apache Airflow, running the SCD type 2 dimension tables and the denormalised fact table.

Read More

Data Build Tool (dbt) Pizza Shop Demo - Part 3 Modelling on BigQuery

featured.png
February 8, 202416 min read Data-Engineering Dbt Pizza Shop DemoBigQueryDbtDockerGCPPython

Model pizza shop data on Google BigQuery with dbt, keeping SCD type 2 dimensions and denormalising the fact table with nested and repeated fields.

Read More

Data Build Tool (dbt) Pizza Shop Demo - Part 2 ETL on PostgreSQL via Airflow

featured.png
January 25, 20249 min read Data-Engineering Dbt Pizza Shop DemoApache AirflowDbtDockerPostgreSQLPython

Orchestrate the PostgreSQL pizza shop dbt project with Apache Airflow, scheduling the SCD type 2 dimension and fact table builds as an ETL process.

Read More

Data Build Tool (dbt) Pizza Shop Demo - Part 1 Modelling on PostgreSQL

featured.png
January 18, 202415 min read Data-Engineering Dbt Pizza Shop DemoDbtDockerPostgreSQLPython

Model fictional pizza shop data on PostgreSQL with dbt, building two SCD type 2 dimension tables and one transactional fact table.

Read More

Kafka Development on Kubernetes - Part 3 Kafka Connect

featured.png
January 11, 20247 min read Data Integration Data-Streaming Kubernetes Kafka Development on KubernetesApache KafkaDockerKafka ConnectKubernetesMinikubePython

Ingest fake customer and order data into Kafka with the MSK Data Generator and save it to S3, with connectors deployed as Strimzi custom resources.

Read More

Kafka Development on Kubernetes - Part 2 Producer and Consumer

featured.png
January 4, 20248 min read Data-Streaming Kubernetes Kafka Development on KubernetesApache KafkaDockerKubernetesMinikubePythonStrimzi

Send and read Kafka topic records from Python client applications built with the kafka-python package and deployed on Kubernetes.

Read More

Kafka Development on Kubernetes - Part 1 Cluster Setup

featured.png
December 21, 20237 min read Data-Streaming Kubernetes Kafka Development on KubernetesApache KafkaDockerKubernetesMinikubePythonStrimzi

Create a Kafka cluster on Kubernetes with Strimzi, the first step in a series on Python clients and Kafka Connect pipelines on Kubernetes.

Read More

Consume data from Kafka using Lambda - Real Time Streaming with Kafka and Flink Lab 6

featured.png
December 14, 20236 min read Data-Streaming Real Time Streaming With Kafka and FlinkAmazon MSKApache KafkaAWSAWS LambdaKpowPython

Consume Kafka messages with an AWS Lambda function, using Amazon MSK as an event source so that Lambda polls the topic and invokes the function.

Read More

Clean, Aggregate, and Enrich Events with Flink - Real Time Streaming with Kafka and Flink Lab 4

featured.png
November 23, 202315 min read Data-Streaming Real Time Streaming With Kafka and FlinkApache FlinkApache KafkaOpenSearchPyFlinkPython

Aggregate taxi ride counts and trip durations by vendor over five second windows with PyFlink, then chart them in an OpenSearch dashboard.

Read More
  • ««
  • «
  • 2
  • 3
  • 4
  • 5
  • 6
  • »
  • »»
Profile
Jaehyeon Kim
Jaehyeon Kim
Data Engineer | Data Streaming Enthusiast | Powering ML & AI in Real Time
Taxonomies
Data Streaming 70 Data Engineering 38 Development 28 Data Analysis 17 Data Integration 12 Open Source 7 Kubernetes 6 Machine Learning 5 Security 5 Data Architecture 3 Data Processing 3 Big Data 2 System Architecture 2 Web Development 2
Python 75 Apache Kafka 68 AWS 50 Docker 50 R 37 Apache Flink 34 Kpow 22 Apache Beam 17 Apache Spark 16 Kafka Connect 15 Amazon MSK 14 AWS Lambda 14 dbt 13 Amazon EMR 11 Kubernetes 8 PyFlink 8 Kotlin 7 PostgreSQL 7 Change Data Capture (CDC) 6 Debezium 6 dynamic-des 6 Amazon DynamoDB 5 Apache Airflow 5 Discrete Event Simulation 5 Factor House Local 5 PySpark 5 Amazon API Gateway 4 Amazon Athena 4 Apache Iceberg 4 AWS Glue 4 AWS Glue Schema Registry 4 BigQuery 4 Digital Twin 4 FastAPI 4 Minikube 4 R Shiny 4 RServe 4 SimPy 4 Amazon EKS 3 Amazon QuickSight 3 ALL 137
Kafka Development with Docker 11 Apache Beam Python Examples 10 Real Time Streaming with Kafka and Flink 7 dbt Pizza Shop Demo 6 Tree Based Methods in R 6 Apache Beam Local Development with Python 5 Building Real-Time Digital Twins with dynamic-des 5 dbt for Effective Data Transformation on AWS 5 Getting Started with Real-Time Streaming in Kotlin 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started with PyFlink on AWS 3 Kafka Development on Kubernetes 3 Parallel processing on single machine 3 Realtime Dashboard with FastAPI, Streamlit and Next.js 3 API development with R 2 dbt Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Download Stock Data 2 ALL 24
2026 11 2025 13 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • featured.png
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • featured.png
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • featured.png
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • featured.png
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • featured.png
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • featured.png
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • featured.png
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • featured.png
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • featured.gif
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
  • featured.png
    Stream Processing with Flink in Kotlin
    December 10, 2025
  • featured.png
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • featured.png
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • featured.png
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • featured.png
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • featured.jfif
    Current London 2026: Building End-to-End Data Lineage
    May 22, 2026
  • featured.png
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • featured.png
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • featured.png
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • featured.png
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • featured.gif
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
Actions
Go back Reload Copy URL

Jaehyeon Kim

Data Engineer | Data Streaming Enthusiast | Powering ML & AI in Real Time

Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.