Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Categories

      List of categories.

    • Tags

      List of tags.

    • Series

      List of series.

    • Archives

  • Projects
  • Slides

/

  • Github Linkedin RSS

  • Font Size
  • Palette
  • Mode
  1. Home
  2. Archives

2026

17 posts in total
  • odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command Oct 12
  • Defining Data-Streaming Simulations in YAML, Without Writing Python Oct 6
  • Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL Oct 2
  • Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect Oct 1
  • Data Streaming and Machine Learning Projects That Run on Your Laptop Sep 30
  • ...

2025

13 posts in total
  • Stream Processing with Flink in Kotlin Dec 10
  • Guide to Building Integrated Web Applications with FastAPI and NiceGUI Nov 19
  • Self-service Data Platform via a Multi-tenant SQL Gateway Jul 17
  • Flink Table API - Declarative Analytics for Supplier Stats in Real Time Jun 17
  • Flink DataStream API - Scalable Event Processing for Supplier Stats Jun 10
  • ...

2024

29 posts in total
  • Develop Streaming File Reader using Splittable DoFn - Apache Beam Python Examples Part 10 Dec 19
  • Develop Batch File Reader and PiSampler using Splittable DoFn - Apache Beam Python Examples Part 9 Dec 5
  • Enhance Sport Activity Tracker with Runner Motivation - Apache Beam Python Examples Part 8 Nov 21
  • Change Data Capture (CDC) Local Development with PostgreSQL, Debezium Server and Pub/Sub Emulator Nov 7
  • Apache Beam Python Examples - Part 7 Separate Droppable Data into Side Output Oct 24
  • ...

2023

39 posts in total
  • Kafka Development on Kubernetes - Part 1 Cluster Setup Dec 21
  • Consume data from Kafka using Lambda - Real Time Streaming with Kafka and Flink Lab 6 Dec 14
  • Setup Local Development Environment for Apache Flink and Spark Using EMR Container Images Dec 7
  • Write data to DynamoDB using Kafka Connect - Real Time Streaming with Kafka and Flink Lab 5 Nov 30
  • Clean, Aggregate, and Enrich Events with Flink - Real Time Streaming with Kafka and Flink Lab 4 Nov 23
  • ...

2022

15 posts in total
  • Data Build Tool (dbt) for Effective Data Transformation on AWS - Part 5 Athena Dec 6
  • EMR on EKS - Data Build Tool (dbt) for Effective Data Transformation on AWS Part 4 Nov 1
  • EMR on EC2 - Data Build Tool (dbt) for Effective Data Transformation on AWS Part 3 Oct 19
  • Data Build Tool (dbt) for Effective Data Transformation on AWS - Part 2 Glue Oct 9
  • Data Build Tool (dbt) for Effective Data Transformation on AWS - Part 1 Redshift Sep 28
  • ...

2021

7 posts in total
  • Implement Data Lake - Data Lake Demo using Change Data Capture (CDC) on AWS Part 3 Dec 19
  • Data Lake Demo using Change Data Capture (CDC) on AWS - Part 2 Implement CDC Dec 12
  • Data Lake Demo using Change Data Capture (CDC) on AWS - Part 1 Local Development Dec 5
  • Local Development of AWS Glue 3.0 and Later Nov 14
  • Yet another serverless solution for invoking AWS Lambda at a sub-minute frequency Oct 13
  • ...

2020

1 post in total
  • Thoughts on Apache Airflow AWS Lambda Operator Apr 13

2019

5 posts in total
  • Dynamic Routing and Centralized Auth with Traefik, Python and R Example Nov 29
  • Distributed Task Queue with Python and R Example Nov 15
  • Linux Dev Environment on Windows Nov 1
  • AWS Local Development with LocalStack Jul 20
  • Cronicle Multi Server Setup Jul 19

2018

2 posts in total
  • Shiny to Vue.js May 26
  • Async Shiny and Its Limitation May 19

2017

6 posts in total
  • API Development with R Part 2 Nov 19
  • API Development with R Part 1 Nov 18
  • Serverless Data Product POC Backend Part 4 - Serving R ML Model via S3 Apr 17
  • Serverless Data Product POC Backend Part 3 - Exposing R ML Model via APIG Apr 13
  • Serverless Data Product POC Backend Part 2 - Deploying R ML Model via Lambda Apr 11
  • ...

2016

6 posts in total
  • Some Thoughts on Shiny Open Source - Render Multiple Pages Jun 27
  • Some Thoughts on Shiny Open Source - Internal Load Balancing May 23
  • Asynchronous Processing Using Job Queue May 12
  • Boost SparkR with Hive Apr 30
  • Quick Start SparkR in Local and Cluster Mode Mar 2
  • ...

2015

15 posts in total
  • Quick Test to Wrap Python in R Nov 21
  • Some Thoughts on Python for R Users Aug 9
  • Some Thoughts on Python Aug 8
  • Setup Random Seeds on Caret Package May 30
  • Packaging Analysis Mar 24
  • ...

2014

5 posts in total
  • Looping without for Dec 17
  • Short R Examples Dec 3
  • Summarise Stock Returns from Multiple Files Nov 27
  • Download Stock Data - Part II Nov 21
  • Download Stock Data - Part I Nov 20
Profile
Jaehyeon Kim
Jaehyeon Kim
Data Engineer | Data Streaming | Powering ML & AI in Real Time
Taxonomies
Data Streaming 70 Data Engineering 43 Development 28 Data Analysis 17 Open Source 13 Data Integration 12 Machine Learning 7 Kubernetes 6 Security 5 Data Architecture 3 Data Processing 3 Big Data 2 System Architecture 2 Web Development 2
Python 77 Apache Kafka 73 AWS 50 Docker 50 Apache Flink 39 Apache Beam 17 Apache Spark 17 Kafka Connect 16 Amazon MSK 14 AWS Lambda 14 Benchtop 13 dbt 13 odctl 12 Amazon EMR 11 dynamic-des 11 PostgreSQL 10 Kubernetes 8 PyFlink 8 Apache Iceberg 7 Change Data Capture (CDC) 7 Debezium 7 Kotlin 7 Apache Airflow 6 Discrete Event Simulation 6 Kafka UI 6 Amazon DynamoDB 5 PySpark 5 SimPy 5 Amazon API Gateway 4 Amazon Athena 4 AWS Glue 4 AWS Glue Schema Registry 4 BigQuery 4 Digital Twin 4 FastAPI 4 Minikube 4 Amazon EKS 3 Amazon QuickSight 3 Amazon S3 3 Apache Hudi 3 ALL 143
Kafka Development with Docker 11 Apache Beam Python Examples 10 Real Time Streaming with Kafka and Flink 7 Building Real-Time Digital Twins with dynamic-des 6 dbt Pizza Shop Demo 6 Apache Beam Local Development with Python 5 dbt for Effective Data Transformation on AWS 5 Getting Started with Real-Time Streaming in Kotlin 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started with PyFlink on AWS 3 Kafka Development on Kubernetes 3 Parallel processing on single machine 3 Realtime Dashboard with FastAPI, Streamlit and Next.js 3 API development with R 2 dbt Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Download Stock Data 2 From Prototype to Production: Real-Time Product Recommendation with Contextual Bandits 2 ALL 25
2026 17 2025 13 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    October 12, 2026
  • Defining Data-Streaming Simulations in YAML, Without Writing Python
    Defining Data-Streaming Simulations in YAML, Without Writing Python
    October 6, 2026
  • Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    October 2, 2026
  • Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    October 1, 2026
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Current London 2026: Building End-to-End Data Lineage
    Current London 2026: Building End-to-End Data Lineage
    May 22, 2026
  • Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • Productionizing an Online Product Recommender using Event Driven Architecture
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
  • Stream Processing with Flink in Kotlin
    Stream Processing with Flink in Kotlin
    December 10, 2025
  • Guide to Building Integrated Web Applications with FastAPI and NiceGUI
    Guide to Building Integrated Web Applications with FastAPI and NiceGUI
    November 19, 2025
  • odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    odctl 1.0: Kafka, Flink, Spark, Iceberg, Trino and MLflow on Your Laptop with One Command
    October 12, 2026
  • Defining Data-Streaming Simulations in YAML, Without Writing Python
    Defining Data-Streaming Simulations in YAML, Without Writing Python
    October 6, 2026
  • Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    Keeping Game Leaderboards Up to Date in Real Time with Kafka and Flink SQL
    October 2, 2026
  • Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    Change Data Capture on a Simulated Online Shop with Debezium and Kafka Connect
    October 1, 2026
  • Data Streaming and Machine Learning Projects That Run on Your Laptop
    Data Streaming and Machine Learning Projects That Run on Your Laptop
    September 30, 2026
  • Learning MLOps with a Feature Store: A New Series
    Learning MLOps with a Feature Store: A New Series
    September 28, 2026
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Dynamic DES: A Declarative API with Postgres and Redis Connectors
    Dynamic DES: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • Running Kafka, Flink, Spark, Trino and Iceberg Locally with One CLI
    Running Kafka, Flink, Spark, Trino and Iceberg Locally with One CLI
    July 16, 2026
  • One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES
    May 25, 2026
Actions
Go back Reload Copy URL

Jaehyeon Kim

Data Engineer | Data Streaming | Powering ML & AI in Real Time

Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.