Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Categories

      List of categories.

    • Tags

      List of tags.

    • Series

      List of series.

    • Archives

  • Projects
  • Slides

/

  • Github Linkedin RSS

  • Font Size
  • Palette
  • Mode

Run Flink SQL Cookbook in Docker

featured.gif
April 15, 20255 min read Data-StreamingApache FlinkDockerDocker ComposeFlink SQL

Docker Compose runs a local Apache Flink cluster and SQL Client so the Ververica Flink SQL Cookbook recipes work without the Ververica Platform.

Read More

Next.js Dashboard - Realtime Dashboard with FastAPI, Streamlit and Next.js Part 3

featured.gif
March 4, 20258 min read Development Realtime Dashboard With FastAPI, Streamlit and Next.jsApache EChartsNext.jsReactWebSocket

Next.js and React with Apache ECharts show the same live order and revenue metrics, reading from the FastAPI WebSocket server in the browser.

Read More

Streamlit Dashboard - Realtime Dashboard with FastAPI, Streamlit and Next.js Part 2

featured.gif
February 25, 20257 min read Development Realtime Dashboard With FastAPI, Streamlit and Next.jsApache EChartsPythonStreamlitWebSocket

Streamlit and Apache ECharts draw a live sales dashboard that reads order counts and revenue by country from a FastAPI WebSocket server.

Read More

Realtime Dashboard with FastAPI, Streamlit and Next.js - Part 1 Data Producer

featured.gif
February 18, 202510 min read Development Realtime Dashboard With FastAPI, Streamlit and Next.jsDockerFastAPIPostgreSQLPythonWebSocket

A Python generator loads theLook eCommerce data into PostgreSQL, and a FastAPI WebSocket server queries it on a timer to serve live dashboards.

Read More

Develop Streaming File Reader using Splittable DoFn - Apache Beam Python Examples Part 10

featured.png
December 19, 202412 min read Data-Streaming Apache Beam Python ExamplesApache BeamApache FlinkPythonSplittable DoFn

A streaming file reader built with Splittable DoFn scans an input folder for new files repeatedly, a pattern for unbounded sources in the Python SDK.

Read More

Develop Batch File Reader and PiSampler using Splittable DoFn - Apache Beam Python Examples Part 9

featured.png
December 5, 202410 min read Data-Streaming Apache Beam Python ExamplesApache BeamApache FlinkPythonSplittable DoFn

Splittable DoFn in Beam Python builds a batch file reader that processes files in parallel, and a PiSampler that estimates pi by Monte Carlo runs.

Read More

Enhance Sport Activity Tracker with Runner Motivation - Apache Beam Python Examples Part 8

featured.png
November 21, 202418 min read Data-Streaming Apache Beam Python ExamplesApache BeamApache FlinkApache KafkaPython

Pacing messages are added to the Beam sport activity tracker by comparing short term speed metrics against their long term counterparts.

Read More

Change Data Capture (CDC) Local Development with PostgreSQL, Debezium Server and Pub/Sub Emulator

featured.png
November 7, 202411 min read Data Integration Data-StreamingChange Data Capture (CDC)DebeziumGCPGoogle Cloud PubSubPostgreSQLPubSub Emulator

Debezium Server streams PostgreSQL row changes into the Google Pub/Sub emulator, with Docker Compose, a theLook data generator and a Python reader.

Read More

Apache Beam Python Examples - Part 7 Separate Droppable Data into Side Output

featured.png
October 24, 202418 min read Data-Streaming Apache Beam Python ExamplesApache BeamApache FlinkApache KafkaPython

Late droppable elements are detected by a timer in a stateful DoFn and sent to a Beam side output instead of being discarded without notice.

Read More

Call RPC Service in Batch with Defined Batch Size using Stateful DoFn - Apache Beam Python Examples Part 6

featured.png
October 2, 202414 min read Data-Streaming Apache Beam Python ExamplesApache BeamApache FlinkApache KafkaGRPCPython

A stateful DoFn with Beam state and timers fixes the gRPC batch size and maximum wait time instead of leaving the bundle size to the runner.

Read More
  • ««
  • «
  • 1
  • 2
  • 3
  • 4
  • 5
  • »
  • »»
Profile
Jaehyeon Kim
Jaehyeon Kim
Data Engineer | Data Streaming Enthusiast | Powering ML & AI in Real Time
Taxonomies
Data Streaming 70 Data Engineering 38 Development 28 Data Analysis 17 Data Integration 12 Open Source 7 Kubernetes 6 Machine Learning 5 Security 5 Data Architecture 3 Data Processing 3 Big Data 2 System Architecture 2 Web Development 2
Python 75 Apache Kafka 68 AWS 50 Docker 50 R 37 Apache Flink 34 Kpow 22 Apache Beam 17 Apache Spark 16 Kafka Connect 15 Amazon MSK 14 AWS Lambda 14 dbt 13 Amazon EMR 11 Kubernetes 8 PyFlink 8 Kotlin 7 PostgreSQL 7 Change Data Capture (CDC) 6 Debezium 6 dynamic-des 6 Amazon DynamoDB 5 Apache Airflow 5 Discrete Event Simulation 5 Factor House Local 5 PySpark 5 Amazon API Gateway 4 Amazon Athena 4 Apache Iceberg 4 AWS Glue 4 AWS Glue Schema Registry 4 BigQuery 4 Digital Twin 4 FastAPI 4 Minikube 4 R Shiny 4 RServe 4 SimPy 4 Amazon EKS 3 Amazon QuickSight 3 ALL 137
Kafka Development with Docker 11 Apache Beam Python Examples 10 Real Time Streaming with Kafka and Flink 7 dbt Pizza Shop Demo 6 Tree Based Methods in R 6 Apache Beam Local Development with Python 5 Building Real-Time Digital Twins with dynamic-des 5 dbt for Effective Data Transformation on AWS 5 Getting Started with Real-Time Streaming in Kotlin 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started with PyFlink on AWS 3 Kafka Development on Kubernetes 3 Parallel processing on single machine 3 Realtime Dashboard with FastAPI, Streamlit and Next.js 3 API development with R 2 dbt Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Download Stock Data 2 ALL 24
2026 11 2025 13 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • featured.png
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • featured.png
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • featured.png
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • featured.png
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • featured.png
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • featured.png
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • featured.png
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • featured.png
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • featured.gif
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
  • featured.png
    Stream Processing with Flink in Kotlin
    December 10, 2025
  • featured.png
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • featured.png
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • featured.png
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • featured.png
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • featured.jfif
    Current London 2026: Building End-to-End Data Lineage
    May 22, 2026
  • featured.png
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • featured.png
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • featured.png
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • featured.png
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • featured.gif
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
Actions
Go back Reload Copy URL

Jaehyeon Kim

Data Engineer | Data Streaming Enthusiast | Powering ML & AI in Real Time

Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.