Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Categories

      List of categories.

    • Tags

      List of tags.

    • Series

      List of series.

    • Archives

  • Projects
  • Slides

/

  • Github Linkedin RSS

  • Font Size
  • Palette
  • Mode
  1. Home
  2. blogs

Develop Camel DynamoDB Sink Connector - Kafka Connect for AWS Services Integration Part 2

Develop Camel DynamoDB Sink Connector - Kafka Connect for AWS Services Integration Part 2
June 4, 202314 min read Data-Streaming Kafka Connect for AWS Services IntegrationAmazon DynamoDBApache CamelApache KafkaAWSDockerKafka Connect

Develop the Camel DynamoDB sink connector on Docker, consuming fake order records from a Kafka topic and writing them into a DynamoDB table.

Read More: Develop Camel DynamoDB Sink Connector - Kafka Connect for AWS Services Integration Part 2

Kafka Development with Docker - Part 4 Producer and Consumer

Kafka Development with Docker - Part 4 Producer and Consumer
June 1, 20237 min read Data-Streaming Kafka Development With DockerApache KafkaDockerPython

Kafka producer and consumer applications written in Python with kafka-python, one of several open source alternatives to the Java clients.

Read More: Kafka Development with Docker - Part 4 Producer and Consumer

Kafka Development with Docker - Part 3 Kafka Connect

Kafka Development with Docker - Part 3 Kafka Connect
May 25, 20239 min read Data Integration Data-Streaming Kafka Development With DockerApache KafkaDockerKafka Connect

A Kafka Connect ingestion pipeline on Docker. The MSK Data Generator creates fake customer and order records, and the Confluent S3 sink saves them.

Read More: Kafka Development with Docker - Part 3 Kafka Connect

Kafka Development with Docker - Part 2 Management App

Kafka Development with Docker - Part 2 Management App
May 18, 20239 min read Data-Streaming Kafka Development With DockerApache KafkaDockerKafka-UiKpow

Kafka management apps compared for local development, covering IAM access control and integration with MSK Connect and Glue Schema Registry.

Read More: Kafka Development with Docker - Part 2 Management App

How I Prepared for Confluent Certified Developer for Apache Kafka as a Non-Java Developer

How I Prepared for Confluent Certified Developer for Apache Kafka as a Non-Java Developer
May 11, 20233 min read Data-StreamingApache KafkaCCDAK

How a non-Java developer prepared for and passed the Confluent Certified Developer for Apache Kafka exam, which assumes the Java APIs throughout.

Read More: How I Prepared for Confluent Certified Developer for Apache Kafka as a Non-Java Developer

Kafka Development with Docker - Part 1 Cluster Setup

Kafka Development with Docker - Part 1 Cluster Setup
May 4, 20239 min read Data-Streaming Kafka Development With DockerApache KafkaDocker

Set up a Kafka cluster with Docker Compose, the base for a series on developing and testing Kafka applications for AWS in containers.

Read More: Kafka Development with Docker - Part 1 Cluster Setup

Kafka Connect for AWS Services Integration - Part 1 Introduction

Kafka Connect for AWS Services Integration - Part 1 Introduction
May 3, 20234 min read Data Integration Data-Streaming Kafka Connect for AWS Services IntegrationAmazon MSKApache KafkaAWSKafka Connect

A survey of the Kafka connectors available for AWS service integration, as the starting point for building real time data pipelines on AWS.

Read More: Kafka Connect for AWS Services Integration - Part 1 Introduction

Self-managed Blog with Hugo and GitHub Pages

Self-managed Blog with Hugo and GitHub Pages
April 24, 202311 min read Web DevelopmentDocker ComposeGitHub ActionsHugoMarkdown

Set up a self-managed blog with Hugo and GitHub Pages, including why Hugo was chosen over Jekyll and Pelican for its themes and features.

Read More: Self-managed Blog with Hugo and GitHub Pages

Integrate Glue Schema Registry with Your Python Kafka App

Integrate Glue Schema Registry with Your Python Kafka App
April 12, 202326 min read Data-StreamingApache KafkaAWS Glue Schema RegistryAWS LambdaKpowPython

Integrate Python Kafka producer and consumer apps running on AWS Lambda with the Glue Schema Registry, which manages and validates message schemas.

Read More: Integrate Glue Schema Registry with Your Python Kafka App

Simplify Streaming Ingestion on AWS - Part 2 MSK and Athena

Simplify Streaming Ingestion on AWS - Part 2 MSK and Athena
March 14, 202313 min read Data-Streaming Simplify Streaming Ingestion on AWSAmazon AthenaAmazon MSKApache KafkaAWSAWS LambdaPython

Stream Kafka records from Amazon MSK into Athena through its direct integration, with a producer Lambda built locally using AWS SAM and Terraform.

Read More: Simplify Streaming Ingestion on AWS - Part 2 MSK and Athena
  • ««
  • «
  • 7
  • 8
  • 9
  • 10
  • 11
  • »
  • »»
Profile
Jaehyeon Kim
Jaehyeon Kim
Data Engineer | Data Streaming | Powering ML & AI in Real Time
Taxonomies
Data Streaming 70 Data Engineering 38 Development 28 Data Analysis 17 Data Integration 12 Open Source 7 Kubernetes 6 Machine Learning 5 Security 5 Data Architecture 3 Data Processing 3 Big Data 2 System Architecture 2 Web Development 2
Python 75 Apache Kafka 68 AWS 50 Docker 50 Apache Flink 34 Apache Beam 17 Apache Spark 16 Kafka Connect 15 Amazon MSK 14 AWS Lambda 14 dbt 13 Amazon EMR 11 Kubernetes 8 PyFlink 8 Kotlin 7 PostgreSQL 7 Change Data Capture (CDC) 6 Debezium 6 dynamic-des 6 Amazon DynamoDB 5 Apache Airflow 5 Discrete Event Simulation 5 Factor House Local 5 PySpark 5 Amazon API Gateway 4 Amazon Athena 4 Apache Iceberg 4 AWS Glue 4 AWS Glue Schema Registry 4 BigQuery 4 Digital Twin 4 FastAPI 4 Minikube 4 SimPy 4 Amazon EKS 3 Amazon QuickSight 3 Amazon S3 3 Apache Hudi 3 EMR on EKS 3 GCP 3 ALL 137
Kafka Development with Docker 11 Apache Beam Python Examples 10 Real Time Streaming with Kafka and Flink 7 dbt Pizza Shop Demo 6 Apache Beam Local Development with Python 5 Building Real-Time Digital Twins with dynamic-des 5 dbt for Effective Data Transformation on AWS 5 Getting Started with Real-Time Streaming in Kotlin 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started with PyFlink on AWS 3 Kafka Development on Kubernetes 3 Parallel processing on single machine 3 Realtime Dashboard with FastAPI, Streamlit and Next.js 3 API development with R 2 dbt Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Download Stock Data 2 From Prototype to Production: Real-Time Product Recommendation with Contextual Bandits 2 ALL 24
2026 11 2025 13 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • Introducing odctl: One CLI for a Local Open Data Stack
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • Building an Event-Driven Hybrid Digital Twin with dynamic-des
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • Why Digital Twins Are Rewiring Industry 4.0
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • Slides as Code: Integrating Reveal.js into my Hugo Blog
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • Productionizing an Online Product Recommender using Event Driven Architecture
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
  • Stream Processing with Flink in Kotlin
    Stream Processing with Flink in Kotlin
    December 10, 2025
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • Introducing odctl: One CLI for a Local Open Data Stack
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • Current London 2026: Building End-to-End Data Lineage
    Current London 2026: Building End-to-End Data Lineage
    May 22, 2026
  • Building an Event-Driven Hybrid Digital Twin with dynamic-des
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • Why Digital Twins Are Rewiring Industry 4.0
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • Slides as Code: Integrating Reveal.js into my Hugo Blog
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • Productionizing an Online Product Recommender using Event Driven Architecture
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
Actions
Go back Reload Copy URL

Jaehyeon Kim

Data Engineer | Data Streaming | Powering ML & AI in Real Time

Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.