Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Archives

    • Series

      List of series.

    • Categories

      List of categories.

    • Tags

      List of tags.


/

  • Github Linkedin Paypal RSS

  • Font Size
  • Palette
  • Mode
  1. Home
  2. Tags
  3. Docker

Thoughts on Apache Airflow AWS Lambda Operator

featured.png
April 13, 20209 min read Data EngineeringApache AirflowAWSAWS LambdaDockerDocker ComposePython

In this post, it is demonstrated how AWS Lambda can be integrated with Apache Airflow using a custom operator inspired by the ECS Operator.

Read More

Dynamic Routing and Centralized Auth With Traefik, Python and R Example

featured.png
November 29, 20199 min read DevelopmentDockerDocker ComposeFastAPIPythonRRserveTraefik

Traefik is a modern HTTP reverse proxy and load balancer. In this post, it'll be demonstrated how path-based routing can be set up by Traefik with Docker. Also a centralized authentication will be illustrated with the Forward Authentication feature of Traefik.

Read More

Distributed Task Queue With Python and R Example

featured.png
November 15, 20198 min read DevelopmentCeleryDockerDocker ComposeFastAPIKubernetesPythonRRedisRserve

In this post, I'll illustrate how a web service is created using FastAPI framework where tasks are sent to multiple workers. The workers are built with Celery and Rserve. Redis is used as a message broker/result backend for Celery and a key-value store for Rserve. Demos can be run in both Docker Compose and Kubernetes.

Read More

Linux Dev Environment on Windows

featured.png
November 1, 201912 min read DevelopmentDockerDocker ComposeFastAPIKubernetesMinikubePythonRRserveVSCodeWSL

In this post, I'll demonstrate how to create a Linux development environment on Windows using WSL. Also an example app (Rserve web service with a sidecar container) on Minikube will be demonstrated.

Read More

AWS Local Development With LocalStack

featured.png
July 20, 20196 min read DevelopmentAWSAWS LambdaDockerDocker ComposeFlaskFlask-RestPlusLocalStackPythonS3SQS

LocalStack provides an easy-to-use test/mocking framework for developing AWS applications. In this post, I'll demonstrate how to utilize LocalStack for development using a web service.

Read More

Cronicle Multi Server Setup

featured.png
July 19, 20195 min read DevelopmentCronicleDockerNginx

Cronicle is a multi-server task scheduler and runner. In this post, multi-server configuration of Cronicle will be demonstrated with Docker and Nginx as load balancer.

Read More
  • ««
  • «
  • 3
  • 4
  • 5
  • 6
  • 7
  • »
  • »»
Profile
Jaehyeon Kim
Jaehyeon Kim
Developer Experience at Factor House | Technical Content Creator
Taxonomies
Data Streaming 52 Apache Kafka 38 Data Engineering 26 Development 22 Apache Beam 17 Apache Flink 13 Apache Spark 13 Machine Learning 7
Docker 66 Python 65 Docker Compose 56 Apache Kafka 46 AWS 43 R 36 Apache Flink 31 Amazon MSK 21 Apache Beam 17 Apache Spark 17 Kafka Connect 17 AWS Lambda 15 Terraform 14 Data Build Tool (DBT) 13 Amazon EMR 11 PyFlink 11 Kubernetes 10 Amazon MSK Connect 9 GRPC 7 Amazon Athena 6 Minikube 6 PySpark 6 Amazon DynamoDB 5 Apache Airflow 5 Rserve 5 Visual Studio Code 5 Amazon API Gateway 4 Amazon Managed Flink 4 Amazon Managed Service for Apache Flink 4 Amazon QuickSight 4 Apache Iceberg 4 AWS Glue 4 BigQuery 4 Data Lake 4 FastAPI 4 GCP 4 OpenSearch 4 PostgreSQL 4 Security 4 Shiny 4 ALL 115
Kafka Development With Docker 11 Apache Beam Python Examples 10 Real Time Streaming With Kafka and Flink 7 DBT Pizza Shop Demo 6 Tree Based Methods in R 6 Apache Beam Local Development With Python 5 DBT for Effective Data Transformation on AWS 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started With Pyflink on AWS 3 Kafka Development on Kubernetes 3 Parallel Processing on Single Machine 3 Realtime Dashboard With FastAPI, Streamlit and Next.js 3 API Development With R 2 DBT Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Integrate Schema Registry With MSK Connect 2 Kafka, Flink and DynamoDB for Real Time Fraud Detection 2 Simplify Streaming Ingestion on AWS 2
2025 5 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • featured.png
    Meet the Streamhouse Trio - Paimon, Fluss, and Iceberg for Unified Data Architectures
    May 6, 2025
  • featured.gif
    Run Flink SQL Cookbook in Docker
    April 15, 2025
  • featured.gif
    Realtime Dashboard With FastAPI, Streamlit and Next.js - Part 1 Data Producer
    February 18, 2025
  • featured.png
    Change Data Capture (CDC) Local Development With PostgreSQL, Debezium Server and Pub/Sub Emulator
    November 7, 2024
  • featured.png
    Guide to Running DBT in Production
    September 13, 2024
  • featured.png
    DBT CI/CD Demo With BigQuery and GitHub Actions
    September 5, 2024
  • featured.png
    Cache Data on Apache Beam Pipelines Using a Shared Object
    August 22, 2024
  • featured.png
    Apache Beam Python Examples - Part 1 Calculate K Most Frequent Words and Max Word Length
    July 4, 2024
  • featured.png
    Deploy Python Stream Processing App on Kubernetes - Part 1 PyFlink Application
    May 30, 2024
  • featured.png
    Apache Beam Local Development With Python - Part 1 Pipeline, Notebook, SQL and DataFrame
    March 28, 2024
  • featured.png
    Meet the Streamhouse Trio - Paimon, Fluss, and Iceberg for Unified Data Architectures
    May 6, 2025
  • featured.gif
    Run Flink SQL Cookbook in Docker
    April 15, 2025
  • featured.gif
    Realtime Dashboard With FastAPI, Streamlit and Next.js - Part 3 Next.js Dashboard
    March 4, 2025
  • featured.gif
    Realtime Dashboard With FastAPI, Streamlit and Next.js - Part 2 Streamlit Dashboard
    February 25, 2025
  • featured.gif
    Realtime Dashboard With FastAPI, Streamlit and Next.js - Part 1 Data Producer
    February 18, 2025
  • featured.png
    Apache Beam Python Examples - Part 10 Develop Streaming File Reader Using Splittable DoFn
    December 19, 2024
  • featured.png
    Apache Beam Python Examples - Part 9 Develop Batch File Reader and PiSampler Using Splittable DoFn
    December 5, 2024
  • featured.png
    Apache Beam Python Examples - Part 8 Enhance Sport Activity Tracker With Runner Motivation
    November 21, 2024
  • featured.png
    Change Data Capture (CDC) Local Development With PostgreSQL, Debezium Server and Pub/Sub Emulator
    November 7, 2024
  • featured.png
    Apache Beam Python Examples - Part 7 Separate Droppable Data Into Side Output
    October 24, 2024
Actions
Go back Reload Copy URL

Jaehyeon Kim

Developer Experience at Factor House | Technical Content Creator

Copyright © 2023-2025 Jaehyeon Kim. All Rights Reserved.