Jaehyeon Kim
Jaehyeon Kim

  • Blog
    • Categories

      List of categories.

    • Tags

      List of tags.

    • Series

      List of series.

    • Archives

  • Projects
  • Slides

/

  • Github Linkedin RSS

  • Font Size
  • Palette
  • Mode
  1. Home
  2. blogs

Asynchronous Processing Using Job Queue

May 12, 20163 min read DevelopmentR

In this post, a way to overcome one of R's limitations of lack of multi-threading is discussed by job queuing using the jobqueue package

Read More: Asynchronous Processing Using Job Queue

Boost SparkR with Hive

April 30, 20168 min read Data-EngineeringApache HiveApache SparkHiveQLRSparkR

Run SparkR in Hive Context to reach the Hive UDFs and window functions the SQL Context lacks, compared against dplyr, plus building Spark with Hive.

Read More: Boost SparkR with Hive

Quick Start SparkR in Local and Cluster Mode

March 2, 20168 min read Data-EngineeringApache SparkRSparkR

Run SparkR in local and cluster mode from an R project, setting environment variables and the library path, then reading JSON and CSV files.

Read More: Quick Start SparkR in Local and Cluster Mode

Spark Cluster Setup on VirtualBox

February 22, 20164 min read Data-EngineeringApache SparkRSparkR

Set up a two node Spark standalone cluster on VirtualBox Ubuntu guests, covering machine preparation, copying the VDI image and password-less SSH.

Read More: Spark Cluster Setup on VirtualBox

Quick Test to Wrap Python in R

November 21, 20154 min read DevelopmentPythonR

Wrap Python boto calls for Amazon S3 in an R package, rs3helper, where the Python scripts return JSON so R parses it as vectors and data frames.

Read More: Quick Test to Wrap Python in R

Some Thoughts on Python for R Users

August 9, 20155 min read DevelopmentPythonR

Call a SOAP web service from Python with the suds library, a job R has no comprehensive client package for, demonstrated on the Sizmek MDX API.

Read More: Some Thoughts on Python for R Users

Some Thoughts on Python

August 8, 20154 min read DevelopmentPythonR

Write Python in an object-oriented style, shown with SOAP API client classes, and a reading order for the Introducing Python book by chapter group.

Read More: Some Thoughts on Python

Setup Random Seeds on Caret Package

May 30, 20156 min read Data AnalysisR

Setting up random seed is important for reproducibility of analysis. In this post, we discuss how to generate random seed using the caret package.

Read More: Setup Random Seeds on Caret Package

Packaging Analysis

March 24, 20158 min read Data AnalysisR

Turn an analysis into an R package, with the treebgg bagging package as the worked example, covering roxygen2 documents, testthat tests and vignettes.

Read More: Packaging Analysis

Parallel Processing on Single Machine - Part 3

March 19, 20158 min read Data Analysis Parallel Processing on Single MachineR

Compare the snow and foreach approaches to parallel processing in R on three practical examples, starting with k-means clustering on the Boston data.

Read More: Parallel Processing on Single Machine - Part 3
  • ««
  • «
  • 12
  • 13
  • 14
  • 15
  • 16
  • »
  • »»
Profile
Jaehyeon Kim
Jaehyeon Kim
Data Engineer | Data Streaming | Powering ML & AI in Real Time
Taxonomies
Data Streaming 70 Data Engineering 38 Development 28 Data Analysis 17 Data Integration 12 Open Source 7 Kubernetes 6 Machine Learning 5 Security 5 Data Architecture 3 Data Processing 3 Big Data 2 System Architecture 2 Web Development 2
Python 75 Apache Kafka 68 AWS 50 Docker 50 Apache Flink 34 Apache Beam 17 Apache Spark 16 Kafka Connect 15 Amazon MSK 14 AWS Lambda 14 dbt 13 Amazon EMR 11 Kubernetes 8 PyFlink 8 Kotlin 7 PostgreSQL 7 Change Data Capture (CDC) 6 Debezium 6 dynamic-des 6 Amazon DynamoDB 5 Apache Airflow 5 Discrete Event Simulation 5 Factor House Local 5 PySpark 5 Amazon API Gateway 4 Amazon Athena 4 Apache Iceberg 4 AWS Glue 4 AWS Glue Schema Registry 4 BigQuery 4 Digital Twin 4 FastAPI 4 Minikube 4 SimPy 4 Amazon EKS 3 Amazon QuickSight 3 Amazon S3 3 Apache Hudi 3 EMR on EKS 3 GCP 3 ALL 137
Kafka Development with Docker 11 Apache Beam Python Examples 10 Real Time Streaming with Kafka and Flink 7 dbt Pizza Shop Demo 6 Apache Beam Local Development with Python 5 Building Real-Time Digital Twins with dynamic-des 5 dbt for Effective Data Transformation on AWS 5 Getting Started with Real-Time Streaming in Kotlin 5 Kafka Connect for AWS Services Integration 5 Serverless Data Product 4 Data Lake Demo Using Change Data Capture 3 Getting Started with PyFlink on AWS 3 Kafka Development on Kubernetes 3 Parallel processing on single machine 3 Realtime Dashboard with FastAPI, Streamlit and Next.js 3 API development with R 2 dbt Guide for Production 2 Deploy Python Stream Processing App on Kubernetes 2 Download Stock Data 2 From Prototype to Production: Real-Time Product Recommendation with Contextual Bandits 2 ALL 24
2026 11 2025 13 2024 29 2023 39 2022 15 2021 7 2020 1 2019 5 2018 2 2017 6 2016 6 2015 15 2014 5
Posts
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • Introducing odctl: One CLI for a Local Open Data Stack
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • Building an Event-Driven Hybrid Digital Twin with dynamic-des
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • Why Digital Twins Are Rewiring Industry 4.0
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • Slides as Code: Integrating Reveal.js into my Hugo Blog
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • Productionizing an Online Product Recommender using Event Driven Architecture
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
  • Stream Processing with Flink in Kotlin
    Stream Processing with Flink in Kotlin
    December 10, 2025
  • Building an Agentic Analytics System over an Iceberg Lakehouse
    Building an Agentic Analytics System over an Iceberg Lakehouse
    July 18, 2026
  • Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    Dynamic DES v0.11.1: A Declarative API with Postgres and Redis Connectors
    July 17, 2026
  • Introducing odctl: One CLI for a Local Open Data Stack
    Introducing odctl: One CLI for a Local Open Data Stack
    July 16, 2026
  • One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    One Simulation, Two Pipelines: Batch Training and Live Inference with Dynamic DES v0.8.1
    May 25, 2026
  • Current London 2026: Building End-to-End Data Lineage
    Current London 2026: Building End-to-End Data Lineage
    May 22, 2026
  • Building an Event-Driven Hybrid Digital Twin with dynamic-des
    Building an Event-Driven Hybrid Digital Twin with dynamic-des
    April 29, 2026
  • Why Digital Twins Are Rewiring Industry 4.0
    Why Digital Twins Are Rewiring Industry 4.0
    April 22, 2026
  • Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    Building a Real-Time Industrial Digital Twin with Apache Flink and Online Machine Learning
    April 21, 2026
  • Slides as Code: Integrating Reveal.js into my Hugo Blog
    Slides as Code: Integrating Reveal.js into my Hugo Blog
    March 9, 2026
  • Productionizing an Online Product Recommender using Event Driven Architecture
    Productionizing an Online Product Recommender using Event Driven Architecture
    February 23, 2026
Actions
Go back Reload Copy URL

Jaehyeon Kim

Data Engineer | Data Streaming | Powering ML & AI in Real Time

Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.