58 posts in total
- Kafka Clients with Avro - Schema Registry and Order Events
May 27, 2025
- Kafka Clients with JSON - Producing and Consuming Order Events
May 20, 2025
- Apache Beam Python Examples - Part 8 Enhance Sport Activity Tracker with Runner Motivation
November 21, 2024
- Apache Beam Python Examples - Part 7 Separate Droppable Data into Side Output
October 24, 2024
- Apache Beam Python Examples - Part 6 Call RPC Service in Batch with Defined Batch Size using Stateful DoFn
October 2, 2024
- Apache Beam Python Examples - Part 5 Call RPC Service in Batch using Stateless DoFn
September 18, 2024
- Apache Beam Python Examples - Part 4 Call RPC Service for Data Augmentation
August 15, 2024
- Apache Beam Python Examples - Part 3 Build Sport Activity Tracker with/without SQL
August 1, 2024
- Apache Beam Python Examples - Part 2 Calculate Average Word Length with/without Fixed Look back
July 18, 2024
- Apache Beam Python Examples - Part 1 Calculate K Most Frequent Words and Max Word Length
July 4, 2024
- Deploy Python Stream Processing App on Kubernetes - Part 2 Beam Pipeline on Flink Runner
June 6, 2024
- Deploy Python Stream Processing App on Kubernetes - Part 1 PyFlink Application
May 30, 2024
- Apache Beam Local Development with Python - Part 4 Streaming Pipelines
May 2, 2024
- Apache Beam Local Development with Python - Part 3 Flink Runner
April 18, 2024
- Kafka Development on Kubernetes - Part 3 Kafka Connect
January 11, 2024
- Kafka Development on Kubernetes - Part 2 Producer and Consumer
January 4, 2024
- Kafka Development on Kubernetes - Part 1 Cluster Setup
December 21, 2023
- Real Time Streaming with Kafka and Flink - Lab 6 Consume data from Kafka using Lambda
December 14, 2023
- Setup Local Development Environment for Apache Flink and Spark Using EMR Container Images
December 7, 2023
- Real Time Streaming with Kafka and Flink - Lab 5 Write data to DynamoDB using Kafka Connect
November 30, 2023
13 posts in total
- Setup Local Development Environment for Apache Flink and Spark Using EMR Container Images
December 7, 2023
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 4 EMR on EKS
November 1, 2022
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 3 EMR on EC2
October 19, 2022
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 2 Glue
October 9, 2022
- Develop and Test Apache Spark Apps for EMR Remotely Using Visual Studio Code
September 7, 2022
- Manage EMR on EKS with Terraform
August 26, 2022
- Data Warehousing ETL Demo with Apache Iceberg on EMR Local Environment
June 26, 2022
- Develop and Test Apache Spark Apps for EMR Locally Using Docker
May 8, 2022
- EMR on EKS by Example
January 17, 2022
- AWS Glue Local Development with Docker and Visual Studio Code
August 20, 2021
- Boost SparkR with Hive
April 30, 2016
- Quick Start SparkR in Local and Cluster Mode
March 2, 2016
- Spark Cluster Setup on VirtualBox
February 22, 2016
50 posts in total
- Data Build Tool (dbt) Pizza Shop Demo - Part 6 ETL on Amazon Athena via Airflow
March 14, 2024
- Data Build Tool (dbt) Pizza Shop Demo - Part 5 Modelling on Amazon Athena
March 7, 2024
- Real Time Streaming with Kafka and Flink - Lab 6 Consume data from Kafka using Lambda
December 14, 2023
- Setup Local Development Environment for Apache Flink and Spark Using EMR Container Images
December 7, 2023
- Real Time Streaming with Kafka and Flink - Lab 5 Write data to DynamoDB using Kafka Connect
November 30, 2023
- Real Time Streaming with Kafka and Flink - Lab 3 Transform and write data to S3 from Kafka using Flink
November 16, 2023
- Real Time Streaming with Kafka and Flink - Lab 2 Write data to Kafka from S3 using Flink
November 9, 2023
- Kafka Connect for AWS Services Integration - Part 5 Deploy Aiven OpenSearch Sink Connector
October 30, 2023
- Real Time Streaming with Kafka and Flink - Lab 1 Produce data to Kafka using Lambda
October 26, 2023
- Kafka Connect for AWS Services Integration - Part 4 Develop Aiven OpenSearch Sink Connector
October 23, 2023
- Real Time Streaming with Kafka and Flink - Introduction
October 5, 2023
- Kafka, Flink and DynamoDB for Real Time Fraud Detection - Part 2 Deployment via AWS Managed Flink
September 14, 2023
- Getting Started with Pyflink on AWS - Part 3 AWS Managed Flink and MSK
September 4, 2023
- Getting Started with Pyflink on AWS - Part 2 Local Flink and MSK
August 28, 2023
- Kafka, Flink and DynamoDB for Real Time Fraud Detection - Part 1 Local Development
August 10, 2023
- Kafka Connect for AWS Services Integration - Part 3 Deploy Camel DynamoDB Sink Connector
July 3, 2023
- Kafka Development with Docker - Part 7 Producer and Consumer with Glue Schema Registry
June 22, 2023
- Kafka Development with Docker - Part 6 Kafka Connect with Glue Schema Registry
June 15, 2023
- Kafka Development with Docker - Part 5 Glue Schema Registry
June 8, 2023
- Kafka Connect for AWS Services Integration - Part 2 Develop Camel DynamoDB Sink Connector
June 4, 2023