14 posts in total
- Kafka Development on Kubernetes - Part 3 Kafka Connect
January 11, 2024
- Kafka Development on Kubernetes - Part 2 Producer and Consumer
January 4, 2024
- Kafka Development on Kubernetes - Part 1 Cluster Setup
December 21, 2023
- Kafka Development with Docker - Part 11 Kafka Authorization
July 20, 2023
- Kafka Development with Docker - Part 10 SASL Authentication
July 13, 2023
- Kafka Development with Docker - Part 9 SSL Authentication
July 6, 2023
- Kafka Development with Docker - Part 8 SSL Encryption
June 29, 2023
- Kafka Development with Docker - Part 7 Producer and Consumer with Glue Schema Registry
June 22, 2023
- Kafka Development with Docker - Part 6 Kafka Connect with Glue Schema Registry
June 15, 2023
- Kafka Development with Docker - Part 5 Glue Schema Registry
June 8, 2023
- Kafka Development with Docker - Part 4 Producer and Consumer
June 1, 2023
- Kafka Development with Docker - Part 3 Kafka Connect
May 25, 2023
- Kafka Development with Docker - Part 2 Management App
May 18, 2023
- Kafka Development with Docker - Part 1 Cluster Setup
May 4, 2023
24 posts in total
- Data Build Tool (dbt) Pizza Shop Demo - Part 6 ETL on Amazon Athena via Airflow
March 14, 2024
- Data Build Tool (dbt) Pizza Shop Demo - Part 5 Modelling on Amazon Athena
March 7, 2024
- Data Build Tool (dbt) Pizza Shop Demo - Part 4 ETL on BigQuery via Airflow
February 22, 2024
- Data Build Tool (dbt) Pizza Shop Demo - Part 3 Modelling on BigQuery
February 8, 2024
- Data Build Tool (dbt) Pizza Shop Demo - Part 2 ETL on PostgreSQL via Airflow
January 25, 2024
- Data Build Tool (dbt) Pizza Shop Demo - Part 1 Modelling on PostgreSQL
January 18, 2024
- Setup Local Development Environment for Apache Flink and Spark Using EMR Container Images
December 7, 2023
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 5 Athena
December 6, 2022
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 4 EMR on EKS
November 1, 2022
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 3 EMR on EC2
October 19, 2022
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 2 Glue
October 9, 2022
- Data Build Tool (dbt) for Effective Data Transformation on AWS – Part 1 Redshift
September 28, 2022
- Develop and Test Apache Spark Apps for EMR Remotely Using Visual Studio Code
September 7, 2022
- Manage EMR on EKS with Terraform
August 26, 2022
- Revisit AWS Lambda Invoke Function Operator of Apache Airflow
August 6, 2022
- Data Warehousing ETL Demo with Apache Iceberg on EMR Local Environment
June 26, 2022
- Develop and Test Apache Spark Apps for EMR Locally Using Docker
May 8, 2022
- EMR on EKS by Example
January 17, 2022
- Data Lake Demo using Change Data Capture (CDC) on AWS – Part 3 Implement Data Lake
December 19, 2021
- Data Lake Demo using Change Data Capture (CDC) on AWS – Part 2 Implement CDC
December 12, 2021
27 posts in total
- Deploy Python Stream Processing App on Kubernetes - Part 2 Beam Pipeline on Flink Runner
June 6, 2024
- Deploy Python Stream Processing App on Kubernetes - Part 1 PyFlink Application
May 30, 2024
- Real Time Streaming with Kafka and Flink - Lab 6 Consume data from Kafka using Lambda
December 14, 2023
- Real Time Streaming with Kafka and Flink - Lab 5 Write data to DynamoDB using Kafka Connect
November 30, 2023
- Real Time Streaming with Kafka and Flink - Lab 4 Clean, Aggregate, and Enrich Events with Flink
November 23, 2023
- Real Time Streaming with Kafka and Flink - Lab 3 Transform and write data to S3 from Kafka using Flink
November 16, 2023
- Real Time Streaming with Kafka and Flink - Lab 2 Write data to Kafka from S3 using Flink
November 9, 2023
- Benefits and Opportunities of Stateful Stream Processing
November 2, 2023
- Kafka Connect for AWS Services Integration - Part 5 Deploy Aiven OpenSearch Sink Connector
October 30, 2023
- Real Time Streaming with Kafka and Flink - Lab 1 Produce data to Kafka using Lambda
October 26, 2023
- Kafka Connect for AWS Services Integration - Part 4 Develop Aiven OpenSearch Sink Connector
October 23, 2023
- Building Apache Flink Applications in Python
October 19, 2023
- Real Time Streaming with Kafka and Flink - Introduction
October 5, 2023
- Kafka, Flink and DynamoDB for Real Time Fraud Detection - Part 2 Deployment via AWS Managed Flink
September 14, 2023
- Getting Started with Pyflink on AWS - Part 3 AWS Managed Flink and MSK
September 4, 2023
- Getting Started with Pyflink on AWS - Part 2 Local Flink and MSK
August 28, 2023
- Getting Started with Pyflink on AWS - Part 1 Local Flink and Local Kafka
August 17, 2023
- Kafka, Flink and DynamoDB for Real Time Fraud Detection - Part 1 Local Development
August 10, 2023
- Kafka Connect for AWS Services Integration - Part 3 Deploy Camel DynamoDB Sink Connector
July 3, 2023
- Kafka Connect for AWS Services Integration - Part 2 Develop Camel DynamoDB Sink Connector
June 4, 2023
19 posts in total
- Serverless Application Model (SAM) for Data Professionals
July 18, 2022
- Simplify Your Development on AWS with Terraform
February 6, 2022
- Yet another serverless solution for invoking AWS Lambda at a sub-minute frequency
October 13, 2021
- Adding Authorization to a Graphql API
July 20, 2021
- Dynamic Routing and Centralized Auth with Traefik, Python and R Example
November 29, 2019
- Distributed Task Queue with Python and R Example
November 15, 2019
- Linux Dev Environment on Windows
November 1, 2019
- AWS Local Development with LocalStack
July 20, 2019
- Cronicle Multi Server Setup
July 19, 2019
- Shiny to Vue.js
May 26, 2018
- Async Shiny and Its Limitation
May 19, 2018
- API Development with R Part II
November 19, 2017
- API Development with R Part I
November 18, 2017
- Serverless Data Product POC Backend Part IV - Serving R ML Model via S3
April 17, 2017
- Serverless Data Product POC Backend Part III - Exposing R ML Model via APIG
April 13, 2017
- Serverless Data Product POC Backend Part II - Deploying R ML Model via Lambda
April 11, 2017
- Serverless Data Product POC Backend Part I - Packaging R ML Model for Lambda
April 8, 2017
- Some Thoughts on Shiny Open Source - Render Multiple Pages
June 27, 2016
- Some Thoughts on Shiny Open Source - Internal Load Balancing
May 23, 2016
15 posts in total
- Asynchronous Processing Using Job Queue
May 12, 2016
- Boost SparkR with Hive
April 30, 2016
- Quick Start SparkR in Local and Cluster Mode
March 2, 2016
- Spark Cluster Setup on VirtualBox
February 22, 2016
- Quick Test to Wrap Python in R
November 21, 2015
- Packaging Analysis
March 24, 2015
- Parallel Processing on Single Machine - Part III
March 19, 2015
- Parallel Processing on Single Machine - Part II
March 17, 2015
- Parallel Processing on Single Machine - Part I
March 14, 2015
- Quick Trial of Adding Column
January 14, 2015
- Looping without for
December 17, 2014
- Short R Examples
December 3, 2014
- Summarise Stock Returns from Multiple Files
November 27, 2014
- Download Stock Data - Part II
November 21, 2014
- Download Stock Data - Part I
November 20, 2014