Data Lake Demo using Change Data Capture (CDC) on AWS - Part 2 Implement CDC

Data Lake Demo using Change Data Capture (CDC) on AWS - Part 2 Implement CDC
December 12, 202117 min read Data-Engineering Data Integration Data-Streaming Data Lake Demo Using Change Data CaptureAmazon EMRAmazon MSKApache HudiApache KafkaAWSChange Data Capture (CDC)DebeziumKafka Connect

Build change data capture on AWS with Amazon MSK and MSK Connect, streaming PostgreSQL row changes into the topics that feed the data lake.

Data Lake Demo using Change Data Capture (CDC) on AWS - Part 1 Local Development

Data Lake Demo using Change Data Capture (CDC) on AWS - Part 1 Local Development
December 5, 202118 min read Data-Engineering Data Integration Data-Streaming Data Lake Demo Using Change Data CaptureAmazon EMRAmazon MSKApache HudiApache KafkaAWSChange Data Capture (CDC)DebeziumKafka Connect

Set up the source PostgreSQL database with an outbox table, then run Debezium and an S3 sink connector locally with Docker Compose.