A survey of the Kafka connectors available for AWS service integration, as the starting point for building real time data pipelines on AWS.

Integrate Python Kafka producer and consumer apps running on AWS Lambda with the Glue Schema Registry, which manages and validates message schemas.

Stream Kafka records from Amazon MSK into Athena through its direct integration, with a producer Lambda built locally using AWS SAM and Terraform.

Stream Kafka records from Amazon MSK into Redshift through its direct integration, with a producer Lambda built locally using AWS SAM and Terraform.

Configure a Kafka consumer to seek offsets by timestamp while partitions are assigned by subscription, on a single node cluster in Docker Compose.

Deploy a Change Data Capture pipeline to AWS with the Apicurio registry on ECS, private subnets over VPN and the Avro converter packaged with the connectors.

A local Change Data Capture setup on Docker Compose with Debezium, the Confluent S3 sink and the Apicurio registry, showing how schema evolution is managed.

Run a Hudi DeltaStreamer application on Amazon EMR, then query the resulting Hudi table with Athena and build a QuickSight dashboard on it.

Build change data capture on AWS with Amazon MSK and MSK Connect, streaming PostgreSQL row changes into the topics that feed the data lake.

Set up the source PostgreSQL database with an outbox table, then run Debezium and an S3 sink connector locally with Docker Compose.