Consume Kafka messages with an AWS Lambda function, using Amazon MSK as an event source so that Lambda polls the topic and invokes the function.

Deploy a Kafka to OpenSearch pipeline on AWS with Terraform, covering a VPC, VPN server, MSK cluster, OpenSearch domain and the MSK Connect connectors.

Produce fake taxi ride data into a Kafka topic on Amazon MSK from an AWS Lambda function invoked repeatedly by an EventBridge schedule rule.

Kafka and PyFlink replace Amazon Kinesis in a real time analytics app from an AWS workshop. The original and the new architecture are compared.

Deploy a PyFlink app that reads and writes Kafka topics on Amazon MSK to Amazon Managed Service for Apache Flink, the managed Flink runtime.

Connect a PyFlink app to an IAM authenticated MSK cluster, building a custom uber jar because Amazon Managed Service for Apache Flink takes only one jar.

Deploy the Camel DynamoDB sink connector and its data generator source on Amazon MSK and MSK Connect, moving the local pipeline onto AWS.

A survey of the Kafka connectors available for AWS service integration, as the starting point for building real time data pipelines on AWS.

Stream Kafka records from Amazon MSK into Athena through its direct integration, with a producer Lambda built locally using AWS SAM and Terraform.

Stream Kafka records from Amazon MSK into Redshift through its direct integration, with a producer Lambda built locally using AWS SAM and Terraform.