Simplify Streaming Ingestion on AWS – Part 2 MSK and Athena

featured.png
March 14, 202312 min read Data Streaming Simplify Streaming Ingestion on AWSAmazon AthenaAmazon EventBridgeAmazon MSKApache KafkaAWSAWS LambdaAWS SAMPythonTerraform

Streaming ingestion from Kafka (MSK) into Redshift and Athena can be much simpler as they now support direct integration. In part 2, we discuss an end-to-end streaming ingestion solution using EventBridge, Lambda, MSK and Athena. We also use AWS SAM integrated with Terraform for developing the producer Lambda function locally.

Simplify Streaming Ingestion on AWS – Part 1 MSK and Redshift

featured.png
February 8, 202318 min read Data Streaming Simplify Streaming Ingestion on AWSAmazon EventBridgeAmazon MSKAmazon RedshiftApache KafkaAWSAWS LambdaAWS SAMPythonTerraform

Streaming ingestion from Kafka (MSK) into Redshift and Athena can be much simpler as they now support direct integration. In part 1, we discuss an end-to-end streaming ingestion solution using EventBridge, Lambda, MSK and Redshift. We also use AWS SAM integrated with Terraform for developing the producer Lambda function locally.