Set up a local Apache Flink and Kafka environment, then build two Beam Python streaming pipelines for top K frequent words and longest word length.

Deploy an Apache Beam Python pipeline to a Flink session cluster on minikube, packaged as a Docker image and submitted as a Kubernetes job.

Deploy a PyFlink application to minikube with the Flink Kubernetes Operator, alongside a Kafka cluster that provides its source and sink topics.

Unit test batch and streaming Apache Beam pipelines in Python, using TestStream to advance watermarks and processing time across scenarios.

A streaming Apache Beam pipeline in Python that aggregates page visits by user in fixed 20 second windows, written with and without Beam SQL.

The Apache Beam portability layer explained, with bash scripts that manage local Flink and Kafka clusters and a streaming pipeline on the Flink Runner.

Ingest fake customer and order data into Kafka with the MSK Data Generator and save it to S3, with connectors deployed as Strimzi custom resources.

Send and read Kafka topic records from Python client applications built with the kafka-python package and deployed on Kubernetes.

Create a Kafka cluster on Kubernetes with Strimzi, the first step in a series on Python clients and Kafka Connect pipelines on Kubernetes.

Consume Kafka messages with an AWS Lambda function, using Amazon MSK as an event source so that Lambda polls the topic and invokes the function.