Batching gRPC calls in a stateless DoFn so one request covers a whole bundle, cutting the time a Beam Python pipeline spends on enrichment.

Deploying a dbt project to dev and prod on BigQuery, covering slim CI, unit tests and a write audit publish step that builds on a cloned dataset.

GitHub Actions gives a dbt project on BigQuery a slim CI run on pull requests and a deploy job that publishes the project as a container image.

The Shared class in the Beam Python SDK caches lookup data in memory for batch and streaming pipelines, with a periodic refresh for the latter.

Data augmentation in Beam Python by calling a gRPC service once per input element, running on a local Flink cluster with Kafka as the source.

A sport activity tracker in Beam Python, built first with native transforms and then with Beam SQL, showing the limits of Beam SQL in the Python SDK.

Two Beam Python pipelines compute average word length from a Kafka topic, one emitting a global average and one using a sliding time window.

Set up a local Apache Flink and Kafka environment, then build two Beam Python streaming pipelines for top K frequent words and longest word length.

Deploy an Apache Beam Python pipeline to a Flink session cluster on minikube, packaged as a Docker image and submitted as a Kubernetes job.

Deploy a PyFlink application to minikube with the Flink Kubernetes Operator, alongside a Kafka cluster that provides its source and sink topics.