Stream Kafka records from Amazon MSK into Redshift through its direct integration, with a producer Lambda built locally using AWS SAM and Terraform.

Configure a Kafka consumer to seek offsets by timestamp while partitions are assigned by subscription, on a single node cluster in Docker Compose.

Extend the Apache Airflow Lambda invoke function operator with a correlation ID, so it reports the invocation result and the exact error message.

Build a serverless data processing app with AWS SAM. A Lambda function runs when an S3 object is created and gets its third party packages from a layer.

Run a data warehousing ETL job with Apache Iceberg for storage and PySpark for processing in an EMR local environment, then verify results in Athena.

Create a development environment for AWS Glue 3.0 and later by building a custom Docker image, because AWS publishes no image for those versions.

Build development environments for AWS Glue 1.0 and 2.0 with the published Docker image and the Visual Studio Code Remote Containers extension.

Run frequent Apache Airflow tasks on AWS Lambda instead of Celery workers or ECS, with a custom operator modelled on the ECS operator.

Set up path-based routing with Traefik and Docker for Python and R services, then add centralized authentication with Forward Authentication.

A FastAPI web service dispatches tasks to Celery and RServe workers, with Redis as broker and store, running on Docker Compose and Kubernetes.