Transform IMDb data with dbt on Amazon EMR on EC2, running layered models through a Spark Thrift server.

Transform IMDb data in an S3 data lake with dbt on AWS Glue, running layered models through Glue interactive sessions with the dbt-glue adapter.

Transform IMDb data in Amazon Redshift Serverless with dbt, building layered models that follow dbt best practices.

Develop Spark apps on an EMR cluster in a private subnet over VPN and the VS Code remote SSH extension, with the cluster shared by several users.

Provision EMR on EKS with Terraform and EKS Blueprints, then compare two Spark jobs with and without Dynamic Resource Allocation under Karpenter.

Extend the Apache Airflow Lambda invoke function operator with a correlation ID, so it reports the invocation result and the exact error message.

Build a serverless data processing app with AWS SAM. A Lambda function runs when an S3 object is created and gets its third party packages from a layer.

Run a data warehousing ETL job with Apache Iceberg for storage and PySpark for processing in an EMR local environment, then verify results in Athena.

Create a Spark local development environment for Amazon EMR with Docker and Visual Studio Code, with Spark examples and Glue Catalog integration.

Deploy a Change Data Capture pipeline to AWS with the Apicurio registry on ECS, private subnets over VPN and the Avro converter packaged with the connectors.