Open source I build and maintain, and the repositories behind the posts. Each card links the code, and most link the writing that walks through it.
dynamic-des
Real-time SimPy control plane to dynamically update parameters and stream outputs via external systems like Kafka, Redis, or Postgres. Built for event-driven digital twins.
odctl
A curated collection of open source technologies and the odctl CLI for running a local Docker-based data platform to experiment with modern data architecture and MLOps.
nicegui-fastapi-template
A template for rapidly building full-stack web applications in Python, featuring a FastAPI backend, a NiceGUI frontend, PostgreSQL, and Docker.
agentic-analytics-system
A local, open source reference for conversational analytics over an Iceberg lakehouse: a Strands agent talks to a WrenAI semantic layer over MCP, with Trino execution and long-term memory.
oml-digital-twin-hotrolling
A streaming Digital Twin of a steel hot rolling mill demonstrating Online Machine Learning (OML) with Apache Kafka, Apache Flink and MOA to handle real-time concept drift.
kafka-pocs
Runnable Kafka examples: Docker Compose and Kubernetes clusters, Python producers and consumers, Kafka Connect pipelines, Glue Schema Registry integration, TLS and SASL security, and MSK ingestion into Redshift and Athena.
flink-demos
Apache Flink examples in PyFlink and Kotlin: local Docker clusters, AWS Managed Flink deployments with Kafka, a Flink SQL cookbook, and worked exercises from Confluent courses and the Stream Processing with Flink book.
dbt-cicd-demo
A dbt project on BigQuery with two GitHub Actions workflows: slim CI builds only modified models on pull requests, and deploy runs unit tests, pushes a Docker image and publishes dbt docs.
dbt-on-aws
Five dbt projects that transform IMDb data on Redshift Serverless, AWS Glue, EMR on EC2, EMR on EKS and Athena. Each folder holds the Terraform for the service plus the dbt models.
streaming-demos
Four streaming demos with Docker: Debezium CDC from PostgreSQL to a Pub/Sub emulator, a FastAPI WebSocket dashboard in Streamlit and Next.js, Kotlin Kafka and Flink clients, and a Flink-trained contextual bandit recommender.
data-lake-demo
Change data capture from PostgreSQL to a Hudi table on S3: Debezium and an S3 sink on Amazon MSK Connect, Hudi DeltaStreamer on EMR, queried with Athena and QuickSight. Includes a local Docker version.
emr-local-dev
A VS Code dev container from the EMR Spark image, plus Kafka, with PySpark examples for spark-submit, pytest, Jupyter and Structured Streaming. Batch examples use S3 and the Glue Data Catalog.