One small model of a shop that packs orders, taken through every desflow command in the order a simulation study uses them: get the model, check it makes sense, look at what it produces, run it many times, compare two options, and base its inputs on measured times. Each command is shown with its real output.

dynamic-des is now desflow. A simulation of a small shop shows what five releases added: customers who give up waiting and machines that break, a check that a model makes sense before it runs, and an answer to a question such as "would a third packer cut the wait?" that accounts for chance.

odctl is a command-line tool that runs an open-source data platform on your laptop: Kafka, Flink, Spark, Iceberg, ClickHouse, Trino, Airflow, MLflow, Feast and more, started with one command and wired together. Version 1.0 adds a documentation site with a guide for every service.

Test data for a streaming pipeline, from a settings file rather than a program: a plain YAML file describes a simulation, and one command runs it and sends its events to Kafka, PostgreSQL, Parquet or Iceberg.

Keeping top 10 leaderboards up to date as game scores arrive: four Flink SQL jobs read the scores from Kafka and keep the rankings in PostgreSQL, and a web dashboard shows them as they change.

Capturing every insert and update in PostgreSQL without changing the application: Debezium streams the changes to Kafka, and an S3 sink connector saves them as files.

Data engineering, stream processing, machine learning and MLOps projects that run on a laptop from a fresh clone, each explaining the system it builds.

Learning MLOps hands-on: the three projects from Jim Dowling's feature store book, rebuilt with open source tools that run locally.

Stopping a language model from inventing SQL over a lakehouse: a semantic layer between the model and Iceberg tables, built with Strands, WrenAI and Trino.

Generating test data that changes while it runs: SimPy simulations written with a declarative API that read settings from and write events to PostgreSQL and Redis.