Apache Iceberg

Defining Data-Streaming Simulations in YAML, Without Writing Python

October 6, 20267 min read Data-Engineering Open Source Building Real-Time Digital Twins With Dynamic-Des Apache Iceberg Apache Kafka Discrete Event Simulation Dynamic-Des PostgreSQL Python SimPy YAML

Test data for a streaming pipeline, from a settings file rather than a program: a plain YAML file describes a simulation, and one command runs it and sends its events to Kafka, PostgreSQL, Parquet or Iceberg.

Learning MLOps with a Feature Store: A New Series

September 28, 20263 min read Machine-Learning Open Source MLOps With a Feature Store Apache Airflow Apache Iceberg Dynamic-Des Feast Feature Store MLflow MLOps Odctl

Learning MLOps hands-on: the three projects from Jim Dowling's feature store book, rebuilt with open source tools that run locally.

Building an Agentic Analytics System over an Iceberg Lakehouse

July 18, 20266 min read Data-Engineering Machine-Learning Open Source Agentic AI Apache Iceberg Dynamic-Des Mem0 Odctl Strands Trino WrenAI

Stopping a language model from inventing SQL over a lakehouse: a semantic layer between the model and Iceberg tables, built with Strands, WrenAI and Trino.

Running Kafka, Flink, Spark, Trino and Iceberg Locally with One CLI

July 16, 20264 min read Data-Engineering Open Source Apache Flink Apache Iceberg Apache Kafka Apache Spark Odctl Open Data Stack Python Trino

Running Kafka, Flink, Spark, Trino, Iceberg and Airflow together on a laptop: one CLI starts them as a single local stack, with MLOps and observability tools.