Dynamic DES

Dynamic DES is a high-performance, real-time control plane for SimPy.

It bridges the gap between static discrete-event simulations and the live world by allowing you to update simulation parameters (arrivals, service times, capacities) and stream telemetry and events via Kafka, Redis, or PostgreSQL without stopping the simulation. It also transforms your models into synchronized forecasting engines by fast-forwarding through simulation time to predict future states or backfill Data Lakes with schema-enforced Parquet or JSONL files directly to AWS S3, Google Cloud Storage (GCS), Azure Blob, and SeaweedFS using PyArrow VFS.

Key Features

• ⚡ Real-Time Control: Synchronize SimPy with the system clock using DynamicRealtimeEnvironment.
• 🔗 Dynamic Registry: Dynamic, path-based updates (e.g., Line_A.arrival.rate) that trigger instant logic changes.
• 🛡️ Enterprise Ready: Native **kwargs passthrough for SASL, mTLS, OAuth, and AWS IAM Kafka clusters.
• 📦 Pluggable Serialization: Stream lightweight JSON by default, or map specific ML topics to lazy-loaded Avro/Schema Registry serializers.
• 🗄️ Data Lake Ready: Write chunked Parquet and JSONL datasets directly to object storage via PyArrow VFS, with built-in schema inference and drift prevention.
• 🦆 Pydantic Duck-Typing: Seamlessly publish strictly-typed Pydantic V2 models straight from your simulation logic.
• 🔋 Flexible Resources: DynamicResource provides prioritized queuing with graceful capacity shrinking.
• 📊 System Observability: Built-in lag monitoring to track simulation drift from real-world time.

Documentation Layout

• Getting Started: Quick installation and zero-setup demo commands.
• Core Concepts: Deep dive into the architecture, the Switchboard Pattern, and the shrinking paradox.
• Examples
  • Local Simulation: A dependency-free approach to testing and deterministic scheduling.
  • Kafka Digital Twin: A full event-driven architecture showcasing dynamic state updates and Pydantic event routing.
  • Historical Data & Forecasting: Fast-forward simulation time to generate datasets or predict future states.
• Advanced Guides
  • Connecting to Secure Kafka Clusters: Connect to enterprise environments natively (SASL, mTLS, OAuth, and AWS IAM).
  • Advanced Serialization: Avro & Pydantic: Leverage pluggable serialization, duck-typing, and schema registries (Confluent & AWS Glue).
  • Data Lake Ingestion & Storage: Configure PyArrow VFS for AWS S3, UUID chunking, and schema drift prevention.
• API Reference: Detailed documentation of classes and methods.