Skip to content

Changelog

Each release, newest first: what changed, then under Upgrading what to change in code, commands or installs that use the previous release.

Version 0.28.0

  • SimulationRegistry.change sets a value and returns a ParamChange that says what was stored, or raises ParamRejected when the registry refuses it. See Know Whether a Change Was Refused.
  • on_change, on the registry and on SimulationContext, is a list of callbacks that receive a ParamChange for every accepted change, with its source: an ingress connector and its class, a scenario step, or the overrides of start(). See Follow Every Change.
  • update, set_exactly and start() take source, actor and reason, which each ParamChange carries.
  • SimulationContext.registry_paths() returns the paths and starting values of a run without running it. See Find the Path.
  • FileIngress, type: File in YAML, changes parameters during a run when you edit a local YAML file, with no service. See File and the File Steering example.
  • The AI skill and Use an AI Assistant teach how to draft a model from a description of a process.

Upgrading

  • A scenario step whose value the registry refuses, such as an exponential rate of 0, now stops the load of a blueprint with its line, and start() of a builder model. start() checks the scenario with its overrides, so an override or an experiment design that makes a later step's value invalid also stops the start. A run stops with ParamRejected at a step's time when an ingress message, an on: action or a process has made the step's value invalid, and a low-level run_scenario raises it for any refused value. Before 0.28.0 the value was logged and ignored, and the run went on. Values sent by ingress connectors and on: actions are still logged and ignored. See Stability and Limits.

Version 0.27.0

  • GlueAvroSerializer and GlueAvroDeserializer talk to AWS Glue with desflow's own client and wire format, so the glue extra no longer depends on aws-glue-schema-registry. The arguments and the bytes on the topic are the same.
  • The kafka extra no longer installs kafka-python. KafkaAdminConnector.create_topics and the topic creation of KafkaEgress use aiokafka.
  • JSON Schema and Protobuf are read and written through both schema registries, next to Avro, by ConfluentJsonSchemaSerializer, ConfluentJsonSchemaDeserializer, ConfluentProtobufSerializer, ConfluentProtobufDeserializer, GlueJsonSchemaSerializer, GlueJsonSchemaDeserializer, GlueProtobufSerializer and GlueProtobufDeserializer. See JSON Schema and Protobuf.
  • The builder and low-level scripts of the Kafka, PostgreSQL, Redis and Related Records examples stop at EXAMPLE_UNTIL simulation seconds when that variable is set, and run until Ctrl + C when it is not.
  • A new How-To group, Storage and Tables, has two pages: Write to Cloud Storage writes Parquet and JSON Lines files to S3, Google Cloud Storage or Azure Blob Storage, and Write to Iceberg Tables appends to a table through a SQLite, REST, PostgreSQL or Glue catalog. Each has YAML, Builder and Low-level forms.

Upgrading

  • The glue extra no longer installs aws-glue-schema-registry. Code that imports aws_schema_registry itself must add it to its own dependencies.
  • Glue schema registration waits up to 30 seconds (10 checks, 3 seconds apart) for a new schema version to become available. aws-glue-schema-registry 1.1.3 waited 3 seconds once and then checked 10 times with no wait between checks.
  • Glue registry and framing failures raise desflow's SchemaRegistryError and SerializationError, importable from desflow.connectors.egress.kafka and desflow.connectors.ingress.kafka, not aws_schema_registry.SchemaRegistryException.
  • GlueAvroDeserializer compiles a JSON schema version with fastjsonschema, as aws-glue-schema-registry 1.1.3 did, and the glue extra needs fastjsonschema 2.22.1 or later, where 1.1.3 accepted any 2.x release from 2.15. A JSON schema version that fastjsonschema cannot compile, or a record that is not JSON, does not match the schema or cannot be checked against it, raises SerializationError, where 1.1.3 raised the exception of fastjsonschema, orjson or Python, or its own ValidationError.
  • The confluent extra also installs jsonschema and protobuf, and the glue extra also installs protobuf.
  • The kafka extra no longer installs kafka-python. KafkaAdminConnector takes aiokafka's admin settings, so a kafka-python-only setting such as ssl_cafile is refused: pass ssl_context. A kafka-python token provider is refused too: pass KafkaAdminConnector the aiokafka provider that KafkaEgress takes, as AWS MSK (IAM) shows. Code that imports kafka itself installs kafka-python separately. KafkaAdminConnector.create_topics raises the aiokafka.errors class for a broker error, such as aiokafka.errors.InvalidReplicationFactorError, where it raised the kafka.errors class: catch aiokafka.errors.KafkaError instead.

Version 0.26.0

  • The confluent extra no longer installs confluent-kafka. desflow writes and reads the Confluent Schema Registry wire format itself with httpx and fastavro, so the extra no longer depends on a package without Python 3.15 support. The bytes on the topic are the same.
  • ConfluentAvroSerializer and ConfluentAvroDeserializer take named arguments for a secured registry: auth, verify, cert, timeout, headers, proxy and max_retries. See Registry Connection.
  • A Parquet or JSON Lines egress writes to Google Cloud Storage and Azure Blob Storage, from a filesystem mapping with type: gcs or type: azure, in YAML, the builder and the low-level API. See Parquet and JSON Lines.
  • An Iceberg egress writes through a SQL catalog on SQLite or PostgreSQL and through a Glue catalog, and the iceberg extra installs what they need. See Iceberg.
  • Python 3.15 is supported. On 3.15 the kafka, confluent and glue extras build aiokafka, and the iceberg extra builds pyiceberg and zstandard, from source, which needs a C compiler. See Installation.
  • GlueAvroSerializer serializes a record. It failed on every record in every release since 0.7.0.
  • The docs are reorganised. Learn holds the tutorial and, after it, the concept pages, each explained from the tutorial's shop. Command Line and SimPy and desflow are sections of their own. The examples are the Runnable Examples group of How-To, and its overview page holds Services for the Examples, the odctl profiles each example needs, which moved from Installation.
  • Model code is shown in YAML, Builder and Low-level tabs on every page except those about one way of writing, such as Builder API. Every How-To script has all three forms, each with a test, except on the two pages about blueprints alone, Call Python from YAML and Set Up Your Editor.
  • The Cheat Sheet and YAML Format pages are removed. YAML Keys is the one list of blueprint keys, and both old addresses lead to it.
  • Each page's address follows the sidebar, such as cli/run/ for desflow run and how-to/modelling/breakdowns/ for Model Breakdowns. Every old address, and every old link to a moved section, leads to the new place.

Upgrading

  • pip install "desflow[confluent]" no longer installs confluent-kafka. Code that imports confluent_kafka itself installs it separately.
  • The confluent and glue extras no longer install dataclasses-avroschema, which desflow's own code does not use. Code that builds its schema with AvroBaseModel installs it with pip install "dataclasses-avroschema[pydantic]".
  • The confluent-kafka keys passed as **registry_kwargs, such as basic.auth.user.info, still work with a FutureWarning and are removed in 0.30.0. Use auth=, verify=, cert=, timeout=, proxy=, headers= and max_retries= instead. OAuth bearer sources other than STATIC_TOKEN are refused: pass the token in headers=. retries.wait.ms, retries.max.wait.ms and url are refused: set max_retries= and pass the URL as registry_url. cache.capacity and cache.latest.ttl.sec are ignored.
  • Registry and framing errors are SchemaRegistryError and SerializationError, importable from desflow.connectors.egress.kafka or desflow.connectors.ingress.kafka, not confluent-kafka's classes.
  • The subject is always <topic>-value. confluent-kafka 2.14.0 and later first asked the registry for a topic association, so on a registry with associations defined it could use another subject.
  • The deserializer looks up a writer schema by its id alone. confluent-kafka 2.11.0 and later also sent the subject, which makes the registry search every schema context, so a schema registered only in a context other than the default one is no longer found.
  • A writer schema with enabled Data Contract read rules (domain or encoding) is refused, including a rule with onFailure: NONE, which confluent-kafka skipped.
  • A connection error to the registry is retried up to max_retries times (3 by default), with a random wait of up to 1, 2 and 4 seconds; it failed at once before.
  • Each step of a request to the registry (connecting, sending, reading) times out after 10 seconds. It never timed out before. Pass timeout=None for the old behaviour.
  • The parquet extra requires pyarrow 16.0.0 or later. It was 14.0.0. Azure Blob Storage needs AzureFileSystem, which pyarrow has from 16.0.0. On Windows, PyPI's pyarrow wheels include it from 24.0.0.

Version 0.25.0

  • desflow explain FILE shows a model as a graph from its arrivals to its exits, without running it. For each resource it gives the utilisation, the status and, where the queueing formulas apply, the waits. It also names the bottleneck and the capacity that brings utilisation to a target. Where a formula does not apply, it says why. --json prints the same result as JSON, which is experimental through 1.x. See Explain the Load.
  • desflow explain --verify also runs the model unpaced, once or with --replications N, and compares its statistics with the formulas. context.analyze() gives the same result in Python. See Check Against Theory.
  • desflow dist lists the distribution families, shows one family's parameters, rules and syntax, and converts its parameters to a mean and standard deviation and back, such as desflow dist gamma --mean 1.5 --std 0.4832. See Measured Times.
  • desflow run FILE --sample N prints the first N events as JSON lines, unpaced, with no ingress and no real sinks. desflow run FILE --stdout runs the model as usual but prints its records as JSON lines instead of sending them to the file's sinks. It prints every record except the system.simulation.lag_seconds readings. In Python, build, SimulationContext.from_yaml and load_context take preview=Preview(sample=N) for --sample and preview=Preview() for --stdout, with Preview from desflow.blueprint.build. See Preview the Records.
  • ConsoleEgress(format="json"), and type: Console with config: {format: json} in YAML, write each record to stdout as one JSON line, while the log stays on stderr. See Console.
  • load_context(folder, entry), exported from desflow, loads a YAML blueprint or a builder Python file from a local folder or from object storage, such as s3://. See Loading a Model.
  • add_arrival(..., spawn="task") and add_arrival_replay(..., spawn="task") start the named task for each arrival, as spawn does in YAML, so a builder model needs no @app.arrival_loop for it. See Start Them from Arrivals.
  • Router.probabilities gives the share of finished instances each next task receives, where the routing rules fix it. See Task Flow.
  • The Reference section gains one CLI page for each command, generated from the command's help.
  • The CLI code moves to desflow.cli. desflow.blueprint.cli re-exports app and ddes for dynamic-des 0.20.0's ddes command, until the release that removes ddes.
  • The skill for AI assistants gains references/cli.md, the help of every command, and references/api.md, every name desflow exports. Its study steps use desflow experiment and desflow fit. See Use an AI Assistant.
  • desflow run prints a Console sink's records as they are, one line each, as the pages show them. Before, the log's time, level and logger name started the first record of each batch. See Console.
  • When a blueprint or the builder gives a resource, store or container a current_cap above its max_cap, the model now lowers it to max_cap with a warning when it is built, as a live update does. desflow explain then analyses the capacity the run uses. Before, the run lowered it silently, and explain used the larger value.
  • The examples, snippets and pages address the odctl services as 127.0.0.1, such as 127.0.0.1:9092, not localhost. odctl 1.1.0 and later publish every port on 127.0.0.1 only. On macOS localhost resolves to ::1 first, so the Kafka clients could not connect. See Services for the Examples.
  • When the installed release's tag is not published yet, desflow examples get says so and suggests --ref main, instead of saying the example was added after this release.
  • desflow examples get --dest DIR prints the run command with DIR in its path. It printed the path without DIR.
  • desflow on its own lists the commands grouped by task, as odctl does. Its help, and the --help of each command, end with examples. See Command Line.
  • desflow examples list lists the examples, as desflow examples on its own still does. desflow examples get also prints the desflow explain command of an example whose bottleneck it names. See Example Projects.
  • The docs gain the tutorial Complete Model, a whole study from observed data to a recommendation. They also gain the How-To page Check Against Theory and the User Guide page Command Line, which uses every command and option on one shop. Every page is rewritten for a first-time reader. Each term of theory is explained in one section, and other pages link to it. The How-To pages are grouped into Modelling, Data and Kafka, and Tools and Workflow. Each page ends with links to the previous and next pages. The tutorials, the User Guide, the How-To pages and the example Quick Starts use each command where its output answers the page's question, such as desflow explain on Priorities and End Conditions and desflow run --stdout on Patience. Links to files in the repository open the files of the release.
  • The sidebar is regrouped. The User Guide has four groups, Writing a Model, Model Parts, Results, and Live Runs and Connectors, then Command Line. The How-To pages are in four groups and are named for their task, such as Model Breakdowns. The User Guide and How-To sections each open on a page that says what the section is for, and the Examples section is named Example Projects. Coming from SimPy moves to Learn, after SimPy and desflow, and gains the two tables that end SimPy and desflow before: what desflow changes, and the form of each SimPy construct in desflow. Every page keeps its address.

Upgrading

  • desflow run FILE.py exits 2 with Error: desflow run takes a YAML blueprint; run a Python file with python FILE. In 0.24.0 it exited 1 with a YAML parse error. Run a builder file with python FILE.py.
  • desflow fit and FitResult.text() label the KS columns KS stat and KS p (approx.), where 0.24.0 printed KS and p (approx.). The two columns are 8 and 14 characters wide, not 7 and 11. A script that reads the table by column position reads the new widths.
  • A tool that read the time and level before a Console record of desflow run reads the record alone. Every other change is an addition or a fix.

Version 0.24.0

Upgrading

  • resource.queue.items is removed, as the 0.23.0 Upgrading section announced. resource.queue is the list of waiting requests in service order, and [request for _rank, request in resource.queue.items] becomes list(resource.queue). Every other change is an addition or a fix.

Version 0.23.0

  • A wait nests any, all and at_least to any depth, and at_least: n with of: is done when n of its branches are, as SimPy's Condition with a count rule. An empty any or all succeeds at once. A request, get or put can be a branch, a won request holds its unit through the service, and a losing branch is cancelled. Request branches count in the order of acquisition, and the requests of one all are made together. See Wait for One or All.
  • A timeout branch can carry a value that the waiter receives, and a running branch waits until the running instances of a task on the waiter's entity, and the entities it spawned, all end. See Signals and Waiting.
  • process.name, process.instance and process.target in a condition or an expression read the running task, as SimPy's active process, and t, the simulation clock, is now checked against SimPy's env.now. See Read the Current Time and Processes and the Low-level API.
  • spawn on a task starts a task as a new entity without waiting for it, keyed <task>-n with the spawning entity's key in attr.parent, and after starts it later, as SimPy's start_delayed. subscribe interrupts a task when an event happens, as SimPy's subscribe_at. See Signals and Waiting.
  • signals: {closing: {once: true}} (add_signal(name, once=True), Signal(env, name, once=True)) is a signal that triggers once, as SimPy's Event does: a wait after the firing resumes at once and a second firing is refused. See One-Shot Signals.
  • on: on a task or a signal runs actions when it fires or ends: fire a signal with a value, or copy another outcome with from, spawn a flow, interrupt a task, or set a value. A fire on a one-shot signal takes urgent and after, and env.schedule(event, priority, delay) schedules a low-level event. See Actions When an Event Happens.
  • run.on_unhandled_failure: stop ends the run with the error when a task, child flow or one-shot signal fails and nobody handles it, as SimPy does; continue, the default, keeps the run going as before. See Failures.
  • The process-events example shows a one-shot signal, a spawn after a delay, a subscribe and an on: action. See Process Events.
  • advance(until) runs a started builder or YAML run to a time or an event and returns with the run open, finish() runs on to the end, and resume_pacing() paces again after a pause. A run in segments writes the same records as one run. See Run in Segments.
  • simulation.environment: !python module.Class (SimulationContext(environment=Class)) runs the model on a subclass of DynamicRealtimeEnvironment, so an override of step or schedule sees its events. See Run on Your Own Environment.
  • simulation.strict: true in YAML and SimulationContext(strict=True) in the builder raise RuntimeError when a paced run falls more than factor seconds behind, as the low-level strict does; both default to False. See Strict Pacing.
  • sync() on the environment resets the wall-clock anchor, as SimPy's does, so strict stops counting the time before it. See Strict Pacing.
  • A resource may leave its capacity out: current_cap is then 1 and max_cap is current_cap, so resources: {packers: {}} and add_resource("packers") declare one unit, and a DynamicResource with no registry entry registers itself with one unit, as simpy.Resource(env) has. current_cap: .inf (float("inf")) is a resource with no limit; it publishes capacity as null and a utilization of 0. A store or container may leave its capacity out too, and then has no limit, as simpy.Store(env) and simpy.Container(env) have: stores: {parcels: {}}, add_store("parcels"), DynamicStore(env, "Shop", "parcels"). A max_cap alone gives the current_cap, and current_cap: .inf also means no limit. A store or container with no limit publishes capacity as null. Before, leaving a capacity out was refused in every API, and a store with an infinite capacity failed in all three. See Resources and Stores and Containers.
  • users.<resource> and queue.<resource> in a condition read how many units are held and how many requests wait, and get_resource(name).count, .users and .queue read them in code. A granted request has usage_since, and a release is an event, DynamicRelease, that holds its request. See Resources.
  • queue_limit on a resource refuses a request made while that many wait, as SimPy's SortedQueue(maxlen): a task balks and publishes balked with the resource, a request branch of a wait fails the wait, and the low-level request() raises QueueFull, a RuntimeError. See Queue Limit.
  • ResourceBase is the class DynamicResource is built on, with SimPy's hooks: subclass it, override _do_put or _do_get, set PutQueue or the request and release classes, and name the class with type: !python module.Class or add_resource(..., type=Class). See Custom Resource Types.
  • StoreBase and ContainerBase are the classes DynamicStore and DynamicContainer are built on, with SimPy's hooks: subclass one, override _do_put or _do_get, set PutQueue or GetQueue, or bind put and get to event classes of your own, and name the class with type: !python module.Class or add_store(..., type=Class) and add_container(..., type=Class). See Stores and Containers.
  • putters.<name> and getters.<name> in a condition read how many puts and gets wait at a store or container, and count(<store>), items(<store>), in and not in read what a store holds. A condition that reads one of them is checked again when a put or get joins or leaves a queue. See Stores and Containers.
  • wait: false on a task's put or get ("wait": False in the builder) makes the request and goes on without waiting for it, as a SimPy put that is not yielded. See Stores and Containers.
  • A store priority in YAML is kept as given, so a list, a mapping, null or a date is accepted, as in the builder. Before, only a number or an expression was, and the others were refused at load. The priorities in one store must still be orderable against each other. See Stores and Containers.
  • The fuel-dock example shows a store type of your own, a put that is not waited for and a wait on putters.. See Fuel Dock.
  • A YAML experiment file whose check sets on: mean or on: interval, the documented key, now loads. In 0.22.0 YAML read the key as True and the load failed with "Keys should be strings".
  • A task that balks with balk: now gives back the units it kept from the task before it with keep; in 0.22.0 such a unit was held for ever.
  • Raising a store's capacity to a finite value granted only one waiting put; it now grants every put that fits, as raising a resource's capacity grants every request. A rise to no limit already did, and 0.22.0 has the defect.
  • The User Guide pages Resources, Tasks and Processes, Time and Pacing, and Experiments are split into one page per topic. A link to a section that moved, such as user-guide/resources/#breakdowns, opens the page that holds it now.
  • Every SimPy 4.1.2 feature on the SimPy Compatibility page is now native in all three APIs or a difference kept by design, so the page has no Not Yet section. A later SimPy release's features are made native in the desflow release that moves to it.
  • A run that ends at a statistics interval time published every value twice at that time in 0.22.0. It now publishes each value once at that time, plus the values that changed after the interval's reading, so the last value of each statistic is the state at the end. This applies when the run ended there on a stop condition, on an event given to run(until=), at the end of a drain, or with close() after step() had reached that time.

Upgrading

  • A task's SimPy process is named after the task, not run. The process of a task in a root flow ends with the instance's finished payload as its value, also when it hands on to the next task; it was None. It is still None when the instance balks, reneges or is abandoned, and when it fails under on_unhandled_failure: continue; under stop a root task that fails with no on_fail raises TaskFailed. A child flow's process still ends with the flow's last payload.
  • Tasks without a priority on a queue: priority resource, and DynamicResource.request() called without a priority, rank 0, not 1, as SimPy's default. A task with no priority now goes ahead of one with priority 1.
  • A preempted event carries preempted_by, the key of the entity that took the unit, and usage_since, the time the preempted task was granted it. Storage sinks gain two columns.
  • resource.queue is a list of the waiting requests in service order, where resource.queue.items held (rank, request) pairs. queue.items still works with a FutureWarning and is removed in 0.24.0.
  • A store priority keeps the value it is given: a priority whose expression gives a numeric string, such as attr.priority set to '10', now compares as text (a literal 10 or '10' in a blueprint is still a number), and priorities of mixed types raise TypeError at the put instead of ValueError. A True priority, which the builder refused and YAML cast to 1.0, is kept as True, with the same service order.
  • putters_waiting and getters_waiting on DynamicStore and DynamicContainer, and queue on DynamicResource, no longer count a put, get or request that is being granted, which SimPy still holds in its queue during the grant. A watch_changes callback called during a grant read 1 in 0.22.0 and reads 0, also for a request granted at once.
  • add_resource and add_store refuse a fractional current_cap or max_cap, as YAML does; 0.22.0 rounded it down.
  • The low-level strict default is True, as SimPy's: a paced run more than factor seconds behind raises RuntimeError. Pass strict=False for live runs with sinks, as the low-level examples now do. The builder and YAML (simulation.strict) default to False. strict is never checked at factor 0, including before go_live_at. See Strict Pacing.
  • step() and peek() count only the model's events, not desflow's statistics, flush, lag, ingress and watcher processes: peek() skips them and step() runs the ones due before the model's event in the same call, so a script steps a model as it would in SimPy. run() still processes every event. A script that counted statistics readings as steps counts fewer. See Run Length.
  • run() and finish() return the value of the event that stopped the run, as SimPy's env.run(until=event) does; they returned None. A stop on a signal that fails now raises SignalFailed after the run closes, where a re-arming signal's failure used to end the run with no error; desflow run exits with an error. See Run Length.

Version 0.22.0

The low-level API is SimPy, with desflow's own resources, stores and containers, so SimPy code runs there. The SimPy Compatibility page lists every modelling feature of SimPy 4.1.2 with its status in each API: 48 are native in YAML, the builder and the low-level API and checked against plain SimPy, 48 are not yet, which 0.23.0 makes native, as the Roadmap says, and 2 are differences kept by design.

  • A SimPy Compatibility page lists every SimPy 4.1.2 modelling feature with its status in each API, its forms and the tests that compare it with SimPy.
  • Patience and balking: a task's patience takes a duration such as 30 s, and orders that leave or refuse to join are counted as tasks.<task>.reneged and .balked. See Patience and Reneging and Balking.
  • A store can start with items: stores.<name>.items, add_store(..., items=[...]) and DynamicStore(..., items=[...]); a priority store takes [priority, item] pairs. initial_time takes a duration such as 8 h. See Stores and Containers and Run Length.
  • A store get can take a filter, such as item.area == attr.area, and patience covers the wait for a get as well as for a resource; item_field adds the item a YAML get took to the payload. See Stores and Containers and Filtered Inventory.
  • Tasks interact as SimPy processes do: signals that tasks fire and wait for, a wait for one or all of signals, child flows, timeouts and conditions, and fail, fail_reason and on_fail for a task that fails. context.spawn returns the process it starts, and context.call starts a child flow. See Process Interaction, Signals and Waiting and Wait for One or All.
  • Interrupts and preemption: an interrupt on a task or scenario step, or context.interrupt, interrupts running task instances; on_interrupt: resume, restart or abandon says what each does; preempt: true on a priority resource lets a lower priority value take a busy unit. See Interruptions and Priority and Preemption.
  • A task can hold several resources, resource: [picker, bench], acquired in that order, and hand a unit to the next task with keep; a model that acquires resources in opposite orders is refused. See Competing Resources and Several Resources.
  • End conditions: run.stop ends a run at a signal or a condition, and run.drain (or desflow run --drain) stops the arrivals at until and finishes the entities already in the model; start(), step(), peek() and close() step a builder or YAML run one event at a time, and finish() runs a started run on to its end. The statistics count flows.<arrival>.started and in_system, and tasks.<task>.in_progress. See End Conditions and Run Length.
  • A low-level request can be cancelled with request.cancel() and given back with resource.release(request), as in SimPy. See Resources.
  • Breakdowns: resources.<name>.breakdown, add_breakdown and run_breakdowns make a resource fail at random and be repaired, interrupting the work it was doing or letting it finish; each failure and repair publishes down and repaired, and the statistics count resources.<name>.down, downtime and failures. The Preemptive Breakdowns guide becomes Breakdowns. See also Priority and Preemption.
  • Schedules in every API: the builder's add_scenario and add_step, and the low-level run_scenario, take the same steps as a blueprint's scenario. See Time-varying Arrivals and Changing Demand.
  • Replay: an arrival's replay, add_arrival_replay and the low-level read_trace take arrivals from the rows of a CSV or Parquet file, or a list of dicts, at their recorded times, with the row's fields as attributes. See Local Replay and Arrivals and Services.
  • Faster runs at factor: 0: SimPy's own step processes each event, a record's timestamp is formatted once per second of the clock, and ConsoleEgress logs a batch with one call. A low-level workload runs 1.14 times faster than in 0.21.0, and printing every record costs about a seventh of what it did. See Performance Tuning, whose measurements benchmarks/throughput.py repeats.
  • simulation.lag_interval and SimulationContext(lag_interval=) set how often system.simulation.lag_seconds is published. See Clocks and Pacing.
  • The record contract: Records and Telemetry gives every field its type, and a docs test fails when the page and the payload models differ. simulation.timezone, SimulationContext(timezone=) and DynamicRealtimeEnvironment(timezone=) name the zone timestamps are written in, and Statistics defines which entities each count covers.
  • Replications: desflow run FILE --replications N and desflow.replicate run a model with seeds spawned from one seed, ending each replication at until, a stop or the end of a drain, and give each statistic's mean and 95% half-width; the model's sinks receive the result once, as <sim_id>.stats.<measure>.replication.<i>, .mean, .half_width and .n. See Replications and Trusting Results.
  • Experiments: desflow experiment and desflow.run_experiment compare designs with paired replications and sweeps, ending each run at until, a stop or the end of a drain, apply checks that fail the command, compare with observed values, and write CSV and JSON results and a report.md; desflow report builds the report again. See Experiments, Comparing Designs and Parameter Sweeps.
  • Input fitting: desflow fit and desflow.fit_column fit a column of measured times to every distribution family, with diagnostics and a plot, and write settings a blueprint loads with the new !include tag. See Input Fitting, Fit the Inputs and Fit a Distribution.
  • An unknown key in a blueprint or an experiment file names the closest key that place takes, such as "did you mean 'patience'?", and a misspelt name of a resource, task, signal or path names the closest one; the errors this release adds say how to fix them.
  • Easier to start: the Home page has a "Where to Start" table, the Cheat Sheet gives the common YAML keys and builder calls side by side, and every tutorial and How-To page gives its level: Basics, Intermediate or Advanced.

Upgrading

  • desflow needs SimPy 4.1.2 or later, where it accepted 4.0.0, and pydantic 2.6.0 or later, where it accepted 2.0.0, with which desflow 0.22.0 does not import. The conformance tests check desflow against SimPy 4.1.2, which chooses the user to preempt differently from 4.1.1 on equal keys; SimPy 4.0 also imports pkg_resources, which a new Python environment does not have.
  • A store or container built with a capacity of 0 now works, as a resource does, and holds nothing until its capacity rises; SimPy's ValueError stopped it before.
  • A priority on a store get is refused; priority orders a put into a priority store.
  • Resources grant units with the same events as SimPy's Resource, so requests at the same simulation time are granted in SimPy's order, which can change seeded output where such ties occur.
  • patience now covers a task's whole wait, its wait, its resource and its get, where it covered only the resource, so a task can renege while it waits for stock. It counts waiting time only, so a preempted task that queues again goes on with the time it has left. A task with patience needs a resource, a get or a wait.
  • A store without priority is now a SimPy FilterStore, so a get can take a filter. While no waiting get has a filter, it grants gets as simpy.Store does, one waiting get each time it is triggered, as before; while a filtered get waits, it looks at every waiting get each time, as FilterStore does, so gets at the same instant can be granted in another order.
  • A builder task whose function raises, or a YAML task whose !python payload function raises, now fails the task, publishing failed with the error as its reason and ending the entity's flow, instead of stopping the run.
  • DynamicResource.pool and DynamicResourceRequest.cancelled are gone: the resource grants units as SimPy's Resource does. Call request.cancel() to leave the queue; resource.queue.items holds (rank, request) pairs, where it held PriorityItems. A request that gives up, by cancel() or by reneging, now leaves the queue at once, so len(resource.queue.items) and the queue_length and queue_length_avg statistics no longer count it.
  • The built-in statistics publish new names: tasks.<task>.in_progress from minimal, and flows.<arrival>.started and in_system from basic. A low-level process that calls env.stats.task_queued, task_started and task_finished publishes in_progress too, and one that calls flow_started publishes the flow counts.
  • Events with the statuses balked, reneged, failed, interrupted, preempted and resumed, and a resource's down and repaired events, keyed by its name, reach the sinks, so a consumer that filters on status should expect them.
  • A scenario step now applies before every other event at its time, as SimPy applies its own until, so an arrival at the step's time draws its next gap from the new value; seeded runs whose steps fall on the time of another event can give other numbers.
  • At factor: 0, system.simulation.lag_seconds is no longer published unless lag_interval (low-level lag_monitor_interval) is set; a run that goes live publishes it from the go-live instant. At a factor above 0 it is published every simulation second, as before.
  • ConsoleEgress logs each batch with one call, one line per record, so the logging format's prefix starts each batch rather than each line. Code that reads the console log record by record splits each message into lines.
  • At factor: 0, with no go_live_at still to come, each event is processed by SimPy's own step and strict=True is not checked, as there is no wall clock to fall behind; while a go_live_at is pending, the real-time step runs as before.
  • Every timestamp now carries a time zone offset, such as 2026-10-03T12:00:00.000+10:00, so a consumer that parses it must accept +HH:MM, and a sink column without a zone, such as a PostgreSQL timestamp, now gets the UTC wall time. By default timestamps are in UTC, +00:00, where they were in the machine's local time with no offset; set timezone to an IANA name to write them in a zone. A logical_start_time or go_live_at without an offset, now and the default start are read in timezone, else in the zone of logical_start_time, else in UTC, not in local time: for a start or go-live of now, pass datetime.now(timezone.utc), not datetime.now(), whose local wall time would be read in that zone and be off by the difference. env.start_datetime always has a time zone. A start and a go-live, one with an offset and one without, no longer fail. The clock runs in UTC, so a daylight-saving change no longer adds or drops simulation time.
  • A Parquet file now has a column for every key of any row in its batch, where the first row's keys alone were written and later keys were lost; a key that first appears in a later batch adds a column to the later files, and an integer column that later gets a float becomes a double. An Iceberg table not named in schemas gains such a key as an optional column. A batch a Parquet or Iceberg sink cannot write is logged at ERROR, where it stopped the sink with no error.
  • A container's starting level keeps its type in YAML, the builder and the low-level API, so level: 1, level=1 or init=1 gives a level of 1, as SimPy does, where it gave 1.0.
  • A payload key that starts and ends with two underscores, such as __table__, is reserved: Kafka, Parquet, JSON Lines, Iceberg and the console drop it, with a warning, where they wrote it as data. Rename such a key, such as __source__ to source, to keep it. PostgreSQL and Redis still route by __table__ and __stream__. Routers, such as a Kafka topic_router or a storage path_router, no longer see the dropped keys; PostgreSQL and Redis still write any other such key as data.
  • Fitting needs the new fit extra, which installs scipy and matplotlib: pip install "desflow[fit]". The all extra includes it, so desflow[all] now installs both.
  • The glue extra needs boto3 1.26.0 and aws-glue-schema-registry 1.0.0 or later, and the glue and confluent extras need dataclasses-avroschema 0.65.7 or later; an environment with older releases upgrades them.

Version 0.21.0

The simulation model grows: more distributions, stores and containers that tasks use, flows of tasks, built-in statistics and ground-truth labels. The docs gain a Learn section with a tutorial, the Shop, and a User Guide with one page per part of the library.

  • Five new distributions, uniform, triangular, gamma, weibull and empirical, and, in YAML, durations such as 5 min for every time-valued parameter. The builder's distribution parameters take numbers in seconds, and its warmup and run(until=) also take durations; the low-level API takes numbers in seconds. See Arrivals and Services and Distributions.
  • Each distribution draws from a random stream of its own, so adding an arrival leaves the other draws unchanged. A run without a seed draws one and logs it, and desflow run --seed N repeats it. See Randomness and Seeds.
  • Stores and containers that tasks take from and add to, with get and put; a container's level is a live path. See Stores and Containers.
  • A queue order per resource, fifo, lifo or priority, and a priority per task. See Resources.
  • Flows of tasks: next by name, condition or weights, entity attributes with attributes and set, and timed steps with a service and no resource. A builder arrival loop takes a task's instance id from context.next_instance(task), the counter next uses too. See Task Flow.
  • Built-in statistics at five levels, stats: none to stats: all, and a warmup that leaves the start of a run out. See Statistics.
  • Ground-truth labels: a scenario step's record publishes what it changed to when: labels sinks only. See Live Parameters.
  • Every ingress connector reads path and value, and a bad message is logged and skipped. See Ingress.

Upgrading

  • Python 3.11 or later is needed.
  • Ingress messages use path and value: Kafka's path_id and Redis's param_path and param_value are renamed, and a Kafka message, or a PostgreSQL ingress row, with a null value is now refused. Call send_config(topic, path=..., value=...) where it was called with path_id=.
  • For each table the PostgreSQL ingress reads, simulation_params by default, rename the columns: ALTER TABLE simulation_params RENAME COLUMN param_path TO path; and ALTER TABLE simulation_params RENAME COLUMN param_value TO value;
  • An event's key is now the entity's key, the arrival name and a counter from 1 (orders-1), not task-<task id>, and every event has two new fields after key: task and instance. PostgreSQL rows gain task and instance columns where the table has them. See Records and Telemetry.
  • telemetry entries in a blueprint no longer take publish. Use stats: minimal, which publishes the same readings as Shop.stats.resources.<name>.<reading>, and function for readings of your own.
  • Seeded runs give other numbers than before, because each distribution has its own random stream, and requests of equal priority are now served first in, first out.
  • A distribution parameter the family does not take, such as rate on a normal, is now refused at build. The builder's distribution parameters are keyword-only: write add_service("packing", "normal", mean=3).
  • Under the default fifo queue a request's priority is ignored. Add the resource with queue="priority" (YAML queue: priority; low-level DynamicResource(env, sim_id, name, queue="priority")) to rank requests; a task priority on another order is refused.
  • context.wait_for_arrival now yields the new Entity. Pass it to the task as entity=, or the task starts an entity of its own keyed by the task's name.
  • A container's current_cap is its capacity, its largest amount; a model that used it as a level moves to level. DynamicContainer(init=None) now starts at the registry's level, 0 unless the ContainerConfig gives one.

Version 0.20.0

The package dynamic-des is renamed desflow. The classes, the YAML format, the records and the extras are the same as in 0.19.0.

Upgrading

  • Install desflow instead of dynamic-des, with the same extras: pip install desflow, uv add desflow, or for example pip install "desflow[kafka]". The last dynamic-des release, 0.20.0, installs desflow with the same extras. To install desflow beside dynamic-des 0.19.0 instead, uninstall dynamic-des first: with pip, uninstalling it afterwards also removes dynamic_des and ddes, and pip install --force-reinstall --no-deps desflow puts them back.
  • Import desflow instead of dynamic_des, for example from desflow import SimulationContext.
  • Run the desflow command instead of ddes, for example desflow run shop.yaml or desflow examples.
  • import dynamic_des still works in this release. It shows a FutureWarning and gives the desflow modules, submodules such as dynamic_des.connectors.egress.kafka included. The ddes command still works too: it prints a warning and runs desflow. Both are removed in 0.30.0.
  • The docs are at jaehyeon.me/desflow, and the repository is jaehyeon-kim/desflow on GitHub.