Upgrading to 1.0
This page is for a model, a script or a pipeline that runs on desflow 0.28.0. Each item says what 0.28.0 did, what 1.0 does and what to change. 1.0 follows SimPy's rules for time, ties and failures, so a seeded model can publish other records than it did on 0.28.0, and some YAML forms now mean something else. From 1.0 the Compatibility Policy applies, and a 1.x release no longer changes a result without listing it.
To move a model to 1.0:
- Install desflow 1.0 and uninstall
dynamic-desif it is still installed, as Removed Names says. - Run
desflow checkon every blueprint. It names each key or value that 1.0 refuses, with its line. - Run the model's tests, such as
desflow test, and compare its stored results. Where a test fails, find the change below that explains it.desflow test --fixwrites the new values once you accept them.
Removed Names
dynamic_desandddes. 0.28.0 kept the import namedynamic_desand theddescommand, each with a warning. 1.0 removes both:import dynamic_desraisesModuleNotFoundError, and noddescommand is installed. Importdesflowand rundesflow. Uninstalldynamic-des; its last release installs a desflow below 1.0.**registry_kwargsofConfluentAvroSerializerandConfluentAvroDeserializer. 0.26.0 deprecated the confluent-kafka keys. 1.0 removes them, and a call that passes one raisesTypeError. Pass the named argument instead:basic.auth.user.info(user:password) becomesauth=("user", "password"),ssl.ca.locationbecomesverify=,ssl.certificate.location,ssl.key.locationandssl.key.passwordbecomecert=, andmax.retriesbecomesmax_retries=. Thebearer.auth.*keys with the sourceSTATIC_TOKENbecomeheaders={"Authorization": "Bearer <token>", "target-sr-cluster": "<logical cluster>"}, with"Confluent-Identity-Pool-Id": "<pool>"whenbearer.auth.identity.pool.idwas set. Deletecache.capacityandcache.latest.ttl.sec, which had no effect. Connect to a Secured Schema Registry shows the arguments.desflow.core.distributions. The module is gone.DISTRIBUTIONSis indesflow.models.distributionsin 1.0, a module path that 1.x does not promise to keep.desflow distlists the families from the command line.
Run and Time
strictdefaults to SimPy's rule. In 0.28.0,strictwas false in YAML and the Builder unless it was set, and it was never checked atfactor: 0. In 1.0 it is true when it is not set andfactoris above 0 orgo_live_atis set, so a paced run that falls behind the wall clock raisesRuntimeError. An explicitstrict: trueatfactor: 0raises once the wall clock has moved, as SimPy does.--sample, replications,desflow experimentanddesflow explain --verifyrun atfactor: 0and ignorestrict. Setsimulation.strict: false(SimulationContext(strict=False)) to keep the old behaviour. See Strict Pacing.on_unhandled_failuredefaults tostop. In 0.28.0, YAML and the Builder defaulted tocontinue, so a failure that nobody handled ended one entity's flow and the run went on. In 1.0 the default isstop, as in SimPy and the Low-level API, and the run ends with the error. A model that relies on the old rule addsrun: {on_unhandled_failure: continue}(run(on_unhandled_failure="continue")). See Handle a Failure Nobody Handles.- Signals fire once. In 0.28.0 a signal re-armed after it fired. In 1.0 a signal fires once by default, as SimPy's
Eventdoes: a wait after the firing resumes at once, and a second firing is refused. Give a signal that fires more than onceonce: false(add_signal(name, once=False),Signal(env, name, once=False)). Aonce: truesignal that fired again dropped the second firing with a warning; it now raises underon_unhandled_failure: stop. See One-Shot Signals. - A run with no
untilends. In 0.28.0 such a run never ended while statistics were on, or with a sink at afactorabove 0, or with an ingress connector. In 1.0 it ends when only desflow's own events are left, and astopthat can no longer happen raises SimPy'sRuntimeError. Give such a run anuntil. A live model meant to run on gives its connector anuntil. See Stop Early. - A Low-level run to an event ends. In 0.28.0,
DynamicRealtimeEnvironment.run(until=<event>)ran for ever on desflow's own timers when the event never came. In 1.0, with no ingress set up, it ends with SimPy'sRuntimeErrorwhen no model event is left; with an ingress it goes on. Name a time as well:env.run(until=event | env.timeout(t)). - No smallest delay. In 0.28.0 a service time or a drawn delay was at least 0.00001 seconds. In 1.0 the floor is 0: a constant 0 takes no time, and a normal draw below 0 is raised to 0. A constant gap of 0 starts entities at one instant without end, so the run never ends and
untildoes not stop it, as in SimPy. Give such an arrival a gap above 0. sim_tsis not rounded. 0.28.0 roundedsim_tsto three decimals. 1.0 writes the clock as it is, so1.357may read1.3565.... Roundsim_tsbefore you compare it. See Field Values.- A number keeps the type it is written in. 0.28.0 converted times, amounts, container capacities and priorities to floats. In 1.0 a number keeps its written type:
env.now,peek()and the readings of a step or segment print5for a time written5, where they printed5.0. A live update keeps the number as written, so3written over2.5is3. The record fieldscapacity,level,wait_maxandusage_sinceare integers when their inputs are, such as38where 0.28.0 wrote38.0.sim_tsstays a float. Iceberg and Parquet sinks still write a statistic'svalueandusage_sinceas doubles. Text written over a parameter is still read as a float.
Order at One Instant
These changes make a task go on in the same step as the SimPy process it stands for, so records at one instant can change order, and a unit or an item can go to another entity at a tie. A model with no events at the same time publishes the same records.
- A task with one
waitbranch now goes on in the step its event is processed, asyield eventdoes in SimPy. To keep the old order, write the branch underany, aswait: {any: {only: {timeout: 2}}}. Await: {timeout: 5}withpatience: 5 snow starts and finishes at 5, where it reneged; theanyform gives the old result. next,on_failandon_interrupt_nextnow run the following task in the same process and the same step, so it asks for a unit before a child spawned at that instant, and a process you wait for ends when its whole flow ends. To let the child go first, give the following taskwait: {timeout: 0}, which matches SimPy. Asubscribeinterrupt or a preempted kept unit now reaches the following task, whichon_interrupt: abandoncuts short.- A task's
on:actions now run in the step the task ends, before its next task starts. A flag they set is read by the next task, but not by anextcondition on the same task, which is chosen before they run. To run them one step later, fire a signal in the task'son:and put the actions in that signal'son:. - Tied replay rows now start in one step, in file order, before any of them takes its first step. 0.28.0 started each row in a separate step.
- An
interruptreaches an instance made in the same step, fromspawn:inon:,context.spawn,context.callor anextchain. In 0.28.0 it found no instance. The task keysspawn:andinterrupt:still run interrupt first, so one task's own pair does not reach the new instance; twoon:entries in the order spawn, interrupt do. - Patience at a tie. When a patience ends at the instant a unit is granted and the patience's event comes first, the task now reneges. 0.28.0 served it. The same holds for a
get, awaitbranch and a request asked again after a preemption. - A grant at the end of a wait. A request, get or put branch counts as won only when it was processed before the wait's condition, as
req in resultis in SimPy. An item agetwas granted after that is lost. A request granted by then keeps its unit until the task ends. Order the events so the grant comes first when the model needs the unit or item. - Equal priorities in a priority store now leave in the order SimPy's heap gives, which is not put order from four items on. To keep put order, give each item a priority that rises, such as
priority: "1 + process.instance / 1000000"; in the Builder and the Low-level API a pair(priority, number)is a valid priority. See Start With Items and Serve by Priority.
Resources, Waits and Interrupts
- A unit is kept through a plain interrupt. With
on_interrupt: resumeorrestart, a plain interrupt no longer gives back a unit the task holds, so the task goes on with it, and other entities wait until the task ends. A task with several resources asks again only for the unit it lost. A unit kept withkeepnow goes back at once when the next task reneges, fails or has no next task. To give the unit back and queue again, useon_interrupt: abandonwithon_interrupt_nextnaming a task that asks for it. See Interrupts. - A granted
requestbranch underrestart. In 0.28.0, an interrupt of a task withon_interrupt: restartgave back every grantedrequestbranch of its wait and asked again at the back of the line. In 1.0 the task keeps them and starts the other branches again. To queue again as before, write the task withon_interrupt: abandonandon_interrupt_nextnaming a copy of it, and name both tasks in everyinterruptthat names the first. - Interrupt or preemption at the end of the work. A job interrupted or preempted at the instant its work ends now ends at that instant, publishes no
resumedand asks for no unit. In 0.28.0 it waited for a further 0 seconds. keepand a child flow. A child flow started bywait: {task: ...}no longer gives back the unit its parent kept when it starts, balks or is abandoned. The unit stays until the parent's next task starts. A model that counted on the early release ends the parent's task with nokeep.- Opposite-order acquisition loads. 0.28.0 refused a model whose tasks take shared resources in opposite orders. 1.0 loads it and logs a WARNING from the logger
desflow.core.context, and the run can stop if two tasks wait for each other. A test that expected theValueErrorreads the warning instead. - Preempting a finished process. A unit held by a process that has ended, such as a Low-level
requestnever released, now fails the flow that preempts it with SimPy'sRuntimeError. 0.28.0 took the unit with no error. Release a unit before its process ends, withresource.release(request)or awithblock. - Builder
preemptas text.task(preempt="f")was kept as text and counted as true. In 1.0,"f","no","off"and"0"are false, as in YAML. Writepreempt=Trueorpreempt=False. queue_limitis a registry path. A resource now has the path<sim>.resources.<r>.queue_limit, andDynamicResource.queue_limitis a read-only property that follows it, soresource.queue_limit = 5raisesAttributeError. Change it withenv.registry.update("<sim>.resources.<r>.queue_limit", 5).compile_parameters()gives a resource as a subclass ofCapacityConfigthat also holds the limit, so compare itscurrent_capandmax_caprather than the whole object. See Queue Limit.- A
batchwith atimeoutwhose members all leave before it closes is dropped, so the next member starts a new batch and itstimeoutandbatch.agecount from that member. Compare the stored results of a model whose batch members can renege, balk or be interrupted while they wait. See Know What Happens to a Member.
Failures
- A task that ends in a run failure runs none of its
on:actions, as no statement runs after an uncaught exception in SimPy. That covers anextcondition that cannot be evaluated, a wait that fails because its child did, and a function that raises, when noon_failhandles it. A task's ownfailcondition, and a failure thaton_failhandles, keep their actions. - A failed evaluation in a task with
on_fail. A generator or a step that raises in a task withon_failnow goes toon_fail, where 0.28.0 stopped the run. Removeon_failfrom the task when the run must stop. - A
failexpression that cannot be evaluated, such aspayload.a == 2on a payload withouta, now fails the task: it publishesfailedwith the error as its reason, and the run stops withTaskFailedunderstopand goes on undercontinue. In 0.28.0 the error left the run with nofailedrecord, even undercontinue. - A child flow that fails after its waiter was abandoned no longer raises
TaskFailedundercontinue; the run goes on. Understopit is unchanged. - An empty or zero failure reason is kept. A child flow's or a wait's failure with
fail_reason: ""or 0 replaced the reason with the error's text, such assignal 's' failed:. In 1.0,upstream.reasonreads the reason as given. Give the failing task a reason that is not empty, or read the empty reason in the handler. on_failwithoutfailis accepted in YAML and the Builder, and catches a task function that raises. Onlyfail_reasonwithoutfailis refused, as "fail_reason needs a fail condition".
Values and Expressions
- A mapping with a
genkey is a generator. In 0.28.0 such a mapping was data inattributes, storeitems, an interruptcause, afire,on:or timeoutvalue, apayload,setand a scenariovalueandrecord. In 1.0 it is a generator, or a load error with the line. A key namedgendirectly underset,attributesorrecordis still a field name. To keep such a mapping as data, write{gen: literal, value: {gen: 5}}, which publishes{"gen": 5}. See Generated Values. item.<field>where nogetreaches is refused at load, with the task named. In 0.28.0 it read nothing and failed when the run reached it.- A name that does not exist in
resumeoropen. Alookup,aggregateorregistry[...]read that names a missing task, row or path in aresumeoropenvalue is refused at load. Correct the name. - Builder times.
TrueorFalseas a time, inSimulationContext(initial_time=),warmup,lag_interval, adrainlimit or a time-valued distribution parameter, was taken as 1 or 0 seconds. In 1.0 it raisesValueError, as a blueprint does. Write the number of seconds. - Builder draws in a
waitoropen. A generator in a Builderwaitoropen, such as aneachcount or a timeout'safter, now draws from the stream of its place in the blueprint, so the Builder gives the numbers of the same blueprint. At the same seed it gives other draws than 0.28.0. Compare the stored results of such a model again. - A sim id with a dot, such as
Shop.v2, draws the same random times as the same model with a plain sim id, so its results for a seed differ from 0.28.0. Record its results again. - An element named like a kind. A live update of an element is stored by its kind alone. A resource named
service,containersorvariableshas its capacity cast to a whole number, a container namedstoreskeeps a fraction, and a whole-number variable namedmax_caporcurrent_caprefusesinf. The same holds for a sim id whose second part is a kind, such asShop.service.
Sinks and Connectors
- A status record (
queued,startedand each other status of a task) carriessim_idandservicebesidepath_id. 0.28.0 wrotepath_idalone. A consumer that checks the fields of a status value, or a table whose columns 0.28.0 records made, gets the two new fields. See Records and Telemetry. - Routing keys in a payload. A payload key
__topic__(Kafka),__table__(Iceberg) or__path__(Parquet, JSON Lines) now chooses the destination, and the sink removes it. 0.28.0 dropped it with a warning. Rename such a key if it was only data. A__table__that PostgreSQL and Iceberg share on one record needs the formnamespace.tablefor Iceberg, or atable_router. See Reserved Payload Keys. - A sink with
upsert_keysnow writes the other records of a batch and skips only those that have none of the key columns, with one warning per flush that gives the counts. 0.28.0 lost the whole batch. Check the warning count after a run. - An Iceberg column that is an integer in the first batch is no longer truncated when a later batch holds a fraction: that batch is logged at ERROR and lost. Name the column as a double in
schemas, such asschemas: {shop.events: {key: string, v: double}}. See Iceberg. - A statistic and a reading need separate destinations. A statistic's
valueis a number and a reading's is a mapping, so atable_routerorpath_routerthat sends telemetry to Iceberg or Parquet gives readings a table or path of their own. KafkaIngresscreates its topic at the start of a run when it is missing. Givetopic_policy: manualto keep it from creating one.- A Kafka record its serializer refuses is skipped with a warning that names its key and the reason, and the rest of the batch is sent. 0.28.0 stopped the batch and retried the connection.
- A PostgreSQL sink with
envelopeon warns when it skips a record whose value is not a mapping. The reason in that warning, for every sink, now reads "which this sink cannot write as a row". - Parquet, JSON Lines and Iceberg sinks load their libraries before the run starts. In a strict paced run without
logical_start_time, the records are stamped behind the wall clock by the load time. A run withstrict: falsekeeps its 0.28.0 timestamps. - A Parquet, JSON Lines or Iceberg sink without its extra is refused when it is built, with an
ImportErrorthat names the extra. 0.28.0 built it, and the run wrote nothing. ConsoleEgress()with no logging at INFO, such aspython FILEwith nologging.basicConfig, prints oneUserWarning. The records are the same.KafkaAdminConnector.get_state()keys a sim id and a service id that hold a dot by their whole names, such asShop.v2, where it took one part of each.
Builder and Low-level API
- A second
run()of one context starts from the model's starting values, as a new context does, not from the values the first run changed. Pass other starting values tostart()asoverrides, or read the values a run ended with fromcontext.env.registrybefore the next run. compile_parameters()gives the config objects as the model declares them, so a capacity or a stream name left out isNone. Read the starting values a run registers fromregistry_paths().- A task function that replaces
context.sampler,context.valuesorcontext.sim_idduring a run no longer changes the arrivals, breakdowns and family members of that run. Set them beforestart(). - After a
start()that fails before it makes the run's holders,get_resource,get_storeandget_containerraiseKeyError, where they gave the previous run's object. egress_*lists ofDynamicRealtimeEnvironmentare read-only: assigning one raisesAttributeError, and changing a list it returns does not reach the egress. Pass the sinks tosetup_egress.fieldsofadd_arrival_replayandread_tracetakes"all"or a list of columns, as a blueprint'sreplay.fieldsdoes. Writefields=["id"]for one column. See Replay.- A Builder
on=entry, intask(on=),task(listen=)andadd_signal(on=), refuses a key other than the actions,valueandwhen, where 0.28.0 ignored it. Remove the key, or move it into the action's own mapping. - An index on a number, such as
users.machine[0], is refused when the Builder context starts (run()orstart()), not attask(). A Low-levelExpression("users.machine[0]")now parses. load_context(folder, "file.py", preview)applies the preview to the Builder file, where 0.28.0 raisedValueError.
Command Line and Installs
- desflow needs pydantic 2.6.1 or later, where it accepted 2.6.0.
desflow check FILE.pyimports a Builder file and printsok FILE (sim_id ID), or the error. 0.28.0 exited 1. A Low-level file is refused.desflow checkexits 1 for a blueprint whosetestsentry has no seed in a file with nosimulation.random_seed, and, in a run with no end, warns about each arrival with nomax_events. The exit code of that warning is still 0. See desflow check.desflow run FILE.py --sampleprints the Builder file's first events. Without--sample, or with--stdout,--varor--watch, it exits 2 as before.desflow fitwrites its settings file in the blueprint layout: the comment wrapped at 88 characters, a blank line between keys and long lists wrapped. The file loads to the same settings.desflow fitandfit_columnskippoissonandgeometricnamed incandidates, as "counts whole numbers".desflow reportandrun_experimentbuild the report without readingresults.csv, so a missingresults.csvno longer stops the report.- An experiment's counts are the task and flow counts that are 0 when no entity reached them; another whole-number statistic is no longer listed among them.
Error Messages
A test that matches the text of an error matches the new text:
- The priority, patience and balk checks name
seizeandopen: "needs a resource, a seize or a request in its wait or open". on_balk"needs a balk, a resource, a seize or an open", and the messages oncause.<name>andupstream.<field>also nameon_interrupt_run,on_renege,on_balkandon_fail.- An index on a number reads "'machine' is not a family; add family: true to its entry".
statswith nolevel: "stats needs a level, unless it sets only trace, when the level is none".- A
telemetryentry with nofunction: "a telemetry entry takes exactly one of function and readings". - An arrival: "an arrival takes exactly one of dist, replay or gap", and "an arrival with replay takes no distribution parameters".
- A task with a resource "needs a service or a service_time".
- A wait timeout: "a wait timeout is seconds, a duration such as '5 min' or an expression".
- The line of an error inside a wait is the line of the key, not of the
wait. - The warning of a sink that drops a reserved payload key names each routing key and its sink.
- A
Templatewhose generator does not parse is refused when it is made, with the parse error.