Skip to content

Queues and Capacity

In its busy hour the shop gets 1.8 orders a second, and packing still takes 1.5 seconds. One packer cannot keep up. This page asks: how many packers keep the mean wait under 10 seconds?

  • Decision: the number of packers for the busy hour.
  • Measures: the packers' utilisation and the mean wait.
  • Runs: four runs of the hour, one for each number of packers, each with the seed 42.
  • Assumptions: orders arrive at random at a steady rate, and every order takes the same time to pack.

What You Will Learn

This page shows how to:

  • work out a resource's utilisation before running the model;
  • read the load, the stability and the bottleneck with desflow explain;
  • choose the fewest packers that keep the mean wait short.

What You Add

This page adds more orders, then more packers, and desflow explain, which works out the load from the model before it runs. The shop is the one from First Simulation, with the arrival rate of the busy hour and the number of packers as a setting you change between runs.


Work Out the Load

Before running anything, work out how busy the packers would be. Utilisation is the share of time a resource is busy:

utilisation = arrival rate × mean service time ÷ number of units

The load is the part above the division: arrival rate × mean service time. Here it is 1.8 orders a second × 1.5 seconds = 2.7, so each second brings 2.7 seconds of packing, enough work for 2.7 packers. One packer would be busy 270% of the time, which is impossible: it falls further behind with every order.

  • With utilisation below 1 a model is stable: the queue rises and falls, but stays bounded.
  • At 1 or above it is not: the queue grows for as long as the run lasts, and every order waits longer than the one before.

The shop needs at least 3 packers, the fewest that bring utilisation below 1: 2.7 ÷ 3 = 0.9. Queueing Basics explains utilisation, and why the queue grows without limit at 1 or above. Each resource in a model has its own utilisation. The one with the highest utilisation is the bottleneck: it decides how long everything waits.

Here is the busy hour. The arrival rate is 1.8 orders a second, and until is one hour of simulation time:

docs/snippets/yaml/queues_and_capacity.yaml:load
simulation:
  sim_id: Shop
  factor: 0  # unpaced: the hour runs in about a second
  random_seed: 42

services:
  packing: {dist: constant, value: 1.5 s}

arrivals:
  # The busy hour: 1.8 orders a second, more than one packer can pack
  orders: {dist: exponential, rate: 1.8, spawn: pack_order}

tasks:
  pack_order:
    service: packing
    resource: packers
    payload: {event_type: order_packed}

egress:
  - {type: Console}

run: {until: 1 h}
docs/snippets/tutorials/queues_and_capacity.py:load
1
2
3
4
5
app = SimulationContext(sim_id="Shop", factor=0, random_seed=42)  # unpaced
app.add_service("packing", dist="constant", value=1.5)
# The busy hour: 1.8 orders a second, more than one packer can pack. Each
# order starts the task pack_order.
app.add_arrival("orders", dist="exponential", rate=1.8, spawn="pack_order")

spawn="pack_order" starts the task for each arrival, as spawn: does in YAML, so this page needs no @app.arrival_loop.

docs/snippets/tutorials/queues_and_capacity_low_level.py:load
1
2
3
service={"packing": DistributionConfig(dist="constant", value=1.5)},
# The busy hour: 1.8 orders a second, more than one packer can pack
arrival={"orders": DistributionConfig(dist="exponential", rate=1.8)},

A blueprint accepts a duration with a unit wherever it takes a time, such as until: 1 h and value: 1.5 s, and reads a plain number as seconds. A text that is a plain number, such as "5" or "1e-3", is read as seconds too. factor: 0 runs the hour unpaced: as fast as the computer can, in about a second. Nobody needs to watch this run in real time.


Explain the Load

desflow explain does this arithmetic from a YAML or builder model, without running it. It reads three numbers for each resource and works out the fourth:

  • λ (lambda), the arrival rate: rate: 1.8 of arrivals.orders;
  • μ (mu), the service rate, one over the mean service time: value: 1.5 s of services.packing gives 1 ÷ 1.5 = 0.667 orders a second for each packer, one order every 1.5 seconds (40 a minute);
  • c, the number of units: current_cap of resources.packers;
  • ρ (rho), the utilisation: λ ÷ (c × μ).

Save the files from Full Files in one folder and run the commands there.

desflow explain queues_and_capacity.yaml

The file is queues_and_capacity.yaml from Full Files. Its number of packers comes from the variable PACKERS, which Run It With More Packers sets. With PACKERS not set, the shop has one packer.

docs/snippets/tutorials/queues_and_capacity.py:explain
1
2
3
def explain(packers: int) -> str:
    """The busy hour with `packers` packers, as `desflow explain` prints it."""
    return render_terminal(shop(packers).analyze())

context.analyze() returns the same analysis as an Explanation, and render_terminal prints it as the command does. desflow explain also loads a builder Python file that defines a SimulationContext at module level.

desflow explain does not support low-level models. It reads which arrival starts which task and which resource each task uses, and a low-level model does not declare those links: its SimPy code makes them as it runs.

For a YAML or builder model, it prints the graph from each arrival, the resources, the utilisation at each number of units the resource allows, and hints:

Shop
`-- orders (arrival, rate 1.8/s)
    `-- spawn -> pack_order (task, rate 1.8/s)
        +-- uses packers (resource)
        `-- next (share 1) -> exit
Resources
+-------------------------------------------------------+
| resource |   λ |      μ | c |   ρ | Wq | L | status   |
|----------+-----+--------+---+-----+----+---+----------|
| packers  | 1.8 | 0.6667 | 1 | 2.7 |  - | - | UNSTABLE |
+-------------------------------------------------------+
packers by units
+-------------------------+
|  c |      ρ | status    |
|----+--------+-----------|
|  1 |    2.7 | UNSTABLE  |
|  2 |   1.35 | UNSTABLE  |
|  3 |    0.9 | CONGESTED |
|  4 |  0.675 | OK        |
|  5 |   0.54 | OK        |
|  6 |   0.45 | OK        |
|  7 | 0.3857 | OK        |
|  8 | 0.3375 | OK        |
|  9 |    0.3 | OK        |
| 10 |   0.27 | OK        |
+-------------------------+
Bottleneck: packers
Not estimated:
  wait at packers: unstable: the queue grows for as long as the run lasts
Hints for utilisation 0.85:
  packers with 4 units: utilisation 0.675, at most 0.85
  packers with 1 unit: utilisation 0.85 at a request every 1.76 seconds (34 a minute, 0.5667 a second)

The output has four parts:

  • The tree shows each order starting pack_order, which uses packers, and the flow ending after it.
  • The resources table gives λ (the arrival rate) 1.8 orders a second, μ (the rate one packer serves) 0.667 orders a second, c (the number of packers) 1 and utilisation ρ 2.7. The status is CONGESTED from utilisation 0.85 and UNSTABLE from 1, so it is UNSTABLE. The wait (Wq) and the number in the shop (L) are left blank, and the reason is printed below the tables: the queue grows for as long as the run lasts.
  • The table by units covers 1 to 10 packers, up to max_cap. Three packers are stable but congested at utilisation 0.9.
  • The first hint names 4 as the fewest packers at or below 0.85. The second hint keeps the current number of units. It gives the rate of requests into the resource at which utilisation reaches the target. With one packer, an arrival rate of 0.5667 orders a second, an order every 1.76 seconds (34 a minute), gives utilisation 0.85. In a network, that rate includes the requests that come back for rework, not only new arrivals.

Run It With More Packers

The number of packers is the resource's current_cap. Each run sets it once:

docs/snippets/yaml/queues_and_capacity.yaml:packers
1
2
3
4
5
resources:
  # The number of packers comes from the PACKERS variable, 1 when it is not set
  packers:
    current_cap: ${PACKERS:-1}
    max_cap: 10

${PACKERS:-1} reads the environment variable PACKERS, and uses 1 when it is not set.

docs/snippets/tutorials/queues_and_capacity.py:packers
app.add_resource("packers", current_cap=packers, max_cap=10)
docs/snippets/tutorials/queues_and_capacity_low_level.py:packers
resources={"packers": CapacityConfig(current_cap=packers, max_cap=10)},

A DynamicResource with a current_cap of 3 behaves like a SimPy resource with three units: a request is granted as soon as any unit is free, so up to three orders are packed at once. Resources describes how it is built and why.

env.run(until=...) runs SimPy's event loop until that time. DynamicRealtimeEnvironment is desflow's form of SimPy's RealtimeEnvironment. A resource's capacity is its number of units.

To read the mean wait, switch on the built-in statistics at the typical level, which keeps the mean wait of every task; Measuring a Run explains the levels.

docs/snippets/yaml/queues_and_capacity.yaml:stats
# The mean wait of pack_order, published once an hour and at the end
stats: {level: typical, interval: 1 h}
docs/snippets/tutorials/queues_and_capacity.py:stats
# The mean wait of pack_order, kept by the run and read at the end
app.with_stats("typical", interval=3600)
docs/snippets/tutorials/queues_and_capacity_low_level.py:stats
# The mean wait of pack_order, kept by the environment and read at the end
env.setup_stats("Shop", "typical", interval=3600)

A low-level process has no task for desflow to watch, so on this page it reports to env.stats itself instead of publishing events: it calls task_queued, task_started and task_finished with the task's name and instance id. Measuring a Run explains these calls.

Now run the hour with 1, 2, 3 and 4 packers:

At the end, the run publishes the mean wait as Shop.stats.tasks.pack_order.wait_mean, among all its other records. --stdout prints the records as JSON lines, grep keeps the mean wait, and tail -1 keeps its last reading:

1
2
3
4
# The mean wait with one to four packers
for n in 1 2 3 4; do
  PACKERS=$n desflow run queues_and_capacity.yaml --stdout | grep -E 'pack_order\.wait_mean"' | tail -1
done
docs/snippets/tutorials/queues_and_capacity.py:table
def mean_wait(packers: int) -> float:
    """Runs the busy hour with `packers` packers and returns the mean wait."""
    app = shop(packers)
    app.run(until=UNTIL)
    return app.env.stats.readings()["Shop.stats.tasks.pack_order.wait_mean"]


if __name__ == "__main__":
    for packers in range(1, 5):
        print(f"packers {packers}: mean wait {mean_wait(packers):8.2f} s")
docs/snippets/tutorials/queues_and_capacity_low_level.py:table
1
2
3
4
5
6
7
8
9
def mean_wait(packers: int) -> float:
    """Runs the busy hour with `packers` packers and returns the mean wait."""
    env = run(packers)
    return env.stats.readings()["Shop.stats.tasks.pack_order.wait_mean"]


if __name__ == "__main__":
    for packers in range(1, 5):
        print(f"packers {packers}: mean wait {mean_wait(packers):8.2f} s")

Each tab prints the mean wait with 1, 2, 3 and 4 packers:

1
2
3
4
{"stream_type":"telemetry","sim_ts":3600.0,"timestamp":"...","path_id":"Shop.stats.tasks.pack_order.wait_mean","value":1139.8758548010155}
{"stream_type":"telemetry","sim_ts":3600.0,"timestamp":"...","path_id":"Shop.stats.tasks.pack_order.wait_mean","value":484.65045270890204}
{"stream_type":"telemetry","sim_ts":3600.0,"timestamp":"...","path_id":"Shop.stats.tasks.pack_order.wait_mean","value":1.8791946460820799}
{"stream_type":"telemetry","sim_ts":3600.0,"timestamp":"...","path_id":"Shop.stats.tasks.pack_order.wait_mean","value":0.24164977417889455}

python queues_and_capacity.py 2>/dev/null

1
2
3
4
packers 1: mean wait  1139.88 s
packers 2: mean wait   484.65 s
packers 3: mean wait     1.88 s
packers 4: mean wait     0.24 s

python queues_and_capacity_low_level.py 2>/dev/null

1
2
3
4
packers 1: mean wait  1139.88 s
packers 2: mean wait   484.65 s
packers 3: mean wait     1.88 s
packers 4: mean wait     0.24 s


Waits Near Full Use

With 2 packers, utilisation is 1.35. The queue grows all hour, so the mean wait depends mainly on how long the run was: run two hours and it roughly doubles. With 3 packers, utilisation is 0.9 and the wait settles. With 4 it is 0.675, and almost nobody waits.

A stable model can still wait a long time, because the wait grows much faster than utilisation near 1, and variable times make it longer still. Queueing Basics gives the formulas for one packer, the effect of variation and Little's law, and checks them against runs of this shop. desflow explain uses them, and Choosing Inputs measures the wait with constant, triangular, empirical and exponential packing times.


Find the Bottleneck

The bottleneck is the resource with the highest utilisation. Relieving it makes the next busiest resource the new bottleneck. In a shop with one resource that is the packer. In a shop with several tasks, desflow explain finds it from each task's rate, as A Flow of Tasks shows for a shop with a checker and rework. The hints that desflow explain prints for tuning a resource, and its --target option, are in Check It Before a Long Run.


What You Get and Why

How many packers keep the mean wait under 10 seconds? 3 packers, as the mean wait for 1 to 4 packers shows:

Mean wait in the busy hour for 1 to 4 packers on a log scale: 1140 seconds with 1, 485 with 2, 1.88 with 3 and 0.24 with 4; the 10-second line separates 2 from 3

With 3 packers the mean wait is 1.88 seconds, from one run of the hour with constant packing and the seed 42. With 2 packers it is over 400 seconds and still growing when the hour ends. The answer is the fewest packers that bring utilisation below 1. At utilisation 0.9 the wait stays under 10 seconds, because the shop's packing time is constant and 0.9 is not yet close enough to 1.

Queueing Basics explains why the wait grows without limit as utilisation reaches 1, as the table shows for 1 and 2 packers. It also gives the formula for several packers sharing one queue. Each term of the utilisation is a live parameter, so a running model can be relieved, or overloaded on purpose, as Live Parameters shows. Statistics describes how the mean wait is measured.

The formulas describe a queue that has settled. A run starts with no orders in the shop, so its early waits are shorter than the settled waits. A live change, such as a rush, makes the waits rise, and they take time to settle again. Trusting Results shows how to leave out a warm-up and how long to run.

Key Takeaways

  • Utilisation is the arrival rate times the mean service time, divided by the number of units. Below 1 the queue settles; at 1 or above it grows for as long as the run lasts.
  • Near utilisation 1 the wait grows much faster than the utilisation, and variable times make it longer still.
  • The resource with the highest utilisation is the bottleneck. Relieving it makes the next busiest resource the bottleneck.

Full Files

Full files
docs/snippets/yaml/queues_and_capacity.yaml
# The Queues and Capacity tutorial: the shop in its busy hour, as a blueprint.
# Run it once per number of packers, and read the mean wait from the last lines:
#   PACKERS=3 desflow run queues_and_capacity.yaml 2>&1 | grep wait_mean
simulation:
  sim_id: Shop
  factor: 0  # unpaced: the hour runs in about a second
  random_seed: 42

services:
  packing: {dist: constant, value: 1.5 s}

arrivals:
  # The busy hour: 1.8 orders a second, more than one packer can pack
  orders: {dist: exponential, rate: 1.8, spawn: pack_order}

tasks:
  pack_order:
    service: packing
    resource: packers
    payload: {event_type: order_packed}

egress:
  - {type: Console}

run: {until: 1 h}

resources:
  # The number of packers comes from the PACKERS variable, 1 when it is not set
  packers:
    current_cap: ${PACKERS:-1}
    max_cap: 10

# The mean wait of pack_order, published once an hour and at the end
stats: {level: typical, interval: 1 h}
docs/snippets/yaml/queues_network.yaml
# Queues and Capacity: the busy shop as a network. Packed orders are checked, and
# one in ten goes back to be packed again.
# Explain it with: desflow explain queues_network.yaml
simulation: {sim_id: Shop, factor: 0, random_seed: 42}

arrivals:
  orders: {dist: exponential, rate: 1.6, spawn: pack_order}

services:
  packing: {dist: constant, value: 1.5 s}
  checking: {dist: exponential, rate: 2.5}  # 0.4 s on average

resources:
  packers: {current_cap: 4, max_cap: 10}
  checkers: {current_cap: 1, max_cap: 4}

tasks:
  pack_order:
    service: packing
    resource: packers
    payload: {event_type: order_packed}
    next: check_order
  check_order:
    service: checking
    resource: checkers
    payload: {event_type: order_checked}
    # Nine in ten pass; the tenth is packed again
    next:
      weights: {ship_order: 9, pack_order: 1}
  ship_order:
    payload: {event_type: order_shipped}

egress:
  - {type: Console}

run: {until: 1 h}
docs/snippets/tutorials/queues_and_capacity.py
"""The Queues and Capacity tutorial: the shop's busy hour, with the builder.

Orders arrive at 1.8 a second and each takes 1.5 seconds to pack. `explain` prints
what `desflow explain` prints for the shop. The script runs the hour once for each
number of packers, unpaced, and prints the mean wait of each run. `network` is the
shop with a checker and rework.
"""

from desflow import SimulationContext
from desflow.explain import render_terminal

UNTIL = "1 h"  # the busy hour


def shop(packers: int) -> SimulationContext:
    """The shop in its busy hour, with `packers` packers."""
    app = SimulationContext(sim_id="Shop", factor=0, random_seed=42)  # unpaced
    app.add_service("packing", dist="constant", value=1.5)
    # The busy hour: 1.8 orders a second, more than one packer can pack. Each
    # order starts the task pack_order.
    app.add_arrival("orders", dist="exponential", rate=1.8, spawn="pack_order")
    app.add_resource("packers", current_cap=packers, max_cap=10)
    # The mean wait of pack_order, kept by the run and read at the end
    app.with_stats("typical", interval=3600)

    @app.task(service_id="packing", resource_id="packers")
    def pack_order(task_id):
        return {"event_type": "order_packed"}

    return app


def explain(packers: int) -> str:
    """The busy hour with `packers` packers, as `desflow explain` prints it."""
    return render_terminal(shop(packers).analyze())


def network() -> SimulationContext:
    """The busy shop as a network: packed orders are checked, one in ten packed again."""
    app = SimulationContext(sim_id="Shop", factor=0, random_seed=42)
    app.add_arrival("orders", dist="exponential", rate=1.6, spawn="pack_order")
    app.add_service("packing", dist="constant", value=1.5)
    app.add_service("checking", dist="exponential", rate=2.5)  # 0.4 s on average
    app.add_resource("packers", current_cap=4, max_cap=10)
    app.add_resource("checkers", current_cap=1, max_cap=4)

    @app.task(service_id="packing", resource_id="packers", next="check_order")
    def pack_order(task_id):
        return {"event_type": "order_packed"}

    # Nine in ten pass; the tenth is packed again
    @app.task(
        service_id="checking",
        resource_id="checkers",
        next={"weights": {"ship_order": 9, "pack_order": 1}},
    )
    def check_order(task_id):
        return {"event_type": "order_checked"}

    @app.task()
    def ship_order(task_id):
        return {"event_type": "order_shipped"}

    # print(render_terminal(app.analyze())) prints what desflow explain prints
    return app


def mean_wait(packers: int) -> float:
    """Runs the busy hour with `packers` packers and returns the mean wait."""
    app = shop(packers)
    app.run(until=UNTIL)
    return app.env.stats.readings()["Shop.stats.tasks.pack_order.wait_mean"]


if __name__ == "__main__":
    for packers in range(1, 5):
        print(f"packers {packers}: mean wait {mean_wait(packers):8.2f} s")
docs/snippets/tutorials/queues_and_capacity_low_level.py
"""The Queues and Capacity tutorial: the shop's busy hour, on the low-level API.

Orders arrive at 1.8 a second and each takes 1.5 seconds to pack. The script runs the
hour once for each number of packers, unpaced, and prints the mean wait of each run.
`run_network` is the shop with a checker and rework.
"""

import itertools

from desflow import (
    CapacityConfig,
    DistributionConfig,
    DynamicRealtimeEnvironment,
    DynamicResource,
    Router,
    Sampler,
    SimParameter,
    Upstream,
)

UNTIL = 3600  # the busy hour, in seconds


def params(packers: int) -> SimParameter:
    """The shop in its busy hour, with `packers` packers."""
    return SimParameter(
        sim_id="Shop",
        service={"packing": DistributionConfig(dist="constant", value=1.5)},
        # The busy hour: 1.8 orders a second, more than one packer can pack
        arrival={"orders": DistributionConfig(dist="exponential", rate=1.8)},
        resources={"packers": CapacityConfig(current_cap=packers, max_cap=10)},
    )


def pack_order(env, sampler, packers, task_id):
    """One order, which reports each step to the statistics."""
    path_id = "Shop.service.packing"
    env.stats.task_queued("pack_order", task_id)
    with packers.request() as request:
        yield request
        env.stats.task_started("pack_order", task_id)
        yield env.timeout(sampler.sample(env.registry.get_config(path_id)))
    env.stats.task_finished("pack_order", task_id)


def order_generator(env, sampler, packers):
    arrival = env.registry.get_config("Shop.arrival.orders")
    task_id = 0
    while True:
        yield env.timeout(sampler.sample(arrival))
        env.process(pack_order(env, sampler, packers, task_id))
        task_id += 1


def run(packers: int, until=UNTIL, factor=0.0):
    """Runs the busy hour with `packers` packers, unpaced, and returns the environment."""
    env = DynamicRealtimeEnvironment(factor=factor)
    env.registry.register_sim_parameter(params(packers))
    # The mean wait of pack_order, kept by the environment and read at the end
    env.setup_stats("Shop", "typical", interval=3600)
    packers_resource = DynamicResource(env, "Shop", "packers")
    sampler = Sampler(seed=42)
    env.process(order_generator(env, sampler, packers_resource))
    try:
        env.run(until=until)
    finally:
        env.teardown()
    return env


NETWORK = SimParameter(
    sim_id="Shop",
    arrival={"orders": DistributionConfig(dist="exponential", rate=1.6)},
    service={
        "packing": DistributionConfig(dist="constant", value=1.5),
        # One check takes 0.4 s on average
        "checking": DistributionConfig(dist="exponential", rate=2.5),
    },
    resources={
        "packers": CapacityConfig(current_cap=4, max_cap=10),
        "checkers": CapacityConfig(current_cap=1, max_cap=4),
    },
)
# Each task's service and resource, and the task after it
STEPS = {
    "pack_order": ("packing", "packers", Router("check_order")),
    # Nine in ten pass; the tenth is packed again
    "check_order": (
        "checking",
        "checkers",
        Router({"weights": {"ship_order": 9, "pack_order": 1}}),
    ),
    "ship_order": (None, None, Router(None)),
}


def step(env, sampler, shop, name, entity, counters):
    """One task of an order, then the task its router chooses."""
    service, resource, router = STEPS[name]
    task_id = next(counters[name])
    if resource is None:
        env.stats.task_finished(name, task_id)
    else:
        env.stats.task_queued(name, task_id)
        with shop[resource].request() as request:
            yield request
            env.stats.task_started(name, task_id)
            config = env.registry.get_config(f"Shop.service.{service}")
            yield env.timeout(sampler.sample(config))
        env.stats.task_finished(name, task_id)
    following = router.choose(Upstream(task_id, entity, {}, {}), env)
    if following is None:
        env.stats.flow_finished("orders")
    else:
        yield from step(env, sampler, shop, following, entity, counters)


def network_orders(env, sampler, shop, counters):
    arrival = env.registry.get_config("Shop.arrival.orders")
    for number in itertools.count(1):
        yield env.timeout(sampler.sample(arrival))
        env.stats.flow_started("orders")
        env.process(
            step(env, sampler, shop, "pack_order", f"orders-{number}", counters)
        )


def run_network(until=UNTIL):
    """Runs the network for the hour, unpaced, and returns the environment."""
    env = DynamicRealtimeEnvironment(factor=0.0)
    env.registry.register_sim_parameter(NETWORK)
    env.setup_stats("Shop", "typical", interval=3600)
    shop = {
        name: DynamicResource(env, "Shop", name) for name in ("packers", "checkers")
    }
    sampler = Sampler(seed=42)
    # Weights draw from the routing task's own sequence, as the builder's do
    STEPS["check_order"][2].rng = sampler.generator("route.check_order")
    counters = {name: itertools.count() for name in STEPS}
    env.process(network_orders(env, sampler, shop, counters))
    try:
        env.run(until=until)
    finally:
        env.teardown()
    return env


def mean_wait(packers: int) -> float:
    """Runs the busy hour with `packers` packers and returns the mean wait."""
    env = run(packers)
    return env.stats.readings()["Shop.stats.tasks.pack_order.wait_mean"]


if __name__ == "__main__":
    for packers in range(1, 5):
        print(f"packers {packers}: mean wait {mean_wait(packers):8.2f} s")