Queueing Laws [18]

Outline

• Model of Single Queue [source ($\lambda$), queue ($L$), server ($\mu$), sink]

• Performance Metrics: [utilization ($\rho$), response time ($R$), waiting time ($W$), throughput ($\lambda$), number ($Q$)]

• Queueing Networks [transition probability ($p_{ij}$)]

• Operational Laws:

• Stream Law: $\lambda_i = \sum p_{ji}\lambda_j$

• Utilization Law: $\rho_i=\lambda_i/\mu_i$

• Visit (Forced Flow) Law: $V_i=\lambda_i/\lambda$

• Bottleneck Law: $\lambda_{max}=\mu_b/V_b$

• Little's Law: $Q=\lambda R$

• M/M/1 Law: $R=1/(\mu-\lambda$)

• General Response Time Law: $R=\sum R_i V_i$

• Case Study: CPU vs. I/O

Queues: Single Queue [19]

Model:

source

• entry for new jobs at arrival rate: $\lambda$ jobs/sec

queue

• waiting line of jobs with queue length: $L$ jobs

server

• processor of jobs at service rate: $\mu$ jobs/sec

sink

• exit for finished jobs with throughput: $\lambda$ jobs/sec

response

• time waiting in queue and in service: $R$ sec

Queues: Performance Metrics [20]

ratio

• utilization of server: $\rho=\lambda/\mu$ % busy

ratio

• traffic intensity=utilization: $\rho=\lambda/\mu$

rate

• throughput: $\lambda$ jobs/sec (job flow balance)

jobs

• number in system (queue and service): $Q$

time

• waiting time in queue: $W$ sec

time

• service time per visit to server: $S=1/\mu$ sec/job

time

• response time: $R=W+S=W+1/\mu$ sec/job

Networks: probability $p_{ij}$ from device $i$ to $j$ [21]

Series:

Split/Merge:

Cycle:

values

• $\lambda=10,\mu_1=15,\mu_2=20,\mu_3=15$

• $p_1=2/3,p_2=1/3,p=0.1,\mu=150$

Queues: Operational Laws [22]

Queue Stream Law: $\lambda_i = \sum p_{ji}\lambda_j$ [23]

Law

• job flow is conserved at split/merge

• Poisson flow in $\Rightarrow$ Poisson flow out

series

• $\lambda_1 = \lambda_2 = \lambda = 10$ jobs/sec

• throughput = $\lambda$ = arrival rate (balance )

split

• $\lambda_1=p_1\lambda=2\lambda/3=6.67$

• $\lambda_2=p_2\lambda=\lambda/3=3.33$

merge

• $\lambda_3=\lambda_1 + \lambda_2 = 2\lambda/3 + \lambda/3=\lambda=10$

• throughput = $\lambda$ = arrival rate (balance)

cycle

• $\lambda_1 = \lambda +$ $=f(\lambda,p)=$

• $\lambda_2 = (1-p) \cdot$ $=$

• throughput =$=$

VISUAL Queues: Sample Output [24]

SERIES:
Time =    1000.00     (n)   (rho)    (Q)     (lambda)       (R)    (p_ij)
     Type   Rate    # Jobs % Busy  Avg jobs  Job rate     Job time  Prob.
   ------ ------ --------- ------ --------- --------- ------------  -----
1  source   10.0     10030  83.21     3.031    10.030     0.302197  1.00 
2    fifo   15.0     10030  67.31     2.070    10.030     0.206351  1.00 
3    fifo   20.0     10030  49.19     0.961    10.028     0.095828  1.00 
4    sink    0.0     10028  83.21     3.030    10.028     0.302197  

SPLIT/MERGE
1  source   10.0      9994  82.69     2.902     9.994     0.290346  0.67 0.33 
2    fifo   15.0      6717  44.16     0.783     6.717     0.116622  1.00 
3    fifo   20.0      3277  16.42     0.198     3.277     0.060310  1.00 
4    fifo   15.0      9994  66.07     1.920     9.992     0.192176  1.00 
5    sink    0.0      9992  82.69     2.901     9.992     0.290346  

CYCLE:
1  source   10.0     10112  58.51     2.170    10.112     0.214624  1.00 
2    fifo  150.0    101138  67.59     2.170   101.136     0.021456  0.90 0.10 
3    sink    0.0     10110  58.51     2.170    10.110     0.214624

Queue Utilization Law: $\rho_i=\lambda_i/\mu_i$ [25]

Law

• utilization depends on flow and service rate

series

• $\rho_1 = \lambda_1/\mu_1=\lambda/\mu_1 = 67\%$ busy

• $\rho_2 = \lambda_2/\mu_2=\lambda/\mu_2 = 50\%$

split/merge

• $\rho_1=\lambda_1/\mu_1=2\lambda/3\mu_1=44\%$

• $\rho_2=$

• $\rho_3=$

cycle

• $\rho=$

Queue Visit Law: $V_i=\lambda_i/\lambda$ [26]

Law

• visit ratio to server is the same as the flow ratio

• measures fraction of jobs which visit server

series

• $V_1=\lambda_1/\lambda=\lambda/\lambda=1$ (all jobs visit once)

• $V_2=\lambda_2/\lambda=\lambda/\lambda=1$

split/merge

• $V_1=\lambda_1/\lambda=2/3$ (some jobs visit)

• $V_2=$

• $V_3=$

cycle

• $V=$ (repeat visits)

Queue Bottleneck Law: $\lambda_{max}=\mu_b/V_b$ [27]

Law

• bottleneck is server with highest utilization: $\rho_{max}$

• bottleneck limits maximum throughput: $\lambda_{max}$

step 1

• Utilization Law: find bottleneck $b$ with $\rho_{max}$

• bottlenext $b$ service rate: $\mu_b$

step 2

• Visit Law: find $V_b=\lambda_b/\lambda$

• maximum system throughput: $\lambda_{max}=\mu_b/V_b$

proof

• stable system: $\lambda_i \leq \mu_i$

• bottleneck: $\rho_{max}=\lambda_b/\mu_b \Rightarrow$ closest $\lambda$ to $\mu$

• Visit Law: $\lambda_i = \lambda V_i$

• bottleneck: $\lambda_b=\lambda V_b \leq \mu_b$

• maximum system throughput: $\lambda_{max} V_b \leq \mu_b$

• $\lambda_{max}=\mu_b/V_b$

Queue Bottleneck Law: $\lambda_{max}=\mu_b/V_b$ [28]

series

• $\rho_{max}=67\%$ at bottleneck $b=1$

• $\lambda_{max}=\mu_1/V_1= 15/1=15$ jobs/sec

split/merge

• $\rho_{max}=67\%$ at bottleneck $b=3$

• $\lambda_{max}=\mu_3/V_3 = 15/1=15$

cycle

• $\rho_{max}=$

• $\lambda_{max}=$

goal

• improve throughput by fixing the bottleneck $b$

• minimize the maximum utilization

• then fix next bottleneck $b_1$

split/merge

• reduce bottleneck $b=3$ to the same as $\rho_1=44\%$

• $\rho_3=\lambda/\mu_3=44\% \Rightarrow \mu_3=\lambda/44\%=22.7$

Little's Law: $Q=\lambda R$ [29]

Little's Law

• mean number = arrival rate x mean response

• queue and server: $Q=\lambda R$

• subsystem in network: $Q_i = \lambda_i R_i$

• network system: $Q_{sys} = \lambda_{sys} R_{sys}$

• queue: queue length $L=\lambda W$ (waiting time)

• assumption: arrivals = departures

• job flow balance

Little's Law: $Q=\lambda R$ [30]

rules

• (a) $y_{arrival}-y_{departure}=$ (b)

• (a) $x_{departure}-x_{arrival}=$ (c)

• (a) Area = (b) Area = (c) Area = $J=9$

Little's Law

• (b) mean number in system $Q=\frac{J}{T}=\frac{N}{T}$ x $\frac{J}{N}=\lambda R$

• $Q=$ arrival rate x mean time in system

• $Q=\lambda R$: 3/7 x 9/3 = 9/7

Little's Law: $Q=\lambda R$ [31]

series

• given: $R_1=0.2,R_2=0.1$ secs

• $Q_1=\lambda R_1=(10)(0.2) = 2$ jobs

• $Q_2=$

• $R_{sys}=0.2 + 0.1=0.3$ secs

• $Q_{sys}=$

split/merge

• given waiting time in queue: $W_1=0.05$ secs

• $L_1=$

cycle

• given: $R=0.02$

• $Q=$

Queue M/M/1 Law: $R=1/(\mu-\lambda)$ [32]

M/M/c

• Markov=Poisson=memoryless

• Markov arrivals at rate $\lambda$

• Markov service at rate $\mu$

• c parallel servers connected to one queue

• high probability of $\Delta$ jobs = $\pm 1$ in small time frame

M/M/1

• Markov arrivals, service to a queue with 1 server

memoryless (Markov)

• time until next event does not depend on the time since the last event

• if long time since job arrival, the expected time until the next job is still $1/\lambda$

• if 1 hr since last job, expected time is 0.1 secs

Queue M/M/1 Law: $R=1/(\mu-\lambda)$ [33]

response

• response time: $R =\frac{1}{\mu-\lambda}$

• Little's Law for system: $Q=\lambda R = \frac{\lambda}{\mu-\lambda}$

length

• expected queue length: $L$

• $Q = L +$ probability server is busy ($\rho$)

• $L=Q-\rho=\frac{\lambda^2}{\mu(\mu-\lambda)}$

waiting

• Little's Law for queue: $L=\lambda W$

• waiting time: $W=\frac{\lambda}{\mu(\mu-\lambda)}$

M/M/1 Law: $R=1/(\mu-\lambda)$ [34]

series

• response: $R_1 = 1/(\mu_1-\lambda_1)=1/(15-10)=0.2$

• waiting: $W_1=\frac{\lambda_1}{\mu_1(\mu_1-\lambda_1)}=\frac{10}{15(15-10)}=0.13$ secs

• service: $S_1=1/\mu_1=1/15=0.07$ sec

• response = waiting + service (0.2=0.13+0.07)

• $R_2 = 1/(\mu_2-\lambda_2)=1/(20-10)=0.1$

split/merge

• $R_1=$

• $R_2=$

• $R_3=$

• $W_1=$

• $L_3=$

cycle

• $R=$

• $W=$

General Response Time Law: $R=\sum R_i V_i$ [35]

Law

• system response time is mean time for jobs: $R$

• $R=$ sum of server times weighted by visit ratio

• Little's Law for system: $Q=\lambda R$

• Little's Law for server: $Q_i = \lambda_i R_i$

• number in system: $Q= Q_1+Q_2+...+Q_n$

• $Q=\lambda R = \lambda_1 R_1 + \lambda_2 R_2+...+\lambda_n R_n$

• $R=R_1\lambda_1/\lambda+\lambda_2/\lambda R_2 +...+\lambda_n/\lambda R_n$

• $R=\sum R_i V_i$

series

• $R=R_1 V_1+R_2 V_2=R_1 + R_2 = 0.2+0.1=0.3$ sec

split/merge

• $R=R_1 V_1 + R_2 V_2 +R_3 V_3$

• $R=(0.12)(2/3)+(0.06)(1/3)+(0.20)(1)=0.30$

cycle

• $R=$

Case Study: CPU vs. I/O [36]

1. Stream

• $\lambda_1 = \lambda +$ $=f(\lambda,p)=$

• $\lambda_2 = (1-p) \cdot$ $=$

• $\lambda_3 =$$=$

• $\lambda_4 =$$=$

2. Visits

• $V_{CPU} = \lambda_1/\lambda=$

• $V_{I/O}=V_2=V_3=V_4=\lambda_4/\lambda=$

Case Study: CPU vs. I/O [37]

3.Utilization

• $\rho_{CPU} =$

• $\rho_{I/O} =\rho_2=\rho_3=\rho_4=$

4.Bottleneck

• step 1: $\rho_{max}=50\%, \mu_b=100$ at $b=CPU$

• step 2: visit ratio $V_b=1/p$

• $\lambda_{max} = \mu_b/V_b = p\mu_b=(0.1)(100) = 10$ jobs/sec

• usually I/O is bottleneck but 3 I/O's

Case Study: CPU vs. I/O [38]

5. M/M/1

• $R_{CPU}=$

• $R_{I/O}=$

6. Little's

• $Q_{CPU}=$

• $Q_{I/O}=$

7. General Response

• $R=\sum R_i V_i=$

• $R=$

question

• what if there are $n$ I/O processors?