Assignment 4

Problems with parameterization (example:keeper usage):

average duration: 1.27, min: 0.106, max: 6.46

possible
outcome
for keeper
and crane
queues?
For validation, simulate long enough.
 Half widths
Half width determination by Arena: statistical analysis of samples. Arena help file:

Half Width (Runtime Confidence Intervals—Within a Replication)

Some sections contain a column called "Half Width". This statistic is included to help you
determine the reliability of the results from your replication. Three results are possible in
the "Half Width" category:
Insufficient: The formula used to calculate half width requires the samples to be normally
distributed. That assumption may be violated if there is a small number (fewer than 320) of
samples. If that is the case, Arena will return the message "Insufficient" for that variable’s half
width, indicating there is insufficient data to accurately calculate the half width. Running the
simulation for a longer period of time should correct this.
Correllated: The formula used to calculate half width requires the samples to be
independently distributed. Data that is correlated (the value of one observation strongly
influences the value of the next observation) results in an invalid confidence interval
calculation. If data is determined to be correlated, the message "Correllated" is returned for
that variable’s half width. Running the simulation for a longer period of time
should correct this.
A value: If a value is returned in the Half Width category, this value may be interpreted by
saying "in 95% of repeated trials, the sample mean would be reported as
within the interval sample mean ± half width".
 Half width calculation
Arena has to be trusted w.r.t. confidence values.
Computations "inside Arena" not clear!
- determine average and variance s^2
- test for insufficient/correlated
- half width = C.s for some constant C.
?
Concepts from probability theory and statistics.

Homework:
Appendices B,C,D of lecture notes.
 Randomization
Random generator is in fact deterministic!
Replaying same model gives same result.
Still, it has all characteristics of "true" random generator
(no "fairness").

DCT case: 15% increase BF trucks

should diminish keeper queues.
Short simulation: keeper queues might get longer!
(see half width)

Increase confidence in simulation.

Divide simulation run into subruns.
Add initial run (move away from initial state).
 Replications
Make sure that replications/subruns are independent
(e.g. no queue length dependencies).

When done right, division into subruns allows to compute

confidence intervals.

Example computation: page 30 of lecture notes.

n = 30 subruns, each with sample of measure x
(occupation rate).
 1 0.914 11 0.894 21 0.898
2 0.964 12 0.962 22 0.912
3 0.934 13 0.973 23 0.943
4 0.978 14 0.984 24 0.953
5 0.912 15 0.923 25 0.923
6 0.956 16 0.932 26 0.914
7 0.958 17 0.967 27 0.923
8 0.934 18 0.924 28 0.936
9 0.978 19 0.945 29 0.945
10 0.976 20 0.936 30 0.934

Sample average/stddev:
x = 0.9408, s = 0.02485. how do you compute s?
s a 
a - confidence interval: x .z  
n 2
Confidence interval matching
Computed 0.95 - confidence interval should match
Arena's half width.
Arena half width should (for large n) approximate

s  0.05 
.z    1.96( s / n )
n  2 

This example: 1.96(0.02485 / 5.51)=0.00451

So with 95% probability, occ.rate in [0.936,0.945]

cf. Chapter 6 of lecture notes

 s a 
a - confidence interval: x .z  
n 2
Function z: normal distribution surface (table lookup).

Consequences:
- You can be 99.99% confident, but not 100%.
- Four times longer simulation halves confidence int.
- high variance = low confidence
 Comparisons
Many simulation studies (e.g. DCT example) are about
relative shortage of resources, leading to queues.
Compare possible solutions through simulation.
Simulation yields to following reports:

Number waiting Average Half Width

Sol1: res1.Queue 4.18 1.20

Number waiting Average Half Width

Sol2: res1.Queue 4.58 1.39

Sol1 better?
 Number waiting Average Half Width
res1.Queue 4.18 1.20

Number waiting Average Half Width

res1.Queue 4.58 1.39

Longer simulation needed to get following result:

Number waiting Average Half Width

res1.Queue 4.11 0.40

Number waiting Average Half Width

res1.Queue 4.69 0.48
 Variance and confidence
Large half widths caused by high subrun variance,
require very long simulations for acceptable confidence.
For instance, compare

1 0.934 11 0.924 21 0.928 1 0.914 11 0.894 21 0.898

2 0.944 12 0.932 22 0.932 2 0.964 12 0.962 22 0.912
3 0.936 13 0.933 23 0.943 3 0.934 13 0.973 23 0.943
4 0.958 14 0.944 24 0.953 4 0.978 14 0.984 24 0.953
5 0.932 15 0.923 25 0.933 5 0.912 15 0.923 25 0.923
6 0.946 16 0.932 26 0.944 6 0.956 16 0.932 26 0.914
7 0.938 17 0.947 27 0.933 7 0.958 17 0.967 27 0.923
8 0.934 18 0.934 28 0.936 8 0.934 18 0.924 28 0.936
9 0.948 19 0.945 29 0.945 9 0.978 19 0.945 29 0.945
10 0.936 20 0.936 30 0.934 10 0.976 20 0.936 30 0.934

First samples:
less variance, higher confidence, shorter simulation run
 Variance reduction 1
Large half widths caused by high subrun variance,
require very long simulations for acceptable confidence.

Various techniques to reduce subrun variance,

e.g. antithetic random numbers (see lecture notes)

Subrun result depends on sequence of random

numbers r1 , r2 ,..., rn
Next subrun: use antithetic sequence: 1  r1 ,1  r2 , ...,1  rn

Frequent arrivals, long processing times in a subrun

compensated by infrequent arrivals and short times
in next subrun.
 Variance reduction 2
Second approach: sharing of e.g. arrival patterns.
Less risk of adopting inferior solution with fewer arrivals.
Both approaches introduce dependency in subruns.

A third approach: compensate for the number of entities.

subruns sorted combined
sr mq #ent sr mq #ent sr mq #ent
1 0.934 3721 5 0.932 3688 5,4 0.945 7442
2 0.944 3696 8 0.934 3694 8,9 0.941 7445
3 0.936 3712 2 0.944 3696 2,10 0.940 7430
4 0.958 3754 7 0.938 3702 7,1 0.936 7423
5 0.932 3688 3 0.936 3712 3,6 0.941 7439
6 0.946 3718 6 0.946 3718
7 0.938 3702 1 0.934 3721
8 0.934 3694 10 0.936 3734
9 0.948 3751 9 0.948 3751
10 0.936 3734 4 0.958 3754
 Sensitivity analysis
Simulation model based on assumptions;
both modeling and parameters.

Assess dependency of simulation result on assumptions.

Simulate with modified parameters and compare.
Sensitive parameters / uncertain assumptions:
show various outcomes.

avql sol A avql sol B

current demand 5.754 5.978
10% increase 7.872 7.901

15% increase 8.869 8.724