2nd Asia-Pacific Forum on Engineering & Technology Education 1999 UICEE

Sydney, Australia, 4 - 7 July, 1999

Addressing complexity and systemic behaviour in engineering management:

A tutorial for real-life problems
K. Linard & A.C. McLucas
University of New South Wales Codarra Advanced Systems
(Australian Defence Force Academy)
Canberra, ACT, Australia Canberra, ACT, Australia
Email: keithlinard#@#yahoo.co.uk
(Remove hashes to email)
ABSTRACT: The human mind is ill-adapted to modelling of problems with delayed feedback. This paper outlines an approach
used in undergraduate and postgraduate programs in the School of Civil Engineering at ADFA to compensate for human cognitive
shortcomings when dealing with complex and systemic engineering management problems. A management problem is presented
and analysed through the medium of a tutorial. It appears easy to solve but has proven otherwise in practice. Logical and well-
intentioned management interventions result in counter-intuitive systemic responses, suggesting that the wrong problem has been
addressed, or inappropriate techniques employed. The proffered methodology, founded in research and practice, addresses problem
conceptualisation, identification of options, strategy development, and evaluation of effectiveness of selected strategies.
Fundamental competencies sought to be enhanced through application of the methodology are: a systemic and holistic perspective;
a skill-set of systemic techniques for enhancing understanding, analysis and decision-making; use of the concepts and language of
systems; and individual and organisational learning.

MISPERCEPTIONS OF FEEDBACK
A UBIQUITOUS HUMAN LIMITATION
Engineering management stresses the importance of project
performance indicators MTBF, BCWP, BCR, etc etc. An
implicit and fundamental assumption behind the selection of
business performance indicators is that the feedback they
provide will cause decision makers to make appropriate
adjustments to the inputs or processes. This presumes decision
makers can judge with reasonable accuracy the consequences
of their decisions.
However, there is abundant research in the field of system
dynamics [1], as well as in the fields of experimental
economics and psychology which suggest that managers have
great difficulty managing dynamically complex tasks. Sterman
argues persuasively from his work at MIT that there is
systematic misperception of feedback especially when there
are delays in the system. Mosekilde, Larsen and Sterman [2]
present the results of 48 simulations of the Beer Game [3] (a
simulation of a simple factory-warehouse-retail system) run
with 192 MBA students and senior business executives. The
results show that their decision-making on the basis of fairly
straight forward performance indicators, but in the face of
delays, resulted on average in costs 10 times the optimum!
Simulations at the Australian Defence Force Academy show a
similar pattern. In both the MIT and ADFA simulations,
managers and graduate students failed to comprehend the
significance of feedback in the face of delay induced dynamics.
In more recent experiments, where graduate students had full
information, training, incentives and opportunities for gaining
experience, Diehl and Sterman still found poor managerial
performance in the face of variations in feedback strength and
delay. [4] They found the subjects were often outperformed by
a simple no-control rule. That is, totally random business
decisions gave better results that the considered judgement of
their MBA students. Diehl and Sterman argue that the mental
constructs and heuristics that managers bring to bear on
complex tasks are fundamentally dynamically deficient:
Subjects were unable to account well for delays and feedback
effects because (1) peoples mental representations of complex
tasks are highly simplified, tending to exclude side effects,
feedback processes, delays, and other elements of dynamic
complexity; and (2) even when these elements are known,
peoples ability to infer correctly the behaviour of even simple
feedback systems is poor.
The implication of this work is that managers require
specialised decision support tools to respond appropriately to
performance indicators impacted by delayed feedback.
The Civil Engineering School at ADFA runs undergraduate
and postgraduate units in system dynamics modelling to give
students the intellectual and analytical tools to address such
cognitive limitations. The following case study forms the
foundation of the undergraduate unit.
THE CASE STUDY SCENARIO
ThreeTwoOne-Ready Pty Ltd owns a small urban concrete
batching plant. Competition is stiff and ThreeTwoOne-Ready
knows that client service is paramount. An inability to respond
on time and on quality to customer demands will lose orders in
the short-term, and clients-base in the longer term.
Initially the company owned three delivery trucks and utilised
additional owner-driver trucks for peak demand, for which it
paid a premium. Company directors invested in two additional
trucks on the basis of a review of demand patterns. However,
even with five company-owned trucks, customer demands
cannot be fully met without supplementation. The company is
struggling to build a reputation as a reliable and responsive
supplier. Directors of ThreeTwoOne-Ready are increasingly
anxious about the companys ability to remain competitive in
the longer-term.
A year ago the directors advised staff of the plan to invest in
additional trucks, increase office staff and simplify the
ordering workflow procedures. Instead of the expected
accolade, they were met with a barrage of complaints about
antiquated plant, excessive working hours and the ignoring of
employee suggestions on productivity improvement.
In the event, these measures did not solve the supply problems.
Truck efficiency in fact decreased, while average response
time to clients remained virtually unaltered. There was no
decrease in the number of client sales lost due to inability to
deliver as required. The intuitively obvious solution increased
costs, lowered efficiency and left ThreeTwoOne-Ready
marginally competitive, at best.
At this point the student consulting group(s) take over.
CASE STUDY OVERVIEW (IISD & SDM)
The tutorial provides a structured methodology, Iterative and
Interactive Strategy Development (IISD) [5] for understanding
the problem and identifying likely value adding solutions. It
supports this with a rigorous computer simulation modelling
approach, System Dynamics Modelling (SDM), to validate the
business rules of the proposed solution(s). Together these
management tools help build the shared vision among the
stakeholders which will underpin implementation of a solution.
Information
Experience Perception
Knowledge
Iterative and
Interactive
Strategy
Development
Strategy
Consensus Commitment
to Action
Figure 1: Iterative and Interactive Strategy Development
The tutorial shows why it is critical to identify unresolved
issues and their influences, and how to set about solving the
'right' problem. The tutorial demonstrates that only after the
nature of the problem has been established can problem
definition and conceptualisation commence.
Each step of the methodology is demonstrated to the point
where a set of remedial strategies is identified. System
dynamics modelling proves ideal for both detailed analysis of
the problem and testing of potential strategies before
implementation. The strategies are simulated and tested in a
benign environment where pre-implementation rehearsal can
be conducted without causing collateral damage. The whole
analysis process requires ongoing involvement of stakeholders.
An opportunity for rethinking permits reconsideration of
strategies before taking the critical step to implementation.
CASE STUDY OBJECTIVES
With an overall aim of communicating ways to address the
complexity and systemic behaviour in socio-technical systems,
the case study highlights:
a. Approaches to eliciting, recording and testing the mental
models that stakeholders have of the problem set.
b. Approaches to eliciting, recording and testing the mental
models that stakeholders have of the problem set.
Policy
implementation
Understandings
of a system
Policy Problem
analysis definition
Simulation
System
conceptualization
Model
formulation
Figure 2: Overview of the system dynamics modelling
approach (Richardson and Pugh, 1981, p. 17)
c. Analytical tools and techniques to overcome the cognitive
Motivation limitations of the human mind in dealing with complex,
systemic, dynamic problems. The capabilities of these
include:
building systemic and holistic views of the problem
Inputs from Client and rather than taking simplistic or reductionist views.
Environment
a technical language which uses systemic concepts in
describing problems and their behaviour.
Desired Outputs at a skill-set of systemic techniques for understanding,
Individual and
Organisational Levels analysing and making decisions problems
individual learning.
organisational learning in the context of group
Learning
decision-making.
skills in conflict resolution and group dynamics.
d. Underpinning the above are:
developing an understanding of the nature of
organised complexity.
developing an understanding of system dynamics
techniques and their application.
building systems thinking and systems thinking skills.
developing insight into and understanding of dynamic
behaviour
QUALITATIVE AND QUANTITATIVE ANALYSIS
The critical first step is to investigate ab initio the nature of the
problem. The problem is subjected to deliberate elicitation of
stakeholder domain knowledge as a pre-cursor to formal
problem conceptualisation. The aim of qualitative analysis is
to produce an influence diagram showing the influences at
work, the interplay of which is the cause of its dynamic
behaviour. [6]
Many problem solving techniques overlook rigorous
qualitative analysis or avoid it because of perceived poor
traceability through to quantitative analysis. IISD permits
strong and logical development, with clear audit trails, from
qualitative through quantitative analysis.
PERSPECTIVES OF THE PROBLEM
Decision-makers cognitive make-up creates preferences for
particular types of reasoning. Often, those from non-technical
backgrounds are averse to quantitative analytical approaches.
[7] During the early stages of problem conceptualisation, and
especially in investigation of individual stake-holdings, it is
important to gauge the stakeholders preferred modes for
gathering and processing information. Cognitive mapping is a
valuable tool in this regard as it combines a rigorous and
structured methodology for capturing ideas with a qualitative
analytical focus.
QUALITATIVE ASSESSMENT OF THE PROBLEM
CONTEXT COGNITIVE MAPPING
This knowledge elicitation step collects and collates
perceived knowledge about the problem from key
stakeholders. It seeks to identify and define stakeholder
perspectives, values, ingrained assumptions, and preferences
for gathering and processing information.
Knowledge elicitation helps identify areas of agreement,
contentious issues, misunderstandings and potential for
misinformation. It collects quantitative facts and qualitative
impressions, beliefs and feelings. In IISD, this is done through
mapping factors impacting on the problem. From the
individual stakeholder maps it is then possible to develop
aggregated, or summary, views that are the basis for gaining
agreement on specification of the problem. The output is an
influence diagram.
Figure 3: First pass influence diagram of ThreeTwoOne-
Readys problem produced from cognitive maps.
With revision and progressive refinement, the influence
diagram graphically maps the problem with increasing
clarity. Dominant mechanisms are identified and the links are
mapped for all to see, appreciate, challenge, and ultimately
Accept or reject. There may be further cycles of analysis and
challenging of assumptions until the diagram is in a form
broadly acceptable to all key stakeholders.
Achieving complete agreement about the nature of the problem
often turns out to be an unattainable ideal. Despite this, the
influence diagram proves to be a valuable record and cue for
more investigative discussions and the foundation for further
detailed analysis. Equally important, it provides the basis for
dialogue with and between stakeholders.
The preferred mapping technique is based on George Kellys
time-honoured, 1955 Psychological Theory of Personal
Constructs as interpreted and employed most effectively by
Ackermann, Williams, and Eden. [8] Our experience is that
clients take readily to these diagrams and find them valuable in
clarifying and summarising their ideas.
Often issues identified in stakeholder interviews and
preliminary qualitative analysis are found not to be 'root-cause'
issues, but distracters. In ThreeTwoOnes case such 'side
issues' have triggered the discontent. Whilst they may only
be distractors they nevertheless must be addressed before
key organisational problems can be confronted. Without the
stakeholder support major initiatives are likely to fail.
QUANTITATIVE ANALYSIS OF THE BUSINESS RULES
SYSTEM DYNAMICS MODELLING
While qualitative analysis identifies causal relationships and
feedbacks , it does not aid understanding of second or higher
order dynamic impacts. Also, qualitative descriptors of the
relationship between variables often disguise complexity in
business rules. To validate the business rules developed
through the interviews, and especially to understand the
implications of delayed feedback mechanisms, quantitative
system dynamics models is often essential.
The models are built in modular form from influence diagrams
accepted by the stakeholders. When fully integrated, modules
replicate observed behaviour to a level of fidelity appropriate
to the problem analysis and the strategy development.
Close interaction with stakeholders is desirable throughout the
model development. This is particularly important when hard
data is difficult to obtain, or the behaviour of elements of the
system does not lend themselves readily to quantification.
Stakeholder knowledge and experience regarding dynamic
behaviour of the system are incorporated into the model as it
evolves. This involvement is also crucial to building
ownership of, and confidence in, strategies identified through
IISD.
The process of model building significantly aids student
understanding of feedback effects, and the counter-intuitive
consequences of delay induced dynamic behaviour.
Figure 4: (Simplified) Truck module of student System
Dynamics model of ThreeTwoOne-Ready operations
The first iterative cycle ends with the presentation to sponsor
and key stakeholders of influence diagrams, system dynamics
models and, if appropriate, prospective strategies.
During subsequent cycles model development continues. The
emphasis shifts to learning, by 'playing' the models repeatedly,
how and why alternate strategies lead to different systemic
behaviour. Cycles of IISD continue as needed in response to
performance monitoring against model-specific metrics.
FLIGHT SIMULATOR
Preferred strategies are identified and then subjected to further
detailed analysis both to validate the assumptions and to fine
tune the proposed business rules. Technologies such as genetic
algorithm optimisation may then be harnessed with the system
dynamics software to identify both the sensitivity of the model
to different decision levers and the optimum setting of those
levers. We now have a solution, but the critical step is still to
get stakeholder buy-in. The development of a flight
simulator is a valuable tool in achieving this.
Figure 5: Flight simulator control panel for
ThreeTwoOne-Ready system dynamics model
The flight-simulator is simply the system dynamics model
converted to a management game by the addition of a user-
friendly control panel and performance indicator display.
Stakeholders can test the impact of their personal wish lists
and compare it with the recommended strategy. If the issue is
sufficiently contentious, the system dynamics software
(Powersim) allows the development of multi-user war-games.
Most adults learn best by doing, by trying things out in real-
time. However, learning by doing only works as long as the
feedback from our actions is rapid and unambiguous. The
flight-simulator is an ideal practice field where stakeholders
have perfect freedom to experiment with alternative strategies.
This improves the likelihood of developing a shared vision for
policy implementation.
STUDENT REACTION TO APPROACH
The UNSW undertakes periodic student ratings of all subjects.
The student assessment of the Systems Dynamics subjects, at
both undergraduate and postgraduate levels is very favourable
in relation to student interest and perceived difficulty.
On a 1-7 scale (7 being most favourable), over 95% of 1997
and 1998 respondents indicated that they would most certainly
recommend the subject to other students. A significant number
of these students, however, also rated the subject as more
difficult than their other (more technical) units. Follow up
questioning suggests that students had mastered the skills to
address technical problems, where the objectives are clear and
unambiguous. However, they were less prepared for a
situation where there are multiple stakeholders and multiple or
even conflicting objectives, and where the paradigm is one of
mutual learning rather than optimisation..
SUMMARY
The system dynamics unit at ADFA seeks to provide a
conceptual basis for, and experience in, rational enquiry into
and design of complex socio-technical systems.
Its specific goal is to increase student effectiveness in
analytical problem solving through increasing their level and
use of systems thinking and system dynamics modelling. This
requires that they understand problem solving as a social
learning process, where the solution is an emergent property of
a group enquiry. This enquiry should be based on systems
thinking principles and methodologies, with particular
reference to soft systems modelling and system dynamics
theory. This is achieved through the application of a structured
methodology (IISD) and powerful computer tools (Decision
Explorer and Powersim) to a realistic case study.
1 Sterman, J. Misperceptions of Feedback in Dynamic Decision
Making. Organisational Behaviour and Human Decision
Processes, 1989, 43(3), 301-335.
Paich, M, & J Sterman. Boom, Bust and Failures to Learn in
Experimental Markets. Management Science, 1993, 39(12), 1439-
1458.
Smith, V, G Suchanek and A Williams. Bubbles, Crashes and
Endogenous Expectations in Experimental Spot Asset Markets,
Econometrica, 1988, 56(5), 1119-1152.
Funke, J, Solving Complex Problems: Exploration and Control of
Complex Systems, in R Sternberg and P Frensch (eds.), Complex
Problem Solving: Principles and Mechanisms. Erlbaum Assoc.,
New Jersey, 1991.
2 Mosekilde, E, E Larsen and J Sterman. Coping With Complexity:
Deterministic Chaos in Human Decision Making Behaviour, in J
Casti and A Karlqvist (eds.), Beyond Belief: Randomness,
Prediction and Exploration in Science. CRC Press, Boston, 1990.
3 The Beer Game is described in detail in Senge, P, The Fifth
Discipline - The Art and Practice of the Learning Organization.
Doubleday, New York, 1990
4 Diehl, E and J Sterman. Effects of Feedback Complexity on
Dynamic Decision Making. MIT Sloan School of Management,
Research Report D-4401-1. March 1994.
5 McLucas, A., Integrating Soft and Hard Systems Analysis:
Seeking a Practical Framework for Addressing Strategic
Problems. Proc Systems Engineering Pragmatic Solutions to
Todays Real World Problems. Systems Engineering Society of
Australia. Nov 1998.
6 (Coyle, R., System Dynamics Modelling: A Practical Approach.
Chapman Hall, London, 1996.
7 (Nutt, P., Making Tough Decisions: Tactics for Improving
Managerial Decision Making. Jossey-Bass, San Francisco, 1989.
Churchman, C., On the Designing of Inquiring Systems: Basic
Concepts in Systems and Organisations. Basic Books, New York,
1971
Mason, R.O. and Mitroff, I.I., A Program for Research on
Management Information Systems. Management Science, 1973,
19(5), pp 475-487.
8 Ackerman, F., Eden, C., and Williams, T., Modelling in
Litigation: Mixing Qualitative and Quantitative Approaches.
Institute for Operations Research in the Management Sciences.
London, 1997.