Advanced Control using

M ATLAB

or Stabilising the
unstabilisable
David I. Wilson
Auckland University of Technology
New Zealand
February 12, 2014
Copyright © 2014 David I. Wilson
Auckland University of Technology
New Zealand

Creation date: February, 2014.

All rights reserved. No part of this work may be reproduced, stored in a retrieval system, or
transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or
otherwise, without prior permission.
C ONTENTS

1 Introduction 1
1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Matlab for computer aided control design . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Alternative computer design aids . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Economics of control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Laboratory equipment for control tests . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4.1 Plants with one input and one output . . . . . . . . . . . . . . . . . . . . . . 6
1.4.2 Multi-input and multi-output plants . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Slowing down Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 From differential to difference equations 13

2.1 Computer in the loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 Sampling an analogue signal . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.2 Selecting a sample rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.3 The sampling theorem and aliases . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.4 Discrete frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Finite difference models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 Difference equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 The z transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.1 z-transforms of common functions . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Inversion of z-transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.1 Inverting z-transforms with symbolically . . . . . . . . . . . . . . . . . . . . 24
2.4.2 The partial fraction method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.3 Long division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.4 Computational approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.5 Numerically inverting the Laplace transform . . . . . . . . . . . . . . . . . . 31
2.5 Discretising with a sample and hold . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.5.1 Converting Laplace transforms to z-transforms . . . . . . . . . . . . . . . . 36
2.5.2 The bilinear transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.6 Discrete root locus diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.7 Multivariable control and state space analysis . . . . . . . . . . . . . . . . . . . . . . 44
2.7.1 States and state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.7.2 Converting differential equations to state-space form . . . . . . . . . . . . . 47
2.7.3 Interconverting between state space and transfer functions . . . . . . . . . . 49
2.7.4 Similarity transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.7.5 Interconverting between transfer functions forms . . . . . . . . . . . . . . . 54
2.7.6 The steady state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.8 Solving the vector differential equation . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.8.1 Numerically computing the discrete transformation . . . . . . . . . . . . . . 61
2.8.2 Using M ATLAB to discretise systems . . . . . . . . . . . . . . . . . . . . . . . 63
2.8.3 Time delay in state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.9 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

iii
iv CONTENTS

2.9.1 Stability in the continuous domain . . . . . . . . . . . . . . . . . . . . . . . . 69

2.9.2 Stability of the closed loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
2.9.3 Stability of discrete time systems . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.9.4 Stability of nonlinear differential equations . . . . . . . . . . . . . . . . . . . 74
2.9.5 Expressing matrix equations succinctly using Kronecker products . . . . . . 80
2.9.6 Summary of stability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3 Modelling dynamic systems with differential equations 85

3.1 Dynamic system models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.1.1 Steady state and dynamic models . . . . . . . . . . . . . . . . . . . . . . . . 86
3.2 A collection of illustrative models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.2.1 Simple models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.3 Chemical process models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.3.1 A continuously-stirred tank reactor . . . . . . . . . . . . . . . . . . . . . . . 92
3.3.2 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.3.3 A binary distillation column model . . . . . . . . . . . . . . . . . . . . . . . 95
3.3.4 Interaction and the Relative Gain Array . . . . . . . . . . . . . . . . . . . . . 103
3.4 Curve fitting experimental data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.1 Polynomial regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.2 Least squares improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.4.3 Nonlinear least-squares model identification . . . . . . . . . . . . . . . . . . 112
3.4.4 Parameter confidence intervals . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.5 Numerical tools for modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.5.1 The solution of nonlinear ordinary differential equations . . . . . . . . . . . 120
3.5.2 Differential/Algebraic equation systems and algebraic loops . . . . . . . . . 123
3.6 Linearisation of nonlinear dynamic equations . . . . . . . . . . . . . . . . . . . . . . 125
3.6.1 Linearising a nonlinear tank model . . . . . . . . . . . . . . . . . . . . . . . 127
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

4 The PID controller 131

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.1.1 P, PI or PID control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.2 The industrial PID algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
4.2.1 Implementing the derivative component . . . . . . . . . . . . . . . . . . . . 133
4.2.2 Variations of the PID algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.2.3 Integral only control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3 Simulating a PID process in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.4 Extensions to the PID algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.4.1 Avoiding derivative kick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.4.2 Input saturation and integral windup . . . . . . . . . . . . . . . . . . . . . . 140
4.5 Discrete PID controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.5.1 Discretising continuous PID controllers . . . . . . . . . . . . . . . . . . . . . 144
4.5.2 Simulating a PID controlled response in Matlab . . . . . . . . . . . . . . . . 146
4.5.3 Controller performance as a function of sample time . . . . . . . . . . . . . 148
4.6 PID tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
4.6.1 Open loop tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
4.6.2 Closed loop tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
4.6.3 Closed loop single-test tuning methods . . . . . . . . . . . . . . . . . . . . . 159
4.6.4 Summary on closed loop tuning schemes . . . . . . . . . . . . . . . . . . . . 166
4.7 Automated tuning by relay feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.7.1 Describing functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
4.7.2 An example of relay tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.7.3 Self-tuning with noise disturbances . . . . . . . . . . . . . . . . . . . . . . . 173
CONTENTS v

4.7.4 Modifications to the relay feedback estimation algorithm . . . . . . . . . . . 176

4.8 Drawbacks with PID controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.8.1 Inverse response processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.8.2 Approximating inverse-response systems with additional deadtime . . . . 184
4.9 Dead time compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.10 Tuning and sensitivity of control loops . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

5 Digital filtering and smoothing 193

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
5.1.1 The nature of industrial noise . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
5.1.2 Differentiating without smoothing . . . . . . . . . . . . . . . . . . . . . . . . 196
5.2 Smoothing measured data using analogue filters . . . . . . . . . . . . . . . . . . . . 197
5.2.1 A smoothing application to find the peaks and troughs . . . . . . . . . . . . 197
5.2.2 Filter types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.2.3 Classical analogue filter families . . . . . . . . . . . . . . . . . . . . . . . . . 201
5.3 Discrete filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.3.1 A low-pass filtering application . . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.3.2 Digital filter approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
5.3.3 Efficient hardware implementation of discrete filters . . . . . . . . . . . . . 214
5.3.4 Numerical and quantisation effects for high-order filters . . . . . . . . . . . 217
5.4 The Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
5.4.1 Fourier transform definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.4.2 Orthogonality and frequency spotting . . . . . . . . . . . . . . . . . . . . . . 224
5.4.3 Using M ATLAB’s FFT function . . . . . . . . . . . . . . . . . . . . . . . . . . 225
5.4.4 Periodogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
5.4.5 Fourier smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
5.5 Numerically differentiating industrial data . . . . . . . . . . . . . . . . . . . . . . . 230
5.5.1 Establishing feedrates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

6 Identification of process models 235

6.1 The importance of system identification . . . . . . . . . . . . . . . . . . . . . . . . . 235
6.1.1 Basic definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6.1.2 Black, white and grey box models . . . . . . . . . . . . . . . . . . . . . . . . 237
6.1.3 Techniques for identification . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
6.2 Graphical and non-parametric model identification . . . . . . . . . . . . . . . . . . 239
6.2.1 Time domain identification using graphical techniques . . . . . . . . . . . . 239
6.2.2 Experimental frequency response analysis . . . . . . . . . . . . . . . . . . . 245
6.2.3 An alternative empirical transfer function estimate . . . . . . . . . . . . . . 252
6.3 Continuous model identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
6.3.1 Fitting transfer functions using nonlinear least-squares . . . . . . . . . . . . 254
6.3.2 Identification of continuous models using successive differentiation . . . . 256
6.3.3 Practical continuous model identification . . . . . . . . . . . . . . . . . . . . 259
6.4 Popular discrete-time linear models . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
6.4.1 Extending the linear model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
6.4.2 Output error model structures . . . . . . . . . . . . . . . . . . . . . . . . . . 264
6.4.3 General input/output models . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6.5 Regressing discrete model parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.5.1 Simple offline system identification routines . . . . . . . . . . . . . . . . . . 269
6.5.2 Bias in the parameter estimates . . . . . . . . . . . . . . . . . . . . . . . . . . 270
6.5.3 Using the System Identification toolbox . . . . . . . . . . . . . . . . . . . . . 271
6.5.4 Fitting parameters to state space models . . . . . . . . . . . . . . . . . . . . 275
6.6 Model structure determination and validation . . . . . . . . . . . . . . . . . . . . . 277
vi CONTENTS

6.6.1 Estimating model order and deadtime . . . . . . . . . . . . . . . . . . . . . . 280

6.6.2 Robust model fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
6.6.3 Common nonlinear model structures . . . . . . . . . . . . . . . . . . . . . . 283
6.7 Online model identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.7.1 Recursive least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
6.7.2 Recursive least-squares in M ATLAB . . . . . . . . . . . . . . . . . . . . . . . 289
6.7.3 Tracking the precision of the estimates . . . . . . . . . . . . . . . . . . . . . . 293
6.8 The forgetting factor and covariance windup . . . . . . . . . . . . . . . . . . . . . . 295
6.8.1 The influence of the forgetting factor . . . . . . . . . . . . . . . . . . . . . . . 297
6.8.2 Covariance wind-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
6.9 Identification by parameter optimisation . . . . . . . . . . . . . . . . . . . . . . . . . 300
6.10 Online estimating of noise models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
6.10.1 A recursive extended least-squares example . . . . . . . . . . . . . . . . . . 306
6.10.2 Recursive identification using the SI toolbox . . . . . . . . . . . . . . . . . . 309
6.10.3 Simplified RLS algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
6.11 Closed loop identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
6.11.1 Closed loop RLS in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
6.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

7 Adaptive Control 321

7.1 Why adapt? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
7.1.1 The adaption scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
7.1.2 Classification of adaptive controllers . . . . . . . . . . . . . . . . . . . . . . . 323
7.2 Gain scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
7.3 The importance of identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
7.3.1 Polynomial manipulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
7.4 Self tuning regulators (STRs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.4.1 Simple minimum variance control . . . . . . . . . . . . . . . . . . . . . . . . 327
7.5 Adaptive pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
7.5.1 The Diophantine equation and the closed loop . . . . . . . . . . . . . . . . . 330
7.5.2 Solving the Diophantine equation in Matlab . . . . . . . . . . . . . . . . . . 331
7.5.3 Adaptive pole-placement with identification . . . . . . . . . . . . . . . . . . 334
7.5.4 Adaptive pole-placement in S IMULINK . . . . . . . . . . . . . . . . . . . . . 339
7.6 Practical adaptive pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
7.6.1 Dealing with non-minimum phase systems . . . . . . . . . . . . . . . . . . . 340
7.6.2 Separating stable and unstable factors . . . . . . . . . . . . . . . . . . . . . . 343
7.6.3 Experimental adaptive pole-placement . . . . . . . . . . . . . . . . . . . . . 346
7.6.4 Minimum variance control with dead time . . . . . . . . . . . . . . . . . . . 346
7.7 Summary of adaptive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

8 Multivariable controller design 357

8.1 Controllability and observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
8.1.1 Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
8.1.2 Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
8.1.3 Computing controllability and observability . . . . . . . . . . . . . . . . . . 361
8.1.4 State reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
8.2 State space pole-placement controller design . . . . . . . . . . . . . . . . . . . . . . 365
8.2.1 Poles and where to place them . . . . . . . . . . . . . . . . . . . . . . . . . . 369
8.2.2 Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
8.3 Estimating the unmeasured states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
8.4 Combining estimation and state feedback . . . . . . . . . . . . . . . . . . . . . . . . 374
8.5 Generic model control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
8.5.1 The tuning parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
8.5.2 GMC control of a linear model . . . . . . . . . . . . . . . . . . . . . . . . . . 380
CONTENTS vii

8.5.3 GMC applied to a nonlinear plant . . . . . . . . . . . . . . . . . . . . . . . . 382

8.6 Exact feedback linearisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
8.6.1 The nonlinear system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
8.6.2 The input/output feedback linearisation control law . . . . . . . . . . . . . 387
8.6.3 Exact feedback example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
8.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

9 Classical optimal control 395

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
9.2 Parametric optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
9.2.1 Choosing a performance indicator . . . . . . . . . . . . . . . . . . . . . . . . 396
9.2.2 Optimal tuning of a PID regulator . . . . . . . . . . . . . . . . . . . . . . . . 397
9.2.3 Using S IMULINK inside an optimiser . . . . . . . . . . . . . . . . . . . . . . . 403
9.2.4 An optimal batch reactor temperature policy . . . . . . . . . . . . . . . . . . 404
9.3 The general optimal control problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
9.3.1 The optimal control formulation . . . . . . . . . . . . . . . . . . . . . . . . . 407
9.3.2 The two-point boundary problem . . . . . . . . . . . . . . . . . . . . . . . . 409
9.3.3 Optimal control examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
9.3.4 Problems with a specified target set . . . . . . . . . . . . . . . . . . . . . . . 414
9.4 Linear quadratic control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
9.4.1 Continuous linear quadratic regulators . . . . . . . . . . . . . . . . . . . . . 417
9.4.2 Analytical solution to the LQR problem . . . . . . . . . . . . . . . . . . . . . 419
9.4.3 The steady-state solution to the matrix Riccati equation . . . . . . . . . . . . 423
9.4.4 The discrete LQR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
9.4.5 A numerical validation of the optimality of LQR . . . . . . . . . . . . . . . . 431
9.4.6 An LQR with integral states . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
9.5 Estimation of state variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
9.5.1 Random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
9.5.2 Combining deterministic and stochastic processes . . . . . . . . . . . . . . . 445
9.5.3 The Kalman filter estimation scheme . . . . . . . . . . . . . . . . . . . . . . . 446
9.5.4 The steady-state form of the Kalman filter . . . . . . . . . . . . . . . . . . . . 450
9.5.5 Current and future prediction forms . . . . . . . . . . . . . . . . . . . . . . . 452
9.5.6 An application of the Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . 455
9.5.7 The role of the Q and R noise covariance matrices in the state estimator . . 456
9.5.8 Extensions to the basic Kalman filter algorithm . . . . . . . . . . . . . . . . . 460
9.5.9 The Extended Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
9.5.10 Combining state estimation and state feedback . . . . . . . . . . . . . . . . . 465
9.5.11 Optimal control using only measured outputs . . . . . . . . . . . . . . . . . 466
9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

10 Predictive control 469

10.1 Model predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
10.1.1 Constrained predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . 472
10.1.2 Dynamic matrix control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
10.2 A Model Predictive Control Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.2.1 A model predictive control GUI . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.2.2 MPC toolbox in M ATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.2.3 Using the MPC toolbox in S IMULINK . . . . . . . . . . . . . . . . . . . . . . 484
10.2.4 MPC control of an unstable 3 degree of freedom helicopter . . . . . . . . . . 485
10.2.5 Further readings on MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
10.3 Optimal control using linear programming . . . . . . . . . . . . . . . . . . . . . . . 489
10.3.1 Development of the LP problem . . . . . . . . . . . . . . . . . . . . . . . . . 490
viii CONTENTS

11 Expert systems and neural networks 499

11.1 Expert systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
11.1.1 Where are they used? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
11.1.2 Features of an expert system . . . . . . . . . . . . . . . . . . . . . . . . . . . 501
11.1.3 The user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502
11.1.4 Expert systems used in process control . . . . . . . . . . . . . . . . . . . . . 503
11.2 Neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
11.2.1 The architecture of the neural network . . . . . . . . . . . . . . . . . . . . . . 507
11.2.2 Curve fitting using neural networks . . . . . . . . . . . . . . . . . . . . . . . 511
11.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515

A List of symbols 517

B Useful utility functions in Matlab 519

C Transform pairs 521

D A comparison of Maple and MuPad 523

D.1 Partial fractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
D.2 Integral transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
D.3 Differential equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
D.4 Vectors and matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525

E Useful test models 527

E.1 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
E.2 Aircraft model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
L IST OF F IGURES

1.1 Traditional vs. Advanced control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Economic improvements of better control . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Blackbox configuration. The manual switch marked will toggle between either 7
or 9 low-pass filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 The “Black-box” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Balance arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6 The flapper wiring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.7 Flapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.8 Helicopter plant with 2 degrees of freedom. See also Fig. 1.9(a). . . . . . . . . . . . 10
1.9 Helicopter control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.10 Helicopter flying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.11 Real-time Simulink simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1 The computer in the control loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 3 bit sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 A time series with unknown frequency components . . . . . . . . . . . . . . . . . . 18
2.4 The frequency component of a sampled signal . . . . . . . . . . . . . . . . . . . . . 18
2.5 Frequency aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 The Scarlet Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.7 Hénon’s attractor in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.8 Hénon’s attractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.9 Inverting z-transforms using dimpulse . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.10 Numerically inverting the Laplace transform using the Bromwich integral . . . . . 32
2.11 Numerically inverting the Laplace transform . . . . . . . . . . . . . . . . . . . . . . 33
2.12 Numerically inverting Laplace transforms . . . . . . . . . . . . . . . . . . . . . . . . 34
2.13 Ideal sampler and zeroth-order hold . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.14 Zeroth-order hold effects on the discrete Bode diagram . . . . . . . . . . . . . . . . 41
2.15 Bode plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.16 The discrete root locus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.17 Various discrete closed loop responses . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.18 A binary distillation column with multiple inputs and multiple outputs . . . . . . 45
2.19 A block diagram of a state-space dynamic system, (a) continuous system: ẋ =
Ax + Bu, and (b) discrete system: xk+1 = Φxk + ∆uk . (See also Fig. 2.20.) . . . . 46
2.20 A complete block diagram of a continuous state-space dynamic system with out-
puts and direct measurement feed-through, Eqn. 2.41. . . . . . . . . . . . . . . . . . 47
2.21 Unsteady and steady states for level systems . . . . . . . . . . . . . . . . . . . . . . 56
2.22 Submarine step response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.23 Issues in assessing system stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.24 Nyquist diagram of Eqn. 2.94 in (a) three dimensions and (b) as typically presented
in two dimensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
2.25 Liapunov (1857–1918) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.26 Regions of stability for the poles of continuous (left) and discrete (right) systems . 83

ix
x LIST OF FIGURES

3.1 A stable and unstable pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.2 Simple buffer tank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.3 The UK growth based on the GDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.4 A continually stirred tank reactor (CSTR) reactor . . . . . . . . . . . . . . . . . . . . 93
3.5 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.6 Schematic of a distillation column . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.7 Wood-Berry step response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.8 Wood-Berry column in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.9 Distillation tower . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.10 Sparsity of the distillation column model . . . . . . . . . . . . . . . . . . . . . . . . 101
3.11 Open loop distillation column control . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.12 Distillation column control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.13 Distillation column control (in detail) . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.14 Distillation interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.15 Dynamic RGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.16 Dynamic RGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.17 Density of Air . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.18 Fitting a high-order polynomial to some physical data . . . . . . . . . . . . . . . . . 111
3.19 A bio-chemical reaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.20 Model of compressed water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.21 Experimental pressure-rate data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.22 Parameter confidence regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.23 Linear and nonlinear trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.24 Linearising a nonlinear tank model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

4.1 Comparing PI and integral-only control for the real-time control of a noisy flapper
plant with sampling time T = 0.08. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.2 PID simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3 PID internals in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.4 Block diagram of PID controllers as implemented in S IMULINK (left) and classical
text books (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.5 Realisable PID controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.6 PID controller in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.7 PID controller with anti-derivative kick. . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.8 Avoiding derivative kick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.9 Illustrating the improvement of anti-derivative kick schemes for PID controllers
when applied to the experimental electromagnetic balance. . . . . . . . . . . . . . . 140
4.10 Derivative control and noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.11 Anti-windup comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
4.12 Discrete PID controller in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.13 Headbox control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.14 Headbox controlled response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.15 A PID controlled process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
4.16 Sample time and discrete PID control . . . . . . . . . . . . . . . . . . . . . . . . . . 149
4.17 The parameters T and L to be graphically estimated for the openloop tuning method
relations given in Table 4.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
4.18 Cohen-Coon model fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.19 Cohen-Coon tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.20 PID tuning using a GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.21 Solving for the ultimate frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.22 Ziegler-Nichols tuned responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
4.23 Typical response of a stable system to a P-controller. . . . . . . . . . . . . . . . . . . 160
4.24 A Yuwana-Seborg closed loop step test . . . . . . . . . . . . . . . . . . . . . . . . . . 163
4.25 Closed loop responses using the YS scheme . . . . . . . . . . . . . . . . . . . . . . . 164
LIST OF FIGURES xi

4.26 A self-tuning PID controlled process . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

4.27 A process under relay tuning with the PID regulator disabled. . . . . . . . . . . . . 167
4.28 An unknown plant under relay feedback exhibits an oscillation . . . . . . . . . . . 169
4.29 Nyquist & Bode diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.30 PID Relay tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
4.31 Relay tuning with noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
4.32 Relay tuning of the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
4.33 Relay tuning results of the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
4.34 A relay with hysteresis width h and output amplitude d. . . . . . . . . . . . . . . . 177
4.35 Relay feedback with hysteresis width h. . . . . . . . . . . . . . . . . . . . . . . . . . 178
4.36 Relay feedback with hysteresis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
4.37 Relay feedback with an integrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
4.38 2-point Relay identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.39 Monotonicity index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.40 The J curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.41 An inverse response process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
4.42 A NMP plant controlled with a PI controller . . . . . . . . . . . . . . . . . . . . . . 183
4.43 Approximating inverse-response systems with additional deadtime . . . . . . . . . 185
4.44 The Smith predictor structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
4.45 The Smith predictor structure from Fig. 4.44 assuming no model/plant mis-match. 186
4.46 Smith predictor in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
4.47 Dead time compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.48 Deadtime compensation applied to the blackbox . . . . . . . . . . . . . . . . . . . . 189
4.49 Closed loop with plant G(s) and controller C(s) subjected to disturbances and
measurement noise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
4.50 Sensitivity transfer functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
4.51 Sensitivity robustness measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

5.1 A filter as a transfer function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

5.2 A noisy measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
5.3 Noise added to a true, but unknown, signal . . . . . . . . . . . . . . . . . . . . . . . 195
5.4 Derivative action given noisy data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
5.5 Smoothing industrial data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
5.6 Low-pass filter specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
5.7 Three single low-pass filters cascaded together to make a third-order filter. . . . . . 199
5.8 Amplitude response for ideal, low-pass, high pass and band-pass filters. . . . . . . 200
5.9 Analogue Butterworth filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
5.10 Analogue Chebyshev filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.11 Butterworth and Chebyshev filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5.12 Using a Butterworth filter to smooth noisy data . . . . . . . . . . . . . . . . . . . . . 211
5.13 The frequency response for Butterworth filters . . . . . . . . . . . . . . . . . . . . . 212
5.14 Various Butterworth filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.15 Advantages of frequency pre-warping . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.16 Hardware difference equation in Direct Form I . . . . . . . . . . . . . . . . . . . . . 215
5.17 An IIR filter with a minimal number of delays, Direct Form II . . . . . . . . . . . . 216
5.18 Cascaded second-order sections to realise a high-order filter. See also Fig. 5.19. . . 217
5.19 A second-order section (SOS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5.20 Comparing single precision second-order sections with filters in direct form II
transposed form. Note that the direct form II filter is actually unstable when run
in single precision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
5.21 Approximating square waves with sine waves . . . . . . . . . . . . . . . . . . . . . 223
5.22 The Fourier approximation to a square wave . . . . . . . . . . . . . . . . . . . . . . 223
5.23 Two signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
5.24 Critical radio frequencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
xii LIST OF FIGURES

5.25 Power spectrum for a signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

5.26 Smoothing by Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
5.27 Differentiating and smoothing noisy measurement . . . . . . . . . . . . . . . . . . . 232
5.28 Filtering industrial data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

6.1 The prediction problem: Given a model and the input, u, can we predict the out-
put, y? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
6.2 A good model, M, duplicates the behaviour of the true plant, S. . . . . . . . . . . . 237
6.3 An experimental setup for input/output identification. We log both the input and
the response data to a computer for further processing. . . . . . . . . . . . . . . . . 238
6.4 Typical open loop step tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
6.5 Areas method for model identification . . . . . . . . . . . . . . . . . . . . . . . . . . 241
6.6 Examples of the Areas method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.7 Identification of the Blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.8 Balance arm step test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
6.9 Random signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
6.10 A 5-element binary shift register to generate a pseudo-random binary sequence. . 245
6.11 Pseudo-random binary sequence generator in S IMULINK . . . . . . . . . . . . . . . 245
6.12 Black box experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6.13 Black box response analysis using a series of sine waves . . . . . . . . . . . . . . . . 247
6.14 Black box response using an input chirp signal. . . . . . . . . . . . . . . . . . . . . . 248
6.15 Black box frequency response analysis using a chirp signal. . . . . . . . . . . . . . . 249
6.16 Flapper response to a chirp signal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6.17 Experimental setup to subject a random input into an unknown plant. The in-
put/output data was collected, processed through Listing 6.2 to give the frequency
response shown in Fig. 6.18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
6.18 The experimental frequency response compared to the true analytical Bode dia-
gram. See the routine in Listing 6.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
6.19 Black box response given a pseudo-random input sequence. . . . . . . . . . . . . . 251
6.20 Black box frequency response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.21 Empirical transfer function estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
6.22 Experimental data from a continuous plant . . . . . . . . . . . . . . . . . . . . . . . 254
6.23 A continuous-time model fitted to input/output data . . . . . . . . . . . . . . . . . 255
6.24 Continuous model identification strategy . . . . . . . . . . . . . . . . . . . . . . . . 257
6.25 Continuous model identification simulation . . . . . . . . . . . . . . . . . . . . . . . 258
6.26 Continuous model identification of the blackbox . . . . . . . . . . . . . . . . . . . . 259
6.27 Identification using Laguerre functions . . . . . . . . . . . . . . . . . . . . . . . . . 261
6.28 A signal flow diagram of an auto-regressive model with exogenous input or ARX
model. Compare this structure with the similar output-error model in Fig. 6.31. . . 262
6.29 A signal flow diagram of a ARMAX model. Note that the only difference between
this, and the ARX model in Fig. 6.28, is the inclusion of the C polynomial filtering
the noise term. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6.30 Discrete time models implemented in S IMULINK . . . . . . . . . . . . . . . . . . . . 264
6.31 A signal flow diagram of an output-error model. Compare this structure with the
similar ARX model in Fig. 6.28. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6.32 A general input/output model structure . . . . . . . . . . . . . . . . . . . . . . . . . 265
6.33 Kaos, Control and regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.34 ARX estimation exhibiting bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
6.35 Offline system identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
6.36 Fitting a polynomial to some (x, y) data . . . . . . . . . . . . . . . . . . . . . . . . . 278
6.37 Using the AIC to establish model structure . . . . . . . . . . . . . . . . . . . . . . . 278
6.38 Using the AIC to establish model structure using separate validation data . . . . . 279
6.39 Identification of deadtime from the step response . . . . . . . . . . . . . . . . . . . 281
6.40 Deadtime estimation at fast sampling . . . . . . . . . . . . . . . . . . . . . . . . . . 281
LIST OF FIGURES xiii

6.41 Deadtime estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

6.42 Blackbox step model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
6.43 Hammerstein and Wiener model structures . . . . . . . . . . . . . . . . . . . . . . . 284
6.44 Ideal RLS parameter estimation. (See also Fig. 6.46(a).) . . . . . . . . . . . . . . . . 289
6.45 Recursive least squares estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
6.46 RLS under S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
6.47 RLS under S IMULINK (version 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
6.48 Estimation of a two parameter plant . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
6.49 Confidence limits for estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
6.50 RLS and an abrupt plant change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
6.51 The ‘memory’ of the forgetting factor . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
6.52 Identification using various forgetting factors . . . . . . . . . . . . . . . . . . . . . . 298
6.53 Covariance wind-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
6.54 Variable forgetting factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
6.55 The MIT rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
6.56 Optimising the adaptation gain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
6.57 Addition of coloured noise to a dynamic process. See also Fig. 6.32. . . . . . . . . . 305
6.58 RLS with coloured noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
6.59 Recursive extended least-squares estimation . . . . . . . . . . . . . . . . . . . . . . 308
6.60 Recursive extended least-squares estimation . . . . . . . . . . . . . . . . . . . . . . 309
6.61 A simplified recursive least squares algorithm . . . . . . . . . . . . . . . . . . . . . 311
6.62 Furnace input/output data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
6.63 Closed loop estimation using Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . 313
6.64 RLS in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

7.1 The structure of an indirect adaptive controller . . . . . . . . . . . . . . . . . . . . . 323

7.2 Varying process gain of a spherical tank . . . . . . . . . . . . . . . . . . . . . . . . . 324
7.3 Simple minimum variance control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
7.4 Simple minimum variance control (zoomed) . . . . . . . . . . . . . . . . . . . . . . 329
7.5 Adaptive pole-placement control structure . . . . . . . . . . . . . . . . . . . . . . . 329
7.6 Adaptive pole-placement control structure with RLS identification . . . . . . . . . 335
7.7 Control of multiple plants with an adapting controller. We desire the same closed
loop response irrespective of the choice of plant. . . . . . . . . . . . . . . . . . . . . 336
7.8 Three open loop plants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.9 Desired closed loop response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.10 Adaptive pole-placement with identification . . . . . . . . . . . . . . . . . . . . . . 338
7.11 Comparing the adaptive pole-placement with the reference trajectory . . . . . . . . 338
7.12 Adaptive pole-placement with identification in S IMULINK . . . . . . . . . . . . . . 339
7.13 Internals of the Adaptive pole-placement design routine . . . . . . . . . . . . . . . 339
7.14 Bursting in adaptive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
7.15 A plant with poorly damped zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
7.16 Adaptive pole-placement with an unstable B . . . . . . . . . . . . . . . . . . . . . . 344
7.17 Migration of poles and zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
7.18 Areas of well-damped poles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
7.19 Adaptive pole-placement of the black-box . . . . . . . . . . . . . . . . . . . . . . . . 347
7.20 Adaptive pole-placement of the black-box . . . . . . . . . . . . . . . . . . . . . . . . 348
7.21 A non-minimum phase plant with an unstable zero which causes an inverse step
response. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
7.22 Moving average control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

8.1 Reconstructing states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366

8.2 Pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
8.3 Pole-placement degradation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
8.4 Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
xiv LIST OF FIGURES

8.5 Simultaneous control and state estimation . . . . . . . . . . . . . . . . . . . . . . . . 375

8.6 Control and estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
8.7 GMC tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
8.8 GMC tuning characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
8.9 Linear GMC response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
8.10 Linear GMC comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
8.11 GMC CSTR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
8.12 A CSTR phase plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
8.13 The configuration of an input/output feedback linearisation control law . . . . . . 388
8.14 Exact feedback linearisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392

9.1 IAE areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

9.2 ITAE breakdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
9.3 Optimal responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
9.4 Optimal PID tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
9.5 Optimum PI tuning of the blackbox plant . . . . . . . . . . . . . . . . . . . . . . . . 402
9.6 S IMULINK model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
9.7 Production of a valuable chemical in a batch reactor. . . . . . . . . . . . . . . . . . . 404
9.8 Temperature profile optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
9.9 Temperature profile optimisation using 3 temperatures . . . . . . . . . . . . . . . . 406
9.10 Optimum temperature profile comparison for different number of temperatures . 406
9.11 Optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
9.12 Optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
9.13 Optimal control with targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
9.14 Steady-state and time-varying LQR control . . . . . . . . . . . . . . . . . . . . . . . 422
9.15 Steady-state continuous LQR controller . . . . . . . . . . . . . . . . . . . . . . . . . 425
9.16 Comparing discrete and continuous LQR controllers . . . . . . . . . . . . . . . . . . 428
9.17 LQR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
9.18 Pole-placement and LQR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
9.19 Pole-placement and LQR showing the input . . . . . . . . . . . . . . . . . . . . . . 433
9.20 Trial pole locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
9.21 Trial pole-placement performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
9.22 State feedback control system with an integral output state . . . . . . . . . . . . . . 436
9.23 State feedback with integral states . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
9.24 Black box servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
9.25 A state-based estimation and control scheme . . . . . . . . . . . . . . . . . . . . . . 439
9.26 PDF of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
9.27 Correlated noisy x, y data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
9.28 2D ellipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
9.29 A block diagram of a stochastic state-space system . . . . . . . . . . . . . . . . . . . 445
9.30 Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
9.31 A block diagram of a steady-state prediction-type Kalman filter applied to a linear
discrete plant. Compare this with the alternative form in Fig. 9.32. . . . . . . . . . . 453
9.32 A block diagram of a steady-state current estimator-type Kalman filter applied to
a linear discrete plant. Compare with the alternative prediction form in Fig. 9.31. . 454
9.33 Kalman filter demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
9.34 The performance of a Kalman filter for different q/r ratios . . . . . . . . . . . . . . 460
9.35 A random walk process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
9.36 LQG in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

10.1 Horizons used in model predictive control . . . . . . . . . . . . . . . . . . . . . . . 470

10.2 Predictions of the Reserve Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
10.3 Inverse plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
10.4 Predictive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
LIST OF FIGURES xv

10.5 Acausal response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475

10.6 Varying the horizons of predictive control . . . . . . . . . . . . . . . . . . . . . . . . 476
10.7 MPC on the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
10.8 Step response coefficients, gi , for a stable system. . . . . . . . . . . . . . . . . . . . . 477
10.9 DMC control details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
10.10DMC control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
10.11Adaptive DMC of the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
10.12An MPC graphical user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.13Multivariable MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
10.14S IMULINK and MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
10.15MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
10.16A 3 degree of freedom helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486
10.17MPC control of a helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
10.18MPC control of a helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
10.19LP constraint matrix dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
10.20LP optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
10.21LP optimal control with active constraints . . . . . . . . . . . . . . . . . . . . . . . . 496
10.22Non-square LP optimal control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
10.23LP optimal control showing acausal behaviour . . . . . . . . . . . . . . . . . . . . . 497

11.1 Possible neuron activation functions. . . . . . . . . . . . . . . . . . . . . . . . . . . 507

11.2 A single neural processing unit with multiple inputs . . . . . . . . . . . . . . . . . . 508
11.3 Single layer feedforward neural network . . . . . . . . . . . . . . . . . . . . . . . . 509
11.4 A 3 layer fully interconnected feedforward neural network . . . . . . . . . . . . . . 509
11.5 Single layer network with feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
11.6 An unknown input/output function . . . . . . . . . . . . . . . . . . . . . . . . . . . 511
11.7 Fitting a Neural-Network to an unknown input/output function . . . . . . . . . . 513
11.8 Tide predictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
xvi LIST OF FIGURES
L IST OF TABLES

1.1 Computer aids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1 Final and initial value theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.2 Inverting a z transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3 Laplace transform pairs used for testing . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1 Standard nomenclature used in modelling dynamic systems . . . . . . . . . . . . . 90

3.2 Parameters of the CSTR model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.3 The important variables in the forced circulation evaporator . . . . . . . . . . . . . 94
3.4 Compressed water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.5 The parameter values for the CSTR model . . . . . . . . . . . . . . . . . . . . . . . . 122
3.6 The initial state and manipulated variables for the CSTR simulation . . . . . . . . . 123

4.1 Alternative PID tuning parameter conventions . . . . . . . . . . . . . . . . . . . . . 132

4.2 Ziegler-Nichols open-loop PID tuning rules . . . . . . . . . . . . . . . . . . . . . . . 152
4.3 PID controller settings based on IMC for a small selection of common plants where
the control engineer gets to chose a desired closed loop time constant, τc . . . . . . . 154
4.4 Various alternative ‘Ziegler-Nichols’ type PID tuning rules as a function of the
ultimate gain, Ku , and ultimate period, Pu . . . . . . . . . . . . . . . . . . . . . . . . 155
4.5 Closed-loop single-test PID design rules . . . . . . . . . . . . . . . . . . . . . . . . . 161
4.6 Relay based PID tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

5.1 Filter transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

6.1 Experimentally determined frequency response of the blackbox . . . . . . . . . . . 247

6.2 Identification in state-space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

8.1 The relationship between regulation and estimation . . . . . . . . . . . . . . . . . . 374

8.2 Litchfield nonlinear CSTR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

9.1 Common integral performance indices . . . . . . . . . . . . . . . . . . . . . . . . . . 399

11.1 Comparing expert systems and neural networks . . . . . . . . . . . . . . . . . . . . 506

xvii
xviii LIST OF TABLES
L ISTINGS

2.1 Symbolic Laplace to z-transform conversion . . . . . . . . . . . . . . . . . . . . . . 37

2.2 Symbolic Laplace to z-transform conversion with ZOH . . . . . . . . . . . . . . . . 37
2.3 Extracting the gain, time constants and numerator time constants from an arbitrary
transfer function format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.4 Submarine simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.5 Example of the Routh array using the symbolic toolbox . . . . . . . . . . . . . . . . 71
2.6 Solve the continuous matrix Lyapunov equation using Kronecker products . . . . 77
2.7 Solve the matrix Lyapunov equation using the lyap routine . . . . . . . . . . . . . 78
2.8 Using Lyapunov to establish stability of a linear system . . . . . . . . . . . . . . . . 79
2.9 Solve the discrete matrix Lyapunov equation using Kronecker products . . . . . . 79
3.1 Computing the dynamic relative gain array analytically . . . . . . . . . . . . . . . . 105
3.2 Computing the dynamic relative gain array numerically as a function of ω. See
also Listing 3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.3 Curve fitting using polynomial least-squares . . . . . . . . . . . . . . . . . . . . . . 109
3.4 Polynomial least-squares using singular value decomposition. This routine fol-
lows from, and provides an alternative to Listing 3.3. . . . . . . . . . . . . . . . . . 111
3.5 Curve fitting using a generic nonlinear optimiser . . . . . . . . . . . . . . . . . . . . 113
3.6 Curve fitting using the O PTI optimisation toolbox. (Compare with Listing 3.5.) . . 113
3.7 Fitting water density as a function of temperature and pressure . . . . . . . . . . . 115
3.8 Parameter confidence limits for a nonlinear reaction rate model . . . . . . . . . . . 117
3.9 Comparing the dynamic response of a pendulum to the linear approximation . . . 120
3.10 Using linmod to linearise an arbitrary S IMULINK module. . . . . . . . . . . . . . . 127
4.1 Constructing a transfer function of a PID controller . . . . . . . . . . . . . . . . . . 133
4.2 Constructing a discrete (filtered) PID controller . . . . . . . . . . . . . . . . . . . . . 145
4.3 A simple PID controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
¿ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.4 Ziegler-Nichols PID tuning rules for an arbitrary transfer function . . . . . . . . . . 159
4.5 Identifies the characteristic points for the Yuwana-Seborg PID tuner from a trial
closed loop response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
4.6 Compute the closed loop model from peak and trough data . . . . . . . . . . . . . 165
4.7 Compute the ultimate gain and frequency from the closed loop model parameters. 165
4.8 Compute the open loop model, Gm , Eqn. 4.31. . . . . . . . . . . . . . . . . . . . . . 165
4.9 Compute appropriate PI or PID tuning constants based on a plant model, Gm ,
using the IMC schemes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
4.10 Calculates the period and amplitude of a sinusoidal time series using least-squares. 174
5.1 Designing Butterworth Filters using Eqn. 5.4. . . . . . . . . . . . . . . . . . . . . . . 203
5.2 Designing a low-pass Butterworth filter with a cut-off frequency of fc = 800 Hz. . 203
5.3 Designing a high-pass Butterworth filter with a cut-off frequency of fc = 800 Hz. . 203
5.4 Designing Chebyshev Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
5.5 Computing a Chebyshev Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.6 Converting a 7th-order Butterworth filter to 4 second-order sections . . . . . . . . . 218
5.7 Comparing DFII and SOS digital filters in single precision. . . . . . . . . . . . . . . 219
5.8 Routine to compute the power spectral density plot of a time series . . . . . . . . . 226

xix
xx LISTINGS

5.9 Smoothing and differentiating a noisy signal . . . . . . . . . . . . . . . . . . . . . . 232

6.1 Identification of a first-order plant with deadtime from an openloop step response
using the Areas method from Algorithm 6.1. . . . . . . . . . . . . . . . . . . . . . . 241
6.2 Frequency response identification of an unknown plant directly from input/out-
put data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
6.3 Non-parametric frequency response identification using etfe. . . . . . . . . . . . . 252
6.4 Function to generate output predictions given a trial model and input data. . . . . 255
6.5 Optimising the model parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.6 Validating the fitted model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.7 Continuous model identification of a non-minimum phase system . . . . . . . . . . 257
6.8 Generate some input/output data for model identification . . . . . . . . . . . . . . 269
6.9 Estimate an ARX model from an input/output data series using least-squares . . . 270
6.10 An alternative way to construct the data matrix for ARX estimation using Toeplitz
matrices. See also Listing 6.9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
6.11 Offline system identification using arx from the System Identification Toolbox . . 272
6.12 Offline system identification with no model/plant mismatch . . . . . . . . . . . . . 272
6.13 Demonstrate the fitting of an AR model. . . . . . . . . . . . . . . . . . . . . . . . . . 273
6.14 Create an input/output sequence from an output-error plant. . . . . . . . . . . . . 274
6.15 Parameter identification of an output error process using oe and arx. . . . . . . . 274
6.16 A basic recursive least-squares (RLS) update (without forgetting factor) . . . . . . . 287
6.17 Tests the RLS identification scheme using Listing 6.16. . . . . . . . . . . . . . . . . . 290
6.18 A recursive least-squares (RLS) update with a forgetting factor. (See also List-
ing 6.16.) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
6.19 Adaption of the plant gain using steepest descent . . . . . . . . . . . . . . . . . . . 303
6.20 Create an ARMAX process and generate some input/output data suitable for sub-
sequent identification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
6.21 Identify an ARMAX process from the data generated in Listing 6.20. . . . . . . . . 308
6.22 Recursively identify an ARMAX process. . . . . . . . . . . . . . . . . . . . . . . . . 308
6.23 Kaczmarz’s algorithm for identification . . . . . . . . . . . . . . . . . . . . . . . . . 310
7.1 Simple minimum variance control where the plant has no time delay . . . . . . . . 327
7.2 A Diophantine routine to solve F A + BG = T for the polynomials F and G. . . . . 332
7.3 Alternative Diophantine routine to solve F A + BG = T for the polynomials F and
G. Compare with Listing 7.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
7.4 Constructing polynomials for the Diophantine equation example . . . . . . . . . . 333
7.5 Solving the Diophantine equation using polynomials generated from Listing 7.4. . 334
7.6 Adaptive pole-placement control with 3 different plants . . . . . . . . . . . . . . . . 337
7.7 The pole-placement control law when H = 1/B . . . . . . . . . . . . . . . . . . . . 341
7.8 Factorising an arbitrary polynomial B(q) into stable, B + (q), and unstable and
poorly damped, B − (q), factors such that B = B + B − and B + is defined as monic. . 345
7.9 Minimum variance control design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
8.1 A simple state reconstructor following Algorithm 8.1. . . . . . . . . . . . . . . . . . 365
8.2 Pole-placement control of a well-behaved system . . . . . . . . . . . . . . . . . . . . 369
8.3 A deadbeat controller simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
8.4 Pole placement for controllers and estimators . . . . . . . . . . . . . . . . . . . . . . 375
8.5 GMC on a Linear Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
8.6 GMC for a batch reactor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
8.7 The dynamic equations of a batch reactor . . . . . . . . . . . . . . . . . . . . . . . . 384
8.8 Find the Lie derivative for a symbolic system . . . . . . . . . . . . . . . . . . . . . . 390
8.9 Establish relative degree, r (ignore degree 0 possibility) . . . . . . . . . . . . . . . . 390
8.10 Design Butterworth filter of order r. . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
8.11 Symbolically create the closed loop expression . . . . . . . . . . . . . . . . . . . . . 391
9.1 Returns the IAE performance for a given tuning. . . . . . . . . . . . . . . . . . . . . 400
9.2 Optimal tuning of a PID controller for a non-minimum phase plant. This script file
uses the objective function given in Listing 9.1. . . . . . . . . . . . . . . . . . . . . . 400
LISTINGS xxi

9.3 Returns the ITSE using a S IMULINK model. . . . . . . . . . . . . . . . . . . . . . . . 403

9.4 Analytically computing the co-state dynamics and optimum input trajectory as a
function of states and co-states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
9.5 Solving the reaction profile boundary value problem using the boundary value
problem solver, bvp4c.m. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
9.6 Computes the full time-evolving LQR solution . . . . . . . . . . . . . . . . . . . . . 421
9.7 The continuous time differential Riccati equation. This routine is called from List-
ing 9.8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
9.8 Solves the continuous time differential Riccati equation using a numerical ODE
integrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
9.9 Calculate the continuous optimal steady-state controller gain. . . . . . . . . . . . . 424
9.10 Closed loop simulation using an optimal steady-state controller gain. . . . . . . . . 424
9.11 Solving the algebraic Riccati equation for P∞ using Kronecker products and vec-
torisation given matrices A, B, Q and R. . . . . . . . . . . . . . . . . . . . . . . . . 425
9.12 Calculate the discrete optimal steady-state gain by ‘iterating until exhaustion’.
Note it is preferable for numerical reasons to use lqr for this computation. . . . . 427
9.13 Comparing the continuous and discrete LQR controllers. . . . . . . . . . . . . . . . 427
9.14 An LQR controller for the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
9.15 Comparing an LQR controller from Listing 9.14 with a pole-placement controller . 432
9.16 Computing the closed loop poles from the optimal LQR controller from Listing 9.14.434
9.17 Comparing the actual normally distributed random numbers with the theoretical
probability density function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
9.18 Probability and inverse probability distributions for the F -distribution. . . . . . . . 442
9.19 Generate some correlated random data. . . . . . . . . . . . . . . . . . . . . . . . . . 442
9.20 Plot a 3D histogram of the random data from Listing 9.19. . . . . . . . . . . . . . . 443
9.21 Compute the uncertainty regions from the random data from Listing 9.20. . . . . . 444
9.22 Validating the uncertainty regions computed theoretically from Listing 9.21. . . . . 444
9.23 Solving the discrete time Riccati equation using exhaustive iteration around Eqn. 9.98
or alternatively using the dare routine. . . . . . . . . . . . . . . . . . . . . . . . . . 451
9.24 Alternative ways to compute the Kalman gain . . . . . . . . . . . . . . . . . . . . . 453
9.25 State estimation of a randomly generated discrete model using a Kalman filter. . . 456
9.26 Computing the Kalman gain using dlqe. . . . . . . . . . . . . . . . . . . . . . . . . 457
9.27 Demonstrating the optimality of the Kalman filter. . . . . . . . . . . . . . . . . . . . 458
9.28 Potter’s algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
10.1 Predictive control with input saturation constraints using a generic nonlinear op-
timiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
10.2 Objective function to be minimised for the predictive control algorithm with input
saturation constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
10.3 Dynamic Matric Control (DMC) control . . . . . . . . . . . . . . . . . . . . . . . . . 479
10.4 Setting up an MPC controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
10.5 Nonlinear dynamics of a helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486
10.6 Optimal control using linear programming . . . . . . . . . . . . . . . . . . . . . . . 494
11.1 Generate some arbitrary data to be used for subsequent fitting . . . . . . . . . . . . 512
B.1 Polynomial addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
B.2 Multiple convolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
B.3 Strip leading zeros from a polynomial. . . . . . . . . . . . . . . . . . . . . . . . . . . 520
xxii LISTINGS
C HAPTER 1

I NTRODUCTION

Mathematicians may flatter themselves that they posses new ideas which mere human language is as yet
unable to express. Let them make the effort to express those ideas in appropriate words without the aid of
symbols, and if they succeed, they will not only lay us laymen under a lasting obligation but, we venture to
say, they will find themselves very much enlightened during the process, and will even be doubtful
whether the ideas expressed as symbols had ever quite found their way out of the equations into their
minds.
James Clerk Maxwell, 1890

Control, in an engineering sense, is where actions are taken to ensure that a particular physi-
cal process responds in some desired manner. Automatic control is where we have relieved the
human operator from the tedium of consistently monitoring the process and supplying the nec-
essary corrections. Control as a technical discipline is therefore important not only in the fields
of engineering, but also in economics, sociology and indeed in most aspects of our life. When
studying control, we naturally assume that we do conceivably have some chance of influencing
things. For example, it is worthwhile to study the operation of a coal fired power plant in order
to minimise possibly polluting emissions, but it is not worth our time to save the world from the
next ice age, or as the results of a special study group who investigated methods designed to pro-
tect the world from a stray comet (such as the one postulated to have wiped out the dinosaurs 80
million years ago) concluded, there was nothing feasible we could do, such as change the earth’s
orbit, or blast the asteroid, to avoid the collision. In these latter examples, the problem exists, but
our influence is negligible.

The teaching of control has changed in emphasis over the last decade from that of linear single-
input/single-output systems elegantly described in the Laplace domain, to general nonlinear
multiple-input/multiple-output systems best analysed in the state space domain. This change
has been motivated by the increasing demands by industry and public to produce more, faster or
cleaner and is now much more attractive due to the impressive improvements in computer aided
tools, such as M ATLAB used in these notes. This new emphasis is called advanced (or modern)
control as opposed to the traditional or classical control. This set of notes is intended for students
who have previously attended a first course in automatic control covering the usual continuous
control concepts like Laplace transforms, Bode diagrams, stability of linear differential equations,
PID controllers, and perhaps some exposure to discrete time control topics like z transforms and
state space.

This book attempts to describe what advanced control is, and how it is applied in engineering
applications with emphasis on the construction of controllers using computer aided design tools
such as the numerical programming environment M ATLAB from the MathWorks, [136]. With

1
2 CHAPTER 1. INTRODUCTION

this tool, we can concentrate on the intentions behind the design procedures, rather than the
mechanics to follow them.

1.1 S COPE

Part one contains some revision material in z-transforms, modelling and PID controller tuning.
The discrete domain, z-transforms, and stability concepts with a brief discussion of appropriate
numerical methods are introduced in chapter 2. A brief potpourri of modelling is summarised
in chapter 3. Chapter 4 is devoted to the most common industrial controller, the three term PID
controller with emphasis on tuning, implementation and limitations. Some basic concepts from
signal processing such as filtering and smoothing are introduced in chapter 5. Identification and
the closely related adaptive control are together in chapters 6 and 7. State space analysis and
optimal control design are given in the larger chapters 8 and 9.

N OTATION CONVENTIONS

Throughout these notes I have used some typographical conventions. In mathematical expres-
sions, scalar variables are written in italic such as a, b, c, or if Greek, γ, ϕ while vectors x, y are
upright bold lower case and matrices, A, ∆ are bold upper case. More notation is introduced as
required.

Computer commands and output and listings are given in a fixed-width font as A=chol(B*B’).
In some cases where you are to type an interactive command, the M ATLAB prompt >> is given,
and the computed solution returned. If no ambiguity exists such as in the case for functions, the
prompt is omitted.

1.2 M ATLAB FOR COMPUTER AIDED CONTROL DESIGN

Modern control design has heavy computing requirements. In particular one needs to:

1. manipulate symbolic algebraic expressions, and

2. perform intensive numerical calculations and simulations for proto-typing and testing quickly
and reliably, and finally

3. to implement the controller at high speed in special hardware such as an embedded con-
troller or a digital signal processing (DSP) chip perhaps using assembler.

To use this new theory, it is essential to use computer aided design (CAD) tools efficiently as
real-world problems can rarely be solved manually. But as [172] point out, ‘the use of computers
in the design of control systems has a long and fairly distinguished history’. This book uses
M ATLAB for the design, simulation and prototyping of controllers.

M ATLAB, (which is short for MATrix LABoratory), is a programming environment that grew
out of an effort to create an easy user-interface to the very popular and well regarded public
domain F ORTRAN linear algebra collection of programmes, L INPACK and E ISPACK. With this
direct interpretive interface, one can write quite sophisticated algorithms in a very high level
language, that are consequently easy to read and maintain. M ATLAB is supported with a variety
1.2. MATLAB FOR COMPUTER AIDED CONTROL DESIGN 3

of toolboxes comprising of collections of source code subroutines organised in areas of specific

interest. The toolboxes we are most interested in, and used in this book are:

Control toolbox containing functions for controller design, frequency domain analysis, conver-
sions between various models forms, pole placement, optimal control etc. (Used through-
out)
Symbolic toolbox which contains a gateway to the symbolic capabilities of M APLE.
Signal processing toolbox containing filters, wave form generation and spectral analysis. (Used
principally in chapter 5.)
System identification toolbox for identifying the parameters of various dynamic model types.
(Used in chapter 6.) You may also find the following free statistics toolbox useful available
at: www.maths.lth.se/matstat/stixbox/.
Optimisation which contains routines to maximise or minimise algebraic or dynamic functions.
One suitable free toolbox is available from www.i2c2.aut.ac.nz/Wiki/OPTI/. This
toolbox is used primarily in chapters 9 and 10.
Real-time toolbox can be used to interface M ATLAB to various analogue to digital converters. A
free version suitable with Measurement Computing’s MCC DAQs is available from www.i2c2.aut.ac.nz/R

Additional documentation to that supplied with M ATLAB is the concise and free summary notes
[186] or the more recent [69]. Recently there has been exponential growth of other texts that heav-
ily use M ATLAB (such as this one), and a current list is available from the Mathwork’s anonymous
ftp server at www.mathworks.com. This server also contains many user contributed codes, as
well as updates, bug fixes etc.

If M ATLAB, or even programming a high level language is new to you, then [205] is a cheap
recommended compendium, similar in form to this, covering topics in numerical analysis, again
with many M ATLAB examples.

1.2.1 A LTERNATIVE COMPUTER DESIGN AIDS

Table 1.1 lists a number of alternatives computer-aided design and modelling environments sim-
ilar and complimentary to M ATLAB.

Product WWW site comment

S CI L AB www.scilab.org Free Matlab/Simulink clone
O CTAVE www.octave.org Free Matlab clone, inactive
RL AB rlabplus.sourceforge.net Matlab clone, Linux
V ISUAL M ODEL Q www.qxdesign.com shareware Simulink clone
M ATH V IEWS www.mathwizards.com shareware
M U PAD www.mupad.de Interfaces with S CI L AB
M APLE www.maplesoft.com commercial CAS
M ATHEMATICA www.mathematica.com commercial CAS

Table 1.1: Shareware or freeware Matlab lookalikes and computer algebra systems

Unlike M ATLAB, symbolic manipulators are computer programs that by manipulating symbols
can perform algebra. Such programs are alternatively known as computer algebra systems or
4 CHAPTER 1. INTRODUCTION

CAS. The most well known examples are M ATHEMATICA, M APLE, M U PAD, and M ACSYMA,
(see Table 1.1). These programs can find analytical solutions to many mathematical problems
involving integrals, limits, special functions and so forth. They are particularly useful in the
controller design stage.

The Numerics in Control1 group in Europe have collect together a freeware F ORTRAN subroutine
library S LICOT for routines relevant in systems and control.

Problem 1.1 1. Familiarise yourself with the fundamentals of M ATLAB. Run the M ATLAB
demo by typing demo once inside M ATLAB.

2. Try the M ATLAB tutorial (part 1).

3. Read through the M ATLAB primer, [186] or [69], and you should get acquainted with the
M ATLAB users manual.

1.3 E CONOMICS OF CONTROL

Most people would agree that Engineers apply technology, but what do these two words really
mean? Technology is derived from two Greek words, techne which means skill or art, and logia
which means science or study. The interesting point here is that the art component is included.
The English language unfortunately confuses the word engine with engineering so that many peo-
ple have the mistaken view that engineers drive engines (mostly). Actually engineer is derived
from the Latin ingeniatorium which means one who is ingenious at devising. A far cry from the
relatively simple act of piloting jumbos. An interesting American perspective of the professional
engineer and modern technology is given as light reading in [2] and Florman’s The Existential
Pleasures of Engineering, [67].

Chemical engineering is, succinctly put, chemistry constrained by cost. The chemist wants the
reaction to proceed, the chemical engineer takes that for granted, but is interested in increasing
the rate, or pushing the equilibrium, or in most cases both at the same time. As the incentive to
produce better products increases, accompanied by an awareness of potent global competition
driving one to reduce costs, process control becomes an important aspect of engineering.

Obviously modern computerised process control systems are expensive. They are especially ex-
pensive compared with other computers such as office or financial computers because the market
is smaller, the environment harsher, the graphical requirements more critical, the duties more var-
ied, and the potential payoffs larger. Process control has at least two main duties to perform; first
to ensure that the plant is operated safely (that is protect the plant, environment and people), and
second that the product quality is consistent with some customer or regulatory body demanded
specifications. There is always a trade off between how much control you apply and the bene-
fits that result. An automobile manufacturer could produce an almost totally indestructible car
(ie, tank), but the expense of raw materials required, and the high running costs would certainly
deem the project an economic failure.

On the other hand, in 1965 Ralph Nader complained in the aptly named Unsafe at Any Speed
about the poor quality of American automotive engineering, the lack of controls and the unsafe
result. This influential book challenged the balance from between commercial profits and more
quality control. A product with less variation in quality may be worth more than a product that
has a higher average quality, but more variation. A potentially dangerous production facility that
regularly destroys equipment, people or surroundings is not usually tolerated by the licensing
authorities.
1 The home page is located at http://www.win.tue.nl/niconet/niconet.html
1.3. ECONOMICS OF CONTROL 5

Safety concerns motivate

better control.

Fig. 1.1, adapted from [6], gives an industrial perspective of the status of process control in 1994.
The techniques are divided into those considered “classical” or traditional, which demand only
modest digital online computing power, (if any), little in the way of explicit process models or
understanding, and those termed loosely “advanced”.

Advanced control

Optimisation Direct search Rule definition Online simulation

Constraint Steady state dynamic methods single variable multivariable

Regulatory Signal condition PID algorithm Deadtime feedforward control

compensation

Basic analogue control PLCs DCS Process computer

Field Valves Transmitters smart transmitters Onstream analysers

Traditional control

Figure 1.1: A comparison of traditional vs. advanced process control techniques. Adapted from
[6].

One of the major concerns for the process control engineer is to reduce the output variance. If
the variation about the setpoint is small, then the setpoint can be shifted closer to the operating
constraint, without increasing the frequency of alarms. Fig. 1.2 demonstrates the ideal case that
while popular in the advertising literature, is harder to achieve unambiguously in practice.

Many text books in the control field are very vague about the actual configuration of real pro-
cess control systems used today. Other books that take a more trade oriented approach, are
vague about the academic side of the control performance. There are a number of reasons for
this. First many texts try hard to describe only the theoretical aspects of digital control, and any-
thing remotely applied is not considered worthy of their attention. Secondly, the control systems
are rapidly changing as the cost of micro-processors drop in price, and different programming
methods come into flavour. Thirdly many industries are deliberately vague about publishing
the details of their control system since they perceive that this information could help their com-
petitors. However some information of this type is given in [64, pp131-149] and [17]. One good
6 CHAPTER 1. INTRODUCTION

upper quality constraint

violations

spt #3
loss ($) spt #2

setpoint #1

process output advanced control

regulatory

manual control

time

Figure 1.2: Economic improvements owing to better control. If the control scheme can reduce
the variance, the setpoint can be shifted closer to the operating or quality constraint, thereby
decreasing operating costs.

balance for the practitioner is [145].

1.4 L ABORATORY EQUIPMENT FOR CONTROL TESTS

Obviously if we are to study automatic control with the aim to control eventually chemical plants,
manufacturing processes, robots, undertake filtering to do active noise cancellation and so forth,
we should practice, preferably on simpler, more well understood, and potentially less hazardous
equipment.

In the Automatic Control Laboratory in the Department of Electrical Engineering at the Karlstad
University, Sweden we have a number of simple bench-scale plants to test identification and
control algorithms on.

1.4.1 P LANTS WITH ONE INPUT AND ONE OUTPUT

T HE BLACKBOX

Fig. 1.3 and Fig. 1.4(a) shows what we perhaps unimaginatively refer to as a “black-box’. It is a
box, and it is coloured black. Subjecting the box to an input voltage from 0 to 5 volts delivers an
output voltage also spanning from around 0 to 5 volts, but lagging behind the input voltage since
the internals of the blackbox are simply either 7 or 9 (depending on the switch position) low-pass
passive filters cascaded together.

The blackbox is a relatively well behaved underdamped stable system with dominant time con-
stants of around 5 to 10 seconds. Fig. 1.4(b) shows the response of the blackbox to two input
steps. The chief disadvantage of using this device for control studies is that the output response
1.4. LABORATORY EQUIPMENT FOR CONTROL TESTS 7

Input Output
From computer
To computer
D/A ✲ B LACK -B OX Sluggish ✲ A/D
(connector #10) (connector #1)

✲
Fast
GND ✲ input indicator blue ✲ Earth
blue
(connector #11)

Figure 1.3: Blackbox configuration. The manual switch marked will toggle between either 7 or 9
low-pass filters.

Blackbox step response

0.45

0.4

0.35

0.3
ipnut/output

0.25

0.2

0.15 input
output
0.1

0.05

−0.05
15 20 25 30 35 40 45 50
time (s)

(a) Black-box wiring to the National Instruments (b) The response of the blackbox to 2 step inputs
LabPC terminator.

Figure 1.4: The “Black-box”

is not ‘visible’ to the naked eye, and that we cannot manually introduce disturbances. One com-
plication you can do is to cascade two blackboxes together to modify the dynamics.

E LECTRO - MAGNETIC BALANCE ARM

The electromagnetic balance arm shown in Fig. 1.5(a) is a fast-acting, highly oscillatory, plant
with little noise. The aim is to accurately weigh small samples by measuring the current required
to keep the balance arm level, or alternatively just to position the arm at different angles. The
output response to a step in input shown in Fig. 1.5(b) indicates how long it would take for the
oscillations to die away.
8 CHAPTER 1. INTRODUCTION

Step response of the balance arm

1

0.8

0.6

arm position
0.4

0.2

−0.2

−0.4
20 30 40 50 60 70 80 90 100

0.3

0.2

0.1
input

−0.1

−0.2

20 30 40 50 60 70 80 90 100
time ∆T=0.05 seconds

(a) The electromagnetic balance arm (b) The response of the arm to a step changes in input.

Figure 1.5: The electromagnetic balance arm

F LAPPER

Contrary to the balance arm, the flapper in Fig. 1.6 and Fig. 1.7(b) has few dynamics, but signif-
icant low-pass filtered measurement noise. An interesting exercise is to place two flapper units
in close proximity. The air from one then disturbs the flapper from the other which makes an
interacting multivariable plant.

S TEPPER MOTORS

A stepper motor is an example of a totally discrete system.

1.4.2 M ULTI - INPUT AND MULTI - OUTPUT PLANTS

It is possible to construct multivariable interacting plants by physically locating two plants close
to each other. One possibility is to locate two flappers adjacent to each other, another possibility
is one flapper and one balance arm. The extent of the interaction can be varied by adjusting the
relative position of the two plants.

H ELICOPTER

The model helicopter, Fig. 1.8, is an example of a highly unstable, multivariable (3 inputs, 2
outputs) nonlinear, strongly interacting plant. It is a good example of where we must apply
control (or crash and burn). Fig. 1.9(b) shows the controlled response using 2 PID controllers to
control the direction and altitude. Fig. 1.10(a) shows a 3 dimensional view of the desired and
actual flight path. Section 10.2.4 illustrates the control of a different three degree of freedom
helicopter.
1.4. LABORATORY EQUIPMENT FOR CONTROL TESTS 9

position transducer
✙

❖
✲ angle
✲ Flapper arm
Motor ✲
✲
✻ ✻
❄

✻ ✻
power to the motor

auto/man
✌ ◆

✻ ✻ switch On/Off
manual

❄ ❄
Digital to Analogue
A/D
(Fan speed)

Figure 1.6: The flapper wiring

Step response of the flapper

0.35

0.3

0.25
output

0.2

0.15

0.1
0 5 10 15 20 25 30 35 40 45 50

0.4

0.3
input

0.2

0.1

−0.1
0 5 10 15 20 25 30 35 40 45 50
time (sec) dt=0.05

(a) The fan/flapper equipment (b) Step response of the flapper

Figure 1.7: The Fan and flapper.

 10 CHAPTER 1. INTRODUCTION

control inputs
z }| {
top rotor

side rotor

✲ ✶ moveable counter weight

 ✙
 azimuth
plant outputs
 ✲

 ✮
elevation angle
support stand

Figure 1.8: Helicopter plant with 2 degrees of freedom. See also Fig. 1.9(a).

1.5 S LOWING DOWN S IMULINK

For some applications like the development of PID controllers you want to be able to slow down
the S IMULINK simulation to have time to manually introduce step changes, add disturbances,
switch from automatic to manual etc. If left alone in simulation mode, S IMULINK will run as fast
as possible, but it will slow down when it needs to sub-sample the integrator around discontinu-
ities or periods when the system is very stiff.

The S IMULINK Execution Control block allows you to specify that the simulation runs at a multi-
ple of real-time. This is most useful when you want to slow down a simulation, or ensure that it
runs at a constant rate.

The block is available from: http://www.mathworks.com/matlabcentral/fileexchange

The implementation is shown in Fig. 1.11 where the Simulation Execution Control block looks
like an A/D card but in fact is not connected to anything, although it does export some diagnostic
timing information.

An alternative method to slow down a Simulink simulation and force it to run at some set rate is
to use the commercial Humusoft real-time toolbox, but not with the A/D card actually interfaced
to anything.
1.5. SLOWING DOWN SIMULINK 11

(a) The ‘flying’ helicopter balanced using 2 PID controllers since it is openloop unstable and would otherwise crash.

Helicopter
1.5

0.5
output

−0.5

−1

−1.5
0 10 20 30 40 50 60 70 80 90 100

0.5
input

−0.5

−1
0 10 20 30 40 50 60 70 80 90 100
Time (sec)
(b) Multivariable PID control of the helicopter exhibiting mediocre controlled response and severe
derivative kick.

Figure 1.9: Multivariable PID control of an unstable helicopter

12 CHAPTER 1. INTRODUCTION

Helicopter flight path

0.5

0
Up/Down

−0.5

−1
1.5
1
0.5
0 250
200
−0.5
150
−1 100
50
East/West −1.5 0
time

(a) Multivariable helicopter control (b) Model helicopter in the trees

Figure 1.10: Helicopter flying results

Scope1
Simulink
Execution
Control

1
s+1
Signal Transfer Fcn Scope
Generator

Figure 1.11: Real-time Simulink simulations. A parameter in the S IMULINK E XECUTION C ON -

TROL block sets the speed of the simulation.
C HAPTER 2

F ROM DIFFERENTIAL TO DIFFERENCE

EQUATIONS

Excerpt from What a sorry state of affairs, Martin Walker, Guardian Weekly, June 29, 1997.

. . . But then he said something sensible, as he quite often does. . . .

“We need to treat individuals as individuals and we need to address discrete problems for what they are,
and not presume them to be part of some intractable racial issue.” Gingrich, properly understood, is a
national treasure, and not only because he is one of the few Americans who understand the difference
between “discreet” and “discrete”.

2.1 C OMPUTER IN THE LOOP

The cost and flexibility advantages of implementing a control scheme in software rather than
fabricating it in discrete components today are simply too large to ignore. However by inserting a
computer to run the software necessitates that we work with discrete regularly sampled signals.
This added complexity, which by the way is more than compensated by the above mentioned
advantages, introduces a whole new control discipline, that of discrete control.

Fig. 2.1 shows a common configuration of the computer in the loop. For the computer to respond
to any outside events, the signals must first be converted from an analogue form to a digital
signal, say 1 to 5 Volts, which can, with suitable processing be wired to an input port of the
computer’s processor. This device that accomplishes this conversion is called an Analogue to
Digital converter or A/D. Similarly, any binary output from the pins on a processor must first be
converted to an analogue signal using a Digital to Analogue converter, or D/A. In some micro-
controllers, (rather than micro-processors), such as Intel’s 8048 or some versions of the 8051, these
converters may be implemented on the microcontroller chip.

Digital to analogue conversion is easy and cheap. One simply loads each bit across different re-
sistors, and sums the resultant voltages. The conversion is essentially instantaneous. Analogue
to digital is not nearly as easy nor cheap, and this is the reason that the common data acquisition
cards you can purchase for your PC will often multiplex the analogue input channel. There are
various schemes for the A/D, one using a D/A inside a loop using a binary search algorithm.
Obviously this conversion is not instantaneous, although this is not normally considered a prob-
lem for process control applications. Any introductory electrical engineering text such as [173]

13
14 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Process control computer

+
✲
r(t) + ✲ A/D ✲ computer ✲ D/A ✲ Plant ✲ y(t)
setpoint ✻
− output

Figure 2.1: The computer in the control loop

will give further details on the implementation details.

2.1.1 S AMPLING AN ANALOGUE SIGNAL

It is the A/D converter that is the most interesting for our analysis of the discrete control loop.
The A/D converter will at periodic intervals defined by the computer’s clock sample the continu-
ous analogue input signal. The value obtained is typically stored in a device called a zeroth-order
hold until it is eventually replaced by the new sample collected one sample time later. Given con-
straints on cost, the A/D converter will only have a limited precision, or limited number of bits
in which to store the incoming discrete value. Common A/D cards such as the PCLabs card,
[3], use 12 bits giving 212 or slightly over 4 thousand discretisation levels. The residual chopped
away is referred to as the quantisation error. For a given number of bits, b, used in the converter,
the amplitude quantisation is
δ = 2−b
Low cost analogue converters may only use 8 bits, while digital audio equipment use between
16 and 18 bits.

Fig. 2.2 shows the steps in sampling an analogue signal with a three bit (8 discrete levels), A/D
sampler. The dashed stair plot gives an accurate representation of the sampled signal, but owing
to the quantisation error, we are left with the solid stair plot. You can reproduce Fig. 2.2 in
M ATLAB using the fix command to do the chopping, and stairs to construct the stair plot.
While other types of hold are possible, anything higher than a first-order hold is rarely used.

Sampling a continuous signal

8
Analogue
7 Sampled
quantisized
6

4
Signal

Figure 2.2: Sampling an analogue signal 2

(heavy solid) with a three bit (8 discrete lev- 1

els) A/D converter and zeroth-order hold. 0

The sampled values, •, are chopped to the −1

next lowest discrete integer level giving the −2

0 2 4 6 8 10 12
sampled and quantisied output Sample time, (k∆T)
2.1. COMPUTER IN THE LOOP 15

2.1.2 S ELECTING A SAMPLE RATE

Once we have decided to implement discrete control, rather than continuous control, we must
decided on a reasonable sampling rate. This is a crucial parameter in discrete control systems.
The sample time, (T or sometimes denoted ∆t ), is measured in time units, say seconds or in
industrial applications, minutes. The reciprocal of the sample time is the sample frequency, f ,
and is usually measured in cycles/second or Hertz. The radial or angular velocity (which some
confusingly also term frequency) is denoted ω and is measured in radians/second. The inter-
relationships between these quantities are
   
1 cycles ω radians/s
f= = (2.1)
T second 2π radians/cycle

The faster the sampling rate, (the smaller the sampling time, T ), the better our discretised signal
approximates the real continuous signal. However, it is uneconomic to sample too fast, as the
computing and memory hardware may become too expensive. When selecting an appropriate
sampling interval, or sample rate, we should consider the following issues:

• The maximum frequency of interest in the signal

• The sampling theorem which specifies a lower limit required on the sampling rate to resolve
any particular frequency unambiguously. (See §2.1.3 following.)

• Any analogue filtering that may be required (to reduce the problem of aliasing)

• The cost of the hardware and the speed of the A/D converters.

Ogata discusses the selection of a sample time qualitatively in [150, p38]. However for most
chemical engineering processes, which are dominated by relatively slow and overdamped pro-
cesses, the sample time should lie somewhere between ten times the computational delay of the
hardware tc and some small fraction of the process dominant time constant τ , say
τ
10tc ≤ T ≤ (2.2)
10
For most chemical engineering applications, the computational delay is negligible compared with
the process time constant, (tc → 0), so we often choose T ≈ τ /10. Thus for a simple first order
rise, we would expect to have about 20–30 data samples from 0 to 99%. Some may argue that
even this sampling rate is too high, and opt for a more conservative (larger) sample time down
to τ /6. Note that commonly used guidelines such as presented in Table 22.1 in [182, p535] span a
wide range of recommended sample times.

O VERLY FAST SAMPLING

Apart from the high cost of fast A/D converters, there is another argument against fast sampling.
When one samples a continuous system, the poles of a stable continuous system map to the poles
of a stable discrete system as T goes to zero. However the zeros in the LHP of a continuous
system may not map to zeros inside the unit disk of a discrete system as T tends to zero. This
nonintuitive result could create problems if one samples a system, and then uses the inverse
of this system within a one step controller, since now the zeros outside the unit circle become
unstable poles inside the controller for small sample times. Note that the inverse continuous
system is stable, and the discrete inverse system will be stable for large sample times, and it will
only be unstable for small sample times.
16 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

2.1.3 T HE SAMPLING THEOREM AND ALIASES

“Dinsdale was a gentleman. And what’s more he knew how to treat a female impersonator”
John Cleese in Monty Python (15-9-1970)

The sampling theorem gives the conditions necessary to ensure that the inverse sampling pro-
cedure is a “one to one” relationship. To demonstrate the potential problems when sampling,
consider the case where we have two sinusoidal signals, but at different frequencies.1
   
7 1
y1 = − sin 2π t and y2 = sin 2π t
8 8

If we sample these two signals relatively rapidly, say at 25Hz (T =40ms) then we can easily see
two distinct sine curves. However if we sample at T = 1, we obtain identical points.

Aliasing
1

1 t = [0:0.04:8]'; 0.5
y1= -sin(2*pi*7/8*t);
signal

y2= sin(2*pi/8*t);
plot(t,[y1, y2]) 0

As shown opposite, we note that y1 −0.5

(solid) and y2 (dashed) just happen to
coincide at t = 0, 1, 2, 3, . . . (•). −1
0 2 4 6 8
time (s)

Consequently, at the slower sampling rate of 1 Hz, we cannot distinguish between the two dif-
ferent signals. This is the phenomenon known as aliasing since one of the frequencies “pretends”
to be another. In conclusion, two specific sinusoids of different frequencies can have identical
sampled signals. Thus in the act of taking samples from a continuous measurement, we have lost
some information. Since we have experienced a problem when we sample too slowly, it is rea-
sonable to ask what the minimum rate is so that no aliasing occurs. This question is answered by
the sampling theorem which states:

To recover a signal from its sample, one must sample at least two times a
period, or alternatively sample at a rate twice the highest frequency of interest
in the signal.

Alternatively, the highest frequency we can unambiguously reconstruct for a given sampling
rate, 1/T , is half of this, or 1/(2T ). This is called the Nyquist frequency, fN .

In the second example above when we sampled at 1 Hz, the sampling radial velocity was ωs =
2π = 6.28 rad/s. This was satisfactory to reconstruct the low frequency signal (f1 = 1/8 Hz) since
2ω1 = 1.58 rad/s. We are sampling faster than this minimum, so we can reconstruct this signal.
However for the faster signal (f2 = 7/8 Hz), we cannot reconstruct this signal since 2ω2 = 11.0
rad/s, which is faster than the sampling radial velocity.

1 This example was adapted from Franklin & Powell p81

2.1. COMPUTER IN THE LOOP 17

2.1.4 D ISCRETE FREQUENCY

If we sample the continuous signal,

x(t) = A cos(ωt)
with a sample time of T ,
x(nT ) = A cos(ωnT ) = A cos(Ωn)
where the digital frequency, Ω, is defined as

def f
Ω = ωT = 2πf T = 2π
fs

then the range of analogue frequencies is 0 < f < ∞ while the range of digital frequencies is
limited by the Nyquist sampling limit, fs /2 giving the allowable range for the digital frequency
as
0≤Ω≤π

M ATLAB unfortunately decided on a slightly different standard in the S IGNAL P ROCESSING tool-
box. Instead of a range from zero to π, M ATLAB uses a range from zero to 2 where 1 corresponds
to half the sampling frequency or the Nyquist frequency. See [42].

In summary:

symbol units
sample time T or ∆t s
sampling frequency fs = T1 Hz
angular velocity ω = 2πf rad/s
digital frequency Ω = ωT = 2πf
fs –

where the allowable ranges are:

0 ≤ω < ∞, continuous
0 ≤Ω ≤ π, sampled

and Nyquist frequency, fN , and the dimensionless Nyquist frequency, ΩN are:

fs 1
fN = = [Hz]
2 2T
2πfN
ΩN = =π
fs

It is practically impossible to avoid aliasing problems when sampling, using only digital filters.
Almost all measured signals are corrupted by noise, and this noise has usually some high or even
infinite frequency components. Thus the noise is not band limited. With this noise, no matter
how fast we sample, we will always have some reflection of a higher frequency component that
appears as an impostor or alias frequency.

If aliasing is still a problem, and you cannot sample at a higher rate, then you can insert a low
pass analogue filter between the measurement and the analogue to digital converter (sampler).
The analogue filter or in this case known as an anti-aliasing filter, will band-limit the signal, but
not corrupt it with any aliasing. Expensive high fidelity audio equipment will still use analogue
filters in this capacity. Analogue and digital filters are discussed in more detail in chapter 5.
18 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Detecting aliases

Consider the trend y(t) in Fig. 2.3 where we wish to estimate the important frequency compo-
nents of the signal. It is evident that y(t) is comprised of one or two dominating harmonics.

3
continuous
Ts=0.7
2
Ts=1.05

output
0

Figure 2.3: Part of a noisy time series with

−1
unknown frequency components. The con-
tinuous underlying signal (solid) is sam- −2
0 1 2 3 4 5 6 7
pled at T = 0.7, (•), and at T = 1.05 s (△). time (s)

The spectral density when sampling at T = 0.7s (• in Fig.2.3) given in the upper trend of Fig. 2.4
exhibits three distinct peaks. These peaks are the principle frequency components of the signal
and are obtained by plotting the absolute value of the Fourier transform of the time signal2 ,
|DF T {y(t)}|. Reading off the peak positions, and for the moment overlooking any potential

2
10 Ts = 0.7 s
power

0
10
fN=0.71

2
10 Ts = 1.05 s
power

0
10
fN=0.48

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

frequency (Hz)

Figure 2.4: The frequency component of a signal sampled at Ts = 0.7s (upper) and Ts = 1.05s
(lower). The Nyquist frequencies for both cases are shown as vertical dashed lines. See also
Fig. 2.5.

problems with undersampling, we would expect y(t) to be something like

y(t) ≈ sin(2π0.1t) + sin(2π0.5t) + sin(2π0.63t)
However in order to construct Fig. 2.4 we had to sample the original time series y(t) possibly
introducing spurious frequency content. The Nyquist frequency fN = 1/(2Ts ) is 0.7143 and is
shown as a vertical dashed line in Fig. 2.4(top). The power spectrum is reflected in this line, but
is not shown in Fig. 2.4.

If we were to re-sample the process at a different frequency and re-plot the power density plot
then the frequencies that were aliased will move in this second plot. The △ points in Fig. 2.3 are
sampled at ∆t = 1.05s with corresponding spectral power plot in Fig. 2.4. The important data
from Fig. 2.4 is repeated below.
2 More about the spectral analysis or the power spectral density of signals comes in chapter 5.
2.2. FINITE DIFFERENCE MODELS 19

Curve Ts (s) fN (Hz) peak 1 peak 2 peak 3

top 0.7 0.7143 0.1 0.50 0.63
bottom 1.05 0.4762 0.1 0.152 0.452

First we note that the low frequency peak (f1 = 0.1Hz) has not shifted from curve a (top) to
curve b (bottom), so we would be reasonably confident that f1 = 0.1Hz and is not corrupted by
the sampling process.

However, the other two peaks have shifted, and this shift must be due to the sampling process.
Let us hypothesize that f2 = 0.5Hz. If this is the case, then it will appear as an alias in curve b
since the Nyquist frequency for curve b (fN (b) = 0.48) is less than the proposed f2 = 0.5, but it
will appear in the correct position on curve a. The apparent frequency fˆ2 (b) on curve b will be

fˆ2 (b) = 2fN (b) − f2 = 2 × 0.4762 − 0.5 = 0.4524

which corresponds to the third peak on curve b. This would seem to indicate that our hypothesis
is correct for f2 . Fig. 2.5 shows this reflection.

2
10 T = 0.7 s
s
power

0
10
f =0.71
N

2
10 T = 1.05 s
s
Figure 2.5: Reflecting the frequency re-
power

sponse in the Nyquist frequency from

0
10 Fig. 2.4 shows why one of the peaks is ev-
f =0.48
N
ident in the frequency spectrum at Ts =
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
frequency (Hz) 1.05, but what about the other peak?

Now we turn our attention to the final peak. f3 (a) = 0.63 and f2 (b) = 0.152. Let us again try
the fact that f3 (a) is the true frequency = 0.63Hz. If this were the case the apparent frequency
on curve b would be fˆ3 (b) = 2fN (b) − f2 = 0.3224Hz. There is no peak on curve b at this
frequency so our guess is probably wrong. Let us suppose that the peak at f3 (a) is the first
harmonic. In that case the true frequency will be f3 = 2fN (a) − fˆ3 (a) = 0.8Hz. Now we check
using curve b. If the true third frequency is 0.8Hz, then the apparent frequency on curve b will
be fˆ3 (b) = 2fN (b) − f3 = 0.153Hz. We have a peak here which indicates that our guess is a good
one. In summary, a reasonable guess for the unknown underlying function is

y(t) ≈ sin(2π0.1t) + sin(2π0.5t) + sin(2π0.8t)

although we can never be totally sure of the validity of this model. At best we could either re-
sample at a much higher frequency, say fs > 10Hz, or introduce an analogue low pass filter to cut
out, or at least substantially reduce, any high frequencies that may be reflected.

2.2 F INITE DIFFERENCE MODELS

To compute complex mathematical operations such as differentiation or integration, on a digital

computer, we must first discretise the continuous equation into a discrete form. Discretising is
where we convert a continuous time differential equation into a difference equation that is then
possible to solve on a digital computer.
20 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

In control applications, we are often required to solve

ordinary differential equations such as
dy
= f (t, y) (2.3)
dt
for y(t). The derivative can be approximated by the sim-
ple backward difference formula
dy yt − yt−T
≈ (2.4)
dt T
where T is the step size in time or the sampling inter-
val. Provided T is kept sufficiently small, (but not zero) Figure 2.6: In Nathaniel Hawthorne’s The
the discrete approximation to the continuous derivative Scarlet Letter, Hester Prynne is forced to
is reasonable. Inserting this approximation, Eqn. 2.4 wear a large, red “A” on her chest when
into our original differential equation, Eqn. 2.3, and re- she is found guilty of adultery and refuses
arranging for yt results in the famous Euler backward to name the father of her illegitimate child.
difference scheme;
yt = yt−T + T f (t − T, yt−T ) (2.5)
Such a scheme is called a recurrence scheme since a previous solution, yt−T , is used to calculate
yt , and is suitable for computer implementation. The beauty of this crude method of differen-
tiation is the simplicity and versatility. Note that we can discretise almost any function, linear
or nonlinear without needing to solve analytically the original differential equation. Of course
whether our approximation to the original problem is adequate is a different story and this com-
plex, but important issue is addressed in the field of numerical analysis.

Problem 2.1 1. What is the finite difference approximation for a third order derivative?
2. Write down the second order finite difference approximation to
d2 y dy
τ2 2
+ 2ζτ + y = ku
dt dt

2.2.1 D IFFERENCE EQUATIONS

Difference equations are the discrete counterpart to continuous differential equations. Often the
difference equations we are interested in are the result of the discretisation of a continuous dif-
ferential equation, but sometimes, as in the case below, we may be interested in the difference
equation in its own right. Difference equations are much easier to simulate and study in a digital
computer than a differential equation for example, since they are already written in a form where
we can just step along following a relation
xk+1 = f (xk )
rather than resorting to integration. This step by step procedure is also known as a mathematical
mapping since we map the old vector xk to a new vector xk+1 .

H ÉNON ’ S CHAOTIC ATTRACTOR

The system of discrete equations known as Hénon’s attractor,

xk+1 = (yk + 1) − 1.4x2k , x0 = 1 (2.6)
yk+1 = 0.3xk , y0 = 1 (2.7)
2.3. THE Z TRANSFORM 21

is an interesting example of a discrete mapping which exhibits chaotic behaviour.

We will start from point x0 = (x0 , y0 ) = (1, 1) and investigate the subsequent behaviour using
S IMULINK. Since Hénon’s attractor is a discrete system, we will use unit delays to shift back
from xk+1 → xk and to better visualise the results we will plot an (x, y) phase plot. The S IMULINK
simulation is given in Fig. 2.7. Compare this S IMULINK construction (which I have drawn de-
liberately to flow from right to left) so that it matches the way we read the system equations,
Eqn. 2.6-Eqn. 2.7.

1.4
Product
Gain2

x 1/z 1
To Workspace x1 Sum1 Constant

XY Graph
y 1/z 0.3
To Workspace2 y1
Gain

Figure 2.7: Hénon’s attractor, (Eqns 2.6–2.7), modelled in S IMULINK. See also Fig. 2.8.

The time response of x and y, (left figures in Fig. 2.8), of this mapping is not particularly inter-
esting, but the result of the phase plot (without joining the dots), (right figure in Fig. 2.8), is an
interesting mapping.

Actually what the chaotic attractor means is not important at this stage, this is after all only a
demonstration of a difference equation. However the points to note in this exercise are that we
never actually integrated any differential equations, but only stepped along using the unit delay
blocks in S IMULINK. Consequently these types of simulations are trivial to program, and run
very fast in a computer. Another example of a purely discrete system is given in later Fig. 6.10.

2.3 T HE z TRANSFORM

The sampling of a continuous function f (t) at regular sampling intervals equally spaced T time
units apart creates a new function f ∗ (t),

∞
X
f ∗ (t) = f (kT ) δ(t − kT ) (2.8)
n=0

where δ(x) is the Dirac delta function which is defined as being a perfect impulse at x = 0,
and zero elsewhere. Note that the actual value of δ(0) is infinity, but the integral of δ(t) is 1.0.
Expanding the summation term in Eqn 2.8 gives

f ∗ (t) = f0 δ(t) + fT δ(t − T ) + f2T δ(t − 2T ) + f3T δ(t − 3T ) + · · ·

22 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

2
1
1 Start point
xk

−1 0.5

k
−2
1

y
0.5 0
k
y

−0.5 −0.5
0 5 10 15 20 25 30 35 −1.5 −1 −0.5 0 0.5 1 1.5
time counter x
k

(a) The time response (b) The (x, y) or phase response

Figure 2.8: The time response (left) and phase plot (right) of Hénon’s attractor as computed by
the S IMULINK block diagram in Fig. 2.7.

Now suppose we wish to know the value of the third sampled function f ∗ (t = 3T ). For simplicity
we will assume the sample time is T = 1.

f ∗ (3) = f0 δ(3) + f1 δ(2) + f2 δ(1) + f3 δ(0) + f4 δ(−1) + · · ·

| {z } | {z }
all zero all zero

All the terms except for the term containing δ(0) are zero, while δ(0) = ∞. Thus the “value”
of the function f ∗ (3) = ∞. Often you will see a graph of f ∗ (t) depicted where the heights of
the values at the sample times are the same as the height of the continuous distribution such as
sketched in Fig. 2.13. Strictly this is incorrect, as it is the ‘strength’ or integral of f ∗ (t) which is the
same as the value of the continuous distribution f (t). I think of the “function” f ∗ (t) as a series of
impulses whose integral is equal to the value of the continuous function f (t).

If we take the Laplace transform of the sampled function f ∗ (t), we get

(∞ )
X
∗
L {f (t)} = L fkT δ(t − kT ) (2.9)
n=0

Now the function fkT is assumed constant so it can be factored out of the Laplace transform,
and the Laplace transform of the δ(0) is simply 1.0. If the impulse is delayed kT units, then the
Laplace transform is e−kT × 1. Thus Eqn. 2.9 simplifies to
∞
X ∞
X
L {f ∗ (t)} = fkT e−kT s = fk z −k
n=0 k=0

where we have defined

def
z = esT (2.10)
∗
In summary, the z-transform of the sampled function f (t) is defined as
∞
X
def
F (z) = Z {f ∗ (t)} = fk z −k (2.11)
k=0

The function F (z) is an infinite series, but can be written in a closed form if f (t) is a rational
function.
2.3. THE Z TRANSFORM 23

2.3.1 z- TRANSFORMS OF COMMON FUNCTIONS

We can use the definition of the z-transform, Eqn. 2.11, to compute the z-transform of common
functions such as steps, ramps, sinusoids etc. In this way we can build for ourselves a table of
z-transforms such as those found in many mathematical or engineering handbooks.

S AMPLED STEP FUNCTION

The unit sampled step function is simply s(kT ) = 1, k ≥ 0. The z-transform of s(kT ) following
the definition of Eqn. 2.11 is
∞
X
S(z) = kn z −k = 1 + 1 · z −1 + 1 · z −2 + 1 · z −3 + · · · (2.12)
k=0

By using the sum to infinity for a geometric series,3 we obtain the closed form equivalent for
Eqn. 2.12 as
1
S(z) = (2.13)
1 − z −1

R AMP AND EXPONENTIAL FUNCTIONS

For the ramp function, x(k) = n for k ≥ 0, and the exponential function, x(k) = e−an , k ≥ 0, we
could go through the same formal analytical procedure, but in this case we can use the ztrans
command from the symbolic toolbox in M ATLAB.

 z-transform
The of the exponential function,
The z-transform of the ramp function, Z {k}, Z e−ak is
using the Symbolic toolbox for M ATLAB is
1 >> syms k T a
>> ztrans(exp(-a*k*T))
1 >> syms k T
ans =
>> ztrans(k*T)
z/exp(-a*T)/(z/exp(-a*T)-1)
ans =
>> simplify(ans)
T*z/(z-1)ˆ2
6 ans =
z*exp(a*T)/(z*exp(a*T)-1)

In a similar manner we can build our own table of z-transforms of common functions.

F INAL AND INITIAL VALUE THEOREMS

Tables and properties of z-transforms of common functions are given in many handbooks and
texts such as [150, p49,67], but two of the more useful theorems are the initial value and the final
3 The sum of a geometric series of n terms is
a (1 − r n )
a + ar + ar 2 + · · · + ar n−1 = ; r 6= 1
1−r
and if |r| < 1, then the sum for an infinite number of terms converges to
a
S∞ =
1−r
24 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

value theorems given in Table 2.1. As in the continuous case, the final value theorem is only
applicable if the transfer function is stable.

Table 2.1: Comparing final and initial value theorems for continuous and discrete systems

Continuous Discrete
Initial value, limt→0 y(t) lims→∞ sY (s) lim
 z→∞ Y (z)


Final value, limt→∞ y(t) lims→0 sY (s) limz→1 1 − z −1 Y (z)

2.4 I NVERSION OF z- TRANSFORMS

The usefulness of the z-transform is that we can do algebraic manipulations on discrete difference
equations in the same way we can do algebraic manipulations on differential equations using the
Laplace transform. Generally we:

1. Convert to z-transforms, then

2. do some relatively simple algebraic manipulations to make it easier to
3. invert back to the time domain.

The final step, the inversion, is the tricky part. The process of obtaining f ∗ (t) back from F (z) is
called inverting the z-transform, and is written as
f ∗ (t) = Z −1 {F (z)} (2.14)
∗
Note that only the sampled function is returned f (t), not the original f (t). Thus the full inver-
sion process to the continuous domain f (t) is a “one to many” operation.

There are various ways to invert z-transforms:

1. Use tables of z-transforms and their inverses in handbooks, or use a symbolic manipulator.
(See §2.4.1.)
2. Use partial-fraction expansion and tables. In M ATLAB use residue to compute the partial
fractions. (See §2.4.2.)
3. Long division for rational polynomial expressions in the discrete domain. In M ATLAB use
deconv to do the polynomial division. (See §2.4.3.)
4. Computational approach while suitable for computers, has the disadvantage that the an-
swer is not in a closed form. (See §2.4.4.)
5. Analytical formula from complex variable theory. (Not useful for engineering applications,
see §2.4.5.)

2.4.1 I NVERTING z- TRANSFORMS WITH SYMBOLICALLY

The easiest, but perhaps not the most instructive, way to invert z-transforms is to use a computer
algebra package or symbolic manipulator such as the symbolic toolbox or M U PAD. One simply
enters the z-transform, then requests the inverse using the iztrans function in a manner similar
to the forward direction shown on page 23.
2.4. INVERSION OF Z -TRANSFORMS 25

>> syms z % Define a symbolic variable z

>> G = (10*z+5)/(z-1)/(z-1/4) % Construct G(z) = (10z + 5)/(z − 1)/(z − 1/4)
3 G =
(10*z+5)/(z-1)/(z-1/4)
>> pretty(G) % check it
10 z + 5
-----------------
8 (z - 1) (z - 1/4)
>> iztrans(G) % Invert the z-transform, Z −1 {G(z)}
ans =
20*kroneckerDelta(n, 0) - 40*(1/4)ˆn + 20

The Kronecker δ function, or kroneckerDelta(n, 0) is a shorthand way of expressing piece-

wise functions in M ATLAB. The expression kroneckerDelta(n, 0) returns 1 when n = 0,
and 0 for all other values.
1
δ(n)

0.5

0
−5 0 5
n

This is a rather messy and needlessly complicated artifact due to the fact that the symbolic tool-
box does not know that n is defined as positive. Also M ATLAB automatically choose n for the
indexing variable, but we might prefer to use k and explicitly inform M ATLAB that k > 0. This
gives us a cleaner solution.

>> syms z
>> syms k positive
>> G = (10*z+5)/(z-1)/(z-1/4);
4

>> y=iztrans(G,k) % Invert to get y[k] = Z −1 {G(z)}

y =
20-40*(1/4)ˆk

9 >> limit(y,k,inf) % Compute final value: y[∞]

ans =
20

The last command computed the steady-state by taking the limit of y[k] as k → ∞.

2.4.2 T HE PARTIAL FRACTION METHOD

Inverting transforms by partial fractions is applicable for both discrete z-transforms and con-
tinuous Laplace transforms. In both cases, we find an equivalent expression for the transform
in simple terms that are summed together. Hopefully we can then consult a set of tables, such
as [150, Table 2.1 p49], to invert each term individually. The inverse of the sum, is the sum of
the inverses meaning that these expressions when summed together give the full inversion. The
M ATLAB function residue forms partial fractions from a continuous transfer function, although
the help file warns that this operation is numerically poorly conditioned. Likewise the routine
residuez (that is residue with a trailing ‘z’) is designed to extract partial fractions in a form
suitable when using z−transforms.
26 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

As an example, suppose we wish to invert

−4s2 − 58s − 24 −4s2 − 58s − 24

G(s) = =
s3 + 2s2 − 24s s(s + 6)(s − 4)
using partial fractions noting that it contains no multiple or complex roots.

It is easy to spot the pole at s = 0, and the two remaining can be found by synthetic division, or
by factorising using the roots([1 2 -24 0]) command. The partial fraction decomposition
of G(s) can be written as

−4s2 − 58s − 24 A B C
G(s) = = + +
s(s + 6)(s − 4) s s+6 s−4
where the coefficients A, B and C can be found either by equating the coefficients or using the
‘cover-up’ rule shown below.
 
−4s2 − 58s − 24  −4s2 − 58s − 24 
A= = 1, B= =3
(s + 6)(s − 4) s=0 s(s − 4) 
s=−6

−4s2 − 58s − 24 
C= = −8
s(s + 6) 
s=4

In M ATLAB we can use the residue command to extract the partial fractions.

>> B = [-4 -58 -24]; % Numerator of G(s) = (−4s2 − 58s − 24)/(s(s + 6)(s − 4))
>> A = [ 1 2 -24 0]; % Denominator of G(s)
>> [r,p,k] = residue(B,A) % Find partial fractions
4 r = % residues (top line)
3
-8
1
p = % factors or poles
9 -6
4
0
k = % No extra parts
[]

The order of the residue and pole coefficient produced by residue is the same for each, so the
partial fraction decomposition is again

−4s2 − 58s − 24 3 −8 1
G(s) = = + +
s3 − 2s − 24s s+6 s−4 s
and we can invert each term individually perhaps using tables to obtain the time domain solution

g(t) = 3e−6t − 8e4t + 1

If the rational polynomial has repeated roots or complex poles, slight modifications to the proce-
dure are necessary so you may need to consult the help file.

S PECIAL CASES FOR z- TRANSFORMS

We can invert z-transforms in a manner similar to Laplace transforms, but if you consult a table
of z-transforms and their inverses in standard control textbooks such as Table 2-1 in [151, p49],
2.4. INVERSION OF Z -TRANSFORMS 27

you will discover that the table is written in terms of factors of the form z/(z + a) or alternatively
1/(1 − z −1) rather than say 1/(z + a). This means that we should first perform the partial fraction
decomposition on X(z)/z, rather than just X(z) as in the continuous case. An outline to follow
is given in Algorithm 2.1.

Algorithm 2.1 Inverting z-transforms by partial fractions

To invert a z transform, X(z), using partial fractions, do the following:

1. Divide X(z) by z obtaining X(z)/z.

2. Find the partial fractions of X(z)/z perhaps using residue.
3. Multiply each partial fraction by z to obtain a fraction of the form z/(z + a)
4. Find the inverse of each fraction separately using tables, and sum together to find x(t).

S YMBOLIC PARTIAL FRACTIONS IN M ATLAB

Suppose we want to invert

10z
G(z) =
z 2 + 4z + 3
to g[k] using partial fractions. Unfortunately there is no direct symbolic partial fraction command,
but Scott Budge from Utah University found the following sneaky trick which involves first inte-
grating, then differentiating the polynomial. This round-about way works because M ATLAB in-
tegrates the expression by internally forming partial fractions and then integrating term by term.
Taking the derivative brings you back to the original expression, but now in partial fraction form.

>> syms z
2 >> G = 10*z/(zˆ2 + 4*z + 3);
>> G= diff(int(G/z)) % Extract partial fractions of G(z)/z
G =
5/(z+1)-5/(z+3)
>> G=expand(G*z)
7 G =
5z 5z
5*z/(z+1)-5*z/(z+3) % Gives us G(z)/z = z+1 − z+3

>> syms k positive; iztrans(G,z,k)

ans =
12 -5*(-3)ˆk+5*(-1)ˆk

While this strategy currently works, we should take note that in future versions of the symbolic
toolbox this may not continue to work. In the above example we also have the option of using
residuez.

2.4.3 L ONG DIVISION

Consider the special case of the rational polynomial where the denominator is simply 1,
c0 + c1 z −1 + c2 z −2 + c3 z −3 + · · ·
F (z) =
1
28 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

The inverse of this transform is trivial since the time solution at sample number k or t = kT is sim-
ply the coefficient of the term z −k . Consequently it follows that f (0) = c0 , f (T ) = c1 , f (2T ) =
c2 , . . . , f (kT ) = ck etc. Thus if we have a rational transfer function, all we must do is to synthet-
ically divide out the two polynomials to obtain a single, albeit possibly infinite, polynomial in
z.

This approach is known as “direct division” [150, p69] and in general will result in an infinite
power series in z. As such it is not a particularly elegant closed-form solution, but for most stable
systems, the power series can be truncated eventually with little error.

So to invert a z-transform using long-division, we:

1. Convert the z-transform to a (possibly infinite) power series by dividing the denominator
into the numerator using long division or deconv.
2. The coefficients of z k are the solutions at the sample times kT .

Long division by hand is tedious and error-prone, but we can use the polynomial division capa-
bilities of M ATLAB’s deconv to do this long division automatically. Since deconv only performs
“integer” division returning a quotient and remainder, we must fool it into giving us the “infi-
nite” series by padding the numerator with zeros.

To invert
z2 + z z2 + z
Y (z) = 2
= 3
(z − 1)(z − 1.1z + 1) z − 2.1z 2 + 2.1z − 1
using deconv for the long division, we pad the numerator on the right with say 5 zeros, (that is,
we multiply by z 5 ), and then do the division.
z7 + z6
= z 4 + 3.1z 3 + 4.41z 2 + 3.751z + 1.7161 + remainder
z3 − 2.1z 2 + 2.1z − 1 | {z }
Q

We are not particularly interested in the remainder polynomial. The more zeros we add, (5 in the
above example), the more solution points we get.

>> Yn = [1,1,0]; % Numerator of G(z)

>> Yd = conv([1,-1],[1 -1.1 1]); % Denominator of G(z) is (z − 1)(z 2 − 1.1z + 1)
3 >> [Q,R] = deconv([Yn,0,0,0,0,0],Yd) % Multiply by z 5 to zero pad, then do long division
Q =
1.0000 3.1000 4.4100 3.7510 1.7161

>> dimpulse(Yn,Yd,7); % Matlab's check

We can also numerically verify the inversion using dimpulse for six samples as shown in Fig. 2.9.

Problem 2.2 1. The pulse transfer function of a process is given by

Y (z) 4(z + 0.3)
= 2
X(z) z − z + 0.4
Calculate the response of y(nT ) to a unit step change in x using long division.
2. Determine the inverse by long division of G(z)
z(z + 1)
G(z) =
(z − 1)(z 2 − z + 1)
2.4. INVERSION OF Z -TRANSFORMS 29

3
Output

1
Figure 2.9: Inverting z-transforms using dimpulse for
0
the first six samples. It is possible also to construct a
discrete transfer function object and then use the generic
0 1 2 3 4 5 6
sample # impulse routine.

2.4.4 C OMPUTATIONAL APPROACH

The computational method is so-called because it is convenient for a computer solution technique
such as M ATLAB as opposed to an analytical explicit solution. In the computational approach we
convert the z-transform to a difference equation and use a recurrence solution.

Consider the following transform (used in the example from page 24) which we wish to invert
back to the time domain.
X(z) 10z + 5 10z + 5 10z −1 + 5z −2
= = 2 = (2.15)
U (z) (z − 1)(z − 0.25) z − 1.25z + 0.25 1 − 1.25z −1 + 0.25z −2
where we have coloured the input blue and the output red. (We will again use this colour con-
vention in chapter 6).

The inverse is equivalent to solving for x(t) when u(t) is an impulse function. The transform can
be expanded and written as a difference equation

xk = 1.25xk−1 − 0.25xk−2 + 10uk−1 + 5uk−2 (2.16)

Since U (z) is an impulse, U (z) = 1 which means that only u(0) = 1 while all other samples
u(k) = 0, k 6= 0. Now we substitute k = 0 to start, and note that u(k) = x(k) = 0 when k < 0 by
definition. We now have enough information to compute x(0).

x0 = 1.25x−1 − 0.25x−2 + 10u−1 + 5u−2

= 1.25 × 0 − 0.25 × 0 + 10 × 0 + 5 × 0
=0

Continuing we substitute k = 1 into Eqn. 2.16 and using our previously computed x0 and noting
that now u0 is nonzero, we can find the next term.

x1 = 1.25x0 − 0.25x−1 + 10u0 + 5u−1

= 1.25 × 0 − 0.25 × 0 + 10 × 1 + 5 × 0
= 10

Just to clarify this recurrence process further, we can try the next iteration of Eqn. 2.16 with k = 2,

x2 = 1.25x1 − 0.25x0 + 10u1 + 5u0

= 1.25 × 10 − 0.2 × 0 + 10 × 0 + 5× 1
= 17.5
30 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Now we can use the full recurrence relation, (Eqn 2.16), to obtain the solution in a stepwise
manner to build up the solution as shown in Table 2.2. All the terms on the right hand side of
Eqn 2.16 are either known initially (u(t)), or involve past information, hence it is possible to solve.
Note that in this inversion scheme, u(t) can be any known input series.

Table 2.2: The inversion of a z transform using the computation method. Only the first 5 samples
and the final value are calculated. The final value is as calculated using the symbolic manipulator
earlier on page 25.

time (kT ) u(k) x(k)

−2 0 0
−1 0 0
0 1 0
1 0 10
2 0 17.5
3 0 19.375
4 0 19.8438
.. .. ..
. . .
∞ 0 20

M ATLAB is well suited for this type of computation. To invert Eqn. 2.15 we can use the discrete
filter command. We rewrite Eqn. 2.15 in the form of a rational polynomial in z −1 ,

b0 + b1 z −1 + b2 z −2
G(z) =
1 + a1 z −1 + a2 z −2
where the numerator and denominator are entered as row vectors in decreasing powers of z. We
then create an impulse input vector u with say 6 samples and then compute the output of the
system

>> b = [0.0, 10.0, 5.0] % Numerator (10z −1 + 5z −2 )

>> a = [1.0, -1.25, 0.25] % Denominator (1 − 1.25z −1 + 0.25z −2 )
3

>> u = [1, zeros(1,5)] % impulse input, u(k) = δ(k)

u =
1 0 0 0 0 0
>> x = filter(b,a,u) % compute y(k)
8 x =
10.0000 17.5000 19.3750 19.8438 19.9609 19.9902

which gives the same results as Table 2.2.

The control toolbox has a special object for linear time invariant (LTI) systems. We can create a
discrete system with the transfer function command, tf, and by specifying a sampling time, we
imply to M ATLAB that it is a discrete system.

1 >> sysd = tf([10 5],[1 -1.25 0.25],1) % System of interest (10z + 5)/(z 2 − 1.25z + 0.25)

Transfer function:
10 z + 5
-------------------
6 zˆ2 - 1.25 z + 0.25
2.4. INVERSION OF Z -TRANSFORMS 31

Sampling time: 1

>> y=impulse(sysd) % compute the impulse response

11 y =
0
10.0000
17.5000
19.3750
16 19.8438
19.9609

Finally, the contour integral method can also be used to invert z-transforms, [150, §3–3], but
Seborg et al maintains it is seldom used in engineering practice, [182, p571] because it is fraught
with numerical implementation difficulties. These difficulties are also present in the continuous
equivalent; some of which are illustrated in the next section.

2.4.5 N UMERICALLY INVERTING THE L APLACE TRANSFORM

Surprisingly, the numerical inversion of continuous transfer functions is considered far less impor-
tant than the computation of the inverse of discrete transfer functions. This is fortunate because
the numerical inversion of Laplace transforms is devilishly tricky. Furthermore, it is unlikely that
you would ever want to numerically invert a Laplace transform in control applications, since
most problems involver little more than a rational polynomial with a possible exponential term.
For these problems we can easily factorise the polynomial and use partial fractions or use the
step orimpulse routines for continuous linear responses.

However in the rare cases where we have a particularly unusual F (s) which we wish to convert
back to the time domain, we might be tempted to use the analytical expression for the inverse
directly
Z σ+j∞
1
f (t) = F (s) est ds (2.17)
2πj σ−j∞
where σ is chosen to be larger than the real part of any singularity of F (s). Eqn. 2.17 is sometimes
known as the Bromwich integral or Mellin’s inverse formula.

The following example illustrates a straight forward application of Eqn. 2.17 to invert a Laplace
transform. However be warned that for all but the most well behaved rational polynomial exam-
ples this strategy is not to be recommended as it results in severe numerical roundoff error due
to poor conditioning.

Suppose we try to numerically invert a simple Laplace transform,

3 3

F (s) = ←→ f (t) = 1 − e−5t
s(s + 5) 5
with a corresponding simple time solution. We can start by defining an anonymous function of
the Laplace transform to be inverted.

Fs = @(s) 3.0./(s+5)./s; % Laplace function to invert F (s) = 3/(s(s + 5)) . . .

Fsexp = @(s,t) Fs(s).*exp(s*t) % and associated integrand F (s)est

Now since the largest singularity of F (s) is 0, we can choose the contour path safely to be say
σ = 0.1. We can also approximate the infinite integration interval by reasonably large numbers.
It is convenient that the M ATLAB routine integral works directly in the complex domain.
32 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

0.7 2

0.6 1.5 n o
0.5 L−1 √ 1
1
s 2 +1

0.4
f(t)

f(t)
0.5
0.3
0
0.2 n o
3
L−1 s(s+5) −0.5
0.1

0 −1
0.02 2

1
error

error
0
0

−0.02 −1
0 2 4 6 8 0 5 10 15 20
t t

(a) Inverting
n o a benign transfer function, (b) Inverting
 a challenging transfer function,
3
L−1 s(s+5) L −1 √ 1
s2 +1

Figure 2.10: Numerically inverting Laplace transforms by direct evaluation of the Bromwich
integral. Since this strategy is suspectable to considerable numerical errors, it is not to be recom-
mended.

c = 0.1; % Value σ > than any singularities of F (s)

a = c-100j; b = c+200j; % Contour path approx: σ − j∞ to σ + j∞
3

t=linspace(0,8); % time range of interest

% Numerically approximate Eqn. 2.17.
ft = 1./(2*pi*1j)*arrayfun(@(t) integral(@(x) Fsexp(x,t),a,b), ti);
plot(t,ft) % Compare results in Fig. 2.10(a).

Due to small numerical roundoff however, the returned solution has a small imaginary compo-
nent, which we can in this instance safely ignore. In this rather benign example, where we know
the analytical solution, we can validate the accuracy as shown in Fig. 2.10(a), which it has to be
admitted, is not that wonderful. We should also note in the simplistic implementation above we
have not adequately validated that our algorithmic choices such as the finite integration limits
are appropriate, so we should treat any subsequent result with caution. In fact if you repeat this
calculation with a larger value of σ, or a smaller integration range such as say 1 − 10j to 1 + 10j
then the quality of the solution drops alarmingly. One way to improve the integration accuracy
is to use the waypoints option in the integral routine.

Over the years there have been many attempts to improve the inversion strategy as reviewed
in [1], but nearly all of the proposals run into the problem√of precision. For example, Fig. 2.11
shows the problem when attempting to invert F (s) = 1/ s2 + 1 using the Gaver-Stehfest ap-
proximation algorithm with a varying number of terms. The Gaver-Stehfest family of schemes
are well known to produce inaccurate results for underdamped systems, but when we increase
the number of terms in an attempt to improve the accuracy, the results deteriorate due to numer-
ical round off. Directly applying the Bromwich integral to this transform as shown in Fig. 2.10(b)
is no better. While the strategy suggested in [1] seemingly circumvents the problem of precision,
it does so by brute-force using multi-precision arithmetic which is hardly elegant.
2.4. INVERSION OF Z -TRANSFORMS 33

No. of terms = 8
1

0.5

−0.5
0 2 4 6 8 10
No. of terms = 16
1

0.5

−0.5
0 2 4 6 8 10
No. of terms = 20
1

0.5

−0.5
0 2 4 6 8 10
No. of terms = 26
1

0
Figure 2.11: Demonstrating the precision problems
when numerically inverting the Laplace transform using
the Gaver-Stehfest algorithm with a varying number of
−1
0 2 4 6 8 10 terms. The exact inversion is the red solid line while the
t approximate inversion is given by –◦–.

An alternative numerical algorithm is the inverse Laplace transform function invlap function
contributed by Karl Hollenbeck from the Technical University of Denmark. You can test this
routine using one of the following Laplace transform pairs in Table 2.3. The first two are standard,
those following are more unusual. A more challenging collection of test transforms is given in
[1].

Table 2.3: A collection of Laplace transform pairs suitable for testing numerical inversion strate-
gies.

Description F (s) f (t) Comment

Easy test 1/s2 t
1 1


Control TF 2 e−t − e−3t
(s + 2)(s + 3)
1
Oscillatory √ J0 (t) Bessel function, see Fig. 2.10(b) & Fig. 2.11.
2
s +1
1 −1/s √
Nasty √ e √1 cos( 4t), See Fig. 2.12.
s πt

n o
The numerical solution to L−1 √1 e−1/s is compared with the analytical solution from Table 2.3
s
in Fig. 2.12 using a modest tolerance of 10−2 . The routine splits the time axis up into decades,
34 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

and inverts each separately which is why we can see some numerical noise starting at t = 10.

1.5

n o
exp(−1/s)

f(t)
0.5 L−1 √
s

0
−0.5
10

−2
10
error

−4
10

Figure 2.12: Testing the numerical inver-

√ −6
sion of the Laplace transform e−1/s / s us- 10
0 5 10 15 20 25 30 35 40
ing the invlap routine. time

All of these numerical inversion strategies, starting from the direct application of the Bromwich
integral in Eqn. 2.17, the Gaver-Stehfest algorithm and variants and the implementation invlap
require some care to get reasonable results. Unfortunately in practical cases it can be difficult to
choose suitable algorithmic tuning constants, which mean that it is difficult to generate any sort
of error bound on the computed curve.

2.5 D ISCRETISING WITH A SAMPLE AND HOLD

In §2.1.1 we saw how the continuous analogue signal must be sampled to be digestible by the
process control computer. To analyse this sampling operation, we can approximate the real sam-
pler with an ideal sampler in cascade with a hold element such as given in Fig. 2.13.

✒ f ∗ (t)
f (t) ✲ ✲ Hold ✲ fh (t)
ideal sampler

✻ ✻ ✻
✻✻✻
✻
✻
✻

✲ ✲ ✲
analogue signal sampled signal sample & held signal

Figure 2.13: Ideal sampler and zeroth-order hold

2.5. DISCRETISING WITH A SAMPLE AND HOLD 35

When we sample the continuous function f (t), we get a sampled function f ∗ (t), that is a series
of spikes that exist only at the sampling instant as shown in the middle plot of Fig. 2.13. The
sampled function, f ∗ (t), is undefined for the time between the sampling instants which is incon-
venient given that this is the bulk of the time. Obviously one sensible solution is to retain the
last sampled value until a new one is collected. During the sample period, the sampled value
f ∗ (t) is the same as the last sampled value of the continuous function; f (kT + t) = f (kT ) where
0 ≤ t < T shown diagrammatically in the third plot in Fig. 2.13. Since the last value collected is
stored or held, this sampling scheme is referred to as a zeroth-order hold. The zeroth-order refers
to the fact that the interpolating function used between adjacent values is just a horizontal line.
Higher order holds are possible, but the added expense is not typically justified even given the
improved accuracy.

We can find the Laplace transform of the zeroth-order hold element by noting that the input is an
impulse function, and the output is a rectangular pulse of T duration, and y(t) high,

output ✻
Rectangular pulse
y(t) ✙

✛ ✲ time, t
0 sample time, T

Directly applying the definition of the Laplace transform,

Z ∞ Z T
−st
L{zoh} = y(t) e dt = yt 1 × e−st dt
0 0
1 − e−sT
= yt ×
| {zs }
zoh

we obtain the transform for the zeroth-order hold. In summary, the transformation of a continu-
ous plant G(s) with zero-order hold is
 
G(s)
G(z) = (1 − z −1 ) Z (2.18)
s

Note that this is not the same as simply the z-transform of G(s).

Suppose we wish to discretise the continuous plant

8
G(s) = (2.19)
s+3
with a zero-order hold at a sample time of T = 0.1 seconds. We can do this analytically using ta-
bles, or we can trust it to M ATLAB. First we try analytically and apply Eqn 2.18 to the continuous
process. The difficult bit is the z-transformation. We note that
   
G(s) 8 3
Z = Z (2.20)
s 3 s(s + 3)

and that this is in the form given in tables of z-transforms such as [150, p50], part of which is
repeated here
36 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

X(s) X(z)
a (1−e−aT )z −1
s(s+a) (1−z −1 )(1−e−aT z −1 )

Inverting using the tables, and substituting in Eqn 2.18 gives

   
G(s) 8 3
G(z) = (1 − z −1 ) · = (1 − z −1 ) · Z
s 3 s(s + 3)
8 (1 − e−3T )z −1
= (1 − z −1 ) ·
3 (1 − z −1 )(1 − e−3T z −1 )
−0.3 −1
8 (1 − e )z
= ·
3 (1 − e−0.3 z −1 )
0.6912z −1
= , for T = 0.1 (2.21)
1 − 0.7408z −1
We can repeat this conversion numerically using the continuous-to-discrete function, c2d. We
first define the continuous system as an LTI object, and then convert to the discrete domain using
a zeroth order hold.

>>sysc = tf(8,[1 3]) % Continuous system Gc (s) = 8/(s + 3)

3 Transfer function:
8
-----
s + 3

8 >> sysd = c2d(sysc,0.1,'zoh') % convert to discrete using a zoh

Transfer function:
0.6912
----------
13 z - 0.7408

Sampling time: 0.1

which once again is Eqn. 2.21. In fact we need not specify the ’zoh’ option when constructing a
discrete model using c2d or even in S IMULINK, since a zeroth-order hold is employed by default.

Methods for doing the conversion symbolically are given next in section 2.5.1.

2.5.1 C ONVERTING L APLACE TRANSFORMS TO z- TRANSFORMS

In many practical cases we may already have a continuous transfer function of a plant or con-
troller which we wish to discretise. In these circumstances we would like to convert from a con-
tinuous transfer function in s, (such as say a Butterworth filter) to an equivalent discrete transfer
function in z. The two common ways to do this are:

1. Analytically by first inverting the Laplace transform back to the time domain, and then take
the (forward) z-transform.
def 
G(z) = Z L−1 {G(s)} (2.22)

2. or by using an approximate method known as the bilinear transform.

2.5. DISCRETISING WITH A SAMPLE AND HOLD 37

The bilinear method whilst approximate has the advantage that it just involves a simple algebraic
substitution for s in terms of z. The bilinear transform is further covered in §2.5.2.

The formal method we have already seen previously, but since it is such a common operation, we
can write a symbolic M ATLAB script to do it automatically for us for any given transfer function.
However we should be aware whether the transformation includes the sample and hold, or if
that inclusion is left up to the user.

Listing 2.1 converts a Laplace expression to a z-transform expression without a zeroth-order hold.
Including the zeroth-order hold option is given in Listing 2.2.

Listing 2.1: Symbolic Laplace to z-transform conversion

function Gz = lap2ztran(G)
% Convert symbolically G(s) to G(z)
syms t k T z
Gz = simplify(ztrans(subs(ilaplace(G),t,k*T),k,z));
5 return

Listing 2.2: Symbolic Laplace to z-transform conversion with ZOH

function Gz = lap2ztranzoh(G)
% Convert symbolically G(s) to G(z) with a ZOH.
syms s t k T z
Gz = simplify((1-1/z)*ztrans(subs(ilaplace(G/s),t,k*T),k,z));
5 return

We can test the conversion routines in Listings 2.1 and 2.2 on the trial transfer function G(s) =
1/s2 .

>> Gz =lap2ztran(1/sˆ2); % Convert G(s) = 1/s2 to the discrete domain G(z).

>> pretty(Gz)
T z
--------
5 2
(z - 1)

>> Gz = lap2ztranzoh(1/sˆ2);% Do the conversion again, but this time include a ZOH.
>> pretty(Gz)
10 2
(z + 1) T
1/2 ----------
2
(z - 1)

2.5.2 T HE BILINEAR TRANSFORM

The analytical ‘one step back–one step forward’ procedure while strictly correct, is a little tedious,
so a simpler, albeit approximate, way to transform between the Laplace domain and the z-domain
is to use the bilinear transform method, sometimes known as Tustin’s method. By definition
def
z = esT or z −1 = e−sT , (from Eqn. 2.10). If we substituted directly natural logs would appear in
38 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

the rational polynomial in z,

ln(z)
s= (2.23)
T
making the subsequent analysis difficult. For example the resulting expression could not be
transformed into a difference equation which is what we desire.

We can avoid the troublesome logarithmic terms by using a Padé approximation for the expo-
nential term as
2 − sT
e−sT = z −1 ≈ (2.24)
2 + sT
or alternatively
2 (1 − z −1 )
s≈ (2.25)
T (1 + z −1 )
This allows us to transform a continuous time transfer function G(s) to a discrete time transfer
function G(z),
G(z) = G(s)|s= 2 1−z−1 (2.26)
T 1+z −1

Eqn. 2.26 is called the bilinear transform owing to the linear terms in both the numerator and
denominator and it has the advantage that a stable continuous time filter will be stable in the dis-
crete domain. The disadvantage is that the algebra required soon becomes unwieldy if attempted
manually. Other transforms are possible and are discussed in §4-2 p308 of Ogata [150]. Always
remember that this transformation is approximate, being equivalent to a trapezoidal integration.

Here we wish to approximately discretise the continuous plant

1
G(s) =
(s + 1)(s + 2)

at a sample time of T = 0.1 using the bilinear transform. The discrete approximate transfer
function is obtained by substituting Eqn. 2.25 for s and simplifying.

1 
G(z) = 
(s + 1)(s + 2) s= 2 1−z−1
T 1+z −1

1 1 T 2 (z + 1)2
=   =
2 1−z −1
+1 2 1−z −1
+2 2 (2T 2 + 6T + 4)z 2 + (4T 2 − 8)z + 2T 2 − 6T + 4
T 1+z −1 T 1+z −1

0.0022z 2 + 0.0043z + 0.0022

= for T = 0.1
z 2 − 1.723z + 0.74

Since this transformation is just an algebraic substitution, it is easy to execute it symbolically in

M ATLAB.

1 >> syms s T z
>> G = 1/(s+1)/(s+2);
>> Gd = simplify(subs(G,s,2/T*(1-1/z)/(1+1/z)))
Gd =
1/2*Tˆ2*(z+1)ˆ2/(2*z-2+T*z+T)/(z-1+T*z+T)
6 >> pretty(Gd)
2 2
T (z + 1)
1/2 -------------------------------------
(2 z - 2 + T z + T) (z - 1 + T z + T)
2.5. DISCRETISING WITH A SAMPLE AND HOLD 39

Alternatively we could use M ATLAB to numerically verify our symbolic solution.

>> G = zpk([],[-1 -2],1)

Zero/pole/gain:
1
-----------
5 (s+1) (s+2)

>> T=0.1;
>> Gd = c2d(G,T,'tustin')

10 Zero/pole/gain:
0.0021645 (z+1)ˆ2
---------------------
(z-0.9048) (z-0.8182)

15 Sampling time: 0.1

>> tf(Gd)

Transfer function:
0.002165 zˆ2 + 0.004329 z + 0.002165
20 ------------------------------------
zˆ2 - 1.723 z + 0.7403

Sampling time: 0.1

The bilinear command in the S IGNAL P ROCESSING toolbox automatically performs this map-
ping from s to z. This is used for example in the design of discrete versions of common analogue
filters such as the Butterworth or Chebyshev filters. These are further described in §5.2.3.

Problem 2.3 1. Use Tustin’s method (approximate z-transform) to determine the discrete time
response of
4
G(s) =
(s + 4)(s + 2)

to a unit step change in input by long division. The sample time T = 1 and solve for 7 time
steps. Compare with the exact solution.

T HE FREQUENCY RESPONSE CHARACTERISTICS OF A HOLD ELEMENT

The zeroth order hold element modifies the transfer function, so consequently influences the
closed loop stability, and frequency characteristics of the discrete process. We can investigate this
influence by plotting the discrete Bode and Nyquist plots for a sampled process with and without
a zeroth order hold.

Suppose we have the continuous process

1.57
Gp (s) = (2.27)
s(s + 1)

which is sampled at a frequency of ω = 4 rad/s. (This corresponds to a sample time of T = π/2

seconds.) First we will ignore the zeroth order hold and transform Eqn 2.27 to the z domain by
40 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

using tables such as say Table 2–1, p49 #8 in DCS.

(1 − e−aT )z −1
Gp (z) = 1.57 (2.28)
(1 − z −1 )(1 − e−aT z −1 )
1.243z
= (2.29)
(z − 1)(z − 0.208)
To get the discrete model with the zeroth order hold, we again use Eqn 2.18, and the tables. Doing
this, in a similar manner to what was done for Eqn 2.21, we get
1.2215z + 0.7306
Gh0 Gp (z) = (2.30)
(z − 1)(z − 0.208)
Now we can plot the discrete frequency responses using M ATLAB to duplicate the figure given
in [111, p649] as shown in Fig. 2.14.

In the listing below, I first construct the symbolic discrete transfer function, substitute the sam-
pling time, then convert to a numeric rational polynomial. This rational polynomial is now in a
suitable form to be fed to the control toolbox routines such as bode.

syms s T z
2 G = 1.57/(s*(s+1))
Gd = lap2ztran(G) % convert to a z−transform without ZOH
Gd2 = subs(Gd,T,pi/2)
[num,den] = numden(Gd2) % extract top & bottom lines

7 B= sym2poly(num); A = sym2poly(den); % convert to polynomials

Gd = tf(B,A,pi/2) % construct a transfer function

To generate the discretisation including the zeroth-order hold, we could use the symbolic lap2ztranzoh
routine given in Listing 2.2, or we could use the built-in c2d function.

1 [num,den] = numden(G) % extract numerator & denominator

Bc= sym2poly(num); Ac = sym2poly(den);
Gc = tf(Bc,Ac)

Gdzoh = c2d(Gc,pi/2,'zoh')
6

bode(Gc,Gd,Gdzoh)
legend('Continuous','No ZOH','with ZOH')

In this case I have used vanilla Bode function which will automatically recognise the difference
between discrete and continuous transfer functions. Note that it also automatically selects both
a reasonable frequency spacing, and inserts the Nyquist frequency at fN = 1/2T = 1/π Hz or
ωN = 2 rad/s.

These Bode plots shown in Fig. 2.14, or the equivalent Nyquist plots, show that the zeroth order
hold destabilises the system. The process with the hold has a larger peak resonance, and smaller
gain and phase margins.

Of course we should compare the discrete Bode diagrams with that for the original continuous
process. The M ATLAB bode function is in this instance the right tool for the job, but I will con-
struct the plot manually just to demonstrate how trivial it is to substitute s = iω, and compute
the magnitude and phase of G(iω) for all frequencies of interest.
2.5. DISCRETISING WITH A SAMPLE AND HOLD 41

Bode Diagram

100

50
Magnitude (dB)

−50

−100
−90
Continuous
No ZOH
with ZOH
Phase (deg)

−135

−180
Figure 2.14: The Bode dia-
gram showing the difference
−225 between the continuous plant,
−2 −1 0 1 2
10 10 10 10 10 a discretisation with and with-
Frequency (rad/sec) out the zeroth-order hold.

num = 1.57; % Plant of interest G(s) = 1.57/(s2 + s + 0)

2 den = [1 1 0];
w = logspace(-2,1)'; % Frequency range 10−2 < ω < 101 rad/s.
iw = 1i*w; % s = jω
G = polyval(num,iw)./polyval(den,iw); % G(s = jω)
loglog(w,abs(G)) % Plot magnitude |G(iω)|, see Fig. 2.15.
7 semilogx(w,angle(G)*180/pi) % and phase φ(iω)

We can compare this frequency response of the continuous system with the discretised version
in Fig. 2.14.

5
10

0
AR

10

−5
10
−2 −1 0 1
10 10 10 10

−50
Phase lag φ [deg]

−100

−150

−200
Figure 2.15: A frequency response plot of
10
−2
10
−1
10
0
10
1 G(s) where s = jω constructed manually
frequency ω [rad/s] without using the bode command.
42 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

2.6 D ISCRETE ROOT LOCUS DIAGRAMS

The root locus diagram is a classical technique used to study the characteristic response of trans-
fer functions to a single varying parameter such as different controller gains. Traditionally the
construction of root locus diagrams required tedious manual algebra. However now modern
programs such as M ATLAB can easily construct reliable root locus diagrams which makes the
design procedure far more attractable.

For many controlled systems, the loop is stable for small values of controller gain Kc , but unstable
for a gain above some critical value Ku . It is therefore natural to ask how does the stability (and
response) vary with controller gain? The root locus gives this information in a plot form which
shows how the poles of a closed loop system change as a function of the controller gain Kc .
Given a process transfer function G and a controller Kc Q, the closed loop transfer function is the
familiar
Kc Q(z)G(z)
Gcl = (2.31)
1 + Kc Q(z)G(z)

where the stability is dependent on the denominator (characteristic equation) 1 + Kc Q(z)G(z),

which is of course itself dependent on the controller gain Kc . Plotting the roots of

1 + Kc Q(z)G(z) = 0 (2.32)

as a function of Kc creates the root locus diagram.

In this section4 we will plot the discrete root locus diagram for the process

0.5(z + 0.6)
Gp (z) = (2.33)
z 2 (z − 0.4)

which is a discrete description of a first order process with dead time. We also add a discrete
integral controller of the form
z
Gc (z) = Kc
z−1
We wish to investigate the stability of the characteristic equation as a function of controller gain.

With no additional arguments apart from the transfer function, rlocus will draw the root locus
selecting sensible values for Kc automatically. For this example, I will constrain the plots to have
a square aspect ratio, and I will overlay a grid of constant shape factor ζ and natural frequency
ωn using the zgrid function.

Q = tf([1 0],[1 -1],-1); % controller without gain, Q(z)

G = tf(0.5*[1 0.6],[1 -0.4 0 0],-1); % plant, Gp (z)
3

Gol = Q*G; % open loop Q(z)Gp (z)

zgrid('new'); xlim([-1.5 1.5]); axis equal
rlocus(Gol) % plot root locus, see Fig. 2.16.
rlocfind(Gol)

Once the plot such as Fig. 2.16 is on the screen, I can use rlocfind interactively to establish the
values of Kc at different critical points in Fig. 2.16. Note that the values obtained are approximate.

4 Adapted from [71, p124]

2.6. DISCRETE ROOT LOCUS DIAGRAMS 43

Pole description Name ≈ Kc

border-line stability ultimate gain 0.6
cross the ζ = 0.5 line design gain 0.2
critically damped breakaway gain 0.097
overdamped, ωn = 36◦ 0.09

Root Locus

1.5

1
0.5π/T Ultimate
0.6π/T 0.4π/T gain
0.7π/T 0.1
0.3π/T
0.2
0.3
0.8π/T 0.4 0.2π/T Design gain
0.5 0.5
0.6
0.7
0.9π/T 0.8 0.1π/T
Imaginary Axis

0.9
π/T
0
π/T

0.9π/T 0.1π/T
breakaway
−0.5 0.8π/T 0.2π/T gain

0.7π/T 0.3π/T
0.6π/T 0.4π/T
0.5π/T
−1

−1.5

−1.5 −1 −0.5 0 0.5 1 1.5

Real Axis

Figure 2.16: The discrete root locus of Eqn. 2.33 plotted using rlocus. See also Fig. 2.17 for the
resultant step responses for various controller gains.

Once I select the gain that corresponds to the ζ = 0.5 crossing, I can simulate the closed loop step
response.

Gol.Ts = 1; % Set sampling time T = 1

Kc = 0.2 % A gain where we expect ζ = 0.5
3 step(Kc*Gol/(1 + Kc*Gol),50) % Simulate closed loop, see Fig. 2.17.

A comparison of step responses for various controller gains is given in Fig. 2.17. The curve in
Fig. 2.17 with Kc = 0.2 where ζ ≈ 0.5 exhibits an overshoot of about 17% which agrees well with
the expected value for a second order response with a shape factor of ζ = 0.5. The other curves
do not behave exactly as one would expect since our process is not exactly a second order transfer
function.
44 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

1
Kc=0.1
0
2

1
Kc=0.2
0
2

Figure 2.17: Various discrete closed loop re-

sponses for gains Kc = 0.6, 0.2 and 0.1. Note that 1
with Kc = 0.6 we observe a response on the sta- Kc=0.6
bility boundary, while for Kc = 0.1 we observe
0
a critically damped response as anticipated from 0 10 20 30 40 50
the root locus diagram given in Fig. 2.16. time [s]

2.7 M ULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

In the mid 1970s, the western world suffered an ‘oil shock’ when the petroleum producers and
consumers alike realised that oil was a scarce, finite and therefore valuable commodity. This had
a number of important implications, and one of these in the chemical processing industries was
the increased use of integrated heat plants. This integration physically ties separate parts of the
plant together and demands a corresponding integrated or ‘plant-wide’ control system. Clas-
sical single-input/single-output (SISO) controller design techniques such as transfer functions,
frequency analysis or root locus were found to be deficient, and with the accompanying devel-
opment of affordable computer control systems that could administer hundreds of inputs and
outputs, many of which were interacting, nonlinear and time varying, new tools were required
and the systematic approach offered by state-space analysis became popular.

Many physical processes are multivariable. A binary distillation column such as given in Fig. 2.18
typically has four inputs; the feed flow and composition and reflux and boil-up rates, and at least
four outputs, the product flow and compositions. To control such an interacting system, mul-
tivariable control such as state space analysis is necessary. The Wood-Berry distillation column
model (Eqn 3.24) and the Newell & Lee evaporator model are other examples of industrial pro-
cess orientated multivariable models.

State space analysis only considers first order differential equations. To model higher order sys-
tems, one needs only to build systems of first order equations. These equations are conveniently
collected, if linear, in one large matrix. The advantage of this approach is a compact represen-
tation, and a wide variety of good mathematical and robust numerical tools to analyse such a
system.

A FEW WORDS OF CAUTION

The state-space analysis is sometimes referred to as the “modern control theory”, despite the fact
that is has been around since the early 1960s. However by the end of the 1970s, the promise of
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 45

Feed composition
✲ ✲ distillation composition, xD
(
Disturbances
✲ ✲ distillate rate, D
Feed rate
Distillation ✲ tray 5 temperature, T5
column ✲ tray 15 temperature, T15
reflux(rate
Manpipulated ✲ ✲ Bottoms rate, B
variables ✲ ✲ Bottoms comp., xB
reboiler heat

Inputs Outputs

Figure 2.18: A binary distillation column with multiple inputs and multiple outputs

the academic advances made in the previous decade were turning out to be ill-founded, and it
was felt that this new theory was ill-equipped to cope with the practical problems of industrial
control. In many industries therefore, ‘Process Control’ attracted a bad smell. Writing a decade
still later in 1987, Morari in [138] attempts to rationalise why this disillusionment was around at
the time, and whether the subsequent decade of activity alleviated any of the concerns. Morari
summarises that commentators such as [70] considered theory such as linear multivariable con-
trol theory, (i.e. this chapter) which seemed to promise so much, actually delivered very little
‘and had virtually no impact on industrial practice’. There were other major concerns such as
the scarcity of good process models, the increasing importance of operating constraints, operator
acceptance etc, but the poor track record of linear multivariable control theory landed top billing.
Incidentally, Morari writing almost a decade later still in 1994, [141], revisits the same topic giving
the reader a nice linear perspective.

2.7.1 S TATES AND STATE SPACE

State space equations are just a mathematical equivalent way of writing the original common dif-
ferential equation. While the vector/matrix construction of the state-space approach may initially
appear intimidating compared with high order differential equations, it turns out to be more con-
venient and numerically more robust to manipulate these equations when using programs such
as M ATLAB. In addition the state-space form can be readily expanded into multivariable systems
and even nonlinear systems.

The state of a system is the smallest set of values such that knowledge of these values, knowledge
of any future inputs and the governing dynamic equations, it is possible to predict everything
about the future (and past) output of the system. The state variables are often written as a column
vector of length n and denoted x. The state space is the n dimensional coordinate space that all
possible state vectors must lie in.

The input to a dynamic system is the vector of m input variables, u, that affect the state vari-
ables. Historically control engineers further subdivided the input variables into those they could
easily and consciously adjust (known as control or manipulated variable inputs), and those vari-
ables that may change, but are outside the engineer’s immediate sphere of influence (like the
weather) known as disturbance variables. The number of input variables need not be the same as
the number of state variables, and indeed m is typically less than n.
46 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Many mathematical control text books follow a standard nomenclature. Vectors are written using
a lower case bold font such as x, and matrices are written in upper case bold such as A.

The state space equations are the n ordinary differential equations that relate the state derivatives
to the state themselves, the inputs if any, and time. We can write these equations as
ẋ = f (x, u, t) (2.34)

where f (·) is a vector function meaning that both the arguments and the result are vectors. In
control applications, the input is given by a control law which is often itself a function of the
states,
u = h(x) (2.35)
which if we substitute into Eqn. 2.34, we get the closed loop response
ẋ = f (x, h(x), t)
For autonomous closed loop systems there is no explicit dependence on time, so the differential
equation is simply
ẋ = f (x) (2.36)
For the continuous linear time invariant case, Eqn. 2.34 simplifies to
ẋ = Ax + Bu (2.37)
or in discrete form at time t = kT
xk+1 = Φxk + ∆uk (2.38)
where the state vector x is an (n × 1) vector, or alternatively written as x ∈ ℜn , the control input
vector has m elements, or u ∈ ℜm . The model (transition) matrix is (n × n) or A, Φ ∈ ℜn×n , and
the control or input matrix is (n × m) or B, ∆ ∈ ℜn×m .

Block diagrams of both the continuous and discrete formulations of the state-space model are
shown in Fig. 2.19 and both are conceptually very similar. Such a form is suitable to implement
state-space systems in a simulator such as S IMULINK for example. Fig. 2.19 also introduces a
convention we will use later in system identification, where I colour the input signals blue, and
the states red.

ẋ R x(t) xk xk+1
u(t) ✲ B ✲+ ✲ ✲ uk ✲ ∆ ✲+ ✲ z −1 ✲
✻ ✻

✛ A ✛ ✛ Φ ✛

Continuous plant Discrete plant

Figure 2.19: A block diagram of a state-space dynamic system, (a) continuous system: ẋ = Ax +
Bu, and (b) discrete system: xk+1 = Φxk + ∆uk . (See also Fig. 2.20.)

The output or measurement vector, y, are the variables that are directly measured from the operat-
ing plant, since often the true states cannot themselves be directly measured. These outputs are
related to the states by
y = g(x) (2.39)
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 47

If the measurement relation is linear, then

y = Cx (2.40)

where the r element measurement vector is ℜr , and the measurement matrix is sized C ∈ ℜr×n .

Sometimes the input may directly affect the output bypassing the process, in which case the full
linear system in state space is described by,

ẋ = Ax + Bu
(2.41)
y = Cx + Du

Eqn. 2.41 is the standard starting point for the analysis of continuous linear dynamic systems, and
are shown in block diagram form in Fig. 2.20. Note that in the diagram, the internal state vector
(bold lines), has typically more elements than either the input or output vectors (thin lines). The
diagram also highlights the fact that as an observer to the plant, we can relatively easily measure
our outputs (or appropriately called measurements), we presumably know our inputs to the
plant, but we do not necessarily have access to the internal state variables since they are always
contained inside the dashed box in Fig. 2.20. Strategies to estimate these hidden internal states is
known as state estimation and are described in section 9.5.

✲ D

ẋ R x(t) ❄
u(t) ✲✲ B ✲+ ✲ ✲ C ✲+ ✲ y(t)
✻

✛ A ✛

Figure 2.20: A complete block diagram of a continuous state-space dynamic system with outputs
and direct measurement feed-through, Eqn. 2.41.

2.7.2 C ONVERTING DIFFERENTIAL EQUATIONS TO STATE - SPACE FORM

If we have a collection or system of interlinked differential equations, it is often convenient and

succinct to group them together in a vector/matrix collection. Alternatively if we start with an
nth order differential equation, for the same reasons it is advisable to rewrite this as a collection
of n first order differential equations known as Cauchy form.

Given a general linear nth order differential equation

y (n) + a1 y (n−1) + a2 y (n−2) + · · · + an−1 ẏ + an y = b0 u(n) + b1 u(n−1) + · · · + bn−1 u̇ + bn u (2.42)

by inspection, we can create the equivalent transfer function as

Y (s) b0 sn + b1 sn−1 + · · · + bn−1 s + bn

= n (2.43)
U (s) s + a1 sn−1 + · · · + an−1 s + an
48 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

We can cast this transfer function (or alternatively the original differential equation) into state
space in a number of equivalent forms. They are equivalent in the sense that the input/output
behaviour is identical, but not necessarily the internal state behaviour.

The controllable canonical form is

      
ẋ1 0 1 0 ··· 0 x1 0
 ẋ2   0
   0 1 ··· 0 


 x2  
  0 

 ..   .. .. .. .. ..   ..   .. 
 . = . . . . .   . + . u (2.44)
       
 ẋn−1   0 0 0 ··· 1   xn−1   0 
ẋn −a0 −a1 −a2 · · · −an−1 xn 1
 
x1
h

i x2 

.. .. .  .. 
y= b n − an b 0 . bn−1 − an−1 b0 . · · · .. b1 − a1 b0 
 .  + b0 u

(2.45)
 xn−1 
xn
which is useful when designing pole-placement controllers. The observable canonical form is
      
ẋ1 0 0 · · · 0 −an x1 b n − an b 0
 ẋ2   1 0 · · · 0 −an−1   x2   bn−1 − an−1 b0 
      
 ..  =  .. . . .. .. ..   ..  +  .. u (2.46)
 .   . . . . .   .   . 
ẋn 0 0 · · · 1 −a1 xn b 1 − a1 b 0
 
x1
 x2 
 
y = 0 0 · · · 0 1  ...  + b0 u
  
(2.47)
 
 xn−1 
xn
and is useful when designing observers.

If the transfer function defined by Eqn. 2.43 has real and distinct factors,
Y (s) b0 sn + b1 sn−1 + · · · + bn−1 s + bn
=
U (s) (s + p1 )(s + p2 ) · · · (s + pn )
c1 c2 cn
= b0 + + + ···+
s + p1 s + p2 s + pn
we can derive an especially elegant state-space form
      
ẋ1 −p1 0 x1 1
 ẋ2  
   −p 2 


 x2  
  1 

 ..  =  ..   .. + .. u (2.48)
 .   .   .   . 
ẋn 0 −pn xn 1
 
x1
 x2

 

y= c1 c2 · · · cn  ..  + b0 u (2.49)
 . 
xn
that is purely diagonal. As evident from the diagonal system matrix, this system is totally decou-
pled and possess the best numerical properties for simulation. For systems with repeated roots,
or complex roots, or both, the closest we can come to a diagonal form is the block Jordan form.

We can interconvert between all these above forms using the M ATLAB canon command.
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 49

>> G = tf([5 6],[1 2 3 4]); % Define transfer function G = (5s2 + 6s)/(s3 + 2s2 + 3s + 4)
2

>> canon(ss(G),'compan') % Convert TF to the companion or observable canonical form

a =
x1 x2 x3
7 x1 0 0 -4
x2 1 0 -3
x3 0 1 -2

b =
12 u1
x1 1
x2 0
x3 0

17 c =
x1 x2 x3
y1 0 5 -4

d =
22 u1
y1 0

These transformations are discussed further in [150, p515]. M ATLAB uses a variation of this form
when converting from a transfer function to state-space in the tf2ss function.

2.7.3 I NTERCONVERTING BETWEEN STATE SPACE AND TRANSFER FUNCTIONS

The transfer function form and the state-space form are, by in large, equivalent, and we can con-
vert from one representation to the other. Starting with our generic state-space model, Eqn. 2.37,

ẋ = Ax + Bu (2.50)

with initial condition x(0) = 0 and taking Laplace transforms gives

sx(s) = Ax(s) + Bu(s)

x(s) (sI − A) = Bu(s)
x(s) = (sI − A)−1 B u(s) (2.51)
| {z }
Gp (s)

where Gp (s) is a matrix of expressions in s and is referred to as the transfer function matrix. The
M ATLAB command ss2tf (state-space to transfer function) performs this conversion.

State-space to transfer function conversion

Suppose we wish to convert the following state-space description to transfer function form.
   
−7 10 4 −1
ẋ = x+ u (2.52)
−3 4 2 −3
50 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

From Eqn. 2.51, the transfer function matrix is defined as

Gp (s) = (sI − A)−1 B

    −1  
1 0 −7 10 4 −1
= s −
0 1 −3 4 2 −3
 −1  
(s + 7) −10 4 −1
= (2.53)
3 (s − 4) 2 −3

Now at this point, the matrix inversion in Eqn. 2.53 becomes involved because of the presence of
the symbolic s in the matrix. Inverting the symbolic matrix expression gives the transfer function
matrix.
  
1 s − 4 −10 4 −1
Gp (s) =
(s + 2)(s + 1) −3 s + 7 2 −3
 
4 s + 26
− 2
 s+2 s + 3s + 2 
= 2

3(6 + s) 
− 2
s+2 s + 3s + 2
We can directly apply Eqn. 2.51 using the symbolic toolbox in M ATLAB.

>> syms s
2 >> A = [-7 10; -3 4]; B = [4 -1; 2 -3];
>> G = (s*eye(2)-A)\B % Pulse transfer function Gp (s) = (sI − A)−1 B.
G =
[ 4/(s+2), -(s+26)/(2+sˆ2+3*s)]
[ 2/(s+2), -3*(s+6)/(2+sˆ2+3*s)]

The method for converting from a state space description to a transfer function matrix described
by Eqn. 2.51 is not very suitable for numerical computation owing to the symbolic matrix in-
version required. However [176, p35] describes a method due to Faddeeva that is suitable for
numerical computation in M ATLAB.

Faddeeva’s algorithm to convert a state-space description, ẋ = Ax + Bu into transfer function

form, Gp (s).

1. Compute a, the characteristic equation of the n × n matrix A. (Use poly in M ATLAB.)

2. Set En−1 = I.
3. Compute recursively the following n − 1 matrices

En−1−k = AEn−k + an−k I, k = 1, 2, · · · , n − 1

4. The transfer function matrix is then given by

sn−1 En−1 + sn−2 En−2 + · · · + E0

Gp (s) = B (2.54)
an sn + an−1 sn−1 + · · · + a1 s + a0

We can repeat the state-space to transfer function example, Eqn. 2.52,

     
ẋ1 −7 10 4 −1
= x+ u
ẋ2 −3 4 2 −3
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 51

given on page 49 using Faddeeva’s algorithm. We start by computing the characteristic polyno-
mial of A
a(s) = s2 + 3s + 2 = (s + 1)(s + 2)
and with n = 2 we can compute
   
1 0 5 −1
E1 = , E0 =
0 1 2 2

Following Eqn. 2.54 gives a matrix of polynomials in s,

    
−1 1 1 0 −4 10
(sI − A) = 2 s +
s + 3s + 2 0 1 −3 7
 
1 s−4 10
=
(s + 1)(s + 2) −3 s+7

which we can recognise as the same expression as we found using the symbolic matrix inverse
on page 50. All that remains is to post multiply by B to obtain the pulse transfer function matrix.

We can implement Faddeeva’s algorithm using the symbolic toolbox as

syms s
a = poly(A); n = size(A,1);
as = poly2sym(a,'s'); % Construct characteristic polynomial, a(s).
4 E = eye(n,n); I = eye(n,n);
Gp = sˆ(n-1)*E; % En−1 = I
for i=n-2:-1:0
E = A*E + a(i+2)*I;
Gp = Gp + sˆi*E;
9 end
Gp = simplify(Gp*B/as); % Eqn. 2.54.

Finally, we can also use the control toolbox for the conversion from state-space to transfer func-
tion form. First we construct the state-space form,

A = [-7 10; -3,4]; B = [4 -1; 2 -3]; % Continuous state-space example from Eqn. 2.52.
C = eye(size(A));
sys = ss(A,B,C,[]) % construct system object

where in this case if leave out the D matrix, M ATLAB assumes no direct feed though path. We
now can convert to the transfer function form:

>> sys_tf = minreal(tf(sys)) % Convert to transfer function & remember to cancel common
factors
2

Transfer function from input 1 to output...

4
#1: -----
s + 2
7

2
#2: -----
s + 2

12 Transfer function from input 2 to output...

52 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

-s - 26
#1: -------------
sˆ2 + 3 s + 2

17 -3 s - 18
#2: -------------
sˆ2 + 3 s + 2

Once again we get the same pulse transfer function matrix. Note however that it is also possible
to use ss2tf.

T RANSFER FUNCTION TO STATE SPACE

To convert in the reverse direction, we can use the M ATLAB function tf2ss (transfer function to
state space) thus converting an arbitrary transfer function description to the state space format of
Eqn 2.37. For example starting with the transfer function

(s + 3)(s + 4) s2 + 7s + 12
G(s) = = 3
(s + 1)(s + 2)(s + 5) s + 8s + 17s + 10
we can convert to state-space form using tf2ss.

1 >>num = [1 7 12]; % Numerator: s2 + 7s + 12

>>den = [1 8 17 10]; % Denominator: s3 + 8s + 17s + 10
>>[A,B,C,D] = tf2ss(num,den) % convert to state-space

which returns the following four matrices

  
−8 −17 −10 1
A= 1 0 0 , B= 0 
0 1 0 0
 
C = 1 7 12 , D=0

This form is a variation of the controllable canonical form described previously in §2.7.2. This
is not the only state-space realisation possible however as the C ONTROL T OOLBOX will return a
slightly different ABCD package

>> Gc = tf([1 7 12],[1 8 17 10]) % Transfer function in polynomial form

2 Transfer function:
sˆ2 + 7 s + 12
-----------------------
sˆ3 + 8 sˆ2 + 17 s + 10

7 >>[A,B,C,D] = ssdata(Gc) % extract state-space matrices

A =
-8.0000 -4.2500 -1.2500
4.0000 0 0
0 2.0000 0
12 B =
2
0
0
C =
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 53

17 0.5000 0.8750 0.7500

D =
0

Alternatively we could start with zero-pole-gain form and obtain yet another equivalent state-
space form.

1 G = zpk([-3 -4],[-1 -2 -5],1) % Transfer function model in factored format

Gss = ss(G)

A later section, (§2.7.4) shows how we can convert between equivalent dynamic forms. The four
matrices in the above description form the linear dynamic system as given in Eqn. 2.41. We can
concatenate these four matrices into one large block thus obtaining a shorthand way of storing
these equations

# of inputs
n m
✛ ✲✛ ✲
✻
A B n # of states
G= (n × n) (n × m)
❄
C D ✻p # of measurements
(p × n) (p × m) ❄

which is often called a packed matrix notation of the quadruplet (A,B,C,D). If you get confused
about the appropriate dimensions of A, B etc, then this is a handy check, or alternatively you
can use the diagnostic routine abcdchk. Note that in practice, the D matrix is normally the
zero matrix since the input does not generally effect the output immediately without first passing
through the system dynamics.

2.7.4 S IMILARITY TRANSFORMATIONS

The state space description of a differential equation is not unique. We can transform from one
description to another by using a linear invertible transformation such as x = Tz. Geometrically
in 2 dimensions, this is equivalent to rotating the axes or plane. When one rotates the axes, the
inter-relationship between the states do not change, so the transformation preserves the dynamic
model.

Suppose we have the dynamic system

ẋ = Ax + Bu (2.55)

which we wish to transform in some manner using the non-singular transformation matrix, T,
where x = Tz. Naturally the reverse transformation z = T−1 x exists because we have restricted
ourselves to consider only the cases when the inverse of T exists. Writing Eqn. 2.55 in terms of
54 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

our new variable z we get

d(Tz)
= ATz + Bu
dt
ż = T−1 ATz + T−1 Bu (2.56)
Eqn 2.56 and Eqn 2.55 represent the same dynamic system. They have the same eigenvalues
hence the similarity transform, but just a different viewpoint. The mapping from A to T−1 AT
is called a similarity transform and preserves the eigenvalues. These two matrices are said to be
similar. The proof of this is detailed in [87, p300] and [150, p513–514].

The usefulness of these types of transformations is that the dynamics of the states are preserved
(since the eigenvalues are the same), but the shape and structure of the system has changed.
The motivation is that for certain operations (control, estimation, modelling), different shapes
are more convenient. A pure, (or nearly), diagonal shape of the A matrix for example has much
better numerical properties than a full matrix. This also has the advantage that less computer
storage and fewer operations are needed.

To convert a system to a diagonal form, we use the transformation matrix T, where the columns
of T are the eigenvectors of A. Systems where the model (A) matrix is diagonal are especially
easy to manipulate. We can find the eigenvectors T and the eigenvalues e of a matrix A using
the eig command, and construct the new transformed, and hopefully diagonal system matrix,

[T,e] = eig(A) % find eigenvectors & eigenvalues of A

V = T\A*T % New system matrix, T−1 AT, or use canon

Another use is when testing new control or estimation algorithms, it is sometimes instructive to
devise non-trivial systems with specified properties. For example you may wish to use as an
example a 3 × 3 system that is stable and interacting, and has one over-damped mode and two
oscillatory modes. That is we wish to construct a full A matrix with specified eigenvalues. We
can use the similarity transformations to obtain these models.

Other useful transformations such as the controllable and observable canonical forms are covered
in [150, §6–4 p646 ]. The M ATLAB function canon can convert state-space models to diagonal
or observable canonical form (sometimes known as the companion form). Note however the
help file for this routine discourages the use of the companion form due to its numerical ill-
conditioning.

M ATLAB comes with some utility functions to generate test models such as ord2 which gener-
ates stable second order models of a specified damping and natural frequency, and rmodel is a
flexible general stable random model builder or arbitrary order.

2.7.5 I NTERCONVERTING BETWEEN TRANSFER FUNCTIONS FORMS

The previous section described how to convert between different representations of linear dy-
namic systems such as differential equations, transfer functions and state-space descriptions.
This section describes the much simpler task of converting between the different ways we can
write transfer functions.

Modellers tend to think in continuous time systems, G(s), and in terms of process gain and time
constants, so will naturally construct transfer functions in factored form as
K Πi (αi s + 1)
(2.57)
Πj (τj s + 1)
2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS 55

where the all the variables of interest such as time constants τj are immediately apparent. On
the other hand, system engineers tend to think in terms of poles and zeros, so naturally construct
transfer functions also in factored form, but with a different scaling

K ′ Πi (s + zi )
(2.58)
Πj (s + pi )

so where now the poles, pj , and zeros zi are immediately apparent. This is the form that M ATLAB
uses in the zeros-pole-gain format, zpk.

Finally the hardware engineer would prefer to operate in the expanded polynomial form, (par-
ticularly in discrete cases), where the transfer function is of the form

bm sm + bm−1 sm−1 + · · · + b0
(2.59)
sn + an−1 sn−1 + · · · + a0

This is the form that M ATLAB uses in the transfer-function format, tf. Note that the leading
coefficient in the denominator is set to 1. As a M ATLAB user, you can define a transfer function
that does not follow this convention, but M ATLAB will quietly convert to this normalised form if
you type something like G = tf(zpk(G)).

The inter-conversions between the forms is not difficult; between expressions Eqn. 2.57 and
Eqn. 2.58 simply require some adjusting of the gains and factors, while to convert from Eqn. 2.59
requires one to factorise polynomials.

For example, the following three transfer function descriptions are all equivalent

2 (10s + 1)(−3s + 1) −1.5 (s − 0.3333)(s + 0.1) −60s2 + 14s + 2

= =
(20s + 1)(2s + 1)(s + 1) (s + 1)(s + 0.5)(s + 0.05) 40s3 + 62s2 + 23s + 1
| {z } | {z } | {z }
time constant zero-pole-gain expanded polynomial

It is trivial to interconvert between the zero-pole-gain, Eqn. 2.58, and the transfer function for-
mats, Eqn. 2.59, in M ATLAB, but it is less easy to convert to the time constant description. List-
ing 2.3 extracts from an arbitrary transfer function form the time constants, τ , the numerator time
constants, α, and the plant gain K.

Listing 2.3: Extracting the gain, time constants and numerator time constants from an arbitrary
transfer function format
Gplant = tf(2*mconv([10 1],[-3 1]),mconv([20 1],[2 1],[1 1])) % TF of interest
G = zpk(Gplant); % Convert TF description to zero-pole-gain
3

Kp = G.k;
p = cell2mat(G.p); z = cell2mat(G.z); % Extract poles & zeros
delay0 = G.iodelay ; % Extract deadtime (if any)

8 tau = sort(-1./p,'descend'); % Convert (& sort) to time constants, 1/(τj s + 1)

ntau = sort(-1./z,'descend'); % Convert to numerator time constants, (αi s + 1)
K = Kp*prod(tau)/prod(ntau); % Adjust plant gain

We could of course use the control toolbox functions pole and zero to extract the poles and
zeros from an arbitrary LTI model.
56 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

2.7.6 T HE STEADY STATE

The steady-state, xss , of a general nonlinear system ẋ = f (x, u) is the point in state space such
that all the derivatives are zero, or the solution of

0 = f (xss , u) (2.60)

If the system is linear, ẋ = Ax+ Bu, then the steady state can be evaluated algebraically in closed
form
xss = −A−1 Bu (2.61)

Consequently to solve for the steady-state one could invert the model matrix A but which may be
illconditioned or computationally time consuming. Using M ATLAB one should use xss = -A\B*u.

If A has no inverse, then no (or alternatively infinite) steady states exist. An example of a process
that has no steady state could be a tank-flow system that has a pump withdrawing fluid from
the outlet at a constant rate independent of liquid height say just exactly balancing an input flow
shown in the left hand schematic in Fig. 2.21. If the input flow suddenly increased, then the level
will rise until the tank eventually overflows. If instead the tank was drained by a valve partially
open at a constant value, then as the level rises, the increased pressure (head) will force more
material out through the valve, (right-hand side of Fig. 2.21.) Eventually the system will rise to a
new steady state. It may however overflow before the new steady state is reached, but that is a
constraint on the physical system that is outside the scope of the simple mathematical description
used at this time.

flow in

❄
❄
✻
✻
h h
❄ ❄

✲ flow = constant ✲ flow = f(height)

constant flow pump restriction

Figure 2.21: Unsteady and steady states for level systems

If the system is nonlinear, there is the possibility that multiple steady states may exist. To solve
for the steady state of a nonlinear system, one must use a nonlinear algebraic solver such as
described in chapter 3.

Example The steady state of the differential equation

d2 y dy
2
+7 + 12y = 3u
dt dt
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 57

where dydt = y = 0 at t = 0 and u = 1 , t ≥ 0 can be evaluated using Laplace transforms and the
final value theorem. Transforming to Laplace transforms we get

s2 Y (s) + 7sY (s) + 12Y (s) = 3U (s)

Y (s) 3 3
= 2 =
U (s) s + 7s + 12 (s + 3)(s + 4)

while for a step input

1 3
Y (s) = ·
s s2 + 7s + 12
The final value theorem is only applicable if the system is stable. To check, we require that the
roots of the denominator, s2 + 7s + 12, lie in the left hand plane,
√
−6 ± 72 − 4 × 12
s= = −4 and − 3
2
Given that both roots have negative real parts, we have verified that our system is stable and we
are allowed to apply the final value theorem to solve for the steady-state, y(∞),
3 3
lim y(t) = lim sY (s) = lim = = 0.25
t→∞ s→0 s→0 s2 + 7s + 12 12
Using the state space approach to replicate the above, we first cast the second order differential
equation into two first order differential equations using the controllable canonical form given in
Eqn. 2.45. Let z1 = y and z2 = dydt , then
   
0 1 0
ż = z+ u
−12 −7 3

and the steady state is now

    
−1 −7/12 −1/12 0 0.25
zss = −A Bu = − 1=
1 0 3 0

Noting that z1 = y, we see that the steady state is also at 0.25. Furthermore, the derivative term
(z2 ) is zero, which is as expected at steady state.

2.8 S OLVING THE VECTOR DIFFERENTIAL EQUATION

Since we can solve differential equations by inverting Laplace transforms, we would expect to be
able to solve state-space differential equations such as Eqn. 2.37 in a similar manner. If we look
at the Laplace transform of a simple linear scalar differential equation, ẋ = ax + bu we find two
terms,

sX(s) − x0 = aX(s) + bU (s)

x0 b
X(s) = + U (s) (2.62)
s−a s−a
One of these terms is the response of the system owing to the initial condition, x0 eat , and is
called the homogeneous solution, and the other term is due to the particular input we happen
to be using. This is called the particular integral, and we must know the form of the input, u(t),
before we solve this part of the problem. The total solution is the sum of the homogeneous and
particular components.
58 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

T HE HOMOGENEOUS SOLUTION

For the moment we will consider just the homogeneous solution to our vector differential equa-
tion. That is, we will assume no driving input, or u(t) = 0. (In the following section we will add
in the particular integral due to a non-zero input.)

Our vector differential equation, ignoring any input, is simply

ẋ = Ax, x(t = 0) = x0 (2.63)

Taking Laplace transforms, and not forgetting the initial conditions, we have

sx(s) − x0 = A x(s)
x(s) (sI − A) = x0 (s)
−1
x(s) = (sI − A) x0

Finally inverting the Laplace transform back to the time domain gives
n o
−1
x(t) = L−1 (sI − A) x0 (2.64)

Alternatively we can solve Eqn. 2.63 by separating the variables and integrating obtaining

x(t) = eAt x0 (2.65)

where the exponent of a matrix, eAt , is itself a matrix of the same size as A. We call this matrix
exponential the transition matrix because it transforms the state vector at some initial time x0 to
some point in the future, xt . We will give it the symbol Φ(t).

The matrix exponential is defined just as in the scalar case as a Taylor series expansion,

A2 t2 A3 t3
eAt or Φ(t) = I + At +
def
+ + ···
2! 3!
∞
X Ak tk
= (2.66)
k!
k=0

although this series expansion method is not recommended as a reliable computational strategy.
Better strategies are outlined on page 62.
n o
Comparing Eqn. 2.64 with Eqn. 2.65 we can see that the matrix eAt = L−1 (sI − A)
−1
.

So to compute the solution, x(t), we need to know the initial condition and a strategy to numeri-
cally compute a matrix exponential.

T HE PARTICULAR SOLUTION

Now we consider the full differential equation with nonzero input ẋ = Ax + Bu. Building on
the solution to the homogeneous part in Eqn. 2.65, we get
Z t
x(t) = Φt x0 + Φt−τ Buτ dτ (2.67)
0

where now the second term accounts for the particular input vector u.
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 59

Eqn. 2.67 is not particularly useful as written as both terms are time varying. However the con-
tinuous time differential equation can be converted to a discrete time differential equation that
is suitable for computer control implementation provided the sampling rate is fixed, and the in-
put is held constant between the sampling intervals. We would like to convert Eqn. 2.37 to the
discrete time equivalent, Eqn. 2.38, repeated here

xk+1 = Φxk + ∆uk (2.68)

where xk is the state vector x at time t = kT where T is the sample period. Once the sample pe-
riod is fixed, then Φ and ∆ are also constant matrices. We have also assumed here that the input
vector u is constant, or held, over the sample interval, which is the norm for control applications.

So starting with a known xk at time t = kT , we desire the state vector at the next sample time,
xk+1 , or
Z (k+1)T
A T
xk+1 = e xk + e A (k+1)T
e−Aτ Buτ dt (2.69)
kT

But as we have assumed that the input u is constant using a zeroth-order hold between the
sampling intervals kT and (k + 1)T , Eqn. 2.69 simplifies to
Z T
xk+1 = eAT xk + eAλ Buk dλ (2.70)
0

where λ = T − t. For convenience, we can define two new matrices as

Φ = eAT
def
(2.71)
Z T !
∆=
def
e Aλ
dλ B (2.72)
0

which gives us our desired transformation in the form of Eqn. 2.68. In summary, to discretise
ẋ = Ax + Bu at sample interval T , we must compute a matrix exponential, Eqn. 2.71, and
integrate a matrix exponential, Eqn. 2.72.

Note that Eqn. 2.70 involves no approximation to the continuous differential equation provided
the input is constant over the sampling interval. Also note that as the sample time tends to zero,
the state transition matrix Φ tends to the identity, I.

If the matrix A is non-singular, then

 
∆ = eAT − I A−1 B (2.73)

is a simpler expression than the more general Eqn. 2.72.

Double integrator example. The single-input/single output double integrator system, G(s) =
1/s2 , can be represented in continuous state space using two states as
   
0 0 1
ẋ = x+ u (2.74)
1 0 0
 
y= 0 1 x (2.75)

At sample time T , the state transition matrix is from Eqn. 2.71,

   
0 0 1 0
Φ = eAT = exp =
T 0 T 1
60 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

and the control matrix is given by Eqn. 2.72

Z !
T
∆= e Aλ
dλ B
0
Z T   ! 
1 0 1
= dλ
0 λ 1 0
   
T 0  1  T
=  T2  =  T2 
T 0
2 2
For small problems such as this, the symbolic toolbox helps do the computation.

>> A = [0 0; 1 0]; B = [1; 0];

>> syms T lambda % Symbolic sample time T and λ
>> Phi = expm(A*T) % Φ(T )
Phi =
5 [ 1, 0]
[ T, 1]
>> Delta = int(expm(A*lambda),lambda,0,T)*B % ∆(T )
Delta =
[ T]
10 [ 1/2*Tˆ2]

Symbolic example with invertible A matrix. We can discretise the continuous state-space sys-
tem    
−1.5 −0.5 1
ẋ = x+ u
1 0 0
analytically at a sample time T by computing matrix exponentials symbolically.

>> A = [-3/2, -1/2; 1, 0]; B = [1;0];

>> syms T
>> Phi = expm(A*T)
5 Phi =
[ -exp(-1/2*T)+2*exp(-T), exp(-T)-exp(-1/2*T)]
[ -2*exp(-T)+2*exp(-1/2*T), 2*exp(-1/2*T)-exp(-T)]

Since A is invertible in this example, we can use the simpler Eqn. 2.73

 
>> Delta = (Phi - eye(2))*A\B % ∆ = eAT − I A−1 B
Delta =
3 [ 2/(exp(-1/2*T)*exp(-T)-exp(-T)-exp(-1/2*T)+1)*(exp(-T)-exp(-1/2*T))]
[ -2*(-exp(-1/2*T)+2*exp(-T)-1)/(exp(-1/2*T)*exp(-T)-exp(-T)-exp(-1/2*T)+1)]
>> simplify(Delta)
ans =
[ 2*exp(-1/2*T)/(exp(-T)-1)]
8 [ -2*(2*exp(-1/2*T)+1)/(exp(-T)-1)]

Of course it is evident from the above example that the symbolic expressions for Φ(T ) and ∆(T )
rapidly become unwieldy for dimensions much larger than about 2. For this reason, analytical
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 61

expressions are of limited practical worth. The alternative numerical schemes are discussed in
the following section.

2.8.1 N UMERICALLY COMPUTING THE DISCRETE TRANSFORMATION

Calculating numerical values for the matrices Φ and ∆ can be done by hand for small dimensions
by converting to a diagonal or Jordan form, or numerically using the exponential of a matrix.
Manual calculations are neither advisable nor enjoyable, but [19, p35] mention that if you first
compute
Z T
AT 2 A2 T 3 Ak−1 T k
Ψ= eAτ dτ = IT + + + ···+ + ···
0 2! 3! k!
then
Φ = I + AΨ and ∆ = ΨB (2.76)
A better approach, at least when using M ATLAB, follows from Eqn. 2.71 and Eqn. 2.72 where

dΦ
= ΦA = AΦ (2.77)
dt
d∆
= ΦB (2.78)
dt
These two equations can be concatenated to give
    
d Φ ∆ Φ ∆ A B
= (2.79)
dt 0 I 0 I 0 0
R R
which is in the same form as da/dt = ab. Rearranging this to da/a = b dt leads to the analytical
solution     
Φ ∆ A B
= exp T (2.80)
0 I 0 0
enabling us to extract the required Φ and ∆ matrices provided we can reliably compute the
exponential of a matrix. The M ATLAB C ONTROL T OOLBOX routine to convert from continuous
to discrete systems, c2d, essentially follows this algorithm.

We could try the augmented version of Eqn. 2.80 to compute both Φ and ∆ with one call to the
matrix exponential function for the example started on page 60.

>> A = [-3/2, -1/2; 1, 0]; B = [1;0];

2 >> syms T

>> [n,m] = size(B); % extract dimensions  

A B
>> Aa = [A,B;zeros(m,n+m)] % Augmented A matrix, Ã =
0 0
Aa =
7 -1.5 -0.5 1.0
1.0 0.0 0.0
0.0 0.0 0.0
>> Phi_a = expm(Aa*T) compute exponential, Φ̃ = exp(ÃT )
Phi_a =
12 [ -exp(-1/2*T)+2*exp(-T), exp(-T)-exp(-1/2*T), -2*exp(-T)+2*exp(-1/2*T)]
[-2*exp(-T)+2*exp(-1/2*T), 2*exp(-1/2*T)-exp(-T),2*exp(-T)-4*exp(-1/2*T)+2]
[ 0, 0, 1]
>> Phi = Phi_a(1:n,1:n)
Phi =
62 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

17 [ -exp(-1/2*T)+2*exp(-T), exp(-T)-exp(-1/2*T)]
[ -2*exp(-T)+2*exp(-1/2*T), 2*exp(-1/2*T)-exp(-T)]
>> Delta = Phi_a(1:n,n+1:end)
Delta =
[ -2*exp(-T)+2*exp(-1/2*T)]
22 [ 2*exp(-T)-4*exp(-1/2*T)+2]

R ELIABLY COMPUTING MATRIX EXPONENTIALS

There are in fact several ways to numerically compute a matrix exponential. Ogata gives three
computational techniques, [150, pp526–533], and a paper by Cleve Moler (one of the original
M ATLAB authors) and Charles Van Loan is titled Nineteen dubious ways to compute the exponential
of a matrix5 , and contrasts methods involving approximation theory, differential equations, matrix
eigenvalues and others. Of these 19 methods, two found their way into M ATLAB, namely expm,
which is the recommended default strategy, and expm1 intended to compute ex − 1 for small x .

The one time when matrix exponentials are trivial to compute is when the matrix is diagonal.
Physically this implies that the system is totally decoupled since the matrix A is comprised of
only diagonal elements and the corresponding exponential matrix is simply the exponent of the
individual elements. So given the diagonal matrix
 
λ1 0 0
D =  0 λ2 0 
0 0 λ3

then the matrix exponential is simply

 
eλ1 0 0
D
e =  0 eλ2 0 
0 0 eλ3

which is trivial and reliable to compute. So one strategy then is to transform our system to a
diagonal form (if possible), and then simply find the standard scalar exponential of the individual
elements. However some matrices, such as those with multiple eigenvalues, is is impossible to
convert to diagonal form, so in those cases the best we can do is convert the matrix to a Jordan
block form as described in [150, p527], perhaps using the jordan command from the Symbolic
toolbox.

However this transformation is very sensitive to numerical roundoff and for that reason is not
used for serious computation. For example the matrix
 
1 1
A=
ǫ 1

with ǫ = 0 has a Jordan form of  

1 1
0 1
but for ǫ 6= 0, then the Jordan form drastically changes to the diagonal matrix
 √ 
1+ ǫ 0√
0 1− ǫ
5 SIAM Review, vol. 20, A 1978, pp802–836 and reprinted in [159, pp649–680]. In fact 25 years later, an update was

published with some recent developments.

2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 63

In summary, for serious numerical calculation we should use the matrix exponential function
expm. Remember not to confuse finding the exponent of a matrix, expm, with the M ATLAB func-
tion exp which simply finds the exponent of all individual elements in the matrix.

2.8.2 U SING M ATLAB TO DISCRETISE SYSTEMS

All of the complications of discretising continuous systems to their discrete equivalent can be
circumvented by using the M ATLAB command c2d which is short for continuous to discrete.
Here we need only to pass the continuous system of interest, and the sampling time. As an
example we can verify the conversion of the double integrator system shown on page 2.8.

G = tf(1,[1 0 0]) % Continuous system G(s) = 1/s2 in transfer function form

Gc = ss(G) % Convert to continuous state-space
3 Gd = c2d(Gc,2) % Convert to discrete state-space with a sample time of T = 2

a =
x1 x2
x1 1 0
8 x2 2 1

b =
u1
x1 2
13 x2 2

c =
x1 x2
y1 0 1
18

d =
u1
y1 0

23 Sampling time (seconds): 2

Discrete-time state-space model.

Unlike in the analytical case presented previously, here we must specify a numerical value for
the sample time, T .

Example: Discretising an underdamped second order system with a sample time of T = 3

following Eqn. 2.80 using M ATLAB to compute the matrix exponential of the augmented system.

1 [A,B,C,D] = ord2(3, 0.5); % generate a second order system

Ts = 3.0; % sample time

[na,nb] = size(B);
X = expm([A,B; zeros(nb,na+nb)]*Ts); % matrix exponential
6 Phi = X(1:na,1:na); % Pull off blocks to obtain Φ & ∆.
Del = X(1:na,na+1:end);

We can use M ATLAB to numerically verify the expression found for the discretised double inte-
grator on page 59 at a specific sample time, say T = 4.
64 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

The double integrator in input/output

form is
1 >>Gc = tf(1,[1,0,0])% G(s) = 1/s2
G(s) = 2 >>Gc_ss = ss(Gc);% state space
s
3 >>dt = 4; % sample time
which in packed state space notation is
  >>Gd_ss = c2d(Gc_ss,dt)
  0 0 1 a =
A B
= 1 0 0  x1 x2
C D
0 1 0 8 x1 1.000 0
x2 4.000 1.000
Using a sample time of T = 4, we can com-
pute b =
    u1
1 0 1 0 13 x1 4.000
Φ= =
T 1 4 1 x2 8.000
   
T 4
∆ = T 2 = 16 c =
2 2 x1 x2
18 y1 0 1.000
We can verify this using c2d in M ATLAB as
demonstrated opposite. d =
u1
y1 0

Once the two system matrices Φ and ∆ are known, solving the difference equation for further
values of k just requires the repeated application of Eqn. 2.38. Starting from known initial condi-
tions x0 , the state vector x at each sample instant is calculated thus

x1 = Φx0 + ∆u0
x2 = Φx1 + ∆u1 = Φ2 x0 + Φ∆u0 + ∆u1
x3 = Φx2 + ∆u2 = Φ3 x0 + Φ2 ∆u0 + Φ∆u1 + ∆u2
.. ..
. .
n−1
X
xn = Φn x0 + Φn−1−k ∆uk (2.81)
k=0

The M ATLAB function ltitr which stands for linear time invariant time response or the more
general dlsim will solve the general linear discrete time model as in 2.81. In special cases such
as step or impulse tests, you can use dstep, or dimpulse.

Problem 2.4 1. Evaluate An where  

1 1
A=
1 0
for different values of n. What is so special about the elements of An ? (Hint: find the
eigenvalues of A.)
2. What is the determinant of An ?
3. Write a couple of M ATLAB m-functions to convert between the A, B, C and D form and the
packed matrix form.
4. Complete the state-space to transfer function conversion analytically started in §2.7.3,
Eqn. 2.53. Compare your answer with using ss2tf.
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 65

A submarine example A third-order model of a submarine from [63, p416 Ex9.17] is

   
0 1 0 0
ẋ =  −0.0071 −0.111 0.12  x +  −0.095  u (2.82)
0 0.07 −0.3 0.072
where the state vector x is defined as
 
x⊤ = θ dθ
dt α
The state θ is the inclination of the submarine and α is the angle of attack above the horizontal.
The scalar manipulated variable u is the deflection of the stern plane. We will assume that of the
three states, we can only actually measure two of them, θ and α, thus
 
1 0 0
C=
0 0 1
In this example, we consider just three states x, one manipulated variable u, and two outputs, y.

The stability of the open-loop system is given by the eigenvalues of A. In M ATLAB, the command
eig(A) returns  
−0.0383 + 0.07i
eig(A) =  −0.0383 − 0.07i 
−0.3343
showing that all eigenvalues have negative real parts, indicating that the submarine is oscillatory,
though stable. In addition, the complex conjugate pair indicates that there will be some oscillation
to the response of a step change in stern plane.

We can simulate the response of our submarine system (Eqn 2.82) to a step change in stern plane
movement, u, using the step command. The smooth plot in Fig. 2.22 shows the result of this
continuous simulation, while the ‘staired’ plot shows the result of the discrete simulation using
a sample time of T = 5. We see two curves corresponding to the two outputs.

Listing 2.4: Submarine simulation

A = [0,1,0;-0.0071,-0.111,0.12;0,0.07,-0.3]; B = [0,-0.095,0.072]';
C = [1 0 0;0 0 1];
3

Gc = ss(A,B,C,0) % Continuous plant, Gc (s)

dt = 5; % sample time T = 5 (rather coarse)

Gd = c2d(Gc,dt) % Create discrete model
8

step(Gc,Gd); % Do step response and see Fig. 2.22.

Fig. 2.22 affirms that the openloop process response is stable, supporting the eigenvalue analysis.
Notice how the step command automatically selected appropriate time scales for the simulation.
How did it do this?

The steady state of the system given that u = 1 is, using Eqn 2.61,

1 >> u = 1;
>>xss = -A\B*u; % Steady-states given by xss = −ABu.
>>yss = C*xss
yss = % See steady-state outputs in Fig. 2.22.
-9.3239
6 0.2400
66 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

0
G
c
G
−5 d

1
x
−10

−15

0.4

2 0.3

0.2
x

0.1
Figure 2.22: Comparing the continuous
0
and discrete step responses of the subma- 0 20 40 60 80
rine model. time

which corresponds to the final values given in Fig. 2.22.

We can verify the results from the c2d routine using Eqns 2.71 and 2.72, or in this case since A is
invertible, Eqn. 2.73.

>> Phi = expm(A*5.0); % Don’t forget the ‘m’ in expmatrix()

 
>> Delta = (Phi-eye(size(A)))*(A\B) % ∆ = eAT − I A−1 B

This script should give the same Φ and ∆ matrices as the c2d function since A is non-singular.

Given a discrete or continuous model, we can compute the response to an arbitrary input se-
quence using lsim:

t = [0:dt:100]'; % time vector

U = sin(t/20); % Arbitrary input U (t)
3

lsim(Gc,U,t) % continuous system simulation

lsim(Gd,U) % discrete system simulation

We could explicitly demand a first-order-hold as opposed to the default zeroth order by setting
the options for c2d.

optn = c2dOptions('Method','foh') % Ask for a first-order hold

Gd2 = c2d(Gc,5,optn) % Compare the step response of this with a zeroth-order hold

Problem 2.5 By using a suitable state transformation, convert the following openloop system

ẋ1 = x2
ẋ2 = 10 − 2x2 + u
2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION 67

where the input u is the following function of the reference r,

 
u = 10 + 9r − 9 4 x

into a closed loop form suitable for simulation in M ATLAB using the lsim function. (ie: ẋ =
Ax + Br) Write down the relevant M ATLAB code segment. (Note: In practice, it is advisable to
use lsim whenever possible for speed reasons.)

Problem 2.6 1. Write an m-file that returns the state space system in packed matrix form that
connects the two dynamic systems G1 and G2 together,

Gall = G1 · G2

Assume that G1 and G2 are already given in packed matrix form.

2. (Problem from [185, p126]) Show that the packed matrix form of the inverse of Eqn. 2.41,
u = G−1 y, is  
A − BD−1 C −BD−1
G−1 = (2.83)
D−1 C D−1
assuming D is invertible, and we have the same number of inputs as outputs.

2.8.3 T HE DISCRETE STATE SPACE EQUATION WITH TIME DELAY

The discrete state space equation, Eqn. 2.37, does not account for any time delay or dead time
between the input u and the output x. This makes it difficult to model systems with delay.
However, by introducing new variables we can accommodate dead time or time delays. Consider
the continuous differential equation,

ẋt = Axt + But−θ (2.84)

where θ is the dead time and in this particular case, is exactly equal to 2 sample times (θ = 2T ).
The discrete time equivalent to Eqn 2.84 for a two sample time delay is

xk+3 = Φxk+2 + ∆uk (2.85)

which is not quite in our standard state-space form of Eqn. 2.38 owing to the difference in time
subscripts between u and x.

Now let us introduce a new vector of state variables, z where

 
xk
def
zk =  xk+1  (2.86)
xk+2

is sometimes known as a shift register or tapped delay line of states. Using this new state vector,
we can write the dynamic system, Eqn. 2.85 compactly as
   
xk+1 Ixk+1
zk+1 =  xk+2  =  Ixk+2  (2.87)
xk+3 Φxk+2 + ∆uk

or in standard state-space form as

   
0 I 0 0
zk+1 = 0 0 I  z k +  0  uk (2.88)
0 0 Φ ∆
68 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

This augmented system, Eqn. 2.88 (with 2 units of dead time) is now larger than the original
Eqn. 2.85 given that we have 2n extra states. Furthermore, note that the new augmented transi-
tion matrix in Eqn. 2.88 is now no longer of full rank (provided it was in the first place). If we
wish to incorporate a delay θ, of β sample times (θ = βT ), then we must introduce β dummy
state vectors into the system creating a new state vector z with dimensions (nβ × 1);
 ⊤
zk = xk xk+1 xk+2 · · · xk+β

The augmented state transition matrix Ξ and augmented control matrix Ψ are of the form
   
0 I 0 ··· 0 0
 0 0 I ··· 0   0 
   
 0 0 0 ··· 0   0 
   
Ξ= . . . . , and Ψ =  0  (2.89)
 .. .. .. . . ... 
  
   .. 
 0 0 0 ··· I   . 
0 0 0 ··· Φ ∆

Now the discrete time equation is

zk+1 = Ξzk + Ψuk (2.90)
Since all the introduced states are unobservable, we have
 
xk = I 0 0 · · · 0 zk (2.91)

and that the output equation is amended to

 
yk = C I 0 0 ··· 0 zk (2.92)

If the dead time is not an exact multiple of the sample time, then more sophisticated analysis is
required. See [71, p174]. Systems with a large value of dead time become very unwieldy as a
dummy vector is required for each multiple of the sample time. This creates large systems with
many states that possibly become numerically ill-conditioned and difficult to manipulate.

Problem 2.7 1. Simulate the submarine example (Eqn 2.82) but now introducing a dead time
of 2 sample time units.
2. Explain how the functions step and dstep manage to guess appropriate sampling rates,
and simulation horizons. Under what circumstances will the heuristics fail?

2.9 S TABILITY

Stability is a most desirable characteristic of a control system. As a control designer, you should
only design control systems that are stable. For this reason, one must at least be able to analyse
a potential control system in order to see if it is indeed theoretically stable at least. Once the
controller has been implemented, it may be too late and costly to discover that the system was
actually unstable.

One definition of stability is as follows:

A system of differential equations is stable if the output is bounded for all

time for any given bounded input, otherwise it is unstable. This is referred to
as bounded input bounded output (BIBO) stability.
2.9. STABILITY 69

System Stability

✙ s

Transfer function Nonlinear

■ q
✙ Lyapunov
non-poly ✌
❫
✌ Marginal Linearise
Pade approx. simulate
yes/no eigenvalues
roots of denominator
Routh array poles
Jury test

Figure 2.23: Issues in assessing system stability

While for linear systems, the concept of stability is well defined and relatively straight forward
to evaluate6 , this is not the case for nonlinear systems which can exhibit complex behaviour. The
next two sections discuss the stability for linear systems. Section 2.9.4 discusses briefly techniques
that can be used for nonlinear systems.

Before any type of stability analysis can be carried out, it is important to define clearly what sort
of analysis is desired. For example is a ‘Yes/No’ result acceptable, or do you want to quantify
how close to instability the current operating point is? Is the system linear? Are the nonlinearities
‘hard’ or ‘soft’? Fig. 2.23 highlights some of these concerns.

2.9.1 S TABILITY IN THE CONTINUOUS DOMAIN

The most important requirement for any controller is that the controller is stable. Here we de-
scribe how one can analyse a continuous transfer function to determine whether it is stable or
not.

First recall the conditions for BIBO stability. In the Laplace domain, the transfer function system
is stable if all the poles (roots of the denominator) have negative real parts. Thus if the roots of the
denominator are plotted on the complex plane, then they must all lie to the left of the imaginary
axis or in the Left Hand Plane (LHP). In other words, the time domain solution of the differential
equation must contain only e−at terms and no terms of the form e+at , assuming a is positive.

Why is the stability only determined by the denominator of the transfer function and not by the
numerator or the input?

1. First the input is assumed bounded and stable (by BIBO definition).

6 Actually, the stability criterion can be divided into 3 categories; Stable, unstable and critically (un)stable. Physically

critical stability never really occurs in practice.

70 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

2. If the transfer function is expanded as a sum of partial fractions, then the time solution
terms will be comprised of elements that are summed together. For the system to be un-
stable, at least one of these individual terms must be unstable. Conversely if all the terms
are stable themselves, then the summation of these terms will also be stable. The input will
also contribute separate fractions that are summed, but these are assumed to be stable by
definition, (part 1).

Note that it is assumed for the purposes for this analysis, that the transfer function is written in
its simplest form, that it is physically realisable, and that any time delays are expanded as a Padé
approximation.

In summary to establish the stability of a transfer function, we could factorise the denomina-
tor polynomial using a computer program to compute numerically the roots such as M ATLAB’s
roots routine. Alternatively, we can just hunt for the signs of the real part of the roots, a much
simpler operation, and this is the approach taken by the Routh and Jury tests described next.

R OUTH STABILITY CRITERION

The easiest way to establish the absolute stability of a transfer function is simply to extract the
roots of the denominator perhaps using M ATLAB ’ S root command. In fact this is an overkill
since to establish absolute stability we need only to know the presence of any roots with positive
real parts, not the actual value.

Aside from efficiency, there are two cases where this strategy falls down. One is where we do not
have access to M ATLAB or the polynomial is particularly ill-conditioned such that simple root
extraction techniques fail for numerical reasons, and the other case is where we may have a free
parameter such as a controller gain in the transfer function. This has prompted the development
of simpler exact algebraic methods such as the Routh array or Lyapunov’s method to assess
stability.

The Routh criterion states that all the roots of the polynomial characteristic equation,

an sn + an−1 sn−1 + an−2 sn−2 + an−3 sn−3 + · · · + a1 s + a0 = 0 (2.93)

have negative real parts if and only if the first column of the Routh table (as described below)
have the same sign. Otherwise the number of sign changes is equal to the number of right-hand
plane roots.

The first two rows of the Routh table is simply formed by listing the coefficients of the character-
istic polynomial, Eqn. 2.93, in the manner following
sn an an−2 an−4 · · ·
sn−1 an−1 an−3 an−5 · · ·
sn−2 b1 b2 b3 ···
sn−3 c1 c2 c3 ···
.. .. ..
. . .

and where the new entries b and c in the subsequent rows are defined as
an−1 an−2 − an an−3 an−1 an−4 − an an−5
b1 = , b2 = , etc.
an−1 an−1
b1 an−3 − an−1 b2 b1 an−5 − an−1 b3
c1 = , c2 = , etc.
b1 b1
The Routh table is continued until only zeros remain.
2.9. STABILITY 71

The Routh array is only applicable for continuous time systems, but for discrete time systems,
the Jury test can be used in a manner similar to the Routh table for continuous systems. The
construction of the Jury table is slightly more complicated than the Routh table and is described
in [61, 117–118].

The Routh array is most useful when investigating the stability as a function of a variable such as
controller gain. Rivera-Santos has written a M ATLAB routine to generate the Routh Array with
the possibility of including symbolic variables.7

Suppose we wish to determine the range of stability for a closed loop transfer function with
characteristic equation
s4 + 3s3 + 3s2 + 2s + K = 0
as a function of the unknown gain K.8

Listing 2.5: Example of the Routh array using the symbolic toolbox
>> syms K % Define symbolic gain K
>> ra = routh([1 3 3 2 K],K) % Build Routh array for s4 + 3s3 + 3s2 + 2s + K
3 ra =
[ 1, 3, K]
[ 3, 2, 0]
[ 7/3, K, 0]
[ -9/7*K+2, 0, 0]
8 [ K, 0, 0]

Since all the elements in the first column of the table must be positive, we know that for stability
−(9/7)K + 2 > 0 and that K > 0, or
0 < K < 14/7

S TABILITY AND TIME DELAYS

Time delay or dead time does not affect the stability of the open loop response since it does not
change the shape of the output curve, but only shifts it to the right. This is due to the non-
polynomial term e−sθ in the numerator of the transfer function. Hence dead time can be ignored
for stability considerations for the open loop. However in the closed loop the dead time term ap-
pears in both the numerator and the denominator and now does affect the stability characteristics
and can now no longer be ignored. Since dead time is a nonpolynomial term, we cannot simply
find the roots of the denominator, since now there are an infinite number of them, but instead we
must approximate the exponential term with a truncated polynomial such as a Padé approxima-
tion and then apply either a Routh array in the continuous time case, or a Jury test in the discrete
time case.

Note however that the Nyquist stability criteria yields exact results, even for those systems with
time delays. The drawback is that the computation is tedious without a computer.

Problem 2.8 Given

15(4s + 1)e−θs
G(s) =
(3s + 1)(7s + 1)
7 The routine, routh.m, is available from the MathWorks user’s group collection at
www.mathworks.com/matlabcentral/fileexchange/.
8 Problem adapted from [152, p288]
72 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Nyquist Diagram
3D Nyquist plot
3

4 2
10

2 1
10

Imaginary Axis
ω [rad/s]

0 0
10

−2 −1
10
2
−2
0
4
−2 2 −3
0 −1 0 1 2 3
ℑ(Gi ω)
−4 −2 ℜ(Gi ω) Real Axis

(a) The Nyquist diagram with frequency information (b) A 2D Nyquist diagram which shows the closed
presented in the third dimension. A ‘flagpole’ is loop is unstable since it encircles the (−1, 0i) point.
drawn at (−1, 0i). Since the curve encircles the flag- In this case both positive and negative frequencies are
pole, the closed loop is unstable. plotted.

Figure 2.24: Nyquist diagram of Eqn. 2.94 in (a) three dimensions and (b) as typically presented
in two dimensions.

Find the value of deadtime θ such that the closed-loop is just stable using both a Padé approxi-
mation and a Bode or Nyquist diagram.

A first order Padé approximation is

1 − 2θ s
e−θs =
1 + 2θ s

2.9.2 S TABILITY OF THE CLOSED LOOP

Prior to the days of cheap computers that can readily manipulate polynomial expressions and
extract the roots of polynomials, one could infer the stability of the closed loop from a Bode or
Nyquist diagram of the open loop.

The openloop system

100(s + 2)
G(s) = e−0.2s (2.94)
(s + 5)(s + 4)(s + 3)
is clearly stable, but the closed loop system, G/(1 + G) may, or may not be stable. If we substitute
s = iω, and compute the complex quantity G(iω) as a function of ω, we can apply either the Bode
stability criteria, or the equivalent Nyquist stability criteria to establish the stability of the closed
loop without deriving an expression for the closed loop, and subsequently solving for the closed
loop poles.

The Bode diagram consists of two plots; the magnitude of G(iω) versus ω, and the phase of
G(iω) versus ω. Alternatively we could plot the real and imaginary parts of G(iω) as a function
of frequency, but this results in a three dimensional plot. Such a plot for the system given in
Eqn. 2.94 is given in Fig. 2.24(a).

However three dimensional plots are difficult to manipulate, so we normally ignore the fre-
2.9. STABILITY 73

quency component, and just view the shadow of the plot on the real/imaginary plane. The
two dimensional Nyquist curve corresponding to Fig. 2.24(a) is given in Fig. 2.24(b). It is evident
from either plot that the curve does encircle the critical (0, −1i) point, so the closed loop system
will be unstable.

Establishing the closed loop stability using the Nyquist criteria is slightly more general than when
using the Bode criteria. This is because for systems that exhibit a non-monotonically decreasing
curves may cross the critical lines more than once leading to misleading results. An interesting
example from [81] is use to illustrate this potential problem.

2.9.3 S TABILITY OF DISCRETE TIME SYSTEMS

The definition of stability for a discrete time system is similar to that of the continuous time
system except that the definition is only concerned with the values of the output and input at the
sample times. Thus it is possible for a system to be stable at the sample points, but be unstable
between the sample points. This is called a hidden oscillation, although rarely occurs in practice.
Also note that the stability of a discrete time system is dependent on the sample time, T .

Recall that at sample time T , by definition z = exp(sT ), the poles of the continuous transfer
function pi map to exp(pi T ) in discrete space. If pi is negative, then exp(pi T ) will be less than
1.0. If pi is positive, then the corresponding discrete pole zi will be larger than 1.0. Strictly if pi
is complex, then it is the magnitude of zi that must be less than 1.0 for the system to be stable.
If the discrete poles zi , are plotted on the complex plane, then if they lie within a circle centered
about the origin with a radius of 1, then the system is stable. This circle of radius 1 is called the
unit circle.

The discrete time transfer function G(z) is stable if G(z) has no discrete poles
on, or outside, the unit circle.

Now the evaluation of stability procedure is the same as that for the continuous case. One simply
factorises the characteristic equation of the transfer function and inspects the roots. If any of the
roots zi lie outside the unit circle, the system will be unstable. If any of the poles lie exactly on
the unit circle, there can be some ambiguity about the stability, see [120, p120]. For example the
transfer function which has poles on the unit circle,

1
G(z) =
z2 + 1
given a bounded input such as a step, u(k) = {1, 1, 1, 1 . . .} produces

y(k) = {0, 0, 1, 1, 0, 0, 1, 1, 0, 0, . . .}

which is bounded and stable. However if we try a different bounded input such as a cycle u(k) =
{1, 0, −1, 0, 1, 0, −1, . . .} we observe

y(k) = {0, 0, 1, 0, −2, 0, 3, 0, −4, 0, 5, 0, −6, 0, . . .}

which is unbounded.

Just as in the continuous case, we can establish the absolute stability of a discrete transfer function
without explicitly solving for the roots of the denominator D(z) = 0, which, in the days before
computers was a main consideration. The main methods for discrete systems are the Jury test,
the bilinear transform coupled with the Routh array, and Lyapunov’s method. All these methods
74 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

are analytical and hence exact. The Jury test is the discrete equivalent to the Routh array and
is covered in Ogata [150, p242]. Lyapunov’s method has the special distinction that it is also
applicable to nonlinear ODEs and is covered in §2.9.4.

Suppose we wish to establish the stability of the discretised version of

1
G(s) =
6s2 + 5s + 1
at a sampling period of T = 2 and a first-order hold.

The discrete transfer function is

0.0746 + 0.2005z −1 + 0.0324z −2
G(z) = >> G = tf(1,[6 5 1]);
1 − 0.8813z −1 + 0.1889z −2
2 >> Gd = c2d(G,2,'foh')
with discrete poles given by the solution of >> pole(Gd)
ans =
1 − 0.8813z −1 + 0.1889z −2 = 0 0.5134
0.3679
or  7 >> pzmap(Gd); axis('equal');
0.5134
z=
0.3679

Since both discrete poles lie inside the unit circle, the discretised transfer function with a first-
order hold is still stable.

If you are using the state space description, then simply check the eigenvalues of the homoge-
neous part of the equation (the A or Φ matrix).

2.9.4 S TABILITY OF NONLINEAR DIFFERENTIAL EQUATIONS

Despite the fact that nonlinear differential equations are in general very difficult, if not impossible
to solve, one can attempt to establish the stability without needing to find the solution. Studies of
this sort fall into the realm of nonlinear systems analysis which demands a high degree of math-
ematical insight and competence. [190] is a good introductory text for this subject that avoids
much of the sophisticated mathematics.

Two methods due to the Russian mathematician Lyapunov9 address this nonlinear system sta-
bility problem. The indirect or linearisation method is based on the fact that the stability near
the equilibrium point will be closely approximated to the linearised approximation at the point.
However it is the second method, or direct method that being exact is a far more interesting and
powerful analysis. Since the Lyapunov stability method is applicable for the general nonlinear
differential equation, it is of course applicable for linear systems as well.

The Lyapunov stability theorem says that if such a differential system exists
ẋ = f (x, t) (2.95)
where f (0, t) = 0 for all t, and if there exists a scalar function V (x) having continuous first partial
derivatives satisfying V (x) is positive definite and the time derivative, V̇ (x), is negative definite,
then the equilibrium position at the origin is uniformly asymptotically stable. If V (x, t) → ∞ as
x → ∞, then it is stable “in the large”. V (x) is called a Lyapunov function of the system.

If the Lyapunov function is thought of as the total energy of the system, then we know that the
total energy must both always be positive, hence the restriction that V (x) is positive definite, and
9 Some authors, e.g. Ogata, tend to it spell as ‘Liapunov’. Note however that M ATLAB uses the lyap spelling.
2.9. STABILITY 75

Figure 2.25: “Alexander Mikhailovich Liapunov (1857–1918) was a Russian

mathematician and mechanical engineer. He had the very rare merit of pro-
ducing a doctoral dissertation of lasting value. This classic work was originally
published in 1892 in Russian but is now available in an English translation Sta-
bility of Motion, Academic Press, NY, 1966. Liapunov died by violence in Odessa,
which cannot be considered a surprising fate for a middle class intellectual in the
chaotic aftermath of the Russian revolution.”
Excerpt from Differential Equations and historical applications, Simmons, p465.

also for the system to be asymptotically stable, this energy must slowly with time die away, hence
the requirement that V̇ (x) < 0. The hard part about this method is finding a suitable Lyapunov
function in the first place. Testing the conditions is easy, but note that if your particular candidate
V (x) function failed the requirements for stability, this either means that your system is actually
unstable, or that you just have not found the right Lyapunov function yet. The problem is that
you don’t know which. In some cases, particularly vibrating mechanical ones, good candidate
Lyapunov functions can be found by using the energy analogy, however in other cases this may
not be productive.

Algorithm 2.2 Lyapunov stability analysis of nonlinear continuous and discrete systems.

The second method of Lyapunov is applicable for both continuous and discrete systems, [190,
p65], [150, p557].

Discrete difference equations,

Given a nonlinear difference equation,

Continuous differential equations xk+1 = f (xk ), f (0) = 0

Given a nonlinear differential equation, and a scalar function V (x), continuous in x, then
if
ẋ = f (x, t)
1. V (x) > 0 for x 6= 0 (or V (x) is positive def-
and a scalar function V (x), then if inite), and
1. V (x) is positive definite, and 2. ∆V (x), the difference of V (x), is negative
definite, where
2. V̇ (x), the derivative of V (x), is negative
definite, and ∆V (xk ) = V (xk+1 ) − V (xk )
3. V (x) → ∞ as kxk → ∞ = −V (f (xk )) − V (xk )
then the system ẋ = f (x, t) is globally asymptoti-
cally stable. and
3. V (x) → ∞ as kxk → ∞
then the system xk+1 = f (xk ) is globally asymp-
totically stable.

Example of assessing the stability of a discrete nonlinear system using Lyapunov’s second
method.
76 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Consider the dynamic equation from Ogata [152, Ex 9-18, p729]

ẋ1 = x2 − x1 x21 + x22


ẋ2 = −x2 − x2 x21 + x22
and we want to establish if it is stable or not. The system is nonlinear and the origin is the only
equilibrium state. We can propose a trial Lyapunov function V (x) = x21 + x22 which is positive
definite. (Note that that was the hard bit!) Now differentiating gives
dx1 dx2

V̇ (x) = 2x1 + 2x2 = −2 x21 + x22
dt dt
which is negative definite. Since V (x) → ∞ as x → ∞, then the system is “stable in the large”.
Relevant problems in [152, p771] include B-9-15, B-9-16 and B-9-17.

The problem with assessing the stability of nonlinear systems is that the stability can change
depending on the values of the states and/or inputs. This is not the case for linear systems,
where the stability is a function of the system, not the operating point. An important and relevant
chemical engineering example of a nonlinear system that can either be stable or unstable is a
continuously stirred tank reactor. The reactor can either be operated in the stable or unstable
regime depending on the heat transfer characteristics. The stability of such reactors is important
for exothermic reactions, since the thermal run-away is potentially very hazardous.

S TABILITY OF LINEAR TIME INVARIANT SYSTEMS USING LYAPUNOV

Sections 2.9.1 and 2.9.3 established stability criteria for linear systems. In addition to these meth-
ods, the second method of Lyapunov can be used since the linear system is simply a special case
of the general nonlinear system. The advantages of using the method of Lyapunov are:

1. The Lyapunov method is analytical even for nonlinear systems and so the criteria is, in
principle, exact.
2. It is computationally simple and robust and does not require the solution to the differential
equation.
3. We do not need to extract eigenvalues of an n × n matrix which is an expensive numerical
computation, (although we may need to check the definiteness of a matrix which does
require at least a Cholesky decomposition).

Of course similar advantages apply to the Routh array procedure, but unlike the Routh array,
this approach can be extended to nonlinear problems.

To derive the necessary condition for stability of ẋ = Ax using the method of Lyapunov, we
follow the procedure outlined in Algorithm 2.2 where we choose a possible Lyapunov function
in quadratic form,
V (x) = xT Px
The Lyapunov function will be positive definite if the matrix P is positive definite. The time
derivative is given by
V̇ (x) = ẋT Px + xT Pẋ
= (Ax)T Px + xT P(Ax)
 
= xT AT P + PA x
| {z }
−Q
2.9. STABILITY 77

where we have defined for convenience the matrix Q = −(AT P + PA).

So if −Q is negative definite, (or alternatively Q is positive definite, Q ≻ 0), then the system is
stable at the origin and hence asymptotically stable in the large. Note that this solution procedure
is analytical.

Unfortunately there is no cheap and easy computational way to establish if an arbitrary matrix is
positive definite or not, but using M ATLAB we would usually attempt a Cholesky factorisation.
Apart from the difficulty in deciding if a matrix is positive definite or not, we also have the
problem that a failure of Q being positive definite does not necessarily imply that the original
system is unstable. All it means is that if the original system does turn out to be stable, then the
postulated P was actually not a Lyapunov function.

A solution procedure that avoids us testing many different Lyapunov candidates is to proceed in
reverse; namely choose an arbitrary symmetric positive definite Q (say the identity matrix), and
then solve
AT P + PA = −Q (2.96)

for the matrix P. If P is positive definite, then the system ẋ = Ax is stable.

Eqn. 2.96 is known as the matrix Lyapunov equation or the more general form, AP + PB = −Q,
is known as Sylvester’s equation.

One obvious way to solve for the symmetric matrix P in Eqn. 2.96 is by equating the coefficients
which results in a n(n + 1)/2 system of linear algebraic equations. It is convenient in M ATLAB
to use the Kronecker tensor product to set up the equations and then use M ATLAB’s backslash
operator to solve them. This has the advantage that it is expressed succinctly in M ATLAB, but it
has poor storage and algorithmic characteristics and is recommended only for small dimensioned
problems. Section 2.9.5 explains Kronecker tensor products and this analytical solution strategy
in further detail.

Listing 2.6: Solve the continuous matrix Lyapunov equation using Kronecker products
n= size(Q,1); % Solve AT P + PA = −Q, Q = QT
I= eye(n); % Identity In , the same size as A
3 P=reshape(-(kron(I,A')+kron(A',I))\Q(:),n,n);% −vec(I⊗AT +AT ⊗I)vec(P) = vec(Q)
P= (P+P')/2; % force symmetric (hopefully unnecessary)

The final line, while strictly unnecessary, simply forces the computed result to be symmetric,
which of course it should be in the first case.

Less restrictive conditions on the form of Q exist that could make the solution procedure easier
to do manually and are discussed in [152, p766], and [150, p554]. Note however, using a compu-
tational tool such as M ATLAB, we would not normally bother with these modifications.

The C ONTROL T OOLBOX for M ATLAB includes the function lyap that solves the matrix Lya-
punov equation which is more numerically robust and efficient that equating the coefficients or
using the Kronecker tensor product method given on page 77. However note that the definition
of the Lyapunov matrix equation used by M ATLAB,

AP + PAT = −Q (2.97)

is not exactly the same as that defined by Ogata and used previously in Eqn. 2.96. (See the M AT-
LAB help file for clarification of this and compare with example 9–20 in [152, p734].)
78 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

Suppose we wish to establish the stability of the continuous system

ẋ = Ax
 
0 1
= x
−1 0.5

using lyap, as opposed to say extracting eigenvalues.

Now we wish to solve A⊤ P + PA = −Q for P where Q is some conveniently chosen positive

definite matrix such as say Q = I. Since M ATLAB’s lyap function solves the equivalent matrix
equation given by Eqn. 2.97, we must pass A⊤ rather than A as the first argument to lyap.

Listing 2.7: Solve the matrix Lyapunov equation using the lyap routine
1 >> A = [0,1;-1,-0.4];
>> P = lyap(A',eye(size(A))) % Use AT instead of A
P =
2.7000 0.5000
0.5000 2.5000
6 >> A'*P + P*A + eye(size(A)) % Does this equal zero ?
ans =
1.0e-015 *
0 -0.4441
-0.4441 0.2220
11

>> Q = eye(size(A)); % alternative method

>> n = max(size(Q)); % solve AT X + XA = C where C ≻ 0
>> I = eye(n);
>> P=reshape(-(kron(I,A')+ kron(A',I))\Q(:),n,n)
16 P =
2.7000 0.5000
0.5000 2.5000

Both methods return the same matrix P,

 
2.7 0.5
P=
0.5 2.5

which has minor determinants of 2.7 and 6.5. Since both are positive, following Sylvester’s crite-
ria, P is positive definite and so the system is stable.

While we can use Sylvester’s criteria to establish if P is positive definite or not, and hence the
system’s stability, it is not efficient for large systems. To establish if a symmetric matrix is pos-
itive definite, the easiest way is to look at the eigenvalues. If they are all positive, the matrix is
positive definite. Unfortunately we were originally trying to avoid to solve for the eigenvalues,
so this defeats the original purpose somewhat. Another efficient strategy to check for positive
definiteness with M ATLAB is to attempt a Cholesky decomposition using the [R,p]=chol(A)
command. The decomposition will be successful if A is positive definite, and will terminate early
with a suitable error message if not.

For example suppose we wanted to establish the stability of a transfer function

s−1
G=
(s + 1)(s + 2)(s − 3)
which is clearly unstable due to the pole at s = +3. In this case we convert to state-space, solve
for P and then attempt a Cholesky decomposition to establish if P ≻ 0.
2.9. STABILITY 79

Listing 2.8: Using Lyapunov to establish stability of a linear system

>> G = ss(zpk(1,[-1 -2 3],1)); % System is clearly unstable since one pole is s = +3
2 >> P = lyap(G.a, eye(size(G.a))) % Extract the A matrix & solve Lyapunov
P =
0.6667 0.1179 -0.1179
0.1179 0.3333 0.1667
-0.1179 0.1667 -0.1667
7

>> chol(P)
Error using chol
Matrix must be positive definite.

>> [¬,p] = chol(P) % If p 6= 0, then P is not positive definite.

12 p =
3

As a final check, we could evaluate the eigenvalues of A in M ATLAB by typing eig(A). Of course
in the above case we will get −1, −2 and +3.

D ISCRETE TIME SYSTEMS

For linear discrete time systems, the establishment of stability is similar to the continuous case
given previously. Given the discrete time system xk+1 = Φxk , we will again choose a positive
definite quadratic function for the trial Lyapunov function, V (x) = xT Px where the matrix P is
positive definite, and compute the forward difference

∆V (x) = V (xk+1 ) − V (xk )

= xTk+1 Pxk+1 − xTk Pxk
= (Φxk )T PΦxk − xTk Pxk
 
= xTk ΦT PΦ − P xk
| {z }
−Q

Now if the matrix Q defined as

 
Q = − ΦT PΦ − P (2.98)

is positive definite, then the system is stable.

The reverse solution procedure, i.e. to solve for the matrix P given Q and Φ is analogous to the
continuous time case given in Listing 2.6 using the Kronecker tensor product.

Listing 2.9: Solve the discrete matrix Lyapunov equation using Kronecker products
1 n = max(size(Q)); % Solve AT PA − P = −Q, with Q ≻ 0
I = eye(n); % identity
P = reshape((kron(I,I)-kron(Phi',Phi'))\Q(:),n,n)
P = (P+P')/2; % force symmetric (hopefully unnecessary)

Once again, the more efficient dlyap routine from the C ONTROL T OOLBOX can also be used. In
fact dlyap simply calls lyap after some minor pre-processing.
80 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

We can demonstrate this establishing the stability of

 
3 −0.5
xk+1 = xk
0 0.8
In this case since Φ is upper triangular, we can read the eigenvalues by inspection, (they are
the diagonal elements, 3 and 0.8). Since one of the eigenvalues is outside the unit circle we
know immediately that the process is unstable. Notwithstanding, by following the method of
Lyapunov,

1 >> Phi = [3, -0.5; 0, 0.8]

>> Q = eye(size(Phi)); % Ensure Q ≻ 0
>> P = dlyap(Phi',Q) % Note Φ is transposed.
P =
-0.1250 -0.1339
6 -0.1339 2.9886
>> [R,rc] = chol(P) % Is P +ve definite?
R =
[]
rc =
11 1 % No it isn’t, unstable!

we verify that the system is indeed unstable.

We can both check the definiteness of P using Sylvester’s criteria [152, pp924-926], or by using
the chol function as in the above example, or even by finding the eigenvalues of P. We then can
compare the stability criteria thus established with that obtained by solving for the eigenvalues
of Φ.

Algorithm 2.3 Stability of linear systems using Lyapunov

We wish to establish the absolute stability of the dynamic linear continuous system, ẋ = Ax or
the discrete counterpart, xk+1 = Φxk

1. Choose a convenient positive definite n × n matrix Q, say the identity, In×n , then
2. Solve either
AT P + PA = −Q
in the case of a continuous system, or

ΦT PΦ − P = −Q

in the case of a discrete system for P by either equating the coefficients or using lyap or
dlyap.
3. The Lyapunov function is V (x) = xT Px, so check the definiteness of P. The system is
stable if and only if P is positive definite. Check using Sylvester’s criteria or attempt a
Cholesky factorisation.

2.9.5 E XPRESSING MATRIX EQUATIONS SUCCINCTLY USING K RONECKER PROD -

UCTS

The strategy employed in Listings 2.6 and 2.9 to solve the Lyapunov equation used Kronecker
products and vectorisation or stacking to express the matrix equation succinctly. This made it easy
2.9. STABILITY 81

to solve since the resulting expression was now reformulated into a system of linear equations
which can be solved using standard linear algebra techniques. Further details on the uses and
properties of Kronecker products (or tensor products) are given in [89, p256], [116, Chpt 13]
and [130]. The review in [34] concentrates specifically on Kronecker products used in control
applications.

The Kronecker product, given the symbol ⊗, of two arbitrarily sized matrices, A ∈ ℜm×n and
B ∈ ℜp×q , results in a new (large) matrix
 
a11 B · · · a1n B
A⊗B=
 .. .. ..  mp×nq
. . . ∈ℜ (2.99)
am1 B · · · amn B
of size (mp × nq). In M ATLAB we would write kron(A,B). Note in general A ⊗ B 6= B ⊗ A.

The vectorisation of a matrix A is an operation that converts a rectangular matrix into a single
column by stacking the columns of A on top of each other. In other words, if A is an (n ×
m) matrix, then vec(A) is the resulting (nm × 1) column vector. In M ATLAB we convert block
matrices or row vectors to columns simply using the colon operator, or A(:).

We can combine this vectorisation operation with Kronecker products to express matrix multi-
plication as a linear transformation. For example, for the two matrices A and B of compatible
dimensions, then
vec (AB) = (I ⊗ A) vec(B)
  (2.100)
= BT ⊗ I vec(A)

and for the three matrices A, B and C of compatible dimensions, then

 
vec (ABC) = CT ⊗ A vec(B)
= (I ⊗ AB) vec(C) (2.101)
 
= CT BT ⊗ I vec(A)

Table II in [34] summarises these and many other properties of the algebra of Kronecker products
and sums.

This gives us an alternative way to express matrix expressions such as the Sylvester equation
AX − XA = Q where we wish to solve for the matrix X given matrix A. In this case, using
Eqn. 2.100, we can write
 
vec(AX − XA) = I ⊗ A − AT ⊗ I vec(X) = vec(Q)

which is in the form of a system of linear equations Gx = q where the vectors x and q are simply

the stacked columns of the matrices X, and Q, and the matrix G is given by I ⊗ A − AT ⊗ I .
We first solve for the unknown vector x using say x = G−1 q or some numerically sound equiv-
alent, and then we reassemble the matrix X by un-stacking the columns from x. Of course this
strategy is memory intensive because the size of the matrix G is (n2 ×n2 ). However [34] describes
some modifications to this approach to reduce the dimensionality of the problem.

U SING fsolve TO SOLVE MATRIX EQUATIONS

The general nonlinear algebraic equation solver fsolve within the O PTIMISATION T OOLBOX has
the nice feature that it can solve matrix equations such as the continuous time Lyapunov equation
82 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

directly. Here we wish to find the square matrix X such that

AX + XAT + Q = 0

given an arbitrary square matrix A and a positive definite matrix Q. For an initial estimate for X
we should start with a positive definite matrix such as I. We can compare this solution with the
one generated by the dedicated lyap routine.

>> n = 4;
>> A = rand(n); % Create a ‘random’ matrix A of dimensions (n × n)
>> Q1 = randn(n); Q = Q1'*Q1; % Create a ‘random’ positive definite matrix Q
4

>> LyEqn = @(X) A*X+X*A'+Q; % Matrix equation to be solved: AX + XAT + Q = 0

>> X = fsolve(LyEqn, eye(n)); % Solve the Lyapunov using fsolve
X1 =
-0.8574 -0.6776 1.0713
9 -0.6776 2.7742 -2.6581
1.0713 -2.6581 0.7611

>> Xx = lyap(A,Q); % Solve the Lyapunov using the control toolbox

Xx =
14 -0.8574 -0.6776 1.0713
-0.6776 2.7742 -2.6581
1.0713 -2.6581 0.7611

2.9.6 S UMMARY OF STABILITY ANALYSIS

The stability of a linear continuous time transfer function is determined by the sign of the real
part of the poles. For the transfer function to be stable, all the poles must lie strictly in the left-
hand-plane. In the discrete case, the stability is determined by the magnitude of the possibly
complex poles. To be stable, the discrete poles must lie within the unit circle. (See Fig. 2.26.)
The key difference between the stability of discrete and continuous transfer functions is that the
sample time plays an important role. Generally as one increases the sample time, the discrete
system tend to instability.

To establish the stability of the transfer functions, one need not solve for the roots of the de-
nominator polynomial since exact algebraic methods such as the Routh Array, Jury blocks and
Lyapunov methods exist. However with the current computer aided tools such as M ATLAB, the
task of reliably extracting the roots of high order polynomials is not considered the hurdle it once
was. The Lyapunov method is also applicable for nonlinear systems. See Ogata’s comments,
[150, p250].

2.10 S UMMARY

This chapter briefly developed the tools needed to analyse discrete dynamic systems. While
most physical plants are continuous systems, most control is performed on a digital computer,
and therefore the discrete description is far more natural for computer implementation or sim-
ulation. Converting a continuous differential equation to a discrete differential equation can be
done either by using a backward difference approximation such as Euler’s scheme, or by using z
transforms. The z transform is the discrete equivalent to the Laplace transform in the continuous
domain.
2.10. SUMMARY 83

ℑ ℑ
✻ Stability boundary ✻

+1 unstable
✾ ③
stable
stable
0 ✲ ℜ ✲ ℜ
+1
unstable
✛ imaginary axis ■ unit circle
−1

Continuous s-plane Discrete z-plane

Figure 2.26: Regions of stability for the poles of continuous (left) and discrete (right) systems

A vector/matrix approach to systems modelling and control has the following characteristics:

1. We can convert any linear time invariant differential equation into the state space form
ẋ = Ax + Bu.
2. Once we have selected a sampling time, T , we can convert from the continuous time do-
main to the discrete form equivalent; xk+1 = Φxk + ∆uk .
3. The stability of both the discrete and continuous time systems is determined by the eigen-
values of the A or Φ matrix.
4. We can transform the mathematical description of the process to dynamically similar de-
scriptions by adjusting our base coordinates, which may make certain computations easier.

Stability is one of the most important concepts for the control engineer. Continuous linear sys-
tems are stable if all the poles lie in the left hand side of the complex plane, ie they have negative
real parts. For discrete linear systems, the poles must lie inside the unit circle. No easy check can
be made for nonlinear systems, although the method due to Lyapunov can possibly be used, or
one can approximate the nonlinear as a linear system and check the stability of the latter.

When we deal with discrete systems, we must sample the continuous function. Sampling can
introduce problems in that we may miss interesting information if we don’t sample fast enough.
The sampling theorem tells us how fast we must sample to reconstruct particular frequencies.
Over sampling is expensive, and could possibly introduce unwanted artifacts such as RHP zeros
etc.
84 CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS
C HAPTER 3

M ODELLING DYNAMIC SYSTEMS

WITH DIFFERENTIAL EQUATIONS

The ability to solve so many problems with a small arsenal of mathematical methods gave rise
to a very optimistic “world view” which in the 18th Century came to be called the “Age of
Reason”. The essential point is that the world was felt to be predictable.
Hans Mark

3.1 D YNAMIC SYSTEM MODELS

Dynamic system models are groups of equations that involve time derivative terms that attempt
to reproduce behaviour that we observe around us. I admire the sense of optimism in the quote
given above by Hans Mark: if we know the governing equations, and the initial conditions, then
the future is assured. We now know that the 18th century thinkers were misguided in this sense,
but nevertheless engineers are a conservative lot, and still today with similar tools, they aim to
predict the known universe.

It is important to realise that with proven process models, control system design becomes sys-
tematic, hence the importance of modelling.

Physically dynamic systems are those that change with time. Examples include vibrating struc-
tures (bridges and buildings in earthquakes), bodies in motion (satellites in orbit, fingers on a
typewriter), changing compositions (chemical reactions, nuclear decay) and even human charac-
teristics (attitudes to religion or communism throughout a generation, or attitudes to the neigh-
bouring country throughout a football match) and of course, there are many other examples. The
aim of many modellers is to predict the future. For many of these types of questions, it is infea-
sible to experiment to destruction, but it is feasible to construct a model and test this. Today, it is
easiest to construct a simulation model and test this on a computer. The types of models referred
to in this context are mathematically based simulation models.

Black-box or heuristic models, where we just fit any old curve to the experimental data, and White-
box or fundamental models, where the curves are fitted to well established physical laws repre-
sent two possible extremes in modelling philosophy. In practice, most engineering models lie
somewhere in the grey zone, known appropriately as ‘grey-box’ models where we combine our
partial prior knowledge with black-box components to fit any residuals. Computer tools for

85
86 Modelling

grey-box modelling are developed in [30].

3.1.1 S TEADY STATE AND DYNAMIC MODELS

Steady state models only involve algebraic equations. As such they are more useful for design
rather than control tasks. However they can be used for control to give broad guidelines about
the operating characteristics of the process. One example is Bristol’s relative gain array (§ 3.3.4)
which uses steady state data to analyse for multivariable interaction. But it is the dynamic model
that is of most importance to the control engineer. Dynamic models involve differential equa-
tions. Solving systems involving differential equations is much harder than for algebraic equa-
tions, but is essential for the study and application of control. A good reference for techniques of
modelling dynamic chemical engineering applications is [199].

3.2 A COLLECTION OF ILLUSTRATIVE MODELS

An economist is an expert who will know tomorrow why the things he predicted yesterday didn’t happen
today.
Laurence J. Peter

This section reviews the basic steps for developing dynamic process models and gives a few
examples of common process models. There are many text books devoted to modelling for en-
gineers, notably [33, 115] for general issues, [43, 198] for solution techniques and [131, 199] for
chemical engineering applications. I recommend the following stages in the model building pro-
cess:

1. Draw a diagram of the system with a boundary and labelling the input/output flows that
cross the boundary.
2. Decide on what are state variables, what are parameters, manipulated variables, uncon-
trolled disturbances etc.
3. Write down any governing equations that may be relevant such as conservation of mass
and energy.
4. Massage these equations in a standard form suitable for simulation.

The following sections will describe models for a flow system into and out of a tank with external
heating, a double integrator such as an inverted pendulum or satellite control, a forced circulation
evaporator, and a binary distillation column.

3.2.1 S IMPLE MODELS

P ENDULUMS

Pendulums provide an easy and fruitful target to model. They can easily be built, are visually
impressive, and the governing differential equations derived from the physics are simple but
nonlinear. Furthermore, if you invert the pendulum such as in a rocket, the system is unstable,
and needs an overlaying control scheme.
3.2. A COLLECTION OF ILLUSTRATIVE MODELS 87

torque, Tc
✒
θ
length, l

❄
mg
θ
❄
mg

Pendulum (stable) Inverted pendulum (unstable)

Figure 3.1: A stable and unstable pendulum

Fig. 3.1 shows the two possible orientations. For the classical stable case, the equation of motion
for the pendulum is
d2 θ
ml2 + mgl sin θ = Tc (3.1)
dt2
where θ is the angle of inclination, and Tc is the applied torque. We can try to solve Eqn. 3.1
analytically using dsolve from the symbolic toolbox,

>> syms m l g positive

>> syms Tc theta
>> dsolve(’m*lˆ2*D2theta + m*g*l*sin(theta) = Tc’)

Warning: Explicit solution could not be found; implicit solution returned.

> In C:\MATLAB6p5p1\toolbox\symbolic\dsolve.m at line 292
ans =
[ Int(m/(m*(2*m*g*l*cos(a)+2*Tc*a+C1*m*lˆ2))ˆ(1/2)*l,a = .. theta)-t-C2 = 0]
[ Int(-m/(m*(2*m*g*l*cos(a)+2*Tc*a+C1*m*lˆ2))ˆ(1/2)*l,a = .. theta)-t-C2 = 0]

but the solution returned is not much help indicating that this particular nonlinear differential
equation probably does not have a simple explicit solution.

Eqn. 3.1 is a second order nonlinear differential equation, but by defining a new state variable
vector, x, as

def
x1 = θ (3.2)
def dθ
x2 = ẋ1 = (3.3)
dt

then we note that ẋ2 = ẍ1 , so the single second order differential equation can now be written as
88 Modelling

two coupled first order equations

ẋ1 = x2 (3.4)
g Tc
ẋ2 = − sin x1 + (3.5)
l ml2
Linearising Eqn. 3.5 is trivial since sin θ ≈ θ for small angles. Consequently, both linear and
nonlinear expressions are compared below
" # " # " # " #
0 x1 0 0 1 0
ẋ = −g + Tc , ẋ = −g x+ Tc (3.6)
sin(x1 ) 0 2
0 2
| l {z ml } | l {z ml }
nonlinear linear

while the numerical results are compared on page 120.

In the inverted case, the equation for ẋ2 is almost the same,
g Tc
ẋ2 = + sin x1 + (3.7)
l ml2
but the sign change is enough to place the linearised pole in the right-hand plane.

A DOUBLE INTEGRATOR

The double integrator is a simple linear example from classical mechanics. It can describe,
amongst other things, a satellite attitude control system,
d2 θ
J =u (3.8)
dt2
where J is the moment of inertia, θ is the attitude angle, u is the control torque (produced by
small attitude rockets mounted on the satellite’s side). We can convert the second order system
in Eqn. 3.8 to two first order systems by again defining a new state vector
   
x1 def θ
x= = (3.9)
x2 θ̇
Substituting this definition into Eqn. 3.8, we get J ẋ2 = u and ẋ1 = x2 , or in a matrix-vector form
     
ẋ1 0 1 0
= x 1 u (3.10)
ẋ2 0 0 J

A LIQUID LEVEL TANK

Buffer storage vessels such as shown in Fig. 3.2 are commonly used in chemical plants between
various pieces of equipment such as reactors and separators to dampen out fluctuations in pro-
duction flows.

Our system boundary is essentially the figure boundary, so we have one input, the input flow
rate, Fin , and one output, the output flow rate, Fout . What interests us is the amount of fluid in
the tank, M , which physically must always be less than the full tank Mmax , and greater than 0.

The governing relation is the conservation of mass,

dM
= Fin − Fout (3.11)
dt
3.2. A COLLECTION OF ILLUSTRATIVE MODELS 89

flow in, Fin

tank with cross sectional area, A

✻
measured height, ĥ ∆P true height, h

❄
valve
✲ flow out, Fout

Figure 3.2: Simple buffer tank

If the tank has a constant cross sectional area, then the amount of water in the tank M is pro-
portional to the height of water h, since most fluids are assumed incompressible. In addition the
mass M is proportional to the volume, M = ρV = ρAh. Rewriting Eqn. 3.11 in terms of height
dh Fin − Fout
= (3.12)
dt ρA
since the area and density are constant. If the liquid passes though a valve at the bottom of the
tank, the flowrate out will be a function of height. For many processes, the flow is proportional
to the square root of the pressure drop, (Bernoulli’s equation)
√ √
Fout ∝ ∆P = k h
This square root relation introduces a mild nonlinearity into the model.

The “tank” level is not actually measured in the tank itself, but in a level leg connected to the
tank. The level in this level leg lags behind the true tank level owing to the restriction valve on
the interconnecting line. This makes the system slightly more difficult to control owing to the
added measurement dynamics. We assume, that the level leg can be modelled using a first order
dynamic equation, with a gain of 1, and a time constant of about 2 seconds. I can estimate this by
simply watching the apparatus. Thus
dĥ h − ĥ
= (3.13)
dt 2
The level signal is a current, I, between 4 and 20 mA. This current is algebraically related (by
some function f (·) although often approximately linearly) to the level in the level leg. Thus the
variable that is actually measured is:
I = f (ĥ) (3.14)
The current I is called the output or measured variable. In summary, the full model equations
are;
√
dh Fin − k h
= (3.15)
dt ρA
dĥ h − ĥ
= (3.16)
dt 2
I = f (ĥ) (3.17)
90 Modelling

The input variable (which we may have some hope of changing) is the input flow rate, Fin , and
the exit valve position which changes k. The dependent (state) variables are the actual tank
level, h, and the measurable level in the level leg, ĥ. The output variable is the current, I. Con-
stant parameters are density, and tank cross-sectional area. In Table 3.1 which summarises this
nomenclature, I have written vectors in a bold face, and scalars in italics. The input variables
can be further separated into variables that can be easily changed or changed on demand such
as the outlet valve position (manipulated variables), and inputs that change outside our control
(disturbance variables). In this case the inlet flowrate.

Table 3.1: Standard nomenclature used in modelling dynamic systems

type symbol variable

independent t time, t
states x h, ĥ
inputs u Fin , k
outputs y I
parameters θ ρ, A

A STIRRED TANK HEATER

Suppose an electrical heater is added causing an increase in water temperature. Now in addition
to the mass balance, we have an energy balance. The energy balance in words is “the rate of
temperature rise of the contents in the tank is proportional to the amount of heat that the heater
is supplying plus the amount of heat that is arriving in the incoming flow minus the heat lost
with the out flowing stream minus any other heat losses”.

We note that for water the heat capacity cp , is almost constant over the limited temperature range
we are considering, and that the enthalpy of water in the tank is defined as

H = M cp ∆T (3.18)

where ∆T is the temperature difference between the actual tank temperature T and some refer-
ence temperature, Tref . Writing an energy balance gives

dT
M cp = Fin cp (Tin − Tref ) − Fout cp ∆T + Q − q (3.19)
dt

where Q is the heater power input (kW) and q is the heat loss (kW). These two equations (Eqn 3.12
and 3.19) are coupled. This means that a change in mass in the tank M, will affect the temperature
in the tank T (but not the other way around). This is called one-way coupling.

For most purposes, this model would be quite adequate. But you may decide that the heat loss q
to the surroundings is not constant and the variation significant. A more realistic approximation
of the heat loss would be to assume that it is proportional to the temperature difference between
the vessel and the ambient (room) temperature Troom . Thus q = k(T − Troom ). Note that the heat
loss now can be both positive or negative. You may also decide that the heat capacity cp , and
density ρ of water are a function of temperature. Now you replace these constants with functions
as cp (T ) and ρ(T ). These functions are tabulated in the steam tables1 . The more complete model

1 Steam tables for M ATLAB are available from http://www.kat.lth.se/edu/kat061/MfileLib.html

3.2. A COLLECTION OF ILLUSTRATIVE MODELS 91

would then be

dh Fin − Fout
= (3.20)
dt ρ(T )A
dT
Aρ(T ) = Fin cp (T ) (Tin − T ref ) − Fout cp (T )∆T + Q − k (T − Troom)
dt

“C AVEAT E MPTOR ”

Finally we should always state under what conditions the model is valid and invalid. For this
example we should note that the vessel temperature should not vary outside the range 0 < T <
100◦ C since the enthalpy equation does not account for the latent heat of freezing or boiling. We
should also note that h, Fin , Fout , M, A, k, cp , ρ are all assumed positive.

As a summary and a check on the completeness of the model, it is wise to list all the variables,
and state whether the variable is a dependent or independent variable, whether it is a dynamic
or constant variable, or whether it is a parameter. The degree of freedom for a well posed prob-
lem should be zero. That is, the number of unknowns should equal the number of equations.
Incomplete models, and/or bad assumptions can give very misleading and bad results.

A great example highlighting the dangers of extrapolation from obviously poor models is the bi-
annual UK government forecast of Britain’s economic performance. This performance or growth,
is defined as the change in the Gross Domestic Product (GDP) which is the total amount of goods
and services produced in the country that year. Figure 3.3 plots the actual growth over the last
few years (solid line descending), compared with the forecasts given by the Chancellor of the Ex-
chequer, (dotted lines optimistically ascending). I obtained this information from a New Scientist
article provocatively titled Why the Chancellor is always wrong, [49]. Looking at this performance,
it is easy to be cynical about governments using enormous resources to model things that are
essentially so complicated as to be unmodellable, and then produce results that are politically
advantageous.

119

118 Forecasts

117
Growth GDP change

116

115

114

113

112 Figure 3.3: The UK growth based on the

Actual
GDP comparing the actual growth (solid)
111
1989 1990 1991 1992 1993 versus Treasury forecasts (dashed) from
Year 1989–1992.
92 Modelling

3.3 C HEMICAL PROCESS MODELS

The best material model of a cat is another, or preferably the same, cat.

Arturo Rosenblueth, Philosophy of Science, 1945

Modelling is very important in the capital-intensive world of chemical manufacture. Perhaps

there is no better example of the benefits of modelling than in distillation. One industry maga-
zine2 reported that distillation accounted for around 3% of the total world energy consumption.
Clearly this provides a large economic incentive to improve operation. Distillation models pre-
dict the extent of the component separation given the type of trays in the column, the column di-
ameter and height and the operating conditions (temperature,pressure etc). More sophisticated
dynamic models predict the concentration in the distillate given changes in feed composition,
boilup rate and reflux rate over time. An example of a simple distillation dynamic model is
further detailed in §3.3.3.

When modelling chemical engineering unit operations, we can consider (in increasing complex-
ity):

1. Lumped parameters systems such as well-mixed reactors or evaporators

2. Staged processes such as distillation columns, multiple effect evaporators, floatation.

3. Distributed parameter systems such as packed columns, poorly mixed reactors, dispersion
in lakes & ponds.

An evaporator is an example of a lumped parameter system and is described in §3.3.2 while a

rigorous distillation column model described on page 98 is an example of a staged process.

At present we will now investigate only lumped parameter systems and staged processes which
involve only ordinary differential equations, as opposed to partial differential equations, albeit
a large number of them. Problems which involve algebraic equations coupled to the ordinary
differential equations are briefly considered in section 3.5.2.

A collection of good test industrial process models is available in the nonlinear model library
from www.hedengren.net/research/models.htm.

3.3.1 A CONTINUOUSLY- STIRRED TANK REACTOR

Tank reactors are a common unit operation in chemical processing. A model presented in [86]
considers a simple A → B reaction with a cooling jacket to adjust the temperature, and hence the
reaction rate as shown in Fig. 3.4.

The reactor model has two states, the concentration of compound A, given the symbol Ca mea-
sured in mol/m3 and the temperature of the reaction vessel liquid, T measured in K. The manip-
ulated variables are the cooling jacket water temperature, Tc , the temperature of the feed Tf and

2 The Chemical Engineer, 21st October, 1999, p16

3.3. CHEMICAL PROCESS MODELS 93

Stirrer
Feed, Tf , Caf

Cooling input, Tc ✲ ❄

A→B
✲ Coolant outflow
❄
Product, T, Ca

Figure 3.4: A continually stirred tank reactor (CSTR) reactor

the concentration of the feed, Caf .

 
dT q −E
= (Caf − Ca ) − k0 exp Ca (3.21)
dt V RT
 
dCa q m∆H −E UA
= (Tf − T ) + k0 exp Ca + (Tc − T ) (3.22)
dt V ρCp RT V ρCp

The values for the states and inputs at steady-state are

   
    Tc 300
def T 324.5 def
xss = = , uss =  Tf  =  350  (3.23)
Ca 0.8772
ss Caf ss 1

The values of the model parameters such as ρ, Cp etc. are given in Table 3.2.

Table 3.2: Parameters of the CSTR model

name variable unit
Volumetric Flowrate q = 100 m3 /sec
Volume of CSTR V = 100 m3
Density of A-B Mixture ρ = 1000 kg/m3
Heat capacity of A-B Mixture Cp = .239 J/kg-K
Heat of reaction for A→B m∆H = 5 · 104 J/mol
E/R = 8750 K
Pre-exponential factor k0 = 7.2 · 1010 1/sec
Overall Heat Transfer Coefficient × Area U A = 5 · 104 W/K

Note: At a jacket temperature of Tc = 305K, the reactor model has an oscillatory response. The
oscillations are characterized by reaction run-away with a temperature spike. When the concen-
tration drops to a low value, the reactor cools until the concentration builds and there is another
run-away reaction.

3.3.2 A FORCED CIRCULATION EVAPORATOR

Evaporators such as illustrated in Fig. 3.5 are used to concentrate fruit juices, caustic soda, alu-
mina and many other mixtures of a solvent and a solute. The incentive for modelling such unit
94 Modelling

operations is that the model can provide insight towards better operating procedures or future
design alternatives. A relatively simple dynamic model of a forced circulation evaporator is de-
veloped in [147, chapter 2] is reproduced here as a test process for a wide variety of control tech-
niques of interest to chemical engineers. Other similar evaporator models have been reported
in the literature; [66] documents research over a number of years at the University of Alberta
on a pilot scale double effect evaporator, and [40] describe a linearised pilot scale double effect
evaporator used in multivariable optimal selftuning control studies.

Vapour Cooling
water
Condenser

Condensate

L
level

Pressure P
Separator

Steam

Evaporator

Condensate circulation pump

composition
A
Feed
Product

Figure 3.5: A forced circulation evaporator from [147, p7].

The nonlinear evaporator model can be linearised into a state space form with the variables de-
scribed in Table 3.3. The parameters of the linearised state space model are given in Appendix E.1.

Table 3.3: The important variables in the forced circulation evaporator

type name variable unit range

level L2 m 0–2
State steam pressure P2 kPa ??
product conc x2 % 0–100
product flow F2 kg/min 0-10
Manipulated steam pressure P100 kPa 100–350
c/w flow F200 kg/min 0–300
circulating flow F3 kg/min 30-70
feed flow F1 kg/min 5-15
Disturbance feed composition x1 % 0-10
◦
feed temperature T1 C 20-60
◦
c/w temperature T200 C 15-35

The distinguishing characteristics of this evaporator system from a control point are that the
detailed plant model is nonlinear, the model is non-square (the number of manipulated variables,
u, do not equal the number of state variables, x), and that one of the states, level, is an integrator.
The dynamic model is both observable and controllable, which means that by measuring the
outputs only, we can at least in theory, control all the important state variables by changing the
3.3. CHEMICAL PROCESS MODELS 95

inputs.

Problem 3.1 1. Look at the development of the nonlinear model given in [147, chapter 2].
What extensions to the model would you suggest? Particularly consider the assumption of
constant mass (M = 50kg) in the evaporator circuit and the observation that the level in the
separator changes. Do you think this change will make a significant difference?

2. Describe in detail what tests you would perform experimentally to verify this model if you
had an actual evaporator available.

3. Construct a M ATLAB simulation of the nonlinear evaporator. The relevant equations can be
found in [147, Chpt 2].

3.3.3 A BINARY DISTILLATION COLUMN MODEL

Distillation columns, which are used to separate mixtures of different vapour pressures into al-
most pure components, are an important chemical unit operation. The columns are expensive
to manufacture, and the running costs are high owing to the high heating requirement. Hence
there is much incentive to model them with the aim to operate them efficiently. Schematics of
two simple columns are given in Figures 3.6 and 3.9.

Wood and Berry experimentally modelled a binary distillation column in [206] that separated
water from ethanol as shown in Fig. 3.6. The transfer function model they derived by step testing
a real column has been extensively studied, although some authors, such as [188], point out
that the practical impact of all these studies has been, in all honesty, minimal. Other distillation
column models that are not obtained experimentally are called rigorous models and are based on
fundamental physical and chemical relations. Another more recent distillation column model,
developed by Shell and used as a test model is discussed in [139].

Columns such as given in Fig. 3.6 typically have at least 5 control valves, but because the hydro-
dynamics and the pressure loops are much faster than the composition dynamics, we can use the
bottoms exit valve to control the level at the bottom of the column, use the distillate exit valve to
control the level in the separator, and the condenser cooling water valve to control the column
pressure. This leaves the two remaining valves (steam reboiler and reflux) to control the top and
bottoms composition.

The Wood-Berry model is written as a matrix of Laplace transforms in deviation variables

   
12.8e−s −18.9e−3s 3.8e−8s
 16.7s + 1 21s + 1  u +  14.9s + 1  d
  
y=
 6.6e−7s (3.24)
−19.4e−3s   4.9e−3.4s 
10.9s + 1 14.4s + 1 13.2s + 1
| {z }
G(s)

where the inputs u1 and u2 are the reflux and reboiler steam flowrates respectively, the outputs
y1 and y2 are the mole fractions of ethanol in the distillate and bottoms, and the disturbance
variable, d is the feed flowrate.

It is evident from the transfer function model structure in Eqn. 3.24 that the plant will be inter-
acting and that all the transfer functions have some time delay associated with them, but the
off-diagonal terms have the slightly larger time delays. Both these characteristics will make con-
trolling the column difficult.
96 Modelling

c/w ❄
✻ Reflux, R
condensor s
✛ ❄ ✲ ✲ Distillate, D, xD

Feed, F , xF ✲ ✲
Distillation column

✛
✻
Steam, S ✒
reboiler
✲
❄ ✻
✲ Bottoms, B, xB

Figure 3.6: Schematic of a distillation column

In M ATLAB, we can construct such a matrix of transfer functions (ignoring the feed) with a matrix
of deadtimes as follows:

>> G = tf({12.8, -18.9; 6.6 -19.4}, ...

{ [16.7 1],[21 1]; [10.9 1],[14.4 1]}, ...
'ioDelayMatrix',[1,3; 7,3], ...
4 'InputName',{'Reflux','Steam'}, ...
'OutputName',{'Distillate','bottoms'})

Transfer function from input "Reflux" to output...

12.8
9 Distillate: exp(-1*s) * ----------
16.7 s + 1

6.6
bottoms: exp(-7*s) * ----------
14 10.9 s + 1

Transfer function from input "Steam" to output...

-18.9
Distillate: exp(-3*s) * --------
19 21 s + 1

-19.4
bottoms: exp(-3*s) * ----------
14.4 s + 1
3.3. CHEMICAL PROCESS MODELS 97

Once we have formulated the transfer function model G, we can perform the usual types of
analysis such as step tests etc as shown in Fig. 3.7. An alternative implementation of the column
model in S IMULINK is shown in Fig. 3.8.

Step Response

From: Reflux From: Steam

10
To: Distillate

−10
Amplitude

−20
To: bottoms

−10 Figure 3.7: Step responses of the four

transfer functions that make up the
−20 Wood-Berry binary distillation col-
0 50 100 0 50 100
Time (sec) umn model in Eqn. 3.24.

A 3-input/3 output matrix of transfer functions model of an 19 plate ethanol/water distillation

column model with a variable side stream draw off from [153] is
 
0.66e−2.6s −0.61e−3.5s −0.0049e−s
    6.7s + 1 8.64s + 1 9.06s + 1 
 
y1  −6.5s −3s −1.2s  u1
 y2  =  1.11e −2.36e −0.012e 
  u2 
 3.25s + 1 5s + 1 7.09s + 1 
y3   u3
 −34.68e−9.2s 46.2e −9.4s
0.87(11.61s + 1)e −2.6s 

8.15s + 1 10.9s + 1 (3.89s + 1)(18.8s + 1)

 −1.2s 
0.14e −0.0011(26.32s + 1)e−2.66s

 6.2s + 1 (3.89s + 1)(14.63s + 1)  
 −10.5s  
 0.53e −0.0032(19.62s + 1)e−3.44s  d1
+  (3.25)
 6.9s + 1 (7.29s + 1)(8.94s + 1)  d2
 
−0.6s −2.6s
 −11.54e 0.32e 
7.01s + 1 7.76s + 1
which provides an alternative model for testing multivariable control schemes. In this model
the three outputs are the overhead ethanol mole fraction, the side stream ethanol mole fraction,
and the temperature on tray 19, the three inputs are the reflux flow rate, the side stream product
rate and the reboiler steam pressure, and the two disturbances are the feed flow rate and feed
temperature. This system is sometimes known as the OLMR after the initials of the authors.

Problem 3.2 1. Assuming no disturbances (d = 0), what are the steady state gains of the
Wood-Berry column model? (Eqn 3.24). Use the final value theorem.
2. Sketch the response for y1 and y2 for;
(a) a change in reflux flow of +0.02
(b) A change in the reflux flow of −0.02 and a change in reboiler steam flow of +0.025
simultaneously.
3. Modify the S IMULINK simulation to incorporate the feed dynamics.
98 Modelling

reflux reflux distillate distillate

steam bottoms

steam Wood Berry Column

bottoms
(a) Overall column mask.

12.8
1 1
16.7s+1
reflux distillate
Transfer Fcn Transport
Delay
−18.9
21s+1
Transfer Fcn1 Transport
Delay1
6.6
10.9s+1
Transfer Fcn2 Transport
Delay2

−19.4
2 2
14.1s+1
steam bottoms
Transfer Fcn3 Transport
Delay3

(b) Inside the Wood Berry column mask. Compare this with Eqn. 3.24.

Figure 3.8: Wood-Berry column model implemented in S IMULINK

M ORE RIGOROUS DISTILLATION COLUMN MODELS

Distillation column models are important to chemical engineers involved in the operation and
maintenance of these expensive and complicated units. While the behaviour of the actual multi-
component tower is very complicated, models that assume ideal binary systems are often good
approximations for many columns. We will deal in mole fractions of the more volatile compo-
nent, x for liquid, y for vapour and develop a column model following [132, p69].

A generic simple binary component column model of a distillation column such as shown in
Fig. 3.9 assumes:

1. Equal molar overflow applies (heats of vapourisation of both components are equal, and
mixing and sensible heats are negligible.)

2. Liquid level on every tray remains above weir height.

3. Relative volatility and the heat of vapourisation are constant. In fact we assume a constant
relative volatility, α. This simplifies the vapour-liquid equilibria (VLE) model to
αxn
yn =
1 + (α − 1)xn
3.3. CHEMICAL PROCESS MODELS 99

on tray n with typically α ≈ 1.2–2.

4. Vapour phase holdup is negligible and the feed is a saturated liquid at the bubble point.

✲
❄

c/w ✲ ✲
condensor
✻
❄
condensor collector
✛ ✲ Distillate, D, xD
tray # N reflux, R, xD

..
Feed, F, xF .
✲
feed tray, NF
..
.

tray #1 ✛
✻ reboiler, V
✛ ✲ Bottoms, xB , B
✻
❄

Figure 3.9: Distillation tower

The N trays are numbered from the bottom to top, (tray 0 is reboiler and tray N + 1 is the con-
denser). We will develop separate model equations for the following parts of the column, namely:

Condenser is a total condenser, where the reflux is a liquid and the reflux drum is perfectly
mixed.

General tray has liquid flows from above, and vapour flows from below. It is assumed to be
perfectly mixed with a variable liquid hold up, but no vapour hold up as it is assumed very
fast.

Feed tray Same as a ‘general’ tray, but with an extra (liquid) feed term.

Top & bottom trays Same as a ‘general’ tray, but with one of the liquid (top) or vapour (bottom)
flows missing, but recycle/reflux added.

Reboiler in a perfectly mixed thermo-siphon reboiler with hold up Mb .

The total mass balance in the condenser and reflux drum is

dMd
=V −R−D
dt
100 Modelling

and the component balance on the top tray

dMd xd
= V yNT − (R + D)xD (3.26)
dt
For the general nth tray, that is trays #2 through N − 1, excepting the feed tray, the total mass
balance is
dMn
= Ln+1 − Ln + Vn−1 − Vn
dt | {z }
≈0
= Ln+1 − Ln (3.27)

and the component balance

dMn xn
= Ln+1 xn+1 − Ln xn + V yn−1 − V yn (3.28)
dt
For the special case of the feed nF th tray,
dMnF
= LnF +1 − LnF + F, mass balance
dt
dMnF xnF
= LnF +1 xnF +1 − LnF xnF + F xF + V ynF −1 − V ynF , component balance
dt
and for the reboiler and column base
dMB
= L1 − V − B
dt
dMB xB
= L1 x1 − BxB − V xB
dt
In summary the number of variables for the distillation column model are:

Tray compositions, xn , yn 2NT

Tray liquid flows, Ln NT
VARIABLES Tray liquid holdups, Mn NT
Reflux comp., flow & hold up, xD , R, D, MD 4
Base comp., flow & hold up, xB , yB , V, B, MB 5
Total # of equations 4NT + 9

and the number of equations are:

Tray component balance, NT

Tray mass balance NT
Equilibrium (tray + base) NT + 1
E QUATIONS
hydraulic NT
Reflux comp. & flow 2
Base comp & flow 2
Total # of equations 4NT + 7

Which leaves two degrees of freedom. From a control point of view we normally fix the boildup
rate, Q̇, and the reflux flow rate (or ratio) with some sort of controller,

R = f (xD ), V ∝ Q̇ = f (xB )

Our dynamic model of a binary distillation column is relatively large with two inputs (R, V ), two
outputs (xB , xD ) and 4N states. Since a typical column has about 20 trays, we will have around
3.3. CHEMICAL PROCESS MODELS 101

n = 44 states. which means 44 ordinary differential equations However the (linearised) Jacobian
for this system while a large 44 × 44 matrix is sparse. In our case the percentage of non-zero
elements or sparsity is ≈ 9%.
 
∂ ẋ .. ∂ ẋ
 ∂x .
|{z} ∂u 
|{z}
44×44 44×2

The structure of the A and B matrices are, using the spy command, given in Fig. 3.10.

Jacobian of the Binary distiilation column model

0

10

15
ODE equations, (zdot=)

20

25

30

35

40

45
0 5 10 15 20 25 30 35 40 45
States & input, ([holdup, comp*holdup| [R V])

Figure 3.10: The incidence of the Jacobian and B matrix for the ideal binary distillation column
model. Over 90% of the elements in the matrix are zero.

There are many other examples of distillation column models around (e.g. [142, pp 459]) This
model has about 40 trays, and assumes a binary mixture at constant pressure and constant rela-
tive volatility.

S IMULATION OF THE DISTILLATION COLUMN

The simple nonlinear dynamic simulation of the binary distillation column model can be used in
a number of ways including investigating the openloop response, interactions and quantify the
extent of the nonlinearities. It can be used to develop simple linear approximate transfer function
models, or we could pose “What if?” type questions such as quantifying the response given feed
disturbances.

An openloop simulation of a distillation column gives some idea of the dominant time constants
and the possible nonlinearities. Fig. 3.11 shows an example of one such simulation where we
step change:

1. the reflux from R = 128 to R = 128 + 0.2, and

2. the reboiler from V = 178 to V = 178 + 0.2

102 Modelling

Step change in Reflux Step change in Vapour

1 1

0.995 0.995
Distill conc

0.99 0.99

0.985 0.985

0.98 0.98

0.975 0.975
0 10 20 30 0 10 20 30

0.1 0.1

0.08 0.08
base conc

0.06 0.06

0.04 0.04

0.02 0.02

0 0
0 10 20 30 0 10 20 30
time (min) time (min)

Figure 3.11: Open loop response of the distillation column control

From Fig. 3.11 we can note that the open loop step results are overdamped, and that steady-state
gains are very similar in magnitude. Furthermore the response looks very like a 2 × 2 matrix of
second order overdamped transfer functions,
    
xD (s) G11 G12 R(s)
=
xB (s) G21 G22 V (s)

So it is natural to wonder at this point if it is possible to approximate this response with a low-
order model rather than requiring the entire 44 states and associated nonlinear differential equa-
tions.

C ONTROLLED SIMULATION OF A DISTILLATION COLUMN

A closed loop simulation of the 20 tray binary distillation column with a feed disturbance from
xF = 0.5 to xF = 0.54 at t = 10 minutes is given in Fig. 3.12.

We can look more closely in Fig. 3.13 at the distillate and base compositions to see if they really
are in control, x⋆b = 0.02, x⋆d = 0.98.

Distillation columns are well known to interact, and these interactions cause difficulties in tuning.
We will simulate in Fig. 3.14 a step change in the distillate setpoint from x⋆D = 0.98 to x⋆D = 0.985
at t = 10 min and a step change in bottoms concentration at 150 minutes. The interactions
are evident in the base composition transients owing to the changing distillate composition and
visa versa. These interactions can be minimised by either tightly tuning one of the loops so
consequently leaving the other ‘loose’, or by using a steady-state or dynamic decoupler, or even
a multivariable controller.
3.3. CHEMICAL PROCESS MODELS 103

Binary distillation column model

1

0.8
Liquid concentration

0.6

0.4

0.2

0
130

128
R

126

124
185
Figure 3.12: A closed loop simulation of a
binary distillation column given a feed dis-
180
V

turbance. The upper plot shows the con-

centration on all the 20 trays, the lower
175 plots shows the manipulated variables (re-
0 20 40 60 80 100
time (min) flux and boil-up) response.

0.99
Distillate

0.985

0.98

0.975
0.04
Bottoms

0.03

0.02 Figure 3.13: Detail of the distillate and base

concentrations from a closed loop simula-
0.01 tion of a binary distillation column given a
0 20 40 60 80 100
time [mins] feed disturbance. (See also Fig. 3.12.)

3.3.4 I NTERACTION AND THE R ELATIVE G AIN A RRAY

The Wood-Berry distillation column model is an example of a multivariable, interacting process.

This interaction is evident from the presence of the non-zero off diagonal terms in Eqn. 3.24. A
quantitative measure of the extent of interaction in a multivariable process is the Relative Gain
Array (RGA) due to Bristol, [35]. The RGA is only applicable to square systems (the number of
manipulated variables equal the number of controlled variables), and is a steady-state measure.

The RGA, given the symbol Λ, is an (n × n) matrix where each element is the ratio of the open
loop gain divided by the closed loop gain. Ideally, if you are the sort who do not like interactions,
you would like Λ to be diagonally dominant. Since all the rows and all the columns must sum to
1.0, then a system with no interaction will be such that Λ = I.

The open loop steady-state gain is easy to determine. Taking the Wood-Berry model, Eqn. 3.24,
104 Modelling

Distillate xD
0.99

0.98

0.04

0.03

Base, xB
0.02

0.01

0
160
Reflux
140

120
200
reboiler

180
Figure 3.14: Distillation interactions are ev-
ident when we step change the distillate 160
0 50 100 150 200 250 300
and bottoms setpoints independently. time (min)

as an example, the final steady-state as a function of the manipulated variable u, is

 
12.8 −18.9
yss = Gss u = u (3.29)
6.6 −19.4

The ijth element of Gss is the open loop gain between yi and uj . The Gss matrix can be evaluated
experimentally or formed by applying the final value theorem to Eqn 3.24. Now the closed loop
gain is a function of the open loop gain, so only the open loop gains are needed to evaluate the
RGA. Mathematically the relative gain array, Λ, is formed by multiplying together Gss and G−⊤ ss
elementwise,3

⊤
Λ = Gss ⊗ G−1 ss (3.30)
where the special symbol ⊗ means to take the Hardamard product (also known as the Schur
product), or simply the elementwise product of two equally dimensioned matrices as opposed to
the normal matrix multiplication.

In M ATLAB, the evaluation of the relative gain array Λ is easy.

1 >>Gss = [12.8, -18.9; 6.6, -19.4] % Steady-state gain from Eqn. 3.29
>>L = Gss.*inv(Gss)' % See Eqn. 3.30. Don’t forget the dot-times (.*)

which should return something like

 
2.0094 −1.0094
Λ=
−1.0094 2.0094

Note that all the columns and rows sum to 1.0. We would expect this system to exhibit severe
interactions, although the reverse pairing would be worse.

The usefulness of the RGA is in choosing which manipulated variable should go with which
control variable in a decentralised control scheme. We juggle the manipulated/control variable
parings until Λ most approaches the identity matrix. This is an important and sensitive topic in
 ⊤
3 Note that this does not imply that Λ = Gss G−1
ss , i.e. without the Hardamard product. See [182, p456] for further
details.
3.3. CHEMICAL PROCESS MODELS 105

process control and is discussed in more detail in [194, pp494–503], and in [182, p457]. The most
frequently discussed drawback of the relative gain array, is that the technique only addresses the
steady state behaviour, and ignores the dynamics. This can lead to poor manipulated/output
pairing in some circumstances where the dynamic interaction is particularly strong. The next
section further illustrates this point.

T HE DYNAMIC RELATIVE GAIN ARRAY

As mentioned above, the RGA is only a steady-state interaction indicator. However we could
use the same idea to generate an interaction matrix, but this time consider the elements of the
transfer function matrices as a function of frequency by substituting s = jω. This now means
that the dynamic relative gain array, Λ(ω), is a matrix where the elements as functions of ω as
opposed to being constants.

Consider the (2 × 2) system from [184]

 
  2.5e−5s 5  
y1  (15s + 1)(2s + 1)  u1
4s + 1 
= (3.31)
y2  1 −4e−5s  u2
3s + 1 20s + 1
| {z }
G(s)

which has as its distinguishing feature significant time delays on the diagonal elements. We could
compute the dynamic relative gain array matrix using the definition

Λ(s) = G(s) ⊗ G(s)−T (3.32)

perhaps using the symbolic toolbox to help us with the possibly unwieldy algebra.

Listing 3.1: Computing the dynamic relative gain array analytically

>>syms s
>>syms w real
3

>>G=[2.5*exp(-5*s)/(15*s+1)/(2*s+1), 5/(4*s+1); ...

1/(3*s+1), -4*exp(-5*s)/(20*s+1)] % Plant from Eqn. 3.31

>>RGA = G.*inv(G') % Λ(s), Eqn. 3.32

8 >>DRGA = subs(RGA,'s',1j*w) % Substitute s = jω

>> abs(subs(DRGA,w,1)) % Magnitude of the DRGA matrix at ω = 1 rad/s

ans =
0.0379 0.9793
13 0.9793 0.0379

You can see that from the numerical values of the elements of Λ at ω = 1 rad/s that this system
is not diagonally dominant at this important frequency. Fig. 3.15 validates this observation.

An alternative numerical way to generate the elements of the DRGA matrix as a function of
frequency is to compute the Bode diagram for the multivariable system, extract the current gains,
and then form the RGA from Eqn. 3.30.
106 Modelling

Listing 3.2: Computing the dynamic relative gain array numerically as a function of ω. See also
Listing 3.1.
G = tf({2.5, 5; 1 -4}, ...
2 {conv([15 1],[2,1]),[4 1]; [3 1],[20 1]}, ...
'ioDelayMatrix',[5,0; 0,5]) % Plant from Eqn. 3.31

[¬,¬,K] = zpkdata(G); sK =sign(K); % get sign of the gains

[Mag,Ph,w] = bode(G); % Compute Bode diagram
7 DRGA = NaN(size(Mag));

for i=1:length(w)
K = Mag(:,:,i).*sK; % Gain, including signs
DRGA(:,:,i) = K.*(inv(K))'; % Λ(ω) = K ⊗ K−T
12 end

OnDiag = squeeze(DRGA(1,1,:)); OffDiag = squeeze(DRGA(1,2,:));

semilogx(w, OnDiag,'-',w,OffDiag,'--'); % See Fig. 3.15

The trends of the diagonal and off-diagonal elements of Λ are plotting in Fig. 3.15. What is
interesting about this example is that if we only consider the steady-state case, ω = 0, then
Λ is diagonally dominant, and our pairing looks suitable. However what we should be really
concentrating on are the values around the corner frequency at ω ≈ 0.1 where now the off-
diagonal terms start to dominate.

1
λ
1,1
0.8 λ
1,2

0.6

0.4

0.2

Figure 3.15: The diagonal and off-diagonal el-

ements of the RGA matrix as a function of 0
−3 −2 −1 0 1
10 10 10 10 10
frequency. frequency

For comparison, Fig. 3.16 shows the dynamic RGA for the (3 × 3) OLMR distillation column
model from Eqn. 3.25. In this case the off-diagonal elements do not dominate at any frequency.

2.5
λ11
2
λ
22 Diagonal elements
1.5
λ33
elements of Λ(ω)

0.5

0 λ
23
λ
−0.5 13 Off−diagonal elements
λ
−1 12
Figure 3.16: Comparing the diagonal and
off-diagonal elements of the (3 × 3) RGA −1.5
−3 −2 −1 0 1
10 10 10 10 10
matrix from Eqn. 3.25. frequency
3.4. CURVE FITTING EXPERIMENTAL DATA 107

3.4 C URVE FITTING EXPERIMENTAL DATA

In most cases of industrial importance, the models are never completely white-box, meaning that
there will always be some parameters that need to be fitted, or at least fine-tuned to experimental
data. Generally the following three steps are necessary:

1. Collect N experimental independent xi , and dependent yi data points.

2. Select a model structure, M(θ) with a vector of parameters θ that are to be estimated.
3. Search for the parameters that optimise some sort performance index such as to maximise
the “goodness of fit”, or equivalently minimise the sum of squared residuals.

In this, and the following sections, we will colour the parameters to be estimated in green to
distinguish them clearly from the experimental (x, y) data pairs.

A common performance index or loss function is to select the adjustable parameters so that the
sum of difference squared between the raw data and your model predictions is minimised. This
is the classical Gaussian least-squares minimisation problem. Formally we can write the perfor-
mance index as,
N
X 2
J = (yi − ŷi ) (3.33)
i=1
where the ith model prediction, ŷi is a function of the model, parameters, θ, and input data,
ŷi = f (M(θ), xi ) (3.34)
We want to find the “best” set of parameters, or in other words the set of θ that minimises J in
Eqn. 3.33,
(N )
X
θ ⋆ = arg min (yi − ŷi )2 (3.35)
θ i=1
The “arg min” part of Eqn 3.35 can be read as “the argument who minimises . . . ” since we
are not too interested in the actual value of the performance function at the optimum, J , but
rather the value of the parameters at the optimum, θ⋆ . This least-squared estimation is an intuitive
performance index, and has certain attractable analytical properties. The German mathematician
Gauss is credited with popularising this approach, and indeed using it for analysing data taken
while surveying Germany and making astronomical observations. Note however other possible
objectives rather than Eqn. 3.33 are possible, such as minimising the sum of the absolute value
of the deviations, or minimising the single maximum deviation. Both these latter approaches
have seen a resurgence of interest of the last few years since the computer tools have enabled
investigators to by-pass the difficulty in analysis.

There are many ways to search for the parameters, and any statistical text book will cover these.
However the least squares approach is popular because it is simple, requires few prior assump-
tions, and typically gives good results.

3.4.1 P OLYNOMIAL REGRESSION

An obvious way to fit smooth curves to experimental data is to find a polynomial that passes
through the cloud of experimental data. This is known as polynomial regression. The nth order
polynomial to be regressed is
M(θ) : ŷ = θn xn + θn−1 xn−1 + · · · + θ1 x + θ0 (3.36)
108 Modelling

where we try different values of the (n+1) parameters in the vector θ until the difference between
the calculated dependent variable ŷ is close to the actual measured value y. Mathematically the
vector of parameters θ is obtained by solving a least squares optimisation problem.

Given a set of parameters, we can compute the ith model prediction using
 
θn
 θn−1 
 
xn−1 · · · x 1 ·  ... 
 n   
ŷi = x (3.37)
 
 θ1 
θ0

or written in compact matrix notation

ŷi = xi θ
where xi is the data row vector for the ith observation. If all n observations are stacked vertically
together, we obtain the matrix system
   n 
ŷ1 x1 x1n−1 · · · x1 1
 ŷ2   xn2 x1 n−1
   · · · x2 1  
 ..  =  .. .. .. .. ..  θ (3.38)
 .   . . . . . 
n−1
ŷN xnN xN · · · xN 1
| {z }
X
or y = Xθ in a more compact matrix notation. The matrix of comprised of the stacked rows
of measured independent data, X, in Eqn. 3.38 is called the Vandermonde matrix or data matrix
and can be easily constructed in M ATLAB using the vander command, although it is well known
that this matrix can become very illconditioned.

We can search for the parameters in a number of ways, but minimising the squared error is
a common and easy approach. In this case our objective function that we want to minimise
(sometimes called the cost function), is the summed squared error. In matrix notation, the error
vector is
def
ǫ = y − Xθ (3.39)
and the scalar cost function then becomes
⊤
J = ǫ⊤ ǫ = (y − Xθ) (y − Xθ)
= y⊤ y − θ ⊤ X⊤ y − y⊤ Xθ + θ⊤ X⊤ Xθ

We want to choose θ such that J is minimised, ie a stationary point, thus we can set the partial
derivatives of J with respect to the parameters to zero,4

∂J
=0
∂θ
= −2X⊤ y + 2X⊤ Xθ (3.40)

which we can now solve for θ as  −1

θ = X⊤ X X⊤ y (3.41)
| {z }
pseudo-inverse
As a consequence of the fact that we carefully chose our model structure in Eqn. 3.36 to be linear
in the parameters, θ, then the solution given by Eqn. 3.41 is analytic and therefore very reliable,
4 Ogata, [150, p938], gives some helpful rules when using matrix-vector differentials.
3.4. CURVE FITTING EXPERIMENTAL DATA 109

and straight forward to implement as opposed to nonlinear regression which require iterative
solution techniques.

This method of fitting a polynomial through experimental data is called polynomial least-squares
regression. In general, the number of measurements must be greater (or equal) to the number
of parameters. Even with that proviso, the data matrix X⊤ X can get very ill-conditioned, and
hence it becomes hard to invert in a satisfactorily manner. This problem occurs more often when
high order polynomials are used or when you are trying to over-parameterise the problem. One
solution to this problem is given in §3.4.2.
 −1
The matrix X⊤ X X⊤ is called the left pseudo-inverse of X, and is sometimes denoted X+ .
The pseudo inverse is a generalised inverse applicable for even non-square matrices and is dis-
 −1
cussed in [152, p928]. M ATLAB can compute X⊤ X X⊤ with the pseudo inverse command,
pinv, enabling the parameters to be evaluated simply by typing theta = pinv(X)*y, al-
though it is more efficient to simply use the backslash command, theta = X\y.

A N EXAMPLE OF POLYNOMIAL CURVE FITTING USING LEAST- SQUARES

Tabulated below is the density of air as a function of temperature. We wish to fit a smooth
quadratic curve to this data.

Temperature [◦ C] −100 −50 0 60 100 160 250 350

Air density [kg/m3 ] 1.98 1.53 1.30 1.067 0.946 0.815 0.675 0.566

2.5
Raw Data
fitted curve
2
Air density, [kg/m ]
3

1.5

1
Figure 3.17: The density of air as a func-
tion of temperature. Experimental data,
0.5
•, and a fitted quadratic curve. (See also
−200 −100 0 100 200 300 400 Fig. 3.18 following for a higher-order poly-
°
Temperature [ C] nomial fit.)

To compute the three model parameters from the air density data we can run the following m-file.

Listing 3.3: Curve fitting using polynomial least-squares

T= [-100 -50 0 60 100, 160 250 350]'; % Temperature, T
rho= [1.98, 1.53, 1.30, 1.067 0.946, 0.815, 0.675, 0.566]'; % Air density, ρ

X =vander(T);X =X(:,end-2:end); % Keep only last 3 columns of the Vandermonde matrix

5 theta = X\rho; % Solve for θ, where ρ̂ = θ1 T 2 + θ2 T + θ3

Ti = linspace(-130,400); % validate with a plot

rho_pred = polyval(theta,Ti);
plot(T,rho,'o',Ti,rho_pred,'r-') % See Fig. 3.17.
110 Modelling

The resulting curve is compared with the experimental data in Fig. 3.17.

In the example above, we constructed the Vandemonde data matrix explicitly, and solved for the
parameters using the pseudo-inverse. In practice however, we would normally simply use the
built-in M ATLAB command polyfit which essentially does the same thing.

3.4.2 I MPROVEMENTS TO THE LEAST SQUARES ESTIMATION ALGORITHM

There are many extensions to this simple multi-linear regression algorithm that try to avoid the
poor numerical properties of the scheme given above. If you consider pure computational speed,
then solving a set of linear equations is about twice as fast as inverting a matrix. Therefore instead
of Eqn. 3.41, the equivalent
X⊤ Xθ = X⊤ y (3.42)
is the preferred scheme to calculate θ. M ATLAB ’ S backslash operator, \, used in the example
above follows this scheme internally. We write it as if we expect an inverse, but it actually solves
a system of linear equations.

A numerical technique that uses singular value decomposition (SVD) gives better accuracy (for
the same number of bits to represent the data), than just applying Eqn 3.41, [163, 179]. Singular
values stem from the property that we can decompose any matrix T into the product of two
orthogonal matrices (U, V) and a diagonal matrix Σ,

T = UΣV⊤ (3.43)
where due to the orthogonality,

UU⊤ = I = U⊤ U and VV⊤ = I = V⊤ V (3.44)

The diagonal matrix Σ consists of the square roots of the eigenvalues of T⊤ T which are called
the singular values of T, and the number that differ significantly from zero is the rank of T. We
can make one further modification to Eqn 3.41 by adding a weighting matrix W. This makes the
solution more general, but often in practice one sets W = I. Starting with Eqn 3.42 including the
weighting matrix W−1 = G⊤ G,

X⊤ W−1 Xθ = X⊤ W−1 y (3.45)

⊤ ⊤ ⊤ ⊤
X G GXθ = X G Gy (3.46)
def def
Now we define T = GX and z = Gy gives

T⊤ Tθ = T⊤ z (3.47)
Now we take the singular value decomposition (SVD) of T in Eqn 3.47 giving
  
VΣU⊤ UΣV⊤ θ = VΣU⊤ z (3.48)
VΣ2 V⊤ θ = (3.49)

Multiplying both sides by V⊤ and noting that Σ is diagonal we get,

Σ2 V⊤ θ = ΣU⊤ z (3.50)
⊤ −1 ⊤
| {z } θ = VΣ U z
VV (3.51)
identity

θ = |VΣ−1 ⊤
{z U } z (3.52)
pseudo inverse
3.4. CURVE FITTING EXPERIMENTAL DATA 111

The inverse of Σ is simply the inverse of each of the individual diagonal (and non-zero) elements,
since it is diagonal. The key point here is that we never in the algorithm needed to calculate the
possibly ill-conditioned matrix T⊤ T. Consequently we should always use Eqn. 3.52 in preference
to Eqn. 3.41 due to the more robust numerical properties as shown in Listing 3.4.

Listing 3.4: Polynomial least-squares using singular value decomposition. This routine follows
from, and provides an alternative to Listing 3.3.
1 [U,S,V] = svd(X,0); % Use the ‘economy sized’ SVD, UΣV⊤ = X
theta2 = V*(S\U')*rho % θ = VΣ−1 U⊤ , Eqn. 3.52.

It does not take much for the troublesome XT X matrix to get ill-conditioned. Consider the tem-
perature/density data for air from Fig. 3.17, but in this case we will use temperature in degrees
Kelvin (as opposed to Celsius), and we will take data over slightly larger range of temperatures.
This is a reasonable experimental request, and so we should expect to be able to fit a polynomial
to this new data in much the same way we did in Listing 3.3.

However as shown in Fig. 3.18, we cannot reliably fit a fifth-order polynomial using standard
least-squares to this data set, although we can reliably fit such a 5th-order polynomial using the
SVD strategy of Eqn. 3.52. Note that M ATLAB’s polyfit uses the reliable SVD strategy.

Polynomial order = 5
3.5
Raw Data
3 Standard least−squares standard LS
θ = (X ′ X)−1 X ′ y SVD fitted
2.5
Air density, [kg/m3]

LS using SVD
Figure 3.18: Fitting a high-
2 θ = V Σ−1 U T y
order polynomial to the den-
1.5 sity of air as a function of tem-
perature in degrees Kelvin.
1 Experimental data, a fitted
quintic curve using standard
0.5 least squares, and one us-
ing the more robust singu-
0
100 200 300 400 500 600 700 lar value decomposition. (See
Temperature [degrees Kelvin] also Fig. 3.17.)

A further refined regression technique is termed partial least squares (PLS), and is summarised
in [75]. Typically these schemes decompose the data matrix X, into other matrices that are
ordered in some manner that roughly corresponds to information. The matrices with the least
information (and typically most noise) are omitted, and the subsequent regression uses only the
remaining information.

The example on page 109 used the least squares method for estimating the parameters of an
algebraic equation. However the procedure for estimating a dynamic equation remains the same.
This will be demonstrated later in §6.7.

Problem 3.3 Accuracy and numerical stability are very important when we are dealing with com-
puted solutions. Suppose we wish to invert
 
1 1
A=
1 1+ǫ
where ǫ is a small quantity (such as nearly the machine eps)
112 Modelling

1. What is the (algebraic) inverse of A?

2. Obviously if ǫ = 0 we will have trouble inverting since A is singular, but what does the
pseudo-inverse, A+ , converge to as ǫ → 0?

3.4.3 N ONLINEAR LEAST- SQUARES MODEL IDENTIFICATION

If the model equation is nonlinear in the parameters, then the solution procedure to find the opti-
mum parameters requires a nonlinear optimiser rather than the relatively robust explicit relation
given by Eqn. 3.41 or equivalent. Nonlinear optimisers are usually based on iterative schemes
additionally often requiring good initial parameter estimates and even then may quite possibly
fail to converge to a sensible solution. There are many algorithms for nonlinear optimisation
problems including exhaustive search, the simplex method due to Nelder-Mead, and gradient
methods.

M ATLAB provides a simple unconstrained optimiser, fminsearch, which uses the Nelder-Mead
simplex algorithm. A collection of more robust algorithms and algorithms to optimise con-
strained systems possibly with integer constraints is the O PTI toolbox5 .

The following biochemical example illustrates the solution of a nonlinear algebraic optimisation
problem. Many biological reactions are of the Michalis-Menton form where the cell number y(t)
is given at time t by the relation
αt
y= (3.53)
βt + 1
Suppose we have some experimental data for a particular reaction given as

time, t 0 0.2 0.5 1 1.5 2 2.5

cell count, y 0 1.2 2.1 2.43 2.52 2.58 2.62

where we wish to estimate the parameters α and β in Eqn 3.53 using nonlinear optimisation tech-
niques. Note that the model equation (3.53) is nonlinear in the parameters. While it is impossible
to write the equation in the linear form y = f (t) · θ as done in §3.4, it is possible to linearise the
equation (Lineweaver-Burke plot) by transforming the equation. However this will also trans-
form and bias the errors around the parameters. This is not good practice as it introduces bias
in the estimates, but does give good starting estimates for the nonlinear estimator. Transforming
Eqn. 3.53 we get
α y
y= −
β βt
Thus plotting y/t against y should result in a straight line with an intercept of α/β and a slope of
−1/β.

3.5

Assuming we have the experimental data stored as

3
column vectors [t,y] in M ATLAB, we could plot
cell count y

2.5

t = [0, 0.2, 0.5:0.5:2.5]';

2
y = [0,1.2,2.1,2.43,2.52,2.58,2.62]';
3 % ignore divide-by-zero 1.5
plot(y./t,y,'o-')
1
0 1 2 3 4 5 6
y/t

5 The O PTI toolbox is available from www.i2c2.aut.ac.nz

3.4. CURVE FITTING EXPERIMENTAL DATA 113

which should given an approximately straight line (ignoring the first and possibly the second
points). To find the slope and intercept, we use the polyfit function to fit a line and we get a
slope = −0.2686 and intercept = 2.98. This corresponds to α = 11.1 and β = 3.723.

Now we can refine these estimates using the fminsearch Nelder-Mead nonlinear optimiser.
We must first construct a a small objection function file which evaluates the sum of the squared
error for a given set of trial parameters and the experimental data. This is succinctly coded as an
anonymous function in Listing 3.5 following.

Since we require the experimental data variables y and t in the objective function, but we are not
optimising with respect to them, we must pass these variables as additional parameters to the
optimiser.

Listing 3.5: Curve fitting using a generic nonlinear optimiser

1 % Compute sum of square errors given trial θ and (t, y) data.
sse= @(theta,t,y) sum((y-theta(1)*t./(theta(2)*t+1)).ˆ2);

optns = optimset('Diagnostics','on','Tolx',1e-5);
theta = fminsearch(sse,[11.1 3.723]',optns,t,y) % Polish the estimates.
6 plot(t,y,'o',t, theta(1)*t./(theta(2)*t+1)) % Check final fit in Fig. 3.19.

Listing 3.5 returns the refined estimates for θ as

   
α 12.05
θ̂ = =
β 4.10
and a comparison of both the experimental data •, and the model’s predictions is given in Fig. 3.19.

3 α/β

2.5

2
cell count

1.5

0.5 Exptal data Figure 3.19: The fitted (dashed) and experi-
Best fit curve mental, •, data for a bio-chemical reaction.
0
0 0.5 1 1.5 2 2.5 3 3.5
The asymptotic final cell count, α/β, is given
time by the dashed horizontal line.

C URVE FITTING USING THE O PTI OPTIMISATION TOOLBOX

The M ATLAB O PTIMISATION toolbox contains more sophisticated routines specifically intended
for least-squares curve fitting. Rather than write an objective function to compute the sum of
squares and then subsequently call a generic optimiser as we did in Listing 3.5, we can solve the
problem in a much more direct manner. The opti_lsqcurvefit is the equivalent routine in
the O PTI toolbox that solves least-squares regression problems.

Listing 3.6: Curve fitting using the O PTI optimisation toolbox. (Compare with Listing 3.5.)
114 Modelling

>> theta0 = [11,3]'; % Initial guess θ 0

>> F = @(x,t) x(1)*t./(x(2)*t+1); % f (θ, t) = θ1 t/(θ2 t + 1)
>> >> theta = opti_lsqcurvefit(F,theta0,t,y)
4 theta =
12.0440
4.1035

Note that in Listing 3.6 we encapsulate the function in an anonymous function which is then
passed to the least-squares curve fit routine, opti_lsqcurvefit.

H IGHER DIMENSIONAL MODEL FITTING

Searching for parameters where we have made two independent measurements is just like search-
ing when we have made one independent measurement, except that to plot the results we must
resort to contour or three-dimensional plots. This makes the visualisation a little more difficult,
but changes nothing in the general technique.

We will aim to fit a simple four parameter, two variable function to model the compressibility
of water. Water, contrary to what we taught in school, is compressible, although this is only
noticeable at very high pressures. If we look up the physical properties for compressed water
in steam tables,6 we will find something like table 3.4. Fig. 3.20(a) graphically illustrates the

Table 3.4: Density of compressed water, (ρ × 10−3 kg/m3 )

Pressure (bar) Temperature, ◦ C

0.01 100 200 250 300 350 374.15
100 1.0050 0.9634 0.8711 0.8065 0.7158 — —
221.2 1.0111 0.9690 0.8795 0.8183 0.7391 0.6120 —
500 1.0235 0.9804 0.8969 0.8425 0.7770 0.6930 0.6410
1000 1.0460 1.0000 0.9242 0.8772 0.8244 0.7610 0.7299

strong temperature influence on the density compared with pressure. Note how the missing data
represented by NaNs in M ATLAB is ignored in the plot.

Our model relates the density of water as a function of temperature and pressure. The proposed
model structure is  
b c
ρ = P k exp a + + 3 (3.54)
T T
where the model parameters
def  ⊤
θ= a b c k (3.55)
need to be determined.

Again the approach is to minimise numerically the sum of the squared errors using an optimiser
such as fminsearch. We can check the results of the optimiser using a contour plot with the
experimental data from Table 3.4 superimposed.

The script file in Listing 3.7 calls the minimiser which in turn calls the anonymous function
J_rhowat which given the experimental data and proposed parameters returns the sum of
squared errors. This particular problem is tricky, since it is difficult to know appropriate starting
guesses for θ, and the missing data must eliminated before the optimisation. Before embarking
6 Rogers & Mayhew, Thermodynamic and Transport properties of Fluids, 3rd Edition, (1980), p11
3.4. CURVE FITTING EXPERIMENTAL DATA 115

density (kg/m3)
1100

1000
1000
1100 900
950 850
800
3

1000
True density kg/m

775
900

Pressure (bar)
700
900
850
600
800
800 500
700
750 400 700
600
0
1000 700 300
100 700
200 200 1e+003 1e+003 925
925 850 775
500 650 1e+003
300 100
° 400 0 Pressure (bar) 0 50 100 150 200 250 300 350 400
Temperature, C Temperature (deg C)

(a) True density of compressed water as a function of tem- (b) Model fit
perature and pressure.

Figure 3.20: A 2D model for the density of compressed water. In Fig. 3.20(b), the • mark experi-
mental data points used to construct the 2-D model given as contour lines.

on the full nonlinear minimisation, I first try a linear fitting to obtain good starting estimates for
θ. I also scale the parameters so that the optimiser deals with numbers around unity.

Listing 3.7: Fitting water density as a function of temperature and pressure

rhowat = @(a,P,T) P.ˆa(1).*exp(a(2) + ...
1e2*a(3)./T + 1e7*a(4)./T.ˆ3) % Assumed model ρ = P k exp a + Tb + Tc3

J_rhowat = @(a,P,T,rho) ...

norm(reshape(rho,[],1)-reshape(rhowat(a,P,T),[],1)); % SSE J = (ρ − ρ̂)2
P
4

T = [0.01, 100, 200, 250,300,350, 374.15]'; % Temp [deg C]

P = [100 221.2 500 1000]'; % Pressure [Bar]
rhof = 1.0e3*[1.00250, 0.9634, 0.8711, 0.8065, 0.7158, NaN, NaN; ...
9 1.0111, 0.9690, 0.8795, 0.8183, 0.7391, 0.6120, NaN; ...
1.0235, 0.9804, 0.8969, 0.8425, 0.7770, 0.6930, 0.6410; ...
1.0460, 1.0000, 0.9242, 0.8772, 0.8244, 0.7610, 0.7299]; % density kg/m3

[TT,PP] = meshgrid(T+273,P); Tv = TT(:); Pv = PP(:); % vectorise

14 A = [ones(size(Tv)), 1.0e2 ./Tv, 1.0e7 ./Tv.ˆ3, log(Pv)]; % Scaled data matrix
idx = isnan(rhof(:)); % find missing data points
rhofv = rhof(:); rhofv(idx) = []; % remove bad points
Tv(idx) = []; Pv(idx) = []; A(idx,:) = [];
theta = A\log(rhofv); % first (linear) estimate
19

% Do nonlinear fit
theta_opt = fminsearch(@(theta) J_rhowat(theta,Pv,Tv,rhofv),theta);

[Ti,Pi] = meshgrid([0.01:10:370]+273,[100:100:1000]); % Compare fit with data

24 rho_est = rhowat(theta_opt,Pi,Ti);

Fig. 3.20(b) compares contour plots of the density of water as a function of pressure and temper-
ature derived from the experimental data and the fitted model. The • shows the location of the
experimental data points. The solid contour lines give the predicted density of water compared
with the contours derived from the experimental data (dashed lines). The optimum parameters
116 Modelling

found above are    

a 5.5383
2
b  =  5.8133 · 10 7
   
θ⋆ = 
 c   −1.8726 · 10

 (3.56)
k 0.0305

3.4.4 T HE CONFIDENCE OF THE OPTIMISED PARAMETERS

Finding the optimum model parameters to fit the data is only part of the task. A potentially
more difficult objective is to try to establish how precise these optimum parameters are, or what
the confidence regions of the parameters are. simply expressing the uncertainty of your model as
parameter values ± some interval is a good first approximation, but it does neglect the interaction
of the other parameters, the so-called correlation effect.

To establish the individual confidence limits of the parameters, we need to know the following:

1. The n optimum parameters θ from the m experimental observations. (For the nonlinear
case, this can be found using a numerical optimiser)
2. A linearised data matrix X centered about the optimised parameters.
3. The measurement noise sȲi (This can be approximated from the sum of the squared error
terms, or from prior knowledge.)
4. Some statistical parameters such as the t-statistic as a function of confidence interval (90%,
95% etc) and degrees of freedom. This is easily obtained from statistical tables or using the
qt routine from the S TIXBOX collection mentioned on page 3 for m-file implementations of
some commonly used statistical functions.

The confidence interval for parameter θi is therefore

p
θi ± t(1−α/2) sȲi Pi,i (3.57)

where t(1−α/2) is the t-statistic evaluated at ν = n − m (number of observations less the number
of parameters) degrees of freedom and Pi,i are the diagonal elements of the covariance matrix P.
The measurement noise (if not already approximately known) is
s
p ǫ⊤ ǫ
sȲi ≈ s2r = (3.58)
n−m

and the covariance matrix P is obtained from the data matrix. Note that if we assume we have
perfect measurements and a perfect model, then we would expect that the sum of the errors
will be exactly zero given the true parameters. This is consistent with Eqn 3.58 when ǫ⊤ ǫ = 0
although of course this occurs rarely in practice!

A N EXAMPLE OF ESTABLISHING CONFIDENCE INTERVALS FOR A NONLINEAR MODEL

This section was taken from [88, p198], but with some modifications and corrections. Himmel-
blau, [88, p198], gives some reaction data as:

Pressure, p 20 30 35 40 50 55 60
rate, r 0.068 0.0858 0.0939 0.0999 0.1130 0.1162 0.1190
3.4. CURVE FITTING EXPERIMENTAL DATA 117

and proposes a two parameter nonlinear model of the form

θ0 p
r̂ = (3.59)
1 + θ1 p
Now suppose that we use a nonlinear optimiser program to search for the parameters θ such that
the sum of squared error is minimised.
n
X
min = (ri − r̂i )2 = (r − r̂)T (r − r̂) (3.60)
θ i=1

If we do this using say the same technique as in §3.4.3, we will find that the optimum parameters
are approximately    
def θ0 5.154 · 10−3
θ= = (3.61)
θ1 2.628 · 10−2
and the raw data and model predictions are given in Fig 3.21.

Now the linearised data matrix is a n by m matrix where n is the number of observations and m
is the number of parameters. The partial derivatives of 3.59 with respect to the parameters are;

∂r p ∂r −θ0 p2
= , =
∂θ0 1 + θ1 p ∂θ1 (1 + θ1 p)2

and the data matrix X is defined as

def ∂ri
Xi,j = (3.62)
∂θj
Thus each row of X are the partial derivatives of that particular observation with respect for the
parameters. The covariance matrix P is defined as
 −1
def
P= X⊤ X (3.63)

and should be positive definite and symmetric just as the variance should always be greater
than, or equal to zero. However in practice this requirement may not always hold owing to poor
numerical conditioning. The diagonal elements are the variances we will use for the individual
confidence limits. When using M ATLAB, it is better to look at the singular values, or check the
rank of X⊤ X before doing the inversion.

Listing 3.8 fits parameters to the nonlinear reaction rate model and also illustrates the uncertain-
ties.

Listing 3.8: Parameter confidence limits for a nonlinear reaction rate model
1 p = [20,30,35,40,50,55,60]'; % Pressure, P
r = [0.068,0.0858,0.0939,0.0999,0.1130,0.1162,0.1190]'; % Reaction rate, r

θ0 p
Rxn_rate = @(x,p) x(1)*p./(1+x(2)*p); % r = 1+θ 1p
theta = lsqcurvefit(Rxn_rate,[5e-3 2e-2]',p,r); % Refine estimate of parameters θ
6 nobs = length(p); mpar = length(theta); % # of observations & parameters

r_est = Rxn_rate(theta,p);% Predicted reaction rate r̂(θ, p)

% sum of squared errors J = i ǫ2
P
j = sum((r-r_est).ˆ2);

11 d1= 1+theta(2)*p;
X = [p./d1, -theta(1)*p.*p./d1.ˆ2]; % Data gradient matrix, Eqn. 3.62.
C = inv(X'*X); % not numerically sound ??
118 Modelling

% t-distribution statistics
16 pt = @(x,v) (x≥0).*(1-0.5*betainc(v./(v+x.ˆ2),v/2,0.5)) + ...
(x<0).*(0.5*betainc(v./(v+x.ˆ2),v/2,0.5))
qt = @(P,v) fsolve(@(x) pt(x,v)-P,sqrt(2)*erfinv(2*P-1), ...
optimset('display','off'));

21 alpha = 0.05; % CI level (97.5%)

nu = nobs-mpar; % Degrees of freedom ν = n − m
t = qt(1-alpha,nu); % t statistic from approx function
sy = sqrt(j/nu); % approx measurement noise std
b_ci = t*sy*diag(C); % ± confidence regions

This gives 95% confidence limits for this experiment7 as

   
5.1061 · 10−3 5.2019 · 10−3
<θ< (3.64)
2.2007 · 10−2 3.0553 · 10−2

A plot of the raw data (◦), the model with the optimum estimated parameters and the associated
error bounds using Eqn. 3.64 is given in Fig. 3.21. Note while the model prediction is quite good,
the region defined by the 95% confidence on the parameters is surprisingly large. The error
bounds were generated by simulating the model with the lower and upper bounds of Eqn 3.64
respectively.

Experimental data (o) & model fit

0.15
Data
0.14 model

0.13

0.12

0.11
rate

0.1

0.09

0.08

Figure 3.21: The experimental raw 0.07

data (•), model (—), and associated
0.06
95% error bounds (dotted) for the 10 20 30 40 50 60 70
optimised model fit. pressure

The confidence region (an ellipse in the two variable case) can also be plotted. This gives a deeper
understanding of the parameter interactions, but the visualisation becomes near impossible as
the number of parameters gets much larger than 3. The region for this example is evaluated the
following section.

7 These figures differ from the worked example in Himmelblau. I think he swapped two matrices by mistake, then

continued with this error.

3.5. NUMERICAL TOOLS FOR MODELLING 119

T HE CONFIDENCE REGIONS OF THE PARAMETERS

An approximate confidence region can be constructed by linearising the nonlinear model about
the optimum parameters. The covariance of the parameter estimate is
 −1
cov(b) ≈ X⊤ X σȲ2 i (3.65)

where b is the estimated parameter and β is the true, (but unknown) parameter. Now the confi-
dence region, an ellipse in two parameter space (an ellipsoid in three diameter space) is the region
that with a confidence limit of say 95% we are certain that the true parameter (β) lies within.

(β − b)⊤ (X⊤ X) (β − b) = s2Ȳi mF1−α [m, n − m] (3.66)

where F [m, n − m] is the upper limit of the F distribution for m and n − m degrees of freedom.
This value can be found in statistical tables. Note that the middle term of Eqn 3.66 is X⊤ X and
not the inverse of this.

For the previous example, we can plot the ellipse about the true optimised parameters. Fig 3.22
shows the 95% confidence ellipse about the optimised parameter (marked with a ∗) and the indi-
vidual confidence lengths superimposed from Eqn. 3.64. Since the ellipse is slanted, we note that
the parameters are correlated with each other.

95% Confidence limit

0.032

0.03

0.028
θ2

0.026

0.024

0.022 Figure 3.22: The approximate 95% confi-

dence region in 2 parameter space with
0.02 the individual confidence intervals super-
4.6 4.8 5 5.2 5.4 5.6 5.8
θ1
imposed (marked as ◦).

The ellipse defines an area in which the probability that the true parameter vector lies is better
than 95%. If we use the parameter points marked as a ⋆ in Fig. 3.22, then the model prediction
will differ from the prediction using the optimised parameter vector. This difference is shown as
the errorbars in the lower plot of Fig. 3.22. Note that these errorbars are much smaller than the
errorbars I obtained from considering only the confidence interval for each individual parameter
given in Fig 3.21. Here, owing to the correlation (slanting of the ellipse in Fig 3.22), the parameters
obtained when neglecting the correlation lie far outside the true 95% confidence region. Hence
the bigger errorbars.

3.5 N UMERICAL TOOLS FOR MODELLING

“Unfortunately there are no known methods of solving Eqn 1 (a nonlinear differential equation).
This, of course, is very disappointing.”
M. Braun, [33, p493]
120 Modelling

Since differential equations are difficult to solve analytically, we typically need to resort to nu-
merical methods as described in many texts such as [39, 41, 46, 95, 105, 171] in addition to [205].

Differential equations are broadly separated into two families:

ODEs Ordinary differential equations (ODEs) where time is the only independent variable. The
solution to ODEs can be described using standard plots where the dependent variables are
plotted on the ordinate or y axis against time on the x axis. Ways to numerically solve these
equations are described in section 3.5.1.

PDEs Partial differential equations (PDEs) are where space (and perhaps time) are independent
variables. To display the solution of PDEs, contour maps or 3D plots are required in general.
Generally PDEs are much harder to solve than ODEs and are beyond the scope of this text.

3.5.1 T HE SOLUTION OF NONLINEAR ORDINARY DIFFERENTIAL EQUATIONS

One of the most important “tools” in the desk-top experimenter’s collection, is a good numerical
integrator. We need numerical integrators whenever we are faced with a nonlinear differential
equation, the solutions of which is intractable by any other means. M ATLAB supplies a number of
different numerical integrators optimised for different classess of problems. Typically the 4th/5th
dual order Runge-Kutta implementation, ode45, is a good first choice.

To use the integrators, you must write a function subroutine that contains your differential system
to be solved which will be called by the numerical integration routine.

1. The script file that calls the integrator solver (eg: ode45) with name of the function subrou-
tine to be integrated and parameters including initial conditions, tolerance and time span.

2. The function subroutine (named in the script file) that calculates the derivatives, ẏ, as a
function of the independent variable (usually time), and the dependent variable, f (t, y). In
many cases we are solving a system of ordinary differential equations where the variables
are coupled. In this case we often use the nomenclature x(t) to represent the vector of
dependent variables.

We will demonstrate this procedure by constructing templates of the above two files to solve a
small nonlinear differential equation. One can reuse this template as the base for solving other
ODE systems.

ODE TEMPLATE EXAMPLE

Suppose our task is to investigate the response of the nonlinear pendulum system given in
Eqn. 3.6 and compare this with the linearised version. The script file given in Listing 3.9 com-
putes the angle θ = x1 (t) trajectory for both the nonlinear and linear approximation.

Listing 3.9: Comparing the dynamic response of a pendulum to the linear approximation
g=8.81; l=1; m = 5; T_c=0; % System constants
xdot = @(t,x) [x(2); -g/l*sin(x(1)) + T_c/m/lˆ2]; % dx/dt

x0 = [1,1]'; % start position x(0)

5 [t,x]= ode45(xdot,[0,10],x0); % Integrate nonlinear
3.5. NUMERICAL TOOLS FOR MODELLING 121

% Now try linear system

A = [0 1; -g/l 0]; B = [0;0]; C = [1,0];D=0;
tv = linspace(0,10)';
10 [y2,t2]=lsim(ss(A,B,C,D),0*tv,tv,x0);

plot(t,x(:,1),t2,y2) % See Fig. 3.23.

Note how the trajectories for the nonlinear (solid) result as calculated from ode45 and linear
approximation calculated using lsim gradually diverge in Fig. 3.23.

1.5

0.5

0
x(t)

−0.5

−1
Non Linear
−1.5 Linearised Figure 3.23: The trajectory of the true non-
0 2 4 6 8 10
linear pendulum compared with the lin-
time earised approximation.

Problem 3.4 1. One of my favourite differential equations is called the Van der Pol equation;

d2 y
dy
m 2
− b 1 − y2 + ky = 0
dt dt
where m = 1, b = 3 and k = 2. Solve the equation for 0 < t < 20 using both ode23 and
ode45. Sketch the solution. Which routine is better? You will need to create a function
called vandepol(t,y) which contains the system of equations to be solved.

2. A damped forced pendulum is an example of a chaotic system. The dynamic equations are

dω ω
= − − sin θ + g cos φ (3.67)
dt q
dθ
=ω (3.68)
dt
dφ
= ωd (3.69)
dt
 ⊤  ⊤
where there are three states, (x = ω θ φ
), and three parameters, (ρ = q g ωd ).
 ⊤
Simulate this system for 0 ≤ t ≤ 500 starting from x = −1 2 0.3 and using
 ⊤
ρ = 2 1.5 2/3 as parameters. Be warned, this may take some computer time!

Problem 3.5 A particularly nasty nonlinear chemical engineering type model is found in [84]. This
is a model of two CSTRs8 in series, cooled with a co-current cooling coil. The model equations
8 continuously stirred tank reactors
122 Modelling

are:
 
q E
Ċa1 = (Caf − Ca1 ) − k0 Ca1 exp −
V1 RT1
 
q (−∆H)k0 Ca1 E
Ṫ1 = (Tf − T1 ) + exp − +
V1 ρcp RT1
  
ρc cpc hA1
qc 1 − exp − (Tcf − T1 )
ρcp V1 qc ρc cpc
 
q E
Ċa2 = (Ca1 − Ca2 ) − k0 Ca2 exp −
V2 RT2
 
q (−∆H)k0 Ca2 E
Ṫ2 = (T1 − T2 ) + exp − +
V2 ρcp RT2
      
ρc cpc hA2 hA1
qc 1 − exp − T1 − T2 + exp − (Tcf − T1 )
ρcp V2 qc ρc cpc qc ρc cpc

where the state, control, disturbance and measured variables are defined as
def  ⊤ def def  ⊤
x= Ca1 T1 Ca2 T2 , u = qc , d= Caf Tcf
def
y = Ca2

and the parameters values are given in Table 3.5.

Table 3.5: The parameter values for the CSTR model

description variables value units

reactant flow q 100 l/min
conc. of feed reactant A Caf 1.0 mol/l
temp. of feed Tf 350 K
temp. of cooling feed Tcf 350 K
volume of vessels V1 = V2 100 l
heat transfer coeff. hA1 = hA2 1.67 · 105 J/min.K
pre-exp. constant k0 7.2 · 1010 min−1
E/R E/R 1.0 · 104 K
reaction enthalpy −∆H 4.78 · 104 J/mol
fluid density ρc = ρ 1.0 g/l
heat capacity cp = cpc 0.239 J/g.K

Note that the model equations are nonlinear since the state variable T1 (amongst others) is a
appears nonlinearly in the differential equation. In addition, this model is referred to as control
nonlinear, since the manipulated variable enters nonlinearly. Simulate the response of the con-
centration in the second tank (Ca2 ) to a step change of ±10% in the cooling flow using the initial
conditions given in Table 3.6.

You will need to write a .m file containing the model equations, and use the integrator ode45.
Solve the system over a time scale of about 20 minutes. What is unusual about this response?
How would you expect a linear system to behave in similar circumstances?

Problem 3.6 Develop a M ATLAB simulation for the high purity distillation column model given in
[142, pp459]. Verify the open loop responses given on page 463. Ensure that your simulation is
easily expanded to accommodate a different number of trays, different relative volatility etc.
3.5. NUMERICAL TOOLS FOR MODELLING 123

Table 3.6: The initial state and manipulated variables for the CSTR simulation

description variable value units

coolant flow qc 99.06 l/min
conc. in vessel 1 Ca1 8.53 · 10−2 mol/l
temp. in vessel 1 T1 441.9 K
conc. in vessel 2 Ca2 5.0 · 10−3 mol/l
temp. in vessel 2 T2 450 K

Problem 3.7 A slight complexity to the simple ODE with specified initial conditions is where we
still have the ODE system, but not all the initial conditions. In this case we may know some of the
end conditions instead, thus the system is in principle able to be solved, but not in the standard
manner;– we require a trial and error approach. These types of problems are called two point
boundary problems and we see them arise in heat transfer problems and optimal control. They
are so called two point boundary problems because for ODEs we have two boundaries for the
independent variable; the start and the finish. Try to solve the following from [199, p178].

A fluid enters an immersed cooling coil 10m long at 200 ◦ C and is required to leave at 40 ◦ C. The
cooling medium is at 20 ◦ C. The heat balance is

d2 T
= 0.01(T − 20)1.4 (3.70)
dx2
where the initial and final conditions are;

Tx=0 = 200, Tx=10 = 40

To solve the system, we must rewrite Eqn 3.70 as a system of two first order differential equations,
and supply a guess for the missing initial condition dTdx x=0 =? We can then integrate the system
until x = 10 and check that the final condition is as required. Solve the system.
dT
Hint: Try −47 < dx x=0 < −42.

3.5.2 D IFFERENTIAL /A LGEBRAIC EQUATION SYSTEMS AND ALGEBRAIC LOOPS

In many cases when deriving models from first principals, we end up with a coupled system
of some differential equations, and some algebraic equations. This is often due to our practice
of writing down conservation type equations (typically dynamic), and constraint equations, say
thermodynamic, which are typically algebraic. Such a system in general can be written as

f (ẋ, x, t) = 0 (3.71)

and is termed a DAE or differential/algebraic equation system. If we assume that our model
is “well posed”, that is we have some hope of finding a solution, then we expect that we have
the same number of variables as equations, and it follows that some of the variables will not
appear in the differential part, but only in the algebraic part. We may be able to substitute those
algebraic variables out, such that we are left with only ODEs, which can then be solved using
normal backward difference formula numerical schemes such as rk2.m. However it is more
likely that we cannot extract the algebraic variables out, and thus we need special techniques to
solve these sorts of problems (Eqn. 3.71) as one.

In a now classic article titled Differential/Algebraic Equations are not ODE’s, [160] gives some in-
sight into the problems. It turns out that even for linear DAE systems, the estimate of the error,
124 Modelling

which is typically derived from the difference between the predictor and the corrector, does not
decrease as the step size is reduced. Since most normal ODE schemes are built around this as-
sumption, they will fail. This state of affairs will only occur with certain DAE structures where
the nilpotency or index is ≥ 3. Index 2 systems also tend to cause BDF schemes to fail, but
can be tackled using other methods. The index problem is important because in many times it
can be changed (preferably reduced to less than two), by rearranging the equations. Automated
modelling tools tend, if left alone, to create overly complex models with a high index that are im-
possible to solve. However if we either by using an intelligent symbolic manipulator or human,
change these equations, we may be able to reduce the index.

T HE DIFFICULTIES OF ALGEBRAIC LOOPS

One reason that computer aided modelling tools such as S IMULINK have taken so long to mature
is the problem of algebraic loops. This is a particular problem when the job of assembling the
many different differential and algebraic equations in an efficient way is left to a computer.

Suppose we want to simulate a simple feedback process where the gain is a saturated function of
the output, say
K(y) = max(min(y, 5), 0.1)

If we simulate this in Simulink using

1
1
s+1
Step Gain Transfer Fcn Product Scope

Saturation Gain1

we run into an Algebraic Loop error. M ATLAB returns the following error diagnostic (or something
similar):

Warning: Block diagram ’sgainae’ contains 1 algebraic loop(s).

Found algebraic loop containing block(s):
’sgainae/Gain1’
’sgainae/Saturation’ (discontinuity)
’sgainae/Product’ (algebraic variable)
Discontinuities detected within algebraic loop(s), may have trouble solving

and the solution stalls.

The simplest way to avoid these types of problems is to insert some dynamics into the feedback
loop. In the example above, we could place a transfer function with a unit gain and very small
time constant in place of the algebraic gain in the feedback loop. While we desire the dynamics
of the gain to be very fast so that it approximates the original algebraic gain, overly fast dynamics
cause numerical stability problems, hence there is a trade off.
3.6. LINEARISATION OF NONLINEAR DYNAMIC EQUATIONS 125

3.6 L INEARISATION OF NONLINEAR DYNAMIC EQUATIONS

While most practical engineering problems are nonlinear to some degree, it is often useful to
be able to approximate the dynamics with a linear differential equation which means we can
apply linear control theory. While it is possible to design compensators for the nonlinear system
directly, this is in general far more complicated, and one has far fewer reliable guidelines and
recipes to follow. One particular version of nonlinear controller design called exact nonlinear
feedback is discussed briefly in §8.6.

Common nonlinearities can be divided into two types; hard nonlinearities such as hysterisis,
“stiction”, and dead zones, and soft nonlinearities such as the Arrhenius temperature depen-
dency, power laws, etc. Hard nonlinearities are characterised by functions that are not differen-
tiable, while soft nonlinearities are. Many strategies for compensating for nonlinearities are only
applicable for systems which exhibit soft nonlinearities.

We can approximate soft nonlinearities by truncating a Taylor series approximation of the origi-
nal system. The success of this approximation depends on whether the original function has any
non-differentiable terms such as hysterisis, or saturation elements, and how far we deviate from
the point of linearisation. This section follows the notation of [77, §3.10].

Suppose we have the general nonlinear dynamic plant

ẋ = f (x(t), u(t)) (3.72)
y = g(x, u) (3.73)
and we wish to find an approximate linear model about some operating point (xa , ua ). A first-
order Taylor series expansion of Eqns 3.72–3.73 is
 
∂f  ∂f 
ẋ ≈ f (xa , ua ) + =xa
(x(t) − xa ) + =xa
(u(t) − ua ) (3.74)
∂x x u =ua ∂u x u =ua
 
∂g  ∂g 
y ≈ g(xa , ua ) + =xa
(x(t) − xa ) + =xa
(u(t) − ua ) (3.75)
∂x x u=ua ∂u x u=ua
where the ijth element of the Jacobian matrix ∂f /∂x is ∂f i /∂xj . Note that for linear systems, the
Jacobian is simply A in this notation, although some authors define the Jacobian as the transpose
of this, or AT .

The linearised system Eqn 3.74–3.75 can be written as

ẋ = Ax(t) + Bu(t) + E (3.76)
y = Cx(t) + Du(t) + F (3.77)
where the constant matrices A, B, C and D are defined as
 
∂f  ∂f 
A= =x a
, B= =xa
∂x xu=ua ∂u x u=ua
 
∂g  ∂g 
C= =x a
, D= =xa
∂x xu=ua ∂u x u=ua
and the bias vectors E and F are
 
∂f  ∂f 
E = f (xa , ua ) − =xa
xa − =xa
ua
∂x x u =ua ∂u x u =ua
 
∂g  ∂g 
F = g(xa , ua ) − =xa
xa − =xa
ua
∂x x u=ua ∂u x u=ua
126 Modelling

Note that Eqns 3.76–3.77 are almost in the standard state-space form, but they include the extra
bias constant matrices E and F. It is possible by introducing a dummy unit input to convert this
form into the standard state-space form,
 
  u
ẋ = Ax + B E (3.78)
1
 
  u
y = Cx + D F (3.79)
1

which we can then directly use in standard linear controller design routines such as lsim.

In summary, the linearisation requires one to construct matrices of partial derivatives with respect
to state and input. In principle this can be automated using a symbolic manipulator provided the
derivatives exist. In both the S YMBOLIC T OOLBOX and in M APLE we can use the jacobian
command.

Example: Linearisation using the S YMBOLIC toolbox. Suppose we want to linearise the nonlin-
ear system
"   #
ax1 exp 1 − xb2
ẋ =
cx1 (x2 − u2 )
at an operating point, xa = [1, 2]T , ua = 20.

First we start by defining the nonlinear plant of interest,

>> syms x1 x2 a b c u real

>> x = [x1 x2]';
3 >> fx = [a*x(1)*exp(1-b/x(2)); c*x(1)*(x(2)-uˆ2)]
fx =
[ a*x1*exp(1-b/x2)]
[ c*x1*(x2-uˆ2)]

Now we are ready to construct the symbolic matrix of partial derivatives,

>> Avar = jacobian(fx,x)

Avar =
[ a*exp(1-b/x2), a*x1*b/x2ˆ2*exp(1-b/x2)]
4 [ c*(x2-uˆ2), c*x1]
>> Bvar = jacobian(fx,u)
Bvar =
[ 0]
[ -2*c*x1*u]
9 >> Evar = fx - Avar*x - Bvar*u
Evar =
[ -a*x1*b/x2*exp(1-b/x2)]
[ -c*x1*x2+2*uˆ2*c*x1]

We can substitute a specific set of constants, say, a = 5, b = 6, c = 7 and an operating point

x = [1, 2]T , u = 20, into the symbolic matrices to obtain the numeric matrices.

>> A = subs(Avar,{a,b,c,x1,x2,u},{5,6,7,1,2,20})
A =
3 1.0e+003 *
3.6. LINEARISATION OF NONLINEAR DYNAMIC EQUATIONS 127

0.0007 0.0010
-2.7860 0.0070
>> B = subs(Bvar,{a,b,c,x1,x2,u},{5,6,7,1,2,20})
B =
8 [ 0]
[ -280]
>> E = subs(Evar,{a,b,c,x1,x2,u},{5,6,7,1,2,20})
E =
1.0e+003 *
13 -0.0020
5.5860

At this point we could compare in simulation the linearised version with the full nonlinear model.

3.6.1 L INEARISING A NONLINEAR TANK MODEL

Suppose we wish to linearise the model of the level in a tank given on page 88 where the tank
geometry is such that ρA = 1. The nonlinear dynamic system for the level h is then simplified to

dh √
= −k h + Fin
dt
For a constant flow in, Finss , the resulting steady state level is given by noting that dh/dt = 0, and
so  ss 2
Fin
hss =
k
We wish to linearise the system about this steady-state, so we will actually work with deviation
variables,
def def
x = h − hss , u = Fin − Finss
Now following Eqn. 3.74, we have

∂f ∂f
ḣ = ẋ = f (hss , Finss ) + (h − hss ) + (Fin − Finss )
| {z } ∂h ∂Fin
=0
√
Note that since ∂f /∂h = −k/(2 hss ), then our linearised model about (Finss , hss ) is

−k
ẋ = √ x+u
2 hss
which is in state-space form in terms of deviation variables x and u.

We can compare the linearised model with the nonlinear model in Fig. 3.24 about a nominal input
flow of Finss = 2 and k = 2 giving a steady-state level of hss = 4. Note that we must subtract and
add the relevant biases when using the linearised model.

An alternative, and much simpler way to linearise a dynamic model is to use linmod which
extracts the Jacobians from a S IMULINK model by finite differences.

Listing 3.10: Using linmod to linearise an arbitrary S IMULINK module.

1 k=1; Fin_ss = 2; % Model parameters and steady-state input
hss = (Fin_ss/k)ˆ2 % Steady-state level, hss
[A,B,C,D] = linmod('sNL_tank_linmod',hss,Fin_ss) % Linearise
128 Modelling

Repeating
Sequence
Stair 1 level, h
s
Add1 Add
level
Fin_ss
Math
Fin_ss Gain Function

k sqrt

Scope

simout
x’ = Ax+Bu To Workspace
u−Fin_ss u+Lss
y = Cx+Du
Bias State−Space Bias1

(a) A S IMULINK nonlinear tank model and linear state-space model for comparison

10
full non−linear response
9

7
level

6
linearised response

3
0 10 20 30 40 50
time

(b) Nonlinear and linearised model comparison

Figure 3.24: Comparing the linearised model with the full nonlinear tank level system

A = √
6 -0.2500 % Note −k/(2 hss ) = −1/4
B =
1
C =
1
11 D =
0

Q UANTIFYING THE EXTENT OF THE NONLINEARITY

It is important for the control designer to be able to quantify, if only approximately, the extent
of the open-loop process nonlinearity. If for example the process was deemed only marginally
nonlinear, then one would be confident a controller designed assuming a linear underlying plant
would perform satisfactory. On the other hand, if the plant was strongly nonlinear, then such a
linear controller may not be suitable. Ideally we would like to be able to compute a nonlinear
3.7. SUMMARY 129

metric, say from 0 (identically linear) to 1 (wildly nonlinear) that quantifies this idea simply by
measuring the openloop input/output data. This of course is a complex task, and is going to be
a function of the type input signals, duration of the experiment, whether the plant is stable or
unstable, and if feedback is present.

One such strategy is proposed in [83] and used to assess the suitability of linear control schemes
in [180]. The idea is to compute the norm of the difference between the best linear approxima-
tion and the true nonlinear response for the worst input trajectory within a predetermined set
of trajectories. This is a nested optimisation problem with a min-max construction. Clearly the
choice of linear model family from which to choose the best one, and the choice of the set of input
trajectories will have an effect on the final computed nonlinear measure.

3.7 S UMMARY
9
Stuart Kauffman in a New Scientist article succinctly paraphrased the basis of the scientific
method. He said

“. . . state the variables, the laws linking the variables, and the initial and boundary
conditions, and from these compute the forward trajectory of the biosphere.”

In actual fact he was lamenting that this strategy, sometimes known as scientific determinism
voiced by Laplace in the early 19th century was not always applicable to our world as we under-
stand it today. Nonetheless, for our aim of modeling for control purposes, this philosophy has
been, and will remain for some time I suspect, to be remarkably successful.

Modelling of dynamic systems is important in industry. These types of models can be used for
design and or control. Effective modelling is an art. It requires mathematical skill and engi-
neering judgement. The scope and complexity of the model is dependent on the purpose of the
model. For design studies a detailed model is usually required, although normally only steady
state models are needed at this stage. For control, simpler models (although dynamic) can be
used since the feedback component of the controller will compensate for any model-plant mis-
match. However most control schemes require dynamic models which are more complicated
than the steady state equivalent.

Many of the dynamic models used in chemical engineering applications are built from conser-
vation laws with thermodynamic constraints. These are often expressed as ordinary differential
equations where we equate the rate of accumulation of something (mass or energy) to the inputs,
outputs and generation in a defined control volume. In addition there may be some restrictions
on allowable states, which introduces some accompanying algebraic equations. Thus general
dynamic models can be expressed as a combination of dynamic and algebraic relations
dx
= f (x, u, θ, t) (3.80)
dt
0 = g(x, u θ, t) (3.81)

which are termed DAEs (differential and algebraic equations), and special techniques have been
developed to solve them efficiently. DAEs crop up frequently in automated computer modelling
packages, and can be numerically difficult to solve. [135] provides more details in this field.

Steady state models are a subset of the general dynamic model where the dynamic term, Eqn
3.80, is set to equal zero. We now have an augmented problem of the form Eqn 3.81 only. Linear
9 Stuart Kauffman, God of creativity, 10 May 2008, NewScientist, pp52-53
130 Modelling

dynamic models are useful in control because of the many simple design techniques that exist.
These models can be written in the form

ẋ = Ax + Bu (3.82)

where the model structure and parameters are linear and often time invariant.

Models are obtained, at least in part, by writing the governing equations of the process. If these
are not known, experiments are needed to characterise fully the process. If experiments are used
the model is said to be heuristic. If the model has been obtained from detailed chemical and
physical laws, then the model is said to be mechanistic. In practice, most models are a mixture
of these two extremes. However what ever model is used, it still is only an approximation to the
real world. For this reason, the assumptions that are used in the model development must be
clearly stated and understood before the model is used.
C HAPTER 4

T HE PID CONTROLLER

4.1 I NTRODUCTION

The PID controller is the most common general purpose controller in the chemical process indus-
try today. It can be used as a stand-alone unit, or it can be part of a distributed computer control
system. Over 30 years ago, PID controllers were pneumatic-mechanical devices, whereas nowa-
days they are implemented in software in electronic controllers. The electronic implementation
is much more flexible than the pneumatic devices since the engineer can easily re-program it to
change the configuration of things like alarm settings, tuning constants etc.

Once we have programmed the PID controller, and have constructed something, either in soft-
ware or hardware to control, we must tune the controller. This is surprisingly tricky to do success-
fully, but some general hints and guidelines will be presented in §4.6. Finally, the PID controller
is not perfect for everything, and some examples of common pitfalls when using the PID are
given in §4.8.

4.1.1 P, PI OR PID CONTROL

For many industrial process control requirements, proportional only control is unsatisfactory
since the offset cannot be tolerated. Consequently the PI controller is probably the most com-
mon controller, and is adequate when the dynamics of the process are essentially first or damped
second order. PID is satisfactory when the dynamics are second or higher order. However the
derivative component can introduce problems if the measured signal is noisy. If the process
has a large time delay (dead time), derivative action does not seem to help much. In fact PID
control finds it difficult to control processes of this nature, and generally a more sophisticated
controller such as a dead time compensator or a predictive controller is required. Processes that
are highly underdamped with complex conjugate poles close to the imaginary axis are also diffi-
cult to control with a PID controller. Processes with this type of dynamic characteristics are rare
in the chemical processing industries, although more common in mechanical or robotic systems
comprising of flexible structures.

My own impressions are that the derivative action is of limited use since industrial measurements
such as level, pressure, temperature are usually very noisy. As a first step, I generally only use a PI
controller, the integral part removes any offset, and the two parameter tuning space is sufficiently
small, that one has a chance to find reasonable values for them.

131
132 CHAPTER 4. THE PID CONTROLLER

4.2 T HE INDUSTRIAL PID ALGORITHM

This section describes how to implement a simple continuous-time PID controller. We will start
with the classical textbook algorithm, although for practical and implementation reasons indus-
trially available PID controllers are never this simple for reasons which will soon become clear.
Further details on the subtlety of implementing a practical useful PID controller are described in
[53] and the texts [13, 15, 99].

The purpose of the PID controller is to measure a process error ǫ and calculate a manipulated ac-
tion u. Note that while u is referred to as an input to the process, it is the output of the controller.

The “textbook” non-interacting continuous PID controller follows the equation

 Z 
1 dǫ
u = Kc ǫ + ǫ dt + τd (4.1)
τi dt

where the three tuning constants are the controller proportional gain, Kc , the integral time, τi
and the derivative time, τd , the latter two constants having units of time, often minutes for in-
dustrial controllers. Personally, I find it more intuitive to use the reciprocal of the integral time,
1/τi , which is called reset and has units of repeats per unit time. This nomenclature scheme has
the advantage that no integral action equates to zero reset, rather than the cumbersome infinite
integral time. Just to confuse you further, for some industrial controllers to turn off the integral
component, rather than type in something like 99999, you just type in zero. It is rare in engineer-
ing where we get to approximate infinity with zero! Table 4.1 summarises these alternatives.

Table 4.1: Alternative PID tuning parameter conventions

Parameter symbol units alternative symbol units

Gain Kc input/output Proportional band PB %
integral time τi seconds reset 1/τi seconds−1
derivative time τd seconds

The Laplace transform of the ideal PID controller given in Eqn. 4.1 is
 
U (s) 1
= C(s) = Kc 1 + + τd s (4.2)
E(s) τi s

and the equivalent block diagram in parallel form is

integrator
❄
ǫ ✲ Kc ✲ ✲ 1
τi s
✲+ ✲ u
error input ✻ controller output
✲ τd s

differentiator

where is is clear that the three terms are computed in parallel which is why this form of the PID
controller is sometimes also known as the parallel PID form.
4.2. THE INDUSTRIAL PID ALGORITHM 133

We could rearrange Eqn. 4.2 in a more familiar numerator/denominator transfer function format,


Kc τi τd s2 + τi s + 1
C(s) = (4.3)
τi s

where we can clearly see that the ideal PID controller is not proper, that is, the order of the
numerator (2), is larger than the order of the denominator (1) and that we have a pole at s = 0.
We shall see in section 4.2.1 that when we come to fabricate these controllers we must physically
have a proper transfer function, and so we will need to modify the ideal PID transfer function
slightly.

4.2.1 I MPLEMENTING THE DERIVATIVE COMPONENT

The textbook PID algorithm of Eqn. 4.2 includes a pure derivative term τd s. Such a term is
not physically realisable, and nor would we really want to implement it anyway, since abrupt
changes in setpoint would cause extreme values in the manipulated variable.

There are several approximations that are used in commercial controllers to address this problem.
Most schemes simply add a small factory-set lag term to the derivative term. So instead of simply
τd s, we would use
τd s
derivative term = τd (4.4)
s+1
N
where N is a large value, typically set to a large number somewhere between 10 and 100. Using
this derivative term modifies the textbook PID transfer function of Eqn. 4.2 or Eqn. 4.3 to
 
1 τd s
C(s) = Kc 1 + + τd (4.5)
τi s Ns+1
 
(τi τd + τi τd N ) s2 + (τi N + τd )s + N
= Kc (4.6)
τi s (τd s + N )

which is now physically realisable. Note that as N → ∞, the practical PID controller of Eqn. 4.6
converges to the textbook version of Eqn. 4.3. The derivative-filtered PID controller of Eqn. 4.6
collapses to a standard PI controller if τd = 0.

We can fabricate in M ATLAB the transfer function of this D-filtered PID controller with the pidstd
command which M ATLAB refers to as a PID controller in standard form.

Listing 4.1: Constructing a transfer function of a PID controller

>> C = pidstd(1,2,3,100) % Construct a standard form PID controller
Continuous-time PIDF controller in standard form:
3
 
1 τd s
% C(s) = Kp 1 + τi s
+ τd
s+1
N

with Kp = 1, Ti = 2, Td = 3, N = 100

The generated controller is of a special class known as a pidstd, but we can convert that to the
more familiar transfer function with the now overlayed tf command.

>> tf(C)
134 CHAPTER 4. THE PID CONTROLLER

Transfer function:
4 101 sˆ2 + 33.83 s + 16.67
-------------------------
sˆ2 + 33.33 s

A related M ATLAB function, pid, constructs a slight modification of the above PID controller in
what M ATLAB refers to as parallel form,
s
Cparallel (s) = P + I/s + D
τf s + 1

where in this case as the derivative filter time constant, τf , approaches zero, the derivative term
approaches the pure differentiator.

4.2.2 VARIATIONS OF THE PID ALGORITHM

The textbook algorithm of the PID controller given in Eqn. 4.2 is sometimes known as the par-
allel or non-interacting form, however due to historical reasons there is another form of the PID
controller that is sometimes used. This is known as the series, cascade or interacting form
 
′ ′ 1
Gc (s) = Kc 1 + ′ (1 + τd′ s) (4.7)
τi s

where the three series PID controller tuning constants, Kc′ , τi′ and τd′ are related to, but distinct
from, the original PID tuning constants, Kc , τi and τd . A block diagram of the series PID controller
is given below.

differentiator integrator

✲ τd′ s ✲ 1
τi′ s

❄ ❄
ǫ ✲ K′ ✲ ✲+ ✲ 1 ✲+ ✲ u
c

error input controller output

A series PID controller in the form of Eqn. 4.7 can always be represented in parallel form

τi′ + τd′ τi′ τd′

Kc = Kc′ , τi = τi′ + τd′ , τd = (4.8)
τi′ τi′+ τd′

but not necessarily the reverse

Kc  p 
Kc′ = 1 + 1 − 4τd /τi (4.9)
2
′ τi  p 
τi = 1 + 1 − 4τd /τi (4.10)
2
′ τi  p 
τd = 1 − 1 − 4τd /τi (4.11)
2
since a series form only exists if τi > 4τd . For this reason, the series form of the PID controller is
less commonly used, although some argue that it is easier to tune, [189]. Note that both controller
forms are the same if the derivative component is not used. A more detailed discussion on the
various industrial controllers and associated nomenclatures is given in [99, pp32-33].
4.3. SIMULATING A PID PROCESS IN SIMULINK 135

PI controlled response Integral only control

0.5 0.4

0.4 0.3
output

output
0.3 0.2

0.2 0.1

0.1 0
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200

0.8 0.6

0.6 0.4
input

input
0.4 0.2

0.2 0

0 −0.2
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200
time (sec) dt=0.08 time (sec) dt=0.08

(a) PI control of a flapper with K = 1 and reset= 0.2 (b) Integral-only control of a flapper.

Figure 4.1: Comparing PI and integral-only control for the real-time control of a noisy flapper
plant with sampling time T = 0.08.

4.2.3 I NTEGRAL ONLY CONTROL

For some types of control, integral only action is desired. Fig. 4.1(a) shows the controlled response
of the laboratory flapper, (§1.4.1), controlled with a PI controller. The interesting characteristic of
this plant’s behaviour is the significant disturbances. Such disturbances make it both difficult to
control and demand excessive manipulated variable action.

If however we drop the proportional term, and use only integral control we obtain much the same
response but with a far better behaved input signal as shown in Fig. 4.1(b). This will decrease the
wear on the actuator.

If you are using Eqn. 4.1 with the gain Kc set to zero, then no control at all will result irrespective
of the integral time. For this reason, controllers either have four parameters as opposed to the
three in Eqn. 4.1, or we can follow the S IMULINK convention shown in Fig. 4.4.

4.3 S IMULATING A PID PROCESS IN S IMULINK

S IMULINK is an ideal platform to rapidly simulate control problems. While it does not have the
complete flexibility of raw M ATLAB, this is more than compensated for in the ease of construction
and good visual feedback necessary for rapid prototyping. Fig. 4.2 shows a S IMULINK block
diagram for the continuous PID control of a third order plant.

3
PID
3s3 +5s2 +6s+1.6
Step PID Controller Plant output

Figure 4.2: A S IMULINK block diagram of a PID controller and a third-order plant.
136 CHAPTER 4. THE PID CONTROLLER

The PID controller block supplied as part of S IMULINK is slightly different from the classical de-
scription given in Eqn. 4.2. S IMULINK’s continuous PID block uses the complete parallel skeleton,

I Ns
Gc (s) = P + +D (4.12)
s τd s + N
where we choose the three tuning constants, P, I and D, and optionally the filter coefficient N
which typically lies between 10 and 100. You can verify this by unmasking the PID controller (via
the options menu) block to exhibit the internals as shown in Fig. 4.3.

Proportional Gain

1
1 I 1
s
u Sum y
Integral Gain
Integrator

Figure 4.3: An unmasked view of a PID D N

SumD
controller block that comes supplied in Derivative Gain Filter Coefficient

S IMULINK. Note how the configuration

1
differs from the classical form. See also s

Fig. 4.4. Filter

Note how the derivative component of the S IMULINK PID controller in Eqn. 4.12 follows the
realisable approximation given in Eqn. 4.4 by using a feedback loop with an integrator and gain
of N = 100 as a default.

Block diagrams of both the S IMULINK implementation and the classical PID scheme are com-
pared in Fig. 4.4. Clearly the tuning constants for both schemes are related as follows:

Kc
P = Kc , I= , D = Kc τd (4.13)
τi
or alternatively
Kc D
Kc = P, τi = , τd = (4.14)
I Kc
The S IMULINK scheme has the advantage of allowing integral-only control without modification.

✲ P

❄ ❄
ǫ ✲ ✲ I ✲+ ✲ u ǫ ✲Kc ✲ ✲ 1 ✲+ ✲ u
s τi s
✻ ✻
✲ Ds ✲ τd s

Simulink PID controller block Classical controller block

Figure 4.4: Block diagram of PID controllers as implemented in S IMULINK (left) and classical text
books (right).
4.3. SIMULATING A PID PROCESS IN SIMULINK 137

If you would rather use the classical textbook style PID controller, then it is easy to modify the
S IMULINK PID controller block. Fig. 4.5 shows a S IMULINK implementation which includes the
filter on the derivative component following Eqn. 4.6. Since PID controllers are very common,
you may like to mask the controller as illustrated in Fig. 4.6, add a suitable graphic, and add this
to your S IMULINK library.

1
1 Kc 1/taui 1
s
error Add 4 u
P−gain reset Integrator

Figure 4.5: A realisable con-

s
taud tinuous PID controller imple-
taud /N.s+1
mented in S IMULINK with a
deriv filtered deriv filter on the derivative action.

3
3s3 +5s2 +6s+1.6
Step Plant output
PID Controller

Figure 4.6: A S IMULINK block diagram of a classical PID controller as a mask

Note that to have a reverse acting controller, we either require all three constants to be negative,
or just add a negative gain to the output of the controller.

Reasonable S IMULINK controller tuning

constants for this example are: 1.5

1
y & ref

P = 2, I = 1.5, D=2
0.5
or in classical form
0

Kc = 2, τi = 1.33, τd = 1 4

2
u

which gives a step response as shown op-

posite. 0

0 2 4 6 8 10
time

Because the derivative term in the PID controller acts on the error rather than the output, we see
a large derivative kick in the controller output1 . We can avoid this by using the PID block with
anti-windup, or by modifying the PID block itself. Section 4.4.1 shows how this modification
works.

1 Note that I have re-scaled the axis in the simulation results

138 CHAPTER 4. THE PID CONTROLLER

4.4 E XTENSIONS TO THE PID ALGORITHM

Industrial PID controllers are in fact considerably more complicated than the textbook formula
of Eqn. 4.1 would lead you to believe. Industrial PID controllers have typically between 15 and
25 parameters. The following describes some of the extra functionality one needs in an effective
commercial PID controller.

4.4.1 AVOIDING DERIVATIVE KICK

If we implement the classical PID controller such as Eqn. 4.3 with significant derivative action, the
input will show jump excessively every time either the output or the setpoint changes abruptly.
Under normal operations conditions, the output is unlikely to change rapidly, but during a set-
point change, the setpoint will naturally change abruptly, and this causes a large, though brief,
spike in the derivative of the error. This spike is fed to the derivative part of the PID controller,
and causes unpleasant transients in the manipulated variable. If left unmodified, this may cause
excessive wear in the actuator. Industrial controllers and derivative kick is further covered in
[182, p191].

It is clear that there is a problem with the controller giving a large kick when the setpoint abruptly
changes. This is referred to as derivative kick and is due to the near infinite derivative of the error
when the setpoint instantaneously changes as when in a setpoint change. One way to avoid
problems of this nature is to use the derivative of the measurement ẏ, rather than the derivative
of the error ė = y˙⋆ − ẏ. If we do this, the derivative kick is eliminated, and the input is much less
excited. Equations of both the classical and the ‘anti-kick’ PID controller equations are compared
below.
 Z 
1 t dǫ
Normal: Gc (s) = Kc ǫ + ǫdτ + τd (4.15)
τi 0 dt
 Z 
1 t dy
Anti-Kick: Gc (s) = Kc ǫ + ǫdτ + τd (4.16)
τi 0 dt
The anti-derivative kick controller is sometimes known as a PI-D controller with the dash indi-
cating that the PI part acts on the error, and the D part acts on the output.

In S IMULINK you can build a PID controller with anti-kick by modifying the standard PID con-
troller block as shown in Fig. 4.7.

Fig. 4.8 compares the controlled performance for the third order plant and tuning constants given
previously in Fig. 4.2 where the derivative term uses the error (lefthand simulation), with the
modification where the derivative term uses the measured variable (righthand simulation).

Evident from Fig 4.8 is that the PID controller using the measurement rather than the error be-
haves better, with much less controller action. Of course, the performance improvement is only
evident when the setpoint is normally stationary, rather than a trajectory following problem. Sta-
tionary setpoints are the norm for industrial applications, but if for example, we subjected the
closed loop to a sine wave setpoint, then the PID that employed the error in the derivative term
would perform better than the anti-kick version.

A N ELECTROMAGNETIC BALANCE ARM

The electromagnetic balance arm described previously in §1.4.1 is extremely oscillatory as shown
in Fig. 1.5(b). The overshoot is about 75% which corresponds to a damping ratio of ζ ≈ 0.1
4.4. EXTENSIONS TO THE PID ALGORITHM 139

input/output

1 3
1/taui Kc
s (s+4)(s+1)(s+0.4)
Sum1 Integrator Sum
Pulse gain Plant
reset
Generator
s
taud
taud/N.s+1
derivative filter
deriv

Figure 4.7: PID controller with anti-derivative kick. See also Fig. 4.8.

Normal PID No derivative Kick

1.5 1.5
output & setpoint

1 1

0.5 0.5

0 0

−0.5 −0.5
0 10 20 30 40 0 10 20 30 40

40 40

20 20
input

0 0

−20 −20

−40 −40
0 10 20 30 40 0 10 20 30 40
time time

Figure 4.8: PID control of a plant given in Fig. 4.7. The derivative kick is evident in the input
(lower trend of the left-hand simulation) of the standard PID controller. We can avoid this ‘kick’
by using the derivative of the output rather than the error as seen in the right-hand trend. Note
that the output controlled performance is similar in both cases
140 CHAPTER 4. THE PID CONTROLLER

assuming a proto-type second order process model.

To stabilise a system with poles so close to the imaginary axis requires substantial derivative
action. Without derivative action, the integral component needed to reduce the offset would
cause instability. Unfortunately however the derivative action causes problems with noise and
abrupt setpoint changes as shown in Fig. 4.9(a).

No derivative kick
Derivative kick 2200
2500
2000
Flapper angle

2000
1800

1500 1600

1400
1000

1600
3000
1400
2000
input

1200
1000 1000

0 800
0 10 20 30 40 50 60 0 10 20 30 40 50
time [s] time [s]
(a) PID control exhibiting significant derivative kick. (b) PID control with anti-kick.

Figure 4.9: Illustrating the improvement of anti-derivative kick schemes for PID controllers when
applied to the experimental electromagnetic balance.

The controller output, u(t) exhibits a kick every time the setpoint is changed. So instead of the
normal PID controller used in Fig. 4.9(a), an anti-kick version is tried in Fig. 4.9(b). Clearly there
are no spikes in the input signal when Eqn. 4.16 is used, and the controlled response is slightly
improved.

Abrupt setpoint changes are not the only thing that can trip up a PID controller. Fig. 4.10 shows
another response from the electromagnetic balance, this time with even more derivative action.
At low levels the balance arm is very oscillatory, although this behaviour tends to disappear at
the higher levels owing to the nonlinear friction effects.

4.4.2 I NPUT SATURATION AND INTEGRAL WINDUP

Invariably in practical cases the actual manipulated variable value, u, demanded by the PID
controller is impossible to implement owing to physical constraints on the system. A control
valve for example cannot shut less than 0% open or open more than 100%. Normally “clamps”
are placed on the manipulated variable to prevent unrealisable input demands occurring such as

umin < u < umax , or u = MIN(max u,MAX(min u,u))

or in other words, if the input u is larger than the maximum input allowed max_u, it will be
reset to that maximum input, and similarly the input is saturated if it is less than the minimum
allowable limit. In addition to an absolute limit on the position of u, the manipulated variable
can not instantaneously change from one value to another. This can be expressed as a limit on
the derivative of u, such as | du
dt | < cd .
4.4. EXTENSIONS TO THE PID ALGORITHM 141

4000

3500
Setpoint & output

3000

2500

2000

1500
5000

4000

3000 Figure 4.10: Derivative

input

control and noise. Note

2000 the difference in re-
sponse character once
1000 the setpoint is increased
0 over 3000. This is
0 5 10 15 20 25 30 35 due to nonlinear fric-
time [s] tion effects.

However the PID control algorithm presented so far assumes that the manipulated variable is
unconstrained so when the manipulated variable does saturate, the integral error term continues
to grow without bound. When finally the plant ‘catches up’ and the error is reduced, the integral
term is still large, and the controller must ‘eat up this error’. The result is an overly oscillatory
controlled response.

This is known as integral windup. Historically with the analogue controllers, integral windup
was not much of a problem since the pneumatic controllers had only a limited integral capacity.
However, this limit is effectively infinite in a digital implementation.

There are a number of ways to prevent integral windup and these are discussed in [15, pp10–14]
and more recently in [140, p60] and [31]. The easiest way is to check the manipulated variable
position, and ignore the integral term if the manipulated variable is saturated.

An alternative modification is to compute the difference between the desired manipulated vari-
able and the saturated version and to feed this value back to the integrator within the controller.
This is known as anti-windup tracking and is shown in Fig. 4.11(a). When the manipulated input is
not saturated, there is no change to normal PID algorithm. Note that in this modified PID config-
uration we differentiate the measured value (not the error), we approximate the derivative term
as discussed in section 4.2.1, and we can turn off the anti-windup component with the manual
switch.

As an example (adapted from [19, Fig 8.9]), suppose we are to control an integrator plant, G(s) =
1/s, with tight limits on the manipulated variable, |u| < 0.1. Without anti-windup, the controller
output rapidly saturates, and the uncompensated response shown in Fig. 4.11(b) is very oscilla-
tory. However with anti-windup enabled, the controlled response is much improved.
142 CHAPTER 4. THE PID CONTROLLER

Manual Switch Constant

0

Sum3
1/taui 1/s
Sum2
Integrator
reset

Sum1
1 Kc 1
error u
gain actuator model

taud.s+1
2 Kc*taud
taud/N.s+1
y
Kc*deriv approx Deriv
(a) A PID controller with anti-windup.

Anti−windup demonstration
3
2
output & setpoint

1
0
−1
−2 Anti−windup off Anti−windup on
−3

0.15
0.1
0.05
input

0
−0.05
−0.1

0 100 200 300 400 500 600 700 800 900 1000
time
(b) Turning on the anti-windup controller at t = 660 dramatically improves the controlled response.

Figure 4.11: The advantages of using anti-windup are evident after t = 660.
4.5. DISCRETE PID CONTROLLERS 143

4.5 D ISCRETE PID CONTROLLERS

To implement a PID controller as given in Eqn. 4.1 on a computer, one must first discretise or
approximate the continuous controller equation. If the error at time t = kT is ǫk , then the contin-
uous expression
 Z 
1 de(t)
u(t) = Kc e(t) + e(t) dt + τd (4.17)
τi dt

can be approximated with

 
 
 k 
 T X τd 
uk = Kc ǫk + ǫj + (ǫk − ǫk−1 ) (4.18)
 τi j=0 T
| {z }
 
| {z } differential
integral

The integral in Eqn. 4.17 is approximated in Eqn. 4.18 by the rectangular rule and the derivative
is approximated as a first order difference, although other discretisations are possible. Normally
the sample time, T , used by the controller is much faster than the dominant time constants of the
process so the approximation is satisfactory and the discrete PID controller is, to all intents and
purposes, indistinguishable from a continuous controller.

It is considerably more convenient to consider the discrete PID controller as a rational polynomial
expression in z. Taking z-transforms of the discrete position form of the PID controller, Eqn. 4.18,
we get
 
 T −i+1
τd 
+ z −i+2 + · · · + z −1 + 1 E(z) + (1 − z −1 )E(z)
 
U (z) = Kc E(z) + z
 τi T
| {z }
| {z }
integral differential

This shows that the transfer function of the PID controller is

 
T 1 τd −1
GPID (z) = Kc 1 + + (1 − z ) (4.19)
τi (1 − z −1 ) T

!
Kc T + τi T + τd τi − τi (T − 2τd ) z −1 + τi τd z −2
2
= (4.20)
T τi 1 − z −1

Eqn. 4.19 is the discrete approximation to Eqn. 4.2 and the approximation is reasonable provided
the sample time, T , is sufficiently short. A block diagram of this discrete approximation is

❄
ǫ ✲ Kc ✲ T ✲ 1 ✲+ ✲ u
τi (1−z −1 )
✻
✲ τd ✲ 1 − z −1
T

which we could further manipulate this using (block diagram algebra rules) to use only delay
elements to give
144 CHAPTER 4. THE PID CONTROLLER

z −1 ✛
+ ✻
❄ ❄
ǫ ✲ Kc ✲ T ✲+ ✲+ ✲ u
τi
✻
✲ τd ✲+
T
✻
−
✲ z −1

There are other alternatives for a discrete PID controller depending on how we approximate the
integral part. For example we could use a backward difference,
ukT = u(k−1)T + T y(k−1)T
| {z } | {z }
old integral add on
T
Gi (z) =
z−1
or a forward difference,
ukT = u(k−1)T + T ykT
| {z } | {z }
old integral add on

Tz
Gi (z) =
z−1
(which is not to be recommended due to stability problems), or the trapezoidal approximation.


ukT = u(k−1)T + T /2 ykT + y(k−1)T
| {z } | {z }
old integral add on

T (z + 1)
Gi (z) =
2(z − 1)
We can insert any one of these alternatives to give the overall discrete z-domain PID controller,
although the trapeziodal scheme
 
T (1 + z −1 ) τd −1
GPID (z) = Kc 1 + + (1 − z ) (4.21)
2τi (1 − z −1 ) T


!
Kc T 2 + 2τi T + 2τd τi + T 2 − 2τi T − 4τi τd z −1 + 2τi τd z −2
= (4.22)
2T τi 1 − z −1

is the most accurate and therefore the preferred implementation.

A S IMULINK discrete PID controller with sample time T using the trapeziodal approximation of
Eqn. 4.21 is given in Fig. 4.12. Note that now, as opposed to the continuous time implementation
of the PID controller, the derivative and integral gain values are now a function of sample time,
T . While continuous versions of PID controllers exist in S IMULINK, discrete versions simulate
much faster.

4.5.1 D ISCRETISING CONTINUOUS PID CONTROLLERS

The easiest way to generate a discrete PID controller is to simply call the M ATLAB standard PID
pidstd with a trailing argument to indicate that you want a discrete controller. Since the default
4.5. DISCRETE PID CONTROLLERS 145

1+z −1
1 Kc T/taui/2 1
error 1−z−1 u
Sum1
gain Discrete Filter1
integral

1−z−1
taud/T
1
Discrete Filter2
derivative

Figure 4.12: A discrete PID controller in S IMULINK using a trapezoidal approximation for the
integral with sample time T following Eqn. 4.21. This controller block is used in the simulation
presented in Fig. 4.13.

discretisation strategy is using the forward Euler, it would be prudent to use explicitly state the
stable backward Euler option for both the integration and differentiation.

Listing 4.2: Constructing a discrete (filtered) PID controller

>> C = pidstd(1,2,3,100,0.1, ...
'IFormula','BackwardEuler','DFormula','BackwardEuler')
Discrete-time PIDF controller in standard form:
4
 
1 Ts z 1
% C(s) = Kp 1 + τi
· z−1
+ τd Ts z
τd /N+ z−1

with Kp = 1, Ti = 2, Td = 3, N = 100, Ts = 0.1

9 >> tf(C)

Transfer function:
24.13 zˆ2 - 47.4 z + 23.31
--------------------------
14 zˆ2 - 1.231 z + 0.2308

Sampling time: 0.1

V ELOCITY FORM OF THE PID CONTROLLER

Eqn. 4.18 is called the position form implementation of the PID controller. An alternative form
is the velocity form which is obtained by subtracting the previous control input uk−1 from the
current input uk to get

∆uk = uk − uk−1
    
T τd 2τd τd
= Kc 1 + + ǫk − 1 + ǫk−1 + ǫk−2 (4.23)
τi T T T

The velocity form in Eqn. 4.23 has three advantages over the position form: (see [194, pp636-637]),
namely it requires no need for initialisation, (the computer does not need to know the current u),
no integral windup problems, and the algorithm offers some protection against computer failure
in that if the computer crashes, the input remains at the previous, presumably reasonable, value.
146 CHAPTER 4. THE PID CONTROLLER

One drawback, however, is that it should not be used for P or PD controllers since the controller
is unable to maintain the reference value.

Fig. 4.13 shows a S IMULINK block diagram of a pressure control scheme on a headbox of a paper
machine. The problem was that the pressure sensor was very slow, only delivering a new pres-
sure reading every 5 seconds. The digital PID controller however was running with a frequency
of 1 Hz. The control engineer in this application faced with poor closed loop performance, was
concerned that the pressure sensor was too slow and therefore should be changed. In Fig. 4.13,
the plant is a continuous transfer function while the discrete PID controller runs with sample
period of T = 1 second, and the pressure sensor is modelled with a zeroth-order hold with T = 5
seconds. Consequently the simulation is a mixed continuous/discrete example, with more than
one discrete sample time.

3.5
error ctrl output
200s2 +30s+1
Signal headbox Scope
Generator Discrete PID

pressure sensor

Figure 4.13: Headbox control with a slow pressure transducer. The discrete PID controller was
given in Fig. 4.12.

Fig. 4.14 shows the controlled results. Note how the pressure signal to the controller lags behind
the true pressure, but the controller still manages to control the plant.

4.5.2 S IMULATING A PID CONTROLLED RESPONSE IN M ATLAB

We can simulate a PID controlled plant in M ATLAB by writing a small script file that calls a gen-
eral PID controller function. The PID controller is written in the discrete velocity form following
Eqn. 4.23 in the M ATLAB function file pidctr.m shown in listing 4.3.

Listing 4.3: A simple PID controller

function [u,e] = pidctr(err,u,dt,e,pidt)
% [u,e] = pidctr(err,u,dt,e,pidt)
% PID controller
4 % err: = ǫ, current error, u = u(t), current manipulated variable
% dt = T sample time, e: row vector of past 3 errors
% pidt = [Kc , 1/τI , τd ] = tuning constants

k = pidt(1); rs = pidt(2); td2=pidt(3)/dt;

9 e = [err,e(1:2)]; % error shift register
du = k*[1+rs*dt+td2,-1-2*td2,td2]*e';
u = du + u; % Update control value unew = uold + ∆u
return

This simple PID controller function is a very naive implementation of a PID controller without
any of the necessary modifications common in robust commercial industrial controllers as de-
scribed in section 4.4.
4.5. DISCRETE PID CONTROLLERS 147

2
y
1.5 y sampled
setpoint
1
y & setpoint

0.5

−0.5

−1

−1.5
4

0
u

−2

−4
0 50 100 150
time [s]

Figure 4.14: Headbox control with a slow pressure transducer measurement sample time of T = 5
while the control sample time is T = 1. Upper: The pressure setpoint (dotted), the actual pressure
and the sampled-and-held pressure fed back to the PID controller. Lower: The controller output.

The plant to be controlled for this simulation is

1.2q −1
Gp (q −1 ) = q −d (4.24)
1 − 0.25q −1 − 0.5q −2

where the sample time is T = 2 seconds and the dead time is d = 3 sample time units, and the
setpoint is a long period square wave. For this simulation, we will try out the PID controller with
tuning constants of Kc = 0.3, 1/τi = 0.2 and τd = 0.1. How I arrived at these tuning constants is
discussed later in §4.6.

The M ATLAB simulation using the PID function from Listing 4.3 is given by the following script
file:

a=[0.25,0.5];b=1.2;theta=[a b]'; dead = 3; % Plant G(q), Eqn. 4.24.

3 dt = 2.0; t = dt*[0:300]'; yspt = square(t/40); % time vector & setpoint

y = zeros(size(yspt));u=zeros(size(y)); % initialise y(t) & u(t)
pid = [0.3, 0.2, 0.1]; % PID tuning constants: Kc = 0.3, 1/τI = 0.2, τd = 0.1
e = zeros(1,3); % initialise error integral

8 for i=3+dead:length(y)
X = [y(i-1), y(i-2), u(i-1-dead)]; % collect i/o
y(i) = X*theta; % system prediction
err = yspt(i) - y(i); % current error
[u(i),e] = pidctr(err,u(i-1),dt,e,pid); % PID controller from Listing 4.3.
13 end % for
plot(t,yspt,'--',t,[y,u]) % Plot results in Fig. 4.15.
148 CHAPTER 4. THE PID CONTROLLER

Figure 4.15 shows the controlled response of this simulation. Note how I have plotted the input
(lower plot of Fig. 4.15) as a series of horizontal lines that show that the input is actually a
piecewise zeroth-order hold for this discrete case using the stairs function.

PID control
2

y & setpoint 1

−1

−2

0.5
Input

Figure 4.15: The output (up-

−0.5
per solid), setpoint (upper
dashed) and discretised input
−1
(lower) of a PID controlled 0 50 100 150 200
process. Time

4.5.3 C ONTROLLER PERFORMANCE AS A FUNCTION OF SAMPLE TIME

Given that the discrete PID controller is an approximation to the continuous controller, we must
expect a deterioration in performance with increasing sample time. Our motivation to use coarse
sampling times is to reduce the computational load. Fig. 4.16 compares the controlled response
of the continuous plant,
s+3
Gp (s) =
(s + 4)(τ 2 s2 + 2ζτ s + 1)
with τ = 4, ζ = 0.4 given the same continuous controller settings at different sampling rates.
Note that the reset and derivative controller settings for the discrete controller are functions of
time, and must be adjusted accordingly. Fig. 4.16 shows that the controller performance improves
as the sampling time is decreased and converges to the continuous case. However if the sampling
time is too small, the discrete PID controller is then suspectable to numerical problems.

4.6 PID TUNING METHODS

Tuning PID controllers is generally considered an art and is an active research topic both in
academia and in industry. Typical chemical processing plants have hundreds or perhaps thou-
sands of control loops of which the majority of these loops are non-critical and are of the PID
type and all these loops require tuning. Årzen [8] asserts that “it is a well known fact that many
4.6. PID TUNING METHODS 149

Ts= 4 Ts= 2 Ts= 1 Ts=0.1

1.5
discrete

1
y

0.5 continuous

2.5

1.5
u

0.5

0
0 20 40 0 20 40 0 20 40 0 20 40

Figure 4.16: The effect of varying sampling time, T , when using a discrete PID controller com-
pared to a continuous PID controller. As T → 0, the discrete PID controller converges to the
continuous PID controller.

control loops in (the) process industry are badly tuned, or run in manual mode.” Supporting this
claim, here is a summary of the rather surprising results that Ender, [65], found after investigating
thousands of control loops over hundreds of plants2 ;

• More than 30% of the controllers are operating in manual.

• More than 60% of all installed loops produce less variance in manual than automatic.

• The average loop costs $40,000 to design, instrument and install.

Note however that it is not only industry that seems unable to tune PID controllers since many
publications in the academic world also give mis-tuned PID controllers. This is most common
when comparing the PID with some other sort of more sophisticated (and therefore hopefully
better performing), controller. So it appears that it would be worthwhile to look more closely at
the tuning of PID regulators.

There are two possibilities that we face when tuning PID controllers. One is that we have a model
of the plant to be controlled, perhaps as a transfer function, so then we need to establish a suitable
PID controller such that when combined with the plant, we obtain an acceptable response. The
second case is where we don’t even have a model of the plant to be controlled, so our task is also
to identify (implicitly or explicitly) this plant model as well.

Tuning PID controllers can be performed in the time domain or in the frequency domain with the
controller either operating (closed loop), or disconnected (open loop), and the tuning parameter
calculations can be performed either online or offline. The online tuning technique is the central
component of automated self tuners, discussed more in chapter 7. Section 4.6.1 considers the two
classical time domain tuning methods, one closed loop, the other open loop.
2 mainly in the US
150 CHAPTER 4. THE PID CONTROLLER

4.6.1 O PEN LOOP TUNING METHODS

Open loop tuning methods are where the feedback controller is disconnected and the experi-
menter excites the plant and measures the response. They key point here is that since the con-
troller is now disconnected the plant is clearly now no longer strictly under control. If the loop
is critical, then this test could be hazardous. Indeed if the process is open-loop unstable, then
you will be in trouble before you begin. Notwithstanding for many process control applications,
open loop type experiments are usually quick to perform, and deliver informative results.

To obtain any information about a dynamic process, one must excite it in some manner. If the
system is steady at setpoint, and remains so, then you have no information about how the process
behaves. (However you do have good control so why not quit while you are ahead?) The type
of excitation is again a matter of choice. For the time domain analysis, there are two common
types of excitation signal;– the step change, and the impulse test, and for more sophisticated
analysis, one can try a random input test. Each of the three basic alternatives has advantages and
disadvantages associated with them, and the choice is a matter for the practitioner.

Step change The step change method is where the experimenter abruptly changes the input to
the process. For example, if you wanted to tune a level control of a buffer tank, you could
sharply increase (or decrease) the flow into the tank. The controlled variable then slowly
rises (or falls) to the new operating level. When I want a quick feel for a new process, I
like to perform a step test and this quickly gives me a graphical indication of the degree of
damping, overshoot, rise time and time constants better than any other technique.

Impulse test The impulse test method is where the input signal is abruptly changed to a new
value, then immediately equally abruptly changed back to the old value. Essentially you
are trying to physically alter the input such that it approximates a Dirac delta function.
Technically both types of inputs are impossible to perform perfectly, although the step test
is probably easier to approximate experimentally.
The impulse test has some advantages over the step test. First, since the input after the ex-
periment is the same as before the experiment, the process should return to the same pro-
cess value. This means that the time spent producing off-specification (off-spec) product
is minimised. If the process does not return to the same operating point, then this indi-
cates that the process probably contains an integrator. Secondly the impulse test (if done
perfectly) contains a wider range of excitation frequencies than the step test. An excitation
signal with a wide frequency range gives more information about the process. However
the impulse test requires slightly more complicated analysis.

Random input The random input technique assumes that the input is a random variable approx-
imating white noise. Pure white noise has a wide (theoretically infinite) frequency range,
and can therefore excite the process over a similarly wide frequency range. The step test,
even a perfect step test, has a limited frequency range. The subsequent analysis of this type
of data is now much more tedious, though not really any more difficult, but it does require
a data logger (rather than just a chart recorder) and a computer with simple regression
software. Building up on this type of process identification where the input is assumed,
within reason, arbitrary, are methods referred to as Time Series Analysis (TSA), or spectral
analysis;– both of which are dealt with in more detail in chapter 6.

O PEN - LOOP OR PROCESS REACTION CURVE TUNING METHODS

There are various tuning strategies based on an open-loop step response. While they all follow
the same basic idea, they differ in slightly in how they extract the model parameters from the
4.6. PID TUNING METHODS 151

✲ Plant ✲
input output

✻
tangent at inflection pt
K ..
..
..
..
..
..
..
..
..
..
..
..
..
. ✲time
✲ L✛ T ✲

Figure 4.17: The parameters T and L to be graphically estimated for the openloop tuning method
relations given in Table 4.2.

recorded response, and also differ slightly as to relate appropriate tuning constants to the model
parameters. This section describes three alternatives, the classic Ziegler-Nichols open loop test,
the Cohen-Coon test, and the Åström-Hägglund suggestion.

The classic way of open loop time domain tuning was first published in the early 40s by Ziegler
and Nichols3 , and is further described in [152, pp596–597] and in [182, chap 13]. Their scheme re-
quires you to apply a unit step to the open-loop process and record the output. From the response,
you graphically estimate the two parameters T and L as shown in Fig. 4.17. Naturally if your
response is not sigmoidal or ‘S’ shaped such as that sketched in Fig. 4.17 and exhibits overshoot,
or an integrator, then this tuning method is not applicable.

This method implicitly assumes the plant can be adequately approximated by a first order trans-
fer function with time delay,
Ke−θs
Gp ≈ (4.25)
τs + 1
where L is approximately the dead time θ, and T is the open loop process time constant τ . Once
you have recorded the openloop input/output data, and subsequently measured the times T and
L, the PID tuning parameters can be obtained directly from Table 4.2.

A similar open loop step tuning strategy due to Cohen and Coon published in the early 1950s is
where you record the time taken to reach 50% of the final output value, t2 , and the time taken to
reach 63% of the final value, t3 . You then calculate the effective deadtime with
t2 − ln(2)t3
θ=
1 − ln(2)
and time constant,
τ = t3 − t1
The open loop gain can be calculated by dividing the final change in output over the change in
the input step.
3 Ziegler & Nichols actually published two methods, one open loop, and one closed loop, [208]. However it is only the

second closed loop method that is generally remembered today as the “Ziegler–Nichols” tuning method.
152 CHAPTER 4. THE PID CONTROLLER

Once again, now that you have a model of the plant to be controlled in the form of Eqn. 4.25,
you can use one the alternative heuristics given in Table 4.2. The recommended range of values
for the deadtime ratio for the Cohen-Coon values is 0.1 < θ/τ < 1. Also listed in Table 4.2 are
the empirical suggestions from [16] known as AMIGO, or approximate M -constrained integral
gain optimisation. These values have the same form as the Cohen-Coon suggestions but perform
slightly better.

Table 4.2: The PID tuning parameters as a function of the openloop model parameters K, τ and
θ from Eqn. 4.25 as derived by Ziegler-Nichols (open loop method), Cohen and Coon, or alterna-
tively the AMIGO rules from [16].

Controller Kc τi τd
τ
P – –
Ziegler-Nichols Kθ
(Open loop) 0.9τ θ
PI –
Kθ 0.3
1.2τ
PID 2θ 0.5θ
Kθ
 
1 τ θ
P 1+ – –
K θ 3τ
Cohen-Coon  
1 τ θ 30 + 3θ/τ
PI 0.9 + θ –
K θ 12τ 9 + 20θ/τ
 
1 τ 4 θ 32 + 6θ/τ 4
PID + θ θ
K θ 3 4τ 13 + 8θ/τ 11 + 2θ/τ

1  τ 0.4θ + 0.8τ 0.5θτ

AMIGO PID 0.2 + 0.45 θ
K θ θ + 0.1τ 0.3θ + τ

Fig. 4.18 illustrates the approximate first-order plus deadtime model fitted to a higher-order over-
damped process using the two points at 50% and 63%. The subsequent controlled response using
the values derived from in Table 4.2 is given in Fig. 4.19.

1.5

fitted model
1
y

Figure 4.18: Fitting a first-order model with

actual
deadtime using the Cohen-Coon scheme.
Note how the fitted model is a reason- 0.5
able approximation to the actual response
t2 t3
just using the two data points and gain.
See Fig. 4.19 for the subsequent controlled 0
0 5 10 15 20 25 30 35
response. time

Conventional thought now considers that both the Zeigler-Nichols scheme in Table 4.2 and the
Cohen-Coon scheme gives controller constants that are too oscillatory and hence other modified
tuning parameters exist, [181, p329]. Problem 4.1 demonstrates this tuning method.
4.6. PID TUNING METHODS 153

P−only PI−control PID control

2 2 2
y & setpoint

0 0 0

−2 −2 −2

5 5 5

0 0 0
u

−5 −5 −5
0 100 200 0 100 200 0 100 200
time time time

Figure 4.19: The closed loop response for a P, PI and PID controlled system using the Cohen-Coon
strategy from Fig. 4.18.

Problem 4.1 Suppose you have a process that can be described by the transfer function

K
Gp =
(3s + 1)(6s + 1)(0.2s + 1)

Evaluate the time domain response to a unit step change in input and graphically estimate the
parameters L and T . Design a PI and PID controller for this process using Table 4.2.

C ONTROLLER SETTINGS BASED ON THE OPEN LOOP MODEL

If we have gone to all the trouble of estimating a model of the process, then we could in principle
use this model for controller design in a more formal method than just rely on the suggestions
given in Table 4.2. This is the thinking behind the Internal Model Control or IMC controller
design strategy. The IMC controller is a very general controller, but if we restrict our attention to
just controllers of the PID form, we can derive simple relations between the model parameters
and appropriate controller settings.

The nice feature of the IMC strategy is that it provides the scope to adjust the tuning with a single
parameter, the desired closed loop time constant, τc , something that is missing from the strategies
given previously in Table 4.2. A suitable starting guess for the desired closed loop time constant
is to set it equal to the dominant open loop time constant.

Table 4.3 gives the PID controller settings based on various common process models. For a more
complete table containing a larger selection of transfer functions, consult [182, p308].

An simplification of the this IMC idea in an effort to make the tuning as effortless as possible is
given in [189].

Perhaps the easiest way to tune a plant when the transfer function is known is to use the M ATLAB
function pidtune, or the GUI, pidtool as shown in Fig. 4.20.
154 CHAPTER 4. THE PID CONTROLLER

Table 4.3: PID controller settings based on IMC for a small selection of common plants where the
control engineer gets to chose a desired closed loop time constant, τc .

Plant PID constants

Kc K τi τd
τ
K/(τ s + 1) τ –
τc
K τ1 + τ2
τ1 + τ2 –
(τ1 s + 1)(τ2 s + 1) τc
2
K/s 2τc –
τc
K 2τc + τ 2τc τ
2
2τc + τ
s(τ s + 1) τc 2τc + τ
τ
Ke−θs τ –
τc + θ
τs + 1 τ + θ/2 τθ
τ + θ/2
τc + θ/2 2τ + θ

Figure 4.20: PID tuning of an arbitrary transfer function using the M ATLAB GUI.
4.6. PID TUNING METHODS 155

4.6.2 C LOSED LOOP TUNING METHODS

The main disadvantage of the open loop tuning methods is that it is performed with the controller
switched to manual, i.e. leaving the output uncontrolled in open loop. This is often unreason-
able for systems that are openloop unstable, and impractical for plants where there is a large
invested interest, and the operating engineers are nervous. The Ziegler-Nichols continuous cy-
cling method described next is a well-known closed loop tuning strategy used to address this
problem, although a more recent single response strategy given later in §4.6.3 is faster, safer, and
easier to use.

Z IEGLER -N ICHOLS CONTINUOUS CYCLING METHOD

The Ziegler-Nichols continuous cycling method is one of the best known closed loop tuning
strategies. The controller is left on automatic, but the reset and derivative components are turned
off. The controller gain is then gradually increased (or decreased) until the process output contin-
uously cycles after a small step change or disturbance. At this point, the controller gain you have
selected is the ultimate gain, Ku , and the observed period of oscillation is the ultimate period,
Pu .

Ziegler and Nichols originally suggested in 1942 PID tuning constants as a function of the ulti-
mate gain and ultimate period as shown in the first three rows of Table 4.4. While these values
give near-optimum responses fort load changes, practical experience and theoretical consider-
ations (i.e. [15, 29]) have shown that these tuning values tend to give responses that are too
oscillatory for step-point changes due to the small phase margin. For this reason, various people
have subsequently modified the heuristics slightly as listed in the remaining rows of Table 4.4
which is expanded from that given in [181, p318].

Table 4.4: Various alternative ‘Ziegler-Nichols’ type PID tuning rules as a function of the ultimate
gain, Ku , and ultimate period, Pu .
Response type PID constants
Kc τi τd
P 0.5Ku – –
Ziegler-Nichols PI 0.45Ku Pu /1.2 –
PID 0.6Ku Pu /2 Pu /8
No overshoot 0.2Ku Pu /2 Pu /2
Modified ZN Some overshoot 0.33Ku Pu /2 Pu /3
PI 0.31Ku 2.2Pu –
Tyreus-Luyben PID 0.45Ku 2.2Pu Pu /6.3
Chien-Hrones-Reswick PI 0.47Ku 1 –
Åström-Hägglund PI 0.32Ku 0.94 –
Specified phase-margin, φm PID Ku cos(φm ) f τd Eqn. 4.56

Experience has shown that the Chien-Hrones-Reswick values give an improved response on the
original Ziegler-Nichols, but the Åström-Hägglund tend, like the ZN, to be overly oscillatory.
While the Tyreus-Luyben values deliver very sluggish responses, they exhibit very little over-
shoot and are favoured by process engineers for that reason.

Algorithm 4.1 summarises the ZN ultimate oscillation tuning procedure.

156 CHAPTER 4. THE PID CONTROLLER

Algorithm 4.1 Ziegler-Nichols closed loop PID tuning

To tune a PID controller using the closed-loop Ziegler-Nichols method, do the following:

1. Connect a proportional controller to the plant to be controlled. I.e. turn the controller on
automatic, but turn off the derivative and integral action. (If your controller uses integral
time, you will need to set τi to the maximum allowable value.)

2. Choose a trial sensible controller gain, Kc to start.

3. Disturb the process slightly and record the output.

4. If the response is unstable, decrease Kc and go back to step 3, otherwise increase Kc and
repeat step 3 until the output response is a steady sinusoidal oscillation. Once a gain Kc
has been found to give a stable oscillation proceed to step 5.

5. Record the current gain, and measure the period of oscillation in time units (say seconds).
These are the ultimate gain, Ku and corresponding ultimate period, Pu .

6. Use Table 4.4 to establish the P, PI, or PID tuning constants.

7. Test the closed loop response with the new PID values. If the response is not satisfactory,
further manual ‘fine-tuning’ may be necessary.

This method has proved so popular, that automatic tuning procedures have been developed that
are based around this theory as detailed in chapter 7. Despite the fact that this closed loop test
is difficult to apply experimentally, gives only marginal tuning results in many cases, it is widely
used and very well known. Much of the academic control literature uses the Z-N tuning method
as a basis on which to compare more sophisticated schemes but often conveniently forgetting in
the process that the Z-N scheme was really developed for load disturbances as opposed to set-
point changes. Finally many practicing instrument engineers (who are the ones actually tuning
the plant) know only one formal tuning method— this one.

Consequently it is interesting to read the following quote from a review of a textbook in Process
control:4 “ . . . The inclusion of tuning methods based on the control loop cycling (Ziegler-Nichols
method) without a severe health warning to the user reveals a lack of control room experience on
behalf of the author. ”

Z IEGLER -N ICHOLS CONTINUOUS CYCLING EXAMPLE

Finding the ultimate gain and frequency in practice requires a tedious ‘trial-and-error’ approach.
If however, we already have the model of the process, say in the form of a transfer function, then
establishing the critical frequency analytically is much easier, although we may still need to solve
a nonlinear algebraic equation.

Suppose we have identified a model of our plant as

1
G(s) = e−3s (4.26)
6s2 + 7s + 1
4 Review of A Real-Time Approach to Process Control by Svrcek, Mahoney & Young, reviewed by Patrick Thorpe in The

Chemical Engineer, January 2002, p30.

4.6. PID TUNING METHODS 157

and we want to control this plant using a PID controller. To use Ziegler-Nichols settings we need
to establish the ultimate frequency, ωu , where the angle of G(s = iωu ) = −π radians, or solve the
nonlinear expression
6 G(iωu ) = −π (4.27)
for ωu . In the specific case given in Eqn. 4.26, we have

6 e−3iωu −6 2
1−6ωu +7iωu = −π
 
7ωu
−3ωu − tan−1 = −π (4.28)
1 − 6ωu2

which is a non-trivial function of ultimate frequency, ωu . However it is easy to graph this relation
as a function of ωu and look for zero crossing such as shown in Fig. 4.21, or use a numerical
technique such as Newton-Rhapson to establish that ωu ≈ 0.4839 rad/s implying an ultimate
period of Pu = 2π/ωu = 12.98 seconds per cycle.

−3 w−atan2(7 w,1−6 w2)+π

1
f(ω)

−1
Figure 4.21: Solving F (ω) = 0 for the
−2
ultimate frequency. In this case the ul-
0 0.2 0.4 0.6 0.8 1
ω timate frequency is ωu ≈ 0.48 rad/s.

A quick way to numerically solve Eqn. 4.28 is to first define an anonymous function and then to
use fzero to find the root.

 
7ω
1 >> F = @(w) -3*w - atan2(7*w,1-6*w.ˆ2)+pi; % F (ω) = −3ω − tan−1 1−6ω 2
+π
>> ezplot(F,[0 1]) % See Fig. 4.21.
>> wu = fzero(F,0.1) % Solve F (ω) = 0 for ω
wu =
0.4839

Once we have found the critical frequency, ωu , we can establish the magnitude at this frequency
by substituting s = iωu into the process transfer function, Eqn. 4.26,

|G(iωu )| = |G(0.48i)| ≈ 0.2931

which gives a critical gain, Ku = 1/0.2931 = 3.412. An easy way to do this calculation in M ATLAB
is to simply use bode at a single frequency, as in

>> [M,ph] = bode(G,wu) % Compute G(s = jωu ), should obtain φ = −180◦ , and Ku .
M =
0.2931
ph =
10 -180.0000

Now that we have both the ultimate gain and frequency we can use the classic Zielger-Nichols
rules in Table 4.4 to obtain our tuning constants and simulate the controlled process as shown in
Fig. 4.22.
158 CHAPTER 4. THE PID CONTROLLER

P control PI control PID control

1.4 1.4 1.4

1.2 1.2 1.2

1 1 1
output & setpoint

0.8 0.8 0.8

0.6 0.6 0.6
0.4 0.4 0.4
0.2 0.2 0.2
0 0 0
−0.2 −0.2 −0.2

2 2 2
input

1 1 1

0 0 0
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
time time time

Figure 4.22: The step and load responses of a P (left) and PI (middle) and PID (right) controlled
process using the closed loop Ziegler-Nichols suggestions from Table 4.4.

The tuned response to both a setpoint change and a load disturbance at t = 100 of all three
candidates shown in Fig. 4.22 is reasonable, but as expected the P response exhibits offset, and
the PI is sluggish. We would expect a good response from the PID controller because the actual
plant model structure (second order plus deadtime), is similar to the assumed model structure
by Ziegler and Nichols, which is first order plus deadtime. However while it could be argued
that the step response is still too oscillatory, the response to the load disturbance is not too bad.

In the very common case of a first order plus deadtime structure, Eqn. 4.25, we can find the
ultimate frequency by solving the nonlinear equation
− θωu − tan−1 (τ ωu ) + π = 0 (4.29)
for ωu , and then calculate the ultimate gain from a direct substitution
√
1 1 + τ 2 ω2
Ku = = (4.30)
|G(iωu )| K

The tedious part of the above procedure is generating the nonlinear equation in ωu for the spe-
cific plant structure of interest. We can however automate this equation generation for standard
polynomial transfer functions with deadtime as shown in the listing below.

[num,den] = tfdata(G,'v') % Or use: cell2mat(G.num)

iodelay = G.iodelay;

% Construct the equation 6 G(iω) = −π and solve for ω

15 F = @(w) angle(polyval(num,1i*w))-iodelay*w-angle(polyval(den,1i*w))+pi;
wc = fzero(F,0) % Now solve for ωc

Rather than solve for the critical frequency algebraically, we could alternatively rely on the M AT-
LAB margin routine which tries to extract the gain and phase margins and associated frequencies
for arbitrary transfer functions. Listing 4.4 shows how we can tune a PID controller for an arbi-
trary transfer function, although note that because this strategy attempts to solve a nonlinear
4.6. PID TUNING METHODS 159

expression for the critival frequency using a numerical iterative scheme, this routine is not infal-
lible.

Listing 4.4: Ziegler-Nichols PID tuning rules for an arbitrary transfer function
1
G = tf(1,conv([6 1],[1 1]),'iodelay',5); % Plant of interest G = (6s+1)(s+1)
e−5s

[Gm,Pm,Wcg,Wcp] = margin(G); % Establish critical points

20 Ku = Gm; Pu = 2*pi/Wcg; % Critical gain, Ku & period, Pu

Kc = 0.6*Ku; taui = Pu/2; taud = Pu/8; % PID tuning rules (Ziegler-Nichols)

Gc = tf(Kc*[taui*taud taui 1],[taui 0]); % PID controller Kc (τi τd s2 + τi s + 1)/(τi s)

Note that while the use of the ultimate oscillation Ziegler-Nichols tuning strategy is generally dis-
couraged by practicitioners, it is the oscillation at near instability which is the chief concern, not
the general idea of relating the tuning constants to the ultimate gain and frequency. If for exam-
ple, we already have a plant model in the form of a transfer function perhaps derived from first
principles, or system identification techniques, then we can by-pass the potentially hazardous
oscillation step, and compute directly the tuning constants as a function of ultimate gain and
frequency.

4.6.3 C LOSED LOOP SINGLE - TEST TUNING METHODS

Despite the fact that Ziegler-Nichols continuous cycling tuning method is performed in closed
loop, the experiment is both tedious and dangerous. The Yuwana-Seborg tuning method de-
scribed here, [207], retains the advantages of the ZN tuning method, but avoids many of the the
disadvantages. Given the attractions of this closed-loop reaction curve tuning methodology, there
have been subsequently many extensions proposed, some of which are summarised in [48, 100].
The following development is based on the modifications of [47] with the corrections noted by
[196].

The Yuwana-Seborg (YS) tuning technique is based on the assumption that the plant transfer
function, while unknown, can be approximated by the first order plus deadtime structure,

Km e−θm s
Gm (s) = (4.31)
τm s + 1
with three plant parameters, process gain Km , time constant, τm , and deadtime θm .

Surrounding this process with a trial, but known, proportional-only controller Kc , gives a closed
loop transfer function between reference r(t) and output y(t) as

Y (s) Kc Km e−θm s
= (4.32)
R(s) 1 + τ s + Kc Km e−θm s

A Padé approximation can be used to expand the non-polynomial term in the denominator, al-
though various people have subsequently modified this aspect of the method such as using a
different deadtime polynomial approximation. Using some approximation for the deadtime, we
can approximate the closed loop response in Eqn. 4.32 with

Ke−θs
Gcl ≈ (4.33)
τ 2 s2 + 2ζτ s + 1
and we can extract the model parameters in Eqn. 4.33 from a single closed loop control step test.
160 CHAPTER 4. THE PID CONTROLLER

peak #1
✻ yp1 peak #2
setpoint ..
.. yp2
..
.. ..
r1 ❲ .. ..
✻ .. .. offset ✻
.. ..
.. .. ❄ yss
.........
.. ..
.. .. ✕
.. ym ..
.. .. typical output
Setpoint .. ..
.. .
change ..✛ ✲...
. .
period, P

r0 ❄
time
✲

Figure 4.23: Typical response of a stable system to a P-controller.

Suppose we have a trial proportional-only feedback controlled response with a known controller
gain Kc . It does not really matter what this gain is so long as the response oscillates sufficiently
as shown in Fig. 4.23.

If we step change the reference variable setpoint from r0 to r1 , we are likely to see an under-
damped response such as shown in Fig. 4.23. From this response we measure the initial output y0 ,
the first peak yp1 , the first trough, ym1 and the second peak yp2 , and associated times. Under these
conditions we expect to see some offset given the proportional-only controlled loop, and that the
controller gain should be sufficiently high such that the response exhibits some underdamped
(oscillatory) behaviour.

The final value of the output, y∞ , is approximately

2
yp2 yp1 − ym
y∞ = (4.34)
yp1 + yp2 − 2ym

or alternatively, if the experimental test is of sufficient duration, y∞ could simply be just the last
value of y collected.

The closed loop gain K is given by

y∞
K= (4.35)
r1 − r0
and the overshoot is given by
 
1 yp1 − y∞ y∞ − ym yp2 − y∞
H= + + , (4.36)
3 y∞ yp1 − y∞ y∞ − ym

The deadtime is
θ = 2tp1 − tm , (4.37)
the shape factor
− ln(H)
ζ=q , (4.38)
π 2 + ln2 (H)
4.6. PID TUNING METHODS 161

and the time constant p

(tm − tp1 ) 1 − ζ 2
τ= . (4.39)
π
Now that we have fitted a closed loop model, we can compute the ultimate gain, Ku , and ultimate
frequency, ωu , by solving the nonlinear expression
 
−1 2τ ζωu
− θωu − tan = −π (4.40)
1 − τ 2 ωu2

for ωu , perhaps using an iterative scheme starting with ωu = 0. It follows then that the ultimate
gain is  
1
Ku = Kc 1 + (4.41)
|Gcl (iωu )|
where
K
|Gcl (iωu )| = p (4.42)
(1 − τ ωu )2 + (2τ ζωu )2
2 2

Alternatively of course we could try to use margin to extract Ku and ωu .

At this point, given Ku and ωu , we can tune PID controllers using the ZN scheme directly fol-
lowing Table 4.4.

Alternatively we can derive the parameters for the assumed open loop plant model, Eqn. 4.31, as

|y∞ − y0 |
Km = (4.43)
Kc (|r1 − r0 | − |y∞ − y0 |)
1 p 2 2
τm = Ku Km − 1 (4.44)
ωu
1  
θm = π − tan−1 (τm ωu ) (4.45)
ωu
and then subsequently we can tune it either using the ZN strategy given in Listing 4.4, or use
the internal model controller strategy from Table 4.3, or perhaps using ITAE optimal tuning con-
stants. The optimal ITAE tuning parameters for P, PI and PID controllers are derived from
 −B  D  F
A θm 1 1 θm θm
Kc = , = , τd = Eτm (4.46)
Km τm τi Cτm τm τm

where the constants A, B, C, D, E, F are given in Table 4.5 for P, PI and PID controllers. We are
now in a position to replace our trial Kc with the hopefully better controller tuning constants.

Table 4.5: Tuning parameters for P, PI, and PID controllers using an ITAE optimisation criteria
where the constants A through E are used in Eqn. 4.46.

mode A B C D E F
P 0.49 1.084 – – – –
PI 0.859 0.977 1.484 0.680 – –
PID 1.357 0.947 1.176 0.738 0.381 0.99

Note that the change in setpoint, |r1 − r0 | must be larger than the actual response |y∞ − y0 | owing
to the offset exhibited by a proportional only controller. If the system does not respond in this
manner, then a negative process gain is implied, which in turn implies nonsensical imaginary
time constants. Another often overlooked point is that if the controlled process response does
not oscillate, then the trial gain Kc is simply not high enough. This gain should be increased, and
162 CHAPTER 4. THE PID CONTROLLER

the test repeated. If the process never oscillates, no matter how high the gain is, then one can use
an infinite gain controller — at least in theory.

Algorithm 4.2 summarises this single-test closed loop tuning strategy.

Algorithm 4.2 Yuwana-Seborg closed loop PID tuning

Tuning a PID controller using the closed-loop Yuwana-Seborg method is very similar to Algo-
rithm 4.1, but avoids the trial and error search for the ultimate gain and period.

1. Connect a proportional only controller to the plant.

2. Choose a trial sensible controller gain, Kc and introduce a set point change and record the
output.

3. Check that the output is underdamped and shaped similar to Fig. 4.23, and if not increase
the trial controller gain, Kc , and repeat step 2.

4. From the recorded underdamped response, measure the peaks and troughs, and note the
demanded setpoint change. You may find Listing 4.5 helpful here.

5. Compute the parameters of the closed loop plant and solve for the ultimate frequency,
Eqn. 4.40 and gain, Eqn. 4.41.

6. Optionally compute the open loop plant model using Eqn. 4.43—Eqn. 4.45.

7. Compute the P, PI or PID tuning constants using Table 4.5 or perhaps using the model and
the ZN strategy in Listing 4.4.

8. Test the closed loop response with the new PID values. If the response is not satisfactory,
further manual ‘fine-tuning’ may be necessary.

A M ATLAB implementation to compute the tuning constants given response data is given in
listings starting on page 164.

A consequence of the assumption that the plant is adequately modelled using first-order dy-
namics with deadtime, the Yuwana-Seborg tuning scheme is not suitable for highly oscillatory
processes, nor plants with inverse response behaviour as illustrated in section 4.8.1. Furthermore,
a consequence of the low order Padé approximation is that the strategy fails for processes with
a large relative deadtime. Ways to use this tuning scheme in the presence of excessive measure-
ment noise is described in section 5.2.1.

An example of a closed loop response tuner

Suppose we are to tune a PID controller for an inverse response process,

−s + 1
G(s) =
(s + 1)2 (2s + 1)

Such processes with right-hand plane zeros are challenging because of the potential instability
owing to the initial response heading off into the opposite direction as described further in §4.8.1.

The response to a unit setpoint change is given in the top figure of Fig. 4.24. By recording the
values of the peaks and troughs, we can estimate the parameters of an approximate model of the
4.6. PID TUNING METHODS 163

Trial closed loop step test, K = 1

c

0.5

0 tp1 tm1 tp2

0 5 10 15 20 25

Closed loop model

1
Amplitude

0.5

Actual
0 Fitted

0 5 10 15 20 25
Time (sec)
Open loop model
Figure 4.24: A Yuwana-Seborg closed loop
1 step test. The top plot shows the response
Amplitude

given a unit change in setpoint with the key

0.5 points marked. The middle plot compares
Actual the estimated with the actual closed loop
0 fitted data, while the bottom plot compares the
0 5 10 15 20 25 30 estimated openloop plant model with the
Time (sec) true plant.

closed loop response, Eqn. 4.33. A comparison of this model with the actual data is given in the
middle plot of Fig. 4.24. It is not a perfect fit because the underlying plant is not a pure first order
with deadtime, and because Eqn. 4.33 is only approximate anyway.

Given the closed loop model, we can now extract the open loop model of the plant which is
compared with the true plant in the bottom trend of Fig. 4.24. Note how the first-order plus
deadtime model, specifically in this case

1.003
Ĝ = e−2.24s
3.128s + 1

approximates within reason the higher order true plant dynamics.

Finally now that we have a plant model,Ĝ, we can either use the ITAE optimal or IMC tuning
scheme as shown in Fig. 4.25.

As evident in Fig. 4.25, the ITAE responses are overly oscillatory which tends to be a common
failing of PID tuning schemes that are designed to satisfy a performance objective.

In the IMC case, the desired closed loop time constant was set equal to the sum of the dead-
time and the open loop time constant, τc = τm + θm , which as it turns out is perhaps overly
conservative in this case. Better performance would be achieved by reducing τc .
164 CHAPTER 4. THE PID CONTROLLER

ITAE PI control ITAE PID control

1.5 1.5

1 1
y&r

0.5 0.5

0 0
2 2

0 0
u

−2 −2

IMC PI control IMC PID control

1.5 1.5

1 1
y&r

0.5 0.5

0 0
2 2

0 0
u

−2 −2
0 10 20 30 40 50 0 10 20 30 40 50

Figure 4.25: The PI (left) and PID (right) closed loop responses using the ITAE (upper) and IMC
(lower) tuning schemes based on the identified model but applied to the actual plant.

A UTOMATING THE Y UWANA -S EBORG TUNER

Because this is such a useful strategy for PID tuning, it is helpful to have a semi-automated
procedure. The first step once you have collected the closed loop data with a trial gain is to
identify the values and timing of the peaks and troughs, yp1 , ym1 and yp2 . Listing 4.5 is a simple
routine that attempts to automatically extract these values. As with all small programs of this
nature, the identification part can fail, particularly given excessive noise or unusual response
curves.

Listing 4.5: Identifies the characteristic points for the Yuwana-Seborg PID tuner from a trial closed
loop response
function [yp1,tp1,ym1,tm1,yp2,tp2,yss_est] = ys_ident(t,Y);
% Identify the peaks & troughs from an underdamped closed loop test
3 % Used for Yuwana-Seborg PID tuning

y0 = Y(1); n = length(Y); % start point & total # of points

if n < 20
fprintf('Probably need more points\n');
8 end % if
Y = Y*((Y(n) > y0)-0.5)*2; % workout direction, 1 if positive)
yss_est = mean(Y(n:-1:n-round(n/50))); % average pt at the end (robust)

[yp1,idx] = max(Y); tp1 = t(idx);
13 Y = Y(idx:n); t = t(idx:n);
[ym1,idx] = min(Y); tm1 = t(idx);
Y(1:idx) = []; t(1:idx)=[];
[yp2,idx] = max(Y); tp2 = t(idx);
% 1st maximum point & time at occurrence
% chop off to keep searching
% 1st minimum point & time
% 2nd maximum point & time at occurrence
4.6. PID TUNING METHODS 165

18 return % end ys_ident.m

Using this peak and trough data, we can compute the parameters in the closed loop model fol-
lowing Listing 4.6.

Listing 4.6: Compute the closed loop model from peak and trough data
yinf = (yp1*yp2 - ym1ˆ2)/(yp1+yp2-2*ym1); % Steady-state output, y∞
2 K = yinf/A; % Closed loop gain, K, Eqn. 4.35 & Overshoot, H
H = 1/3*((yp1-yinf)/yinf+(yinf-ym1)/(yp1-yinf)+(yp2-yinf)/(yinf-ym1));

d = 2*tp1 - tm1; % Deadtime, θ, Eqn. 4.37

zeta = -log(H)/sqrt(piˆ2+log(H)ˆ2); % shape factor, ζ, Eqn. 4.38
7 tau = (tm1-tp1)*sqrt(1-zetaˆ2)/pi; % time constant, τ , Eqn. 4.39

Gcl = tf(K,[tauˆ2 2*zeta*tau 1],'iodelay',d); % Closed loop model, Gcl (s), Eqn. 4.33

Now that we have the closed loop model, we can compute the ultimate gain, Ku , and ultimate
frequency, ωu .

Listing 4.7: Compute the ultimate gain and frequency from the closed loop model parameters.
10 fwu = @(w) pi - d*w - atan2(2*zeta*tau*w,1-tauˆ2*w.ˆ2); %

[Gm,Pm,Wcg,Wcp] = margin(Gcl) % Use as a good starting estimate for ωu

wu = fsolve(fwu,Wcg); % Solve nonlinear Eqn. 4.40 for ωu

15 B= K/sqrt((1-tauˆ2*wuˆ2)ˆ2+(2*tau*zeta*wu)ˆ2);
Kcu = Kc*(1+1/B); % Ultimate gain, Ku , Eqn. 4.41

Finally we can now extract the open loop first-order plus deadtime model using Listing 4.8.

Listing 4.8: Compute the open loop model, Gm , Eqn. 4.31.

Km = yinf/(Kc*(A-yinf)); % Plant gain, Gm , Eqn. 4.43
taum = 1/wu*sqrt(Kcuˆ2*Kmˆ2-1); % Plant time constant, τm , Eqn. 4.44
dm = 1/wu*(pi - atan(taum*wu)); % Plant delay, θm , Eqn. 4.45
20 Gm = tf(Km,[taum 1],'iodelay',dm);% Open loop model estimate, Gm (s), Eqn. 4.31

Now that we have an estimate of the plant model, it is trivial to use say the IMC relations to
compute reasonable PI or PID tuning constants. We do need however to decide on an appropriate
desired closed loop time constant.

Listing 4.9: Compute appropriate PI or PID tuning constants based on a plant model, Gm , using
the IMC schemes.
tauc = (tau+dm)/3; % IMC desired closed loop time constant, τc
controller = 'PID'; % Type of desired controller, PI or PID

switch controller % See IMC tuning schemes in Table 4.3.

25 case 'PI'
Kc = 1/Km*(tau)/(tauc+dm);
taui = taum; rs = 1/taui; % Integral time, τi , and reset, 1/τi .
166 CHAPTER 4. THE PID CONTROLLER

taud = 0;
case 'PID'
30 Kc = 1/Km*(tau+dm/2)/(tauc+dm/2);
taui = tau+dm/2; rs = 1/taui;
taud = tau*dm/(2*tau+dm);
end % switch

Now all that is left to do is load the controller tuning constants into the PID controller, and set
the controller to automatic.

4.6.4 S UMMARY ON CLOSED LOOP TUNING SCHEMES

These automated PID tuners are, not surprisingly, quite popular in industry and also in academic
circles. Most of the schemes are based on what was described above, that is the unknown process
is approximated by a simple transfer function, where the ultimate gain and frequency is known
as a function of parameters. Once these parameters are “curve fitted”, then the PID controller is
designed via a Ziegler–Nichols technique. There are numerous modifications to this basic tech-
nique, although most of them are minor. One extension is reported in [48] while [196] summarises
subsequent modifications, corrects errors, and compares alternatives. The relay feedback method
of Hägglund and Åström, [15], described in next in §4.7 is also a closed loop single test method.

Problem 4.2 1. Modify the ys_ident.m function to improve the robustness. Add improved
error checking, and try to avoid other common potential pitfalls such as negative controller
gain, integrator responses, sever nonlinearities, excessive noise, spikes etc.

2. Test your auto closed loop tuner on the following three processes Gp (s) (adapted from [48]);

e−4s
G1 (s) = (4.47)
s+1
e−3s
G2 (s) = (4.48)
(s + 1)2 (2s + 1)
e−0.5s
G3 (s) = (4.49)
(s − 1)(0.15s + 1)(0.05s + 1)

G1 and G2 show the effect of some dead time, and G3 is an open loop unstable process.

3. Create a m file similar to the one above, that implements the closed loop tuning method as
described by Chen, [48]. You will probably need to use the M ATLAB fsolve function to
solve the nonlinear equation.

4.7 A UTOMATED TUNING BY RELAY FEEDBACK

Because the manual tuning of PID controllers is troublesome and tedious, it is rarely done which
is the motivation behind the development of self-tuning controllers. This “tuning on demand”
behaviour after which the controller reverts back to a standard PID controller is different from
adaptive control where the controller is consistently monitoring and adjusting the controller tun-
ing parameters or even the algorithm structure. Industrial acceptance of this type of smart con-
troller was much better than for the fully adaptive controller, partly because the plant engineers
were far more confident about the inherent robustness of a selftuning controller.
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 167

One method to selftune employed by the ECA family of controllers manufactured by ABB shown
in Fig. 4.26 uses a nonlinear element, the relay. The automated tuning by relays is based on the
assumption that the Ziegler-Nichols style of controller tuning is a good one, but is flawed in
practice since finding the ultimate frequency ωu is a potentially hazardous tedious trial and error
experiment. The development of this algorithm was a joint project between SattControl and Lund
University, Sweden [15]. This strategy of tuning by relay feedback is alternatively known as the
Auto Tune Variation or ATV method.

Figure 4.26: A PID con-

troller with a self-tuning
option using a relay from
ABB.

The PID controller with selftuning capability using relay feedback is really two controllers in one
as shown in Fig. 4.27. Here the PID component is disabled, and the relay is substituted. Once the
executive software is confident that the updated controller parameters are an improvement, the
switch is toggled, and the controller reverts back to a normal PID regulator.

relay (active)
d
✲
Plant
setpoint −d

r ✲+ ǫ ■ u ✲
Gp (s) ✲ y
✻−

✲ PID

PID (inactive)

Figure 4.27: A process under relay tuning with the PID regulator disabled.

A relay can be thought of as an on/off controller or a device that approximates a proportional

controller with an infinite gain, but hard limits on the manipulated variable. Thus the relay has
two outputs: if the error is negative, the relay sends a correcting signal of −d units, and if the
error is positive, the relay sends a correcting signal of +d units.
168 CHAPTER 4. THE PID CONTROLLER

4.7.1 D ESCRIBING FUNCTIONS

A relay is a nonlinear element that, unlike the situation with a linear component, the output
to a sinusoidal input will not be in general sinusoidal. However for many plants that are low-
pass in nature, the higher harmonics are attenuated, and we need only consider the fundamental
harmonic of the nonlinear output.

The describing function N , of a nonlinear element is defined as the complex ratio of the funda-
mental harmonic of the output to the input, or

def Y1
N = 6 φ (4.50)
X1

where X1 , Y1 are the amplitudes of the input and output, and φ is the phase shift of the fun-
damental harmonic component of the output, [152, p652]. If there is no energy storage in the
nonlinear element, then the describing function is a function only of the input amplitude.

By truncating a Fourier series of the output, and using Eqn. 4.50, one can show that the describing
function for a relay with amplitude d is

4d
N (a) = (4.51)
πa
where a is the input amplitude.

Now a sustained oscillation or limit cycle of a linear plant G(iω) together with a nonlinear ele-
ment with describing function N will occur when 1 + N G(iω) = 0 or

1
G(iω) = − (4.52)
N
Now since in the case of a relay, N given by Eqn. 4.51 is purely real, then the intersection of G(iω)
and −1/N is somewhere along the negative real axis. This will occur at a gain of πa/(4d), so the
ultimate gain, Ku , which is the reciprocal of this, is

4d
Ku = (4.53)
πa
The ultimate frequency, ωu = 2π/Pu , (in rad/s) is simply the frequency of the observed output
oscillation and can be calculated from counting the time between zero crossings, and the ampli-
tude, a, by measuring the range of the output. We now know from this simple single experiment
one point on the Nyquist curve and a reasonable PID controller can be designed based on this
point using, say, the classical Ziegler-Nichols table.

In summary, under relay feedback control, most plants will exhibit some type of limit oscillation
such as a sinusoidal-like wave with a phase lag of −180◦. This is at the point where the open loop
transfer function crosses the negative real axis on a Nyquist plot, although not necessarily at the
critical −1 point. A typical test setup and response is given in Fig. 4.28.

Given that we know a single point on the Nyquist curve, how do we obtain controller param-
eters? It is convenient to restrict the controller to the PID structure since that is what is mostly
about in practice. The original ZN tuning criteria gave responses that should give a specified
gain and phase margin. For good control a gain margin of about 1.7 and a phase margin of about
φm = 30◦ is recommended. Note that by using different settings of the PID controller, one can
move a single point on the Nyquist curve to any other arbitrary point. Let us assume that we
know the point where the open loop transfer function of the process, Gp (s), crosses the negative
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 169

Relay Unknown plant

u
✲+ ✲ ✲ Plant ✲ output
✻−

✻
output, y output amplitude 2a

❄
✲

✻
input, u ✻ relay amplitude 2d
❄
✲
✛ ✲ time, t
P
period

Figure 4.28: An unknown plant under relay feedback exhibits an oscillation

real axis, ωu . This point has a phase (or argument) of −π or −180◦. We wish the open loop trans-
fer function, Gc Gp to have a specified phase margin of φm by choosing appropriate controller
tuning constants. Thus equating the arguments of the two complex numbers gives
 
1
arg 1 + + iωu τd − π = φm − π (4.54)
iωu τi
which simplifies to5
1
ωu τd − = tan φm
ωu τi
Since we have two tuning parameters (τi , τd ), and one specified condition, (φm ), we have many
solutions. Since their are many PID controller constants that could satisfy this phase margins,
Åström, [11], chose that the integral time should be some factor of the derivative time,
τi = f τd (4.55)
Then the derivative time, which must be real and positive, is
p
f tan φm + f 2 tan2 φm + 4f
τd = (4.56)
2f ωu
where the arbitrary factor, f ≈ 4. Given the ultimate gain Ku , the controller gain is
Kc = Ku cos φm (4.57)
5 Remember that def
for a complex number z = x + iy, the argument of z is the angle z makes with the real axis;
y
arg(z) = arg(x + iy) = tan−1
x
170 CHAPTER 4. THE PID CONTROLLER

Algorithm 4.3 Relay auto-tuning

In summary, to tune a system by relay feedback, you must:

1. Turn off the current PID controller, and swap over to a relay feedback controller with known
relay gain d.
2. Apply some excitation if necessary to get the process to oscillate under relay control. Mea-
sure the resultant error amplitude a, and period Pu . Calculate the ultimate gain from
Eqn. 4.53 and associated frequency ωu .
3. Decide on a phase margin, (φm ≈ 30◦ ), the factor, f ≈ 4, and calculate the PID controller
tuning constants using Eqns 4.55–4.57.
4. Download these parameters to the PID controller, turn off the relay feedback, and swap
back over to the PID controller.
5. Wait until the operator again initiates the self-tuning program after which return to step 1.

4.7.2 AN EXAMPLE OF RELAY TUNING

We can compare the experimentally determined ultimate gain, Ku , and ultimate frequency, ωu ,
with the theoretical values for the plant
1
G(s) = (4.58)
(τ s + 1)4
where the time constant is τ = 0.25. The ultimate gain and frequency are analytically obtained
by solving the two equations
   
Ku  Ku 
arg = −π, and   =1
(τ s + 1)4 s=iωu  (τ s + 1)4 
s=iωu

simultaneously which is somewhat involved. An alternative graphical solution is obtained using

Bode and/or Nyquist diagrams in M ATLAB.

G = zpk([],[-4 -4 -4 -4],4ˆ4);
2 nyquist(G,logspace(-1,3,200));
bode(G) % Bode diagram is an alternative

The Nyquist diagram given in the upper plot of Fig Fig. 4.29 shows the curve crossing the nega-
tive real axis at −1/Ku ≈ −0.25, thus Ku ≈ 4. To find the ultimate angular frequency, we need to
plot the phase lag of the Bode diagram shown in the bottom figure of Fig. 4.29. The phase angle
crosses the −180◦ ≡ −π radians point at a frequency of about ωu = 4 rad/s. Thus the ultimate
period is Pu = 2π/ωu = 1.57 seconds. Note that it was unnecessary to plot the Nyquist diagram
since we could also extract the ultimate gain information from the magnitude part of the Bode
diagram.

E STIMATING Ku AND ωu BY RELAY EXPERIMENT

Now we can repeat the evaluation of Ku and ωu , but this time by using the relay experiment.
I will use a relay with a relay gain of d = 2 but with no hysteresis (h = 0) and a sample time
of T = 0.05 seconds. To get the system moving, I will choose a starting value away from the
setpoint, y0 = 1.
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 171

Nyquist Diagram

0.5 1/K
u
Imaginary Axis

0 6
4
0.1
3
−0.5 0.5
2
1.5 1
−1
−1 −0.5 0 0.5 1
Real Axis

Bode Diagram
Gm = 12 dB (at 4 rad/sec) , Pm = −180 deg (at 0 rad/sec)

0
Magnitude (dB)

−50

−100

−150
0
Figure 4.29: Nyquist (top) and Bode (bot-
Phase (deg)

−180
tom) diagrams for 1/(0.25s + 1)4 . The
Nyquist curve crosses the negative ℜ-axis
at about 0.25. The frequency at which the
−360
10
−1
10
0 ω 1 2 Bode curve passes though φ = −180◦ is
u 10 10
Frequency (rad/sec) ω ≈ 4 rad/s.

G = zpk([],[-4 -4 -4 -4],4ˆ4); % Plant: G(s) = 1/(0.25s + 1)4

2 Ts=0.05; t = [0:Ts:7]'; % sample time
u = 0*t; y=NaN*t; % initialise u & y
d=2; % relay gain

[Phi,Del,C,D] = ssdata(c2d(G,Ts));
7 x = [0,0,0,0.1]'; % some non-zero initial state

for i=1:length(t)-1; % start relay experiment

x = Phi*x + Del*u(i); % integrate model 1 step
y(i) = C*x; % output
12 u(i+1) = -d*sign(y(i)); % Relay controller
end % for

plot(t,y,t,u,'r--')

The result of the relay simulation is given in Fig. 4.30 where we can see that once the transients
have died out, the amplitude of the oscillation of the output (or error since the setpoint is con-
stant) is a = 0.64 units, with a period of P = 1.6 seconds. Using Eqn. 4.53, we get estimates for
the ultimate gain and angular frequency

4d
P̂u = P = 1.6 ≈ 1.57s, and K̂u = = 3.978 ≈ 4
πa
172 CHAPTER 4. THE PID CONTROLLER

which approximately equal the estimates that I graphically found using the Bode and Nyquist
diagrams of the true plant.

2
output & setpoint

0
a = 0.64
−1
relay off
−2
5
input

0
Pu = P = 1.6s
−5
0 5 10 15 20 25 30
time [s]

Figure 4.30: PID tuning using relay feedback. For the first 10 seconds the relay is enabled, and
the process oscillates at the ultimate frequency. After 10s, the relay is turned off, the PID tuner
with the updated constants is enabled, and the plant is under control.

Once we have the ultimate gain and frequency, the PID controller constants are obtained from
Eqn. 4.55 to 4.57 and are shown in Table 4.6. Alternatively we can use any Z-N based tuning
scheme.

Table 4.6: The PID controller tuning parameters obtained from a relay based closed loop experi-
ment
PID parameter Hägglund Ziegler-Nichols units
Kc 3.445 0.6Ku = 3.4 –
τi 0.866 Pu /2 = 0.8 s
τd 0.217 Pu /8 = 0.2 s

Now that we have the PID tuning constants, we can simulate the closed loop response to setpoint
changes. We could create the closed loop transfer function using G*Gpid/(1 + Gpid*G), how-
ever the preferred approach is to use the routines series and feedback. This has the advantage
that it will cancel common factors and perform a balanced realisation.

G = zpk([],[-4 -4 -4 -4],4ˆ4); % Plant to be controlled, Eqn. 4.58.

K = 3.4; ti = 0.8; td = 0.2; % Controller constants from Table 4.6.

Gpid = tf(K*[ti*td ti 1],[ti 0])
5 Gcl = feedback(series(G,Gpid),1); % Create closed loop for simulation

R = [zeros(10,1);ones(100,1);-ones(100,1)]; % Design setpoint vector

T = 0.05*[0:1:length(R)-1]'; % time scale
y = lsim(Gcl,R,T); % Simulate continuous
10 plot(T,[y,R])
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 173

The remainder of Fig. 4.30 for the period after 10 seconds shows the quite reasonable response us-
ing the PID constants obtained via a relay experiment. Perhaps the overshoot at around 40% and
oscillation is a little high, but this is a well known problem of controller tuning via the classical
Z–N approach, and can be addressed by using the modified ZN tuning constants.

4.7.3 S ELF - TUNING WITH NOISE DISTURBANCES

The practical implementation of the relay feedback identification scheme requires that we mea-
sure the period of oscillation P , and the error amplitude a automatically. Both these parameters
are easily established manually by visual inspection if the process response is well behaved and
noise free, but noisy industrial outputs can make the identification of even these simple charac-
teristics difficult to obtain reliably and automatically.

When considering disturbances, [11] develops two modifications to the period and amplitude
identification part of the selftuning algorithm. The first involves a least squares fit, (described
below) and the second involves an extended Kalman filter described in problem 9.7. Of course
hybrids of both these schemes with additional heuristics could also be used. Using the same pro-
cess and relay as in §4.7.2, we can introduce some noise into the system as shown in the S IMULINK
model given in Fig. 4.31(a). Despite the low noise power (0.001), and the damping of the fourth-
order process, the oscillations vary in amplitude and frequency as shown in Fig. 4.31(b).

To obtain reliable estimates of the amplitude and period we can use the fact that a sampled
sinusoidal function with period P and sample time T satisfies

y(t) − θ1 y(t − T ) + y(t − 2T ) + θ2 = 0 (4.59)

with  
2πT
θ1 = 2 cos (4.60)
P
The extra constant, θ2 is introduced to take into account non-zero means. Eqn. 4.59 is linear
in the parameters, and we can solve the least-squares regression problem in M ATLAB using the
backslash command.  
θ1  +
θ= = yt−T , −1 (yt + yt−2T ) (4.61)
θ2
Once we have fitted θ1 , (we are uninterested in θ2 ), the period is obtained by inverting Eqn. 4.60,

2πT
P = (4.62)
cos−1 (θ1 /2)

Now that we know the period, we can solve another linear least-squares regression problem
given the N output data points,

N      2
X 2πkT 2πkT
min y(kT ) − θ3 sin − θ4 cos − θ5 (4.63)
θ k=1 P P

where the amplitude is given by

q
a= θ32 + θ42

By doing the regression in this two step approach, we avoid the nonlinearity of Eqn. 4.63 if we
were to estimate P as well in an attempt to do the optimisation in one step.

Running the S IMULINK simulation given in Fig. 4.31(a) gives a trend something like Fig. 4.31(b).
174 CHAPTER 4. THE PID CONTROLLER

Band−Limited
White Noise

256
0
(s+4)(s+4)(s+4)(s+4)
Constant Relay Zero−Pole output

(a) Relay feedback of a fourth-order plant with band-limited noise added

Relay tuning with noise
2

1
input & output

−1

−2
0 2 4 6 8 10 12 14 16 18 20
time

(b) The controlled output and relay response.

Figure 4.31: Relay oscillations with noise

The only small implementation problem is that S IMULINK does not necessarily return equally
sampled data for continuous system simulation. The easiest work-around is to define a regularly
spaced time vector and do a table lookup on the data using interp1. The script file given in
listing 4.10 calculates the period and amplitude using least-squares.

Listing 4.10: Calculates the period and amplitude of a sinusoidal time series using least-squares.
N = length(t); % Length of data series
tr = linspace(t(1),t(length(t)),1.5*N)'; dt = tr(2)-tr(1);
yr = interp1(t,y,tr,'linear'); % Simulink is not regular in time
y=yr; t = tr; % Now sample time T is regularly spaced.
5

% Construct data matrix

npts = length(y); % # of data points
rhs = y(3:npts) + y(1:npts-2);
X = [y(2:npts-1),-ones(size(rhs))];
10 theta = X\rhs; % Solve LS problem, Eqn. 4.61.
P = 2*pi*dt/acos(theta(1)/2); % Now find period from Eqn. 4.62.

omega = 2*pi/P; % 2nd round to find amplitude .....

k = [1:npts]';
15 X = [sin(omega*k*dt) cos(omega*k*dt) ones(size(k))]; % data matrix
theta = X\y; % Solve 2nd LS problem, Eqn. 4.63.
a = norm(theta(1:2)); % Extract amplitude.

In my case, the computed results are:

P = 1.822, a = 0.616

which is not far off the values obtained with no disturbances, from page 171 of P = 1.6 and
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 175

t
Clock To Workspace1 Auto−Scale
Graph1
Adapter Mux yout
To Workspace
PID Mux
0.6
PID Controller
setpoint
+ +
+ Output Plug
− 0 RT Out RT In
Sum1 Sum
Signal Constant Switch RT Out RT In
generator1 Input Plug

Relay

(a) A S IMULINK implementation of the relay feedback experiment.

Relay based tuning (Black Box)

0.8

0.7
Output & setpoint

0.6

0.5

0.4

0.3
0 10 20 30 40 50 60 70

0.5
input

−0.5

−1
0 10 20 30 40 50 60 70
time (s)
(b) The output of the relay experiment

Figure 4.32: Experimental implementation of relay tuning of the blackbox

a = 0.64.

Relay based tuning of a ‘black-box’

Using a data acquisition card and a real-time toolbox extension6 to M ATLAB, we can repeat the
trial of the relay based tuning method this time controlling a real process, in this case a “black
box”. The S IMULINK controller and input/output data are given in Fig. 4.32.

From Fig. 4.32, we can measure the period and amplitude of the oscillating error
Pu = 7 seconds, a = 0.125
giving an approximate ultimate gain as
4d
Ku = , with d = 1
πa
4
= = 10.2
0.125π
which leads to appropriate Kc , τi , τd , using the modified ZN rules
6 Available from Humusoft
176 CHAPTER 4. THE PID CONTROLLER

Kc τi (s) τd (s)
0.33Ku = 3.4 Pu /2 = 3.5 Pu /3 = 2.3

Since S IMULINK uses a slightly different PID controller scheme from the classical formula,

Kc
P = Kc = 3.4, I= = 0.97, D = Kc τd = 7.8
τi

Using these values in the PID controller with the black-box, we obtain the controlled results
shown in Fig. 4.33 which apart from the noisy input and derivative kick (which is easily removed,
refer §4.4.1), does not look too bad.

PID control based on relay tune

1
Output & setpoint

0.8

0.6

0.4

0.2
0 20 40 60 80 100 120

1.5

0.5
input

−0.5

−1

−1.5
0 20 40 60 80 100 120
time (s)

Figure 4.33: Experimental PID control of the blackbox using parameters obtained by the relay-
based tuning experiment. (See also Fig. 4.32.)

4.7.4 M ODIFICATIONS TO THE RELAY FEEDBACK ESTIMATION ALGORITHM

The algorithm using a relay under feedback as described in section 4.7 establishes just one point
on the Nyquist curve. However it is possible by using a richer set of relay feedback experiments
to build up a more complex model, and perhaps better characterise a wider collection of plants.

R ELAYS WITH HYSTERESIS

Physical relays always have some hysteresis h, to provide mechanical robustness when faced
with noise. By adjusting the hysteresis width we can excite the plant at different points on the
Nyquist curve.
4.7. AUTOMATED TUNING BY RELAY FEEDBACK 177

The response of a relay with hysteresis is given in Fig. 4.34. Note that the relay will not flip to
+d from the −d position until we have a small positive error, in this case h. Likewise, from the
reverse direction, the error must drop below a small negative error, −h, before the relay flips from
+d to −d.

✛
+d relay amplitude

output 0

−d ✲

−h 0 h
error

Figure 4.34: A relay with hysteresis width h and output amplitude d.

The describing function for a relay with amplitude d and hysteresis width h is

1 πp 2 πh
− =− a − h2 − j (4.64)
N (a) 4d 4d

which is a line parallel to the real axis in the complex plane. Compare this with the describing
function for the pure relay in Eqn. 4.51.

The intersection of this line and G(iω) is the resultant steady-state oscillation due to the relay,
refer Fig. 4.35. By adjusting the relay hysteresis width, h, we can move the −1/N (a) line up and
down, thereby establishing different points on the Nyquist diagram. Of course if we increase the
hysteresis too much, then we may shift the 1/N (a) curve too far down so that it never intercepts
the G(iω) path. In this situation, we will not observe any limit cycle in the closed loop experiment.

Fig. 4.36 shows the estimated frequency response for the plant G(s) = 256/(s + 4)4 calculated
using a relay with d = 2 and varying amounts of hysteresis from h = 0 to h = 1.4. Note that the
calculated points do follow the true frequency response of the plant, at least for small values of h.
Obviously if h is too large, then the line −1/N (a) will not intersect the frequency response curve,
G(iω), so there will be no oscillations.

I NSERTING KNOWN DYNAMICS INTO THE FEEDBACK LOOP

A second simple modification is to insert an integrator prior to the relay. The integrator subtracts
−π/2 or −90◦ from the phase, and multiplies the gain by a factor of 1/ω. In this way, under relay
feedback we can estimate the point where the Nyquist curve crosses the negative imaginary
axis as well as the negative real axis. Such a scheme is shown in Fig. 4.37 and is described by
[202] which is a specific case of the general schemes described by [118, 146]. A survey of other
modifications to the basic relay feedback idea for controller tuning is given in [44].
178 CHAPTER 4. THE PID CONTROLLER

ℑ
✻

G(iω)
0 ✲ ℜ

−1
N (a) ✰ increasing ω
✒
operating point under
relay feedback

Figure 4.35: Relay feedback with hysteresis width h.

0.4

0.2

0 0
0.2
0.4
−0.2 0.6
ℑG(i ω)

0.8
−0.4 1
1.2
1.4
−0.6

−0.8
Figure 4.36: Using a relay feedback with varying
−1
amounts of hysteresis from h = 0 to h = 1.4 to ob-
tain multiple points, ✷, on a Nyquist curve. The −1.2
−1 −0.5 0 0.5 1
true frequency response G(iω) is the solid line. ℜG(i ω)

Relay

K(s)
0
den(s)
Constant Manual Switch Transfer Fcn Transport output
1 Delay
s
Integrator Relay1

Figure 4.37: Relay feedback with an integrator option

4.7. AUTOMATED TUNING BY RELAY FEEDBACK 179

Establishing multiple points on the Nyquist curve enables one to extract a transfer function mod-
els. In the case of a first-order plus deadtime model

Kp
G(s) = e−Ls (4.65)
τs + 1
we need two arbitrary distinct points. That is we establish the magnitude and phase of G(iω) at
two frequencies ω1 and ω2 via two relay experiments.

Defining the magnitudes and angles to be

1 1
|G(iω1 )| = , |G(iω2 )| =
k1 k2
6 G(iω1 ) = φ1 , 6 G(iω2 ) = φ2

then the plant gain is s

ω12 − ω22
Kp = , (4.66)
k22 ω12 − k12 ω12
the time constant is
1 q 2 2
τ= k1 Kp − 1, (4.67)
ω1
and the deadtime is
1

L=− φ1 + tan−1 (ω1 τ ) . (4.68)
ω1
In the case of a pure relay and a pure relay with an integrator,

φ1 = −π, φ2 = −π/2.

The case for a second order plant

Kp
G(s) = e−Ls (4.69)
s2 + as + b
is more complicated. The following algorithm, due to [118], first estimates the deadtime via a
nonlinear function, then extracts the coefficients Kp , a and b using least-squares.

First we define the matrices

     
A1 −ω12 0 1   sin Lω1
 A2   −ω22 1/Kp
0 1   cos Lω1 
A=  A3  =  0
  , X =  a/Kp  , Y=  (4.70)
ω1 0   sin Lω2 
b/Kp
A4 0 ω2 0 cos Lω2

and
  
B1 ℑ{G−1 (jω1 )} ℜ{G−1 (jω1 )} 0 0
−1
 B2   0 0 ℑ{G (jω2 )} ℜ{G−1 (jω2 )} 
B=
 B3  =  −ℜ{G−1 (jω1 )} ℑ{G−1 (jω1 )}
   (4.71)
0 0 
B4 0 0 −ℜ{G−1 (jω2 )} ℑ{G−1 (jω2 )}

Now provided we know the deadtime L, we can solve for the remaining parameters in X using
 −1  
A1 B1
X =  A2   B2  Y (4.72)
A3 B3
180 CHAPTER 4. THE PID CONTROLLER

0.4
Step Response

2
0.2 G
G
1est
1.5 G
2est
0

1
ℑ{G(i ω)}

−0.2

Amplitude
−0.4 0.5

−0.6 0

−0.8 −0.5

−1 −1
−1 −0.5 0 0.5 1 1.5 2 0 20 40 60 80 100 120 140 160 180
ℜ{G(i ω)} Time (sec)

(a) The resultant Nyquist plots (b) The resultant step responses

Figure 4.38: Identification of transfer function models using a two-point relay experiment.

However since we do not know L, we must solve the nonlinear relation

(A4 C − B4 )Y = 0 (4.73)

Once given the model, any standard model-based controller tuning scheme such as internal
model control (IMC) can be used to establish appropriate PID tuning constants.

Fig. 4.38 illustrates the procedure for when the true, but unknown plant is the third-order system
1
G= e−6s
(9s + 1)(6s + 1)(4s + 1)
with estimated first and second order models.

Problem 4.3 1. Draw a Nyquist diagram for the transfer function Gp (s). Investigate what hap-
pens to the Nyquist diagram when a PI controller is added. How is the curve shifted
4
Gp (s) =
(τ s + 1)3
where τ = 2.0.
2. Write down pseudo code for the automatic tuning (one button tuning) of a PID controller
using the ZN tuning criteria. Ensure that your algorithm will work under a wide variety of
conditions and will not cause too much instability if any. Consider safety and robustness
aspects for your scheme.
3. Write a .m file to implement a relay feedback tuning scheme using the Åström and Hägglund
tuning method. Use the pidsim file as a starting point. Pass as parameters, the relay gain
d, and the hysteresis delay h. Return the ultimate period and gain and controller tuning
parameters.

4.8 D RAWBACKS WITH PID CONTROLLERS

The PID controller despite being a simple, intuitive and flexible controller does have some draw-
backs. For example PID control is not particularly suitable for processes where the open loop
4.8. DRAWBACKS WITH PID CONTROLLERS 181

1.5

Impulse response Impulse response Impulse response

0.5
α = 0.39 α = 0.71 α = 1.00

0 5 10 15 0 5 10 15 0 5 10 15

Figure 4.39: The step responses and associated monotonicity index, α, for three representative
systems.

step response is highly oscillatory. A way to quantify the extent of the wobble proposed by [13,
p226] is to computing a monotonicity index
R∞
0 g(t) dt
α = R∞ (4.74)
0 |g(t)| dt

where g(t) is the impulse response of the system. Note that if the step response wobbles, then
the impulse response goes negative. For systems with no wobble, the area under the impulse
response and the area under the absolute value of the impulse response is the same, so α = 1.
For systems that do wobble, the area under the impulse curve will be less due to the periods of
‘negative areas’, so α < 1. The step response and associated monotonicity index for three systems
are compared in Fig. 4.39. PID control is deemed suitable for process responses where α > 0.8.

Further limitations of PID control are the inability to handle responses with large deadtimes, or
inverse responses. These are highlighted when trying to control an inverse response as demon-
strated in §4.8.1, and a simple add-on for deadtime processes is given in §4.9.

4.8.1 I NVERSE RESPONSE PROCESSES

An inverse response is where the response to a step change first heads in one direction, but then
eventually arrives at a steady state in the reverse direction as shown in the middle step response
of Fig. 4.39 or Fig. 4.41(b). Naturally this sort of behaviour is very difficult to control, and if one
tends to overreact, perhaps because the controller’s gain is too high, then excessive oscillation is
likely to result. Inverse responses are common in the control of the liquid water level in a steam
drum boiler as described in [20] due to a phenomena known as the swell-and-shrink effect. For
example the water level first increases when the steam valve is opened because the drum pressure
will drop causing a swelling of the steam bubbles below the surface.

Transfer functions that have an odd number of right-hand-plane zeros have inverse responses.
Inverse response processes belong to a wider group of transfer functions called Non-Minimum
Phase (NMP) transfer functions. Other NMP examples are those processes that have dead time
which are investigated later in §4.9. The term “non-minimum phase” derives from the fact that
for these transfer functions, there will exist another transfer function that has the same amplitude
ratio at all frequencies, but a smaller phase lag.

I like to think of inverse response systems as the sum of two transfer functions: one that goes
downwards a short way quickly, and one that goes up further but much more leisurely. The
182 CHAPTER 4. THE PID CONTROLLER

Figure 4.40: The J curve by Ian Bremmer popularised the

notion that things like private investments, or even the
openness/stability state of nations often get worse before
they get better. ‘J curves’ are inverse responses.

addition of these two gives the inverse response. For example;

K1 K2
G(s) = − (4.75)
τ s+1 τ s+1
| 1 {z } | 2 {z }
G1 G2
(K1 τ2 − K2 τ1 )s + K1 − K2
= (4.76)
(τ1 s + 1)(τ2 s + 1)

If K1 > K2 , then for the system to have right hand plane zeros (numerator polynomial equals
zero in Eqn. 4.76), then
K1 τ2 < K2 τ1 (4.77)
If K1 < K2 , then the opposite of Eqn. 4.77 must hold.

A S IMULINK simulation of a simple inverse response process,

4 3 (−5s + 1)
G(s) = − = (4.78)
|3s{z
+ 1} |s {z
+ 1} (3s + 1)(s + 1)
G1 G2

of the form given in Eqn. 4.75 and clearly showing the right-hand plane zero at s = 1/5 is shown
in Fig. 4.41(a). The simulation results for the two individual transfer functions that combine to
give the overall inverse response are given in Fig. 4.41(b).

C ONTROLLING AN INVERSE RESPONSE PROCESS

Suppose we wish to control the inverse response of Eqn. 4.78 with a PI controller tuned using the
ZN relations based on the ultimate gain and frequency. The following listing adapts the strategy
described in Listing 4.4 for the PID tuning of an arbitrary transfer function.

G= tf(4,[3 1]) - tf(3,[1 1]); % Inverse response plant, Eqn. 4.78.

3 [Ku,Pm,Wcg,Wcp] = margin(G); % Establish critical gain, Ku , and frequency, ωu .

Pu = 2*pi/Wcg; % Critical period, Pu
Kc = 0.45*Ku; taui = Pu/1.2; % PI tuning rules (Ziegler-Nichols)
Gc = tf(Kc*[taui 1],[taui 0]);

8 Gcl = feedback(G*Gc,1); % Closed loop

step(Gcl); % Controlled response as shown in Fig. 4.42.
4.8. DRAWBACKS WITH PID CONTROLLERS 183

(a) A S IMULINK simulation of the inverse response process de-

scribed by Eqn. 4.78

Step Response

2 G
1
Amplitude

G
−1 1
G +G G2
1 2
G
2 G1+G2
−2

−3

0 2 4 6 8 10 12 14 16 18
Time (sec)

(b) The inverse response G made up of G1 + G2 .

Figure 4.41: An inverse response process is comprised of two component curves, G1 + G2 . Note
that the inverse response first decreases to a minimum y = −1 at around t = 1.5 but then increases
to a steady-state value of y∞ = 1.

0.5

Figure 4.42: A NMP plant controlled with a

−0.5
0 20 40 60 80 100 120
PI controller is very sluggish but is stable, and
time [s] does eventually reach the setpoint.
184 CHAPTER 4. THE PID CONTROLLER

The controlled performance in Fig. 4.42 is extremely sluggish taking about 100 seconds to reach
the setpoint while the open loop response as shown in Fig. 4.41(b) takes only about one tenth of
that to reach steady-state. The controlled response also exhibits an inverse response.

The crucial problem with using a PID controller for an inverse response process is that if the
controller is tuned too tight, in other words the gain is too high and the controller too fast acting,
then the controller will look at the early response of the system, which being an inverse system is
going the wrong way, and then try to correct it, thus making things worse. If the controller were
more relaxed about the correction, then it would wait and see the eventual reversal in gain, and
not give the wrong action.

In reality, when engineers are faced with inverse response systems, the gains are deliberately de-
turned giving a sluggish response. While this is the best one can do with a simple PID controller,
far better results are possible with a model based predictive controller such as Dynamic Matrix
Control (DMC). In fact this type of process behaviour is often used as the selling point for these
more advanced schemes. Processes G(s) that have RHP zeros are not open-loop unstable (they
require RHP poles for that), but the inverse process (G(s)−1 ) will be unstable. This means that
some internal model controllers, that attempt to create a controller that approximates the inverse
process, will be unstable, and thus unsuitable in practice.

4.8.2 A PPROXIMATING INVERSE - RESPONSE SYSTEMS WITH ADDITIONAL DEAD -

TIME

We cannot easily remove the inverse response, but we can derive an approximate transfer func-
tion that has no right-hand plane zeros using a Padé approximation. We may wish to do this to
avoid the problem of an unstable inverse in a controller design for example. The procedure is to
replace the non-minimum phase term (γs − 1) in the numerator with (γs−1)(γs+1) (γs + 1), and then
approximate the first term with a deadtime element. In some respects we are replacing one evil
(the right-hand plane zeros), with another, an additional deadtime.

For example the inverse response plant

(2s − 1)
G= e−3s (4.79)
(3s + 1)(s + 1)
has a right-hand zero at s = 1/2 which we wish to approximate as an additional deadtime ele-
ment. Now we can re-write the plant as
(2s + 1) −(2s − 1) −3s
G= · e
(3s + 1)(s + 1) (2s + 1)
| {z }
Pade

and approximate the second factor with a Padé approximation7 in reverse

−(2s + 1)
G≈ e−4s e−3s
(3s + 1)(s + 1)
−(2s + 1)
≈ e−7s (4.80)
(3s + 1)(s + 1)
Note that the original system had an unstable zero and 3 units of deadtime while the approximate
system has no unstable zeros, but whose deadtime has increased to 7.

We can compare the two step responses with:

7 Recall that e−θs ≈ (1 − (2/θ)s)/(1 + (2/θ)s) using a 1-1 Padé approximation.
4.9. DEAD TIME COMPENSATION 185

1 >> G = tf([2 -1],conv([3 1],[1 1]),'iodelay',3); % Original system, Eqn. 4.79.

>> G2 = tf(-[2 1],conv([3 1],[1 1]),'iodelay',7);% No inverse response, Eqn. 4.80.

>> step(G,G2); legend('G','G_{pade}') % Refer Fig. 4.43.

in Fig. 4.43 which, apart from the fact that the approximation version failed to capture the inverse
response, look reasonably similar.

Step Response

0.5
True system G
G
pade

0
Pade approximation
Amplitude

−0.5

−1 Figure 4.43: Approximating inverse-

0 5 10 15 20 25 response systems with additional
Time (sec) deadtime

Problem 4.4 A model of a flexible beam pinned to a motor shaft with a sensor at the free end
repeated in [52, p12] is approximated by the non-minimum phase model,

1 −6.475s2 + 4.0302s + 175.77

Gp (s) = · 3 (4.81)
s 5s + 3.5682s2 + 139.5021s + 0.0929

1. Verify that Gp (s) is non-minimum phase.

2. Plot the root locus diagram of Gp (s) for the cases (a) including the integrator, and (b) ex-
cluding the integrator. Based on the root locus, what can you say about the performance of
stabilising the system with a proportional controller?
3. Simulate the controlled response of this system with an appropriately tuned PID controller.

4.9 D EAD TIME COMPENSATION

Processes that exhibit dead time are difficult to control and unfortunately all too common in
the chemical processing industries. The PID controller, when controlling a process with dead-
time falls prey to the sorts of problems exhibited by the inverse response processes simulated in
§4.8.1, and in practice the gain must be detuned so much that the controlled performance suf-
fers. One successful technique for controlling processes with significant dead time, and that has
subsequently spawned many other types of controllers, is the Smith predictor.

To implement a Smith predictor, we must have a model of the process, which we can artificially
split into two components, one being the estimated model without deadtime, G(s) and one com-
186 CHAPTER 4. THE PID CONTROLLER

ponent being the estimated dead time,

Gp (s) = G(s) e−θs ≈ Ĝ(s) e−θ̂s (4.82)

The fundamental idea of the Smith predictor is that we do not control the actual process,Gp (s),
but we control a model of the process without the deadtime, Ĝ(s). Naturally in real life we
cannot remove the deadtime easily, (otherwise we would of course), but in a computer we can
easily remove the deadtime. Fig. 4.44 shows a block diagram of the deadtime compensator where
D(s) is our standard controller such as say a PID controller. The important feature of the Smith
predictor scheme is that the troublesome pure deadtime component is effectively removed from
the feedback loop. This makes the controller’s job much easier.

Dead time compensator

+
r ✲+ ✲+ ✲+ ✲ D(s) ✲ G(s)e−θs ✲ y
✻− ✻− ✻+
Plant
Ĝ(s)e−θ̂s ✛

Ĝ(s) ✛

Figure 4.44: The Smith predictor structure

Using block algebra on the Smith predictor structure given in Fig. 4.44 we can easily show that
the closed loop transfer function is
Y (s) D(s) G(s)e−θs
= (4.83)
R(s) 1 − D(s)Ĝ(s)e−θ̂s + D(s)G(s)e−θs + Ĝ(s)D(s)

and assuming our model closely apprtoximates the true plant,

D(s)G(s)
= e−θs if G = Ĝ and θ = θ̂ (4.84)
1 + G(s)D(s)
The nice property of the Smith predictor is now apparent in Fig. 4.45 in that the deadtime has
been cancelled from the denominator of Eqn. 4.84, and now sits outside the feedback loop.

+
r ✲+ ✲ D(s) G(s) ✲ e−θs ✲y
✻−

Figure 4.45: The Smith predictor structure from Fig. 4.44 assuming no model/plant mis-match.

For a successful implementation, the following requirements should hold:

4.9. DEAD TIME COMPENSATION 187

1. The plant must be stable in the open loop.

2. There must be little or no model/plant mismatch.
3. The delay or dead time must be ‘realisable’. In a discrete controller this may just be a
shift register, but if the dead time is equivalent to a large number of sample times, or a
non-integral number of sample times, or we plan to use a continuous controller, then the
implementation of the dead time element is complicated.

S MITH PREDICTORS IN S IMULINK

Suppose we are to control a plant with significant deadtime,

2
Gp = e−50s (4.85)
τ 2 s2 + 2τ ζs + 1
where τ = 10, ζ = 0.34. A PID controller with tuning constants K = 0.35, I = 0.008, D = 0.8 is
unlikely to manage very well, so we will investigate the improvement using a Smith predictor.

A S IMULINK implementation of a Smith predictor is given in Fig. 4.46 which closely follows the
block diagram of Fig. 4.44. To turn off the deadtime compensation, simply double click on the
manual switch. This will break the compensation loop leaving just the PID controller.

2
PID
100 s2 +6.8s+1
setpoint PID Controller delayp =50 Scope
process

Sum 3
Manual Switch
delay =50
2
100 s2 +6.8s+1
model

Figure 4.46: A S IMULINK implementation of a Smith predictor. As shown, the deadtime compen-
sator is activated. Refer also to Fig. 4.44.

The configuration given in Fig. 4.46 shows the predictor active, but if we change the constant
block to zero, the switch will toggle removing the compensating Smith predictor, and we are left
with the classical feedback controller. Fig. 4.47 highlights the usefulness of the Smith predictor
showing what happens when we turn on the predictor at t = 1750. Suddenly now the controller
finds it easier to drive the process since it needs not worry about the deadtime. Consequently the
controlled performance improves. We could further improve the tuning when using the Smith
predictor, but then in this case the uncompensated controlled response would be unstable. In
the simulation given in Fig. 4.47, there is no model/plant mis-match, but it is trivial to re-run
the simulation with a different dead time in the model, and test the robustness capabilities of the
Smith Predictor.

Fig. 4.48 extends this application to the control of an actual plant, (but with artificially added
deadtime simply because the plant has little natural deadtime) and a deliberately poor model
(green box in Fig. 4.48(a)). Fig. 4.48(b) demonstrates that turning on the Smith predictor at t = 100
shows a big improvement in the controlled response. However the control input shows a little
too much noise.
188 CHAPTER 4. THE PID CONTROLLER

Smith predictor on at t=2100

2
output & setpoint

−1
Smith on
−2
2

1
Input u

−1
Smith on
−2
0 500 1000 1500 2000 2500 3000 3500 4000
time

Figure 4.47: Dead time compensation. The Smith predictor is turned on at t = 2100 and the
controlled performance subsequently rapidly improves.

4.10 T UNING AND SENSITIVITY OF CONTROL LOOPS

Fig. 4.49 depicts a plant G controlled by a controller C to a reference r in the presence of process
disturbances v and measurement noise w. A good controller must achieve a number of different
aims. It should encourage the plant to follow a given setpoint, it should reject disturbances
introduced by possible upstream process upsets and minimise the effect of measurement noise.
Finally the performance should be robust to reasonable changes in plant dynamics.

The output Y (s) from Fig. 4.49 is dependent on the three inputs, R(s), V (s) and W (s) and using
block diagram simplification, is given by

CG G CG
Y (s) = R(s) + V (s) − W (s) (4.86)
1 + CG 1 + CG 1 + CG
Ideally a successful control loop is one where the error is small, where the error is E = R − Y , or

1 G CG
E(s) = R(s) − V (s) + W (s) (4.87)
1 + CG 1 + CG 1 + CG
We can simplify this expression by defining the open loop transfer function, or sometimes re-
ferred to just as the loop transfer function, as

L(s) = C(s)G(s) (4.88)

which then means Eqn. 4.87 is now

1 G L
E(s) = R(s) − V (s) + W (s) (4.89)
1+L 1+L 1+L

By rearranging the blocks in Fig. 4.49, we can derive the effect of the output on the error, the input
on the setpoint and so on. These are known as sensitivity functions and they play an important
role in controller design.
4.10. TUNING AND SENSITIVITY OF CONTROL LOOPS 189

PID In1 Out1

Signal PID Controller Transport Scope

Generator blackbox Delay

model
Delay model Switch Step
1
s+1

(a) Deadtime compensation using S IMULINK and the real-time toolbox

Smith predictor off

0.5
setpoint & output

−0.5
Smith predictor on

−1
0 20 40 60 80 100 120 140 160 180 200

0.5
input

−0.5

−1
0 20 40 60 80 100 120 140 160 180 200
time (s)

(b) Controlled results without, and with, deadtime compensation.

Figure 4.48: Deadtime compensation applied to the blackbox. The compensator is activated at
time t = 100 after which the controlled performance improves.

The sensitivity function is defined as the transfer function from setpoint to error
1
S(s) = = Ger (s) (4.90)
1 + L(s)
The complementary sensitivity function is defined as the transfer function from reference to out-
put
L(s)
T (s) = = Gyr (s) = Gyw (s) (4.91)
1 + L(s)
The disturbance sensitivity function is
G(s)
GS(s) = = Gyv (s) (4.92)
1 + L(s)
and the control sensitivity function is
C(s)
CS(s) = = Gur (s) = Guw (s) (4.93)
1 + L(s)
190 CHAPTER 4. THE PID CONTROLLER

process disturbance
v

reference error
u ❄ controlled output
✲+ e ✲ ✲ + ✲ G(s) ✲ y
r C(s)
✻
−
controller plant

❄
+✛ w
measurement noise

Figure 4.49: Closed loop with plant G(s) and controller C(s) subjected to disturbances and mea-
surement noise.

The error from Eqn. 4.89 (which we desire to keep as small as practical), now in terms of the the
sensitivity and complementary sensitivity transfer functions is

E(s) = S(s)R(s) − S(s)G(s)V (s) + T (s)W (s) (4.94)

If we are to keep the error small for a given plant G(s), then we need to design a controller C(s)
to keep both S(s) and T (s) small. However there is a problem because a direct consequence of
the definitions in Eqn. 4.90 and Eqn. 4.91 is that

S(s) + T (s) = 1

which means we cannot keep both small simultaneously. However we may be able to make S(s)
small over the frequencies when R(s) is large, and T (s) small when the measurement noise W (s)
dominates. In many practical control loops the servo response R(s) dominates at low frequencies,
and the measurement noise dominates the high frequencies. This is known as loop shaping.

We are primarily interested in the transfer function T (s), since it describes how the output changes
given changes in reference command. At low frequencies we demand that T (s) = 1 which means
that we have no offset at steady-state and that we have an integrator somewhere in the process.
At high frequencies we demand that T (s) → 0 which means that fast changes in command signal
are ignored.

Correspondingly we demand that since S(s) + T (s) = 1, then S(s) should be small at low fre-
quencies, and 1 for high frequencies such as shown in Fig. 4.50.

The maximum values of |S| and |T | are also useful robustness measures. The maximum sensitiv-
ity function
kSk∞ = max |S(jω)| (4.95)
ω

is shown as a ∗ in Fig. 4.50 and is inversely proportional to the minimal distance from the loop
transfer function to the critical (−1, 0i) point. Note how the circle of radius 1/||S||∞ centered at
(−1, 0i) just touches the Nyquist curve shown in Fig. 4.51. A large peak in the sensitivity plot in
Fig. 4.50, corresponds to a small distance between the critical point and the Nyquist curve which
means that the closed loop is sensitive to modelling errors and hence not very robust. Values of
kSk∞ = 1.7 are considered reasonable, [109].
4.11. SUMMARY 191

Bode plot of S & T

1
10
||S||
∞

0
10
Magnitude

−1
10

Sensitivity, S(s)
−2
10
Complementary, T(s)

S
Figure 4.50: Typical sensitiv-
T
10
−3 ity transfer functions S(s) and
10
−2
10
−1
10
0
10
1
T (s). The maximum of the |S|
ω [rad/s] is marked ∗.

Nyquist plot of L(s)

1
circle radius=1/||S||
∞
0.5

−0.5

−1
ℑ{L(i ω)}

−1.5

−2

−2.5

−3

−3.5
Figure 4.51: A circle of radius 1/kSk∞ cen-
−4
−3 −2 −1 0 1 2 3
tered at (−1, 0i) just touches the Nyquist
ℜ{L(i ω)} curve. See also Fig. 4.50.

4.11 S UMMARY

PID controllers are simple and common in process control applications. The three terms of a PID
controller relate to the magnitude of the current error, the history of the error (integral), and the
current direction of the error (derivative). The integral component is necessary to remove offset,
but can destabilise the closed loop. While this may be countered by adding derivative action,
noise and abrupt setpoint changes may cause problems. Commercial PID controllers usually add
a small time constant to the derivative component to make it physically realisable, and often only
differentiate the process output, rather than the error.

Many industrial PID controllers are poorly tuned which is clearly uneconomic. Tuning con-
trollers can be difficult and time consuming and there is no correct fail-safe idiot-proof procedure.
The single experiment in closed loop (Yuwana-Seborg) is the safest way to tune a critical loop,
although the calculations required are slightly more complicated than the traditional methods
and is designed for processes that approximate a first-order system with time delay. The tuning
192 CHAPTER 4. THE PID CONTROLLER

parameters obtained by all these methods should only be used as starting estimates. The fine
tuning of the tuning parameters is best done by an enlightened trial and error procedure on the
actual equipment.
C HAPTER 5

D IGITAL FILTERING AND

SMOOTHING

Never let idle hands get in the way of the devil’s work.
Basil Fawlty (circ. ’75)

5.1 I NTRODUCTION

This chapter is concerned with the design and use of filters. Historically analogue filters were the
types of filters that electrical engineers have been inserting for many years in radios fabricated
out of passive components such as resisters, capacitors and inductors or more recently and reli-
ably out of operational amplifiers. When we filter something, be it home made wine, sump oil,
or undesirable material on the Internet, we are really interested in purifying, or separating the
‘noise’ or unwanted, from the desired.

The classical approach to filtering assumes that the useful signals lie in one frequency band, and
the unwanted noise in another. Then simply by constructing a transfer function as our filter,
we can pass the wanted band (signal), while rejecting the unwanted band (noise) as depicted in
Fig. 5.1.

input, u(t) ✲ Filter, H(s) ✲ output, y(t)

(noisy) (smooth)

| {z }
low-pass filter

Figure 5.1: A filter as a transfer function

193
194 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Unfortunately even if the frequency bands of the signal and noise are totally distinct, we still
cannot effect complete separation in an online filtering operation. In practice however, we can
design the filters with such sharp cut-offs that a practical separation is achieved. More commonly
the signal is not distinct from the noise (since the noise tends to inhabit all frequency bands), so
now the filter designer must start to make some trade-offs.

The more sophisticated model-based filters that require computer implementation are described
in chapter 9. While the classical filters can be realised (manufactured) with passive analogue
components such as resisters, capacitors, and more rarely inductors, today they are almost always
implemented digitally. This chapter describes these methods. The S IGNAL P ROCESSING toolbox
contains many useful filter design tools which we will use in this chapter, although the workhorse
function for this chapter, filter, is part of M ATLAB’s kernel.

5.1.1 T HE NATURE OF INDUSTRIAL NOISE

Real industrial measurements are always corrupted with noise. This noise (or error) can be at-
tributed to many causes, but even if these causes are known, the noise is usually still unpre-
dictable. Causes for this noise can include mechanical vibrations, poor electrical connections
between instrument and transducer sensor, electrical interference of other equipment or a com-
bination of the above. What ever the cause, these errors are usually undesirable. Remember, if it
is not unpredictable, then it is not noise.

An interesting example of the debilitating affect of noise is shown in the controlled response
in Fig. 5.2. In this real control loop, the thermocouple measuring the temperature in a sterilising
furnace was disturbed by a mobile phone a couple of meters away receiving a call. Under normal
conditions the standard deviation of the measurement is around 0.1 degrees which is acceptable,
but with the phone operating, this increases to 2 or 3 degrees.

118
temperature [deg C]

116

114

112
mobile phone range here
110

1
Power [%]

0.5

Figure 5.2: A noisy temperature

measurement due to a nearby mobile 0
phone causes problems in the feed- 0 5 10 15 20 25
back control loop. time [min]

We can mathematically describe the real measurement process as

y = y⋆ + v

where y ⋆ is the true value which is corrupted with some other factor which we call v to give the
“as received” measured value y.
5.1. INTRODUCTION 195

true signal, y ⋆ ✲

❄
measurement noise, v ✲+

❄
corrupted measured signal, y

Figure 5.3: Noise added to a true, but unknown, signal

Naturally we never know precisely either v or y ⋆ . The whole point about filtering or signal
processing is to try and distinguish the “signal”, y ⋆ from the noise, v. There is another reason why
one should study filtering, and that is to disguise horrible experimental data. As one operator
said, “I control the plant by looking at trends of the unfiltered data, but I only give filtered data
to management”.

While the noise itself is unpredictable (non deterministic) it often has constant, known or pre-
dictable statistical properties. The noise is often described by a Gaussian distribution, with
known and stable statistical properties such as mean and variance.

Noise is generally considered a nuisance because

1. It is not a true reflection of the actual process variables, since the transducer, or other exter-
nal factors have introduced unwanted information.

2. It makes it difficult to differentiate the data accurately such as say for the ‘D’ part of a PID
controller, or extracting the flowrate from the rate of change of a level signal.

3. Often for process management applications, only an average value or smoothed character-
istic value is required.

4. Sometimes noise is not undesirable! In some specialised circumstances noise can actually
counter-intuitively improve the strength of a signal. See the article entitled The Benefits of
Background Noise [144] for more details of this paradox.

Some care should be taken with point 1. No one really knows what the true value of the process
is (except possibly Mother Nature), so the assertion that a particular time series is not true is
difficult to rigorously justify. However for most applications, it is a reasonable and defendable
assumption. When filtering the data for process control applications, it is often assumed that the
true value is dominated by low frequency dynamics (large time constants etc), and that anything
at a high frequency is unwanted noise. This is a fair assumption for measurements taken on most
equipment in the processing industries. Large industrial sized vessels, reactors, columns etc
often have large holding volumes, large capacities, and hence slow dynamics. This tendency to
favour the low frequencies by low-pass filtering everything, while common for chemical process
engineers, is certainly not the norm in say telecommunications.

Differentiating (i.e. estimating the slope of) noisy data is difficult. Conversely, integrating noisy
data is easy. Differentiating experimental data is required for the implementation of the ‘D’ part
of PID controllers, and in the analysis of rate data in chemical reaction control amongst other
things. Differentiation of actual industrial data from a batch reaction is demonstrated in §5.5, and
as a motivating example, differentiating actual data without smoothing first is given in §5.1.2.
196 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

5.1.2 D IFFERENTIATING WITHOUT SMOOTHING

As seen in chapter 4, the Proportional-Integral-Differential controller uses the derivative compo-

nent, or D part, to stabilise the control scheme and offers improved control. However in practice,
the D part is rarely used in industrial applications, because of the difficulty in differentiating the
raw measured signal in real-time.

The electromagnetic balance arm shown in Fig. 1.5 is a very oscillatory device, that if left un-
compensated would wobble for a considerable time. If we were to compensate it with a PID
controller such that the balance arm closely follows the desired setpoint, we find that we would
need substantial derivative action to counter the oscillatory poles of the plant. The problem is
how we can reliably generate a derivative of a noisy signal for out PID controller.

Fig. 5.4 demonstrates experimentally what happens when we attempt to differentiate a raw and
noisy measurement, using a crude backward difference with a sample time T

dy yt − yt−1
≈
dt T
The upper plot shows the actual position of the balance arm under real-time closed-loop PID con-
trol using a PC with a 12 bit (4096 discrete levels) analogue input/output card. In this instance,
I tuned the PID controller with a large amount of derivative action, and it is this that dominates
the controller output (lower plot). While the controlled response is reasonable, (the arm position
follows the desired setpoint, the input is too noisy), and will eventually wear the actuators if used
for any length of time. Note that this noise is a substantial fraction of the full scale input range.

2500
setpoint & output

2000

1500

1000
5000

4000

3000
Input

2000

1000

0
0 5 10 15 20 25 30
time [s]

Figure 5.4: The significant derivative action of a PID controller given the slightly noisy data from
an electromagnetic balance arm (solid line in upper plot) causes the controller input signal (lower
plot) to go exhibit very high frequencies.

One solution to minimise the excitedness of the noisy input signal whilst still retaining the dif-
ferentiating characteristic of the controller is to low-pass filter (smooth) the measured variable
before we differentiate it. This is accomplished with a filter, and the design of these filters is what
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 197

this chapter is all about. Alternatively the controller could be completely redesigned to avoid the
derivative step.

5.2 S MOOTHING MEASURED DATA USING ANALOGUE FILTERS

If we have all the data at our disposal, we can draw a smooth curve through the data. In the past,
draughtsmen may have used a flexible ruler, or spline, to construct a smoothing curve “by eye”.
Techniques such as least squares splines or other classes of piecewise low-order polynomials
could be used if a more mathematically rigorous fitting is required. Techniques such as these fall
into the realm of regression and examples using M ATLAB functions such as least-squares fitting,
splines, and smoothing splines are covered more in [205]. If the noise is to be discarded and
information about the signal in the future is used, this technique is called smoothing. Naturally
this technique can only be applied offline after all the data is collected since the “future” data
must be known. A smoothing method using the Fourier transform is discussed in §5.4.5.

Unfortunately for real time applications such as the PID controller application in §5.1.2, we can-
not look into the future and establish what the signal will be for certain. In this case we are
restricted to smoothing the data using only output historical values. This type of noise rejection
is called real-time filtering and it is this aspect of filtering that is of most importance to control
engineers. Common analogue filters are discussed in §5.2.3, but to implement using digital hard-
ware we would rather use an equivalent digital description. The conversion from the classical
analogue filter to the equivalent digital filter using the bilinear transform is described in §5.3.

Finally we may wish to predict data in the future. This is called prediction.

5.2.1 A SMOOTHING APPLICATION TO FIND THE PEAKS AND TROUGHS

Section 4.6.3 described one way to tune a PID controller where we first to subject the plant to a
closed loop test with a trial gain, and then record the peaks and troughs of the resultant curve.
However as shown in the actual industrial example in Fig. 5.5(a), due to excessive noise on the
thermocouple (introduced by mobile phones in the near vicinity!), it is difficult to extract the
peaks and troughs from the measured data. It is especially difficult to write a robust algorithm
that will do this automatically with such noise.

The problem is that to achieve a reasonable smoothing we also introduce a large lag. This in turn
probably does not affect the magnitude of the peaks and troughs, but it does affect the timing, so
our estimates of the model parameters will be wrong.

One solution is to use an acausal filter which does not exhibit any phase lag. M ATLAB provides
a double-pass filter called filtfilt which is demonstrated in Fig. 5.5(b). Using the smooth
data from this acausal filter generates reasonable estimates for both the magnitude and timing of
the characteristic points needed for the tuning. The drawback is that the filtering must be done
off-line.

5.2.2 F ILTER TYPES

There are four common filter types: the low-pass, high-pass, band-pass and notch-pass. Each of
these implementations can be derived from the low-pass filter, meaning that it is only necessary
198 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Output & setpoint 4

2
raw value
0
Butterworth filter smoothed, n=5, ω =0.03
c
Key points
−2
0 100 200 300 400 500 600 700 800
time [s]

(a) To extract the key points needed for PID tuning, we must first
smooth the noisy data. Unfortunately, to achieve good smoothing
using a causal filter such as the 5th order Butterworth used here,
we introduce a large lag.

6
Output & setpoint

raw value
0
Acausal smoothed
Key points
−2
0 100 200 300 400 500 600 700 800
time [s]

(b) Using an acausal filter removes the phase lag problem

Figure 5.5: In this industrial temperature control of a furnace, we need to smooth the raw mea-
surement data in order to extract characteristic points required for PID tuning.

to study in detail the design of the first. A comparison of the magnitude response of these filters
is given in Fig. 5.8.

L OW- PASS FILTERS

A linear filter is mathematically equivalent to a transfer function, just like a piece of processing
equipment such as a series of mixing tanks, or distillations columns. Most of the filter applica-
tions used in process industry are low-pass in nature. That is, ideally the filter passes all the low
frequencies, and above some cut off frequency ωc , they attenuate the signal completely. These
types of filters are used to smooth out noisy data, or to retain the long term trends, rather than
the short term noisy contaminated transients.

The ideal low pass filter is a transfer function that has zero phase lag at any input frequency, and
an infinitely sharp amplitude cut off above a specified frequency ωc . The Bode diagram of such
a filter would have a step function down to zero on the amplitude ratio plot and a horizontal
line on the phase angle plot as shown in Fig. 5.8. Clearly it is physically impossible to build
such a filter. However, there are many alternatives for realisable filters that approximate this
behaviour. Fig. 5.6 shows a reasonable attempt to approximate the ideal low-pass filter. Typically
the customer will specify a cut-off frequency, and it is the task of the filter designer to select
sensible values for the pass and stop band oscillations or ripple and the pass and stop band
frequencies to achieve these balancing filter performance with minimal hardware cost.
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 199

|H(iω)| ✻

1 
pass band ripple
1 ..
1+ǫ2 ..
..
..
..
..
..
pass band .. stop band
.
✛ ✲... ✛ ✲
..
..
..
..
..
..
..
..
..
..
1 ..
A2 ..
.
0 ✲ ω
0 ωp ωs
corner frequency

Figure 5.6: Low-pass filter specification

The most trivial low-pass filter approximation to the ideal filter is a single lag with a cut off
frequency (ωc ) of 1/τ . If you wanted a higher order filter, say a nth order filter, you could simply
cascade n of these filters together as shown in Fig. 5.7.
1
Gn (s) =  n (5.1)
s
ωc +1

cascaded 3rd order low-pass filter

z }| {

input, u ✲ 1 ✲ 1 ✲ 1 ✲ output, y
s/ωc +1 s/ωc +1 s/ωc +1

| {z }
single low-pass filter

Figure 5.7: Three single low-pass filters cascaded together to make a third-order filter.

Actually Eqn 5.1 is a rather poor approximation to the ideal filter and better approximations exist
for the same complexity or filter order, although they are slightly more difficult to design and
200 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

implement in hardware. Two classic analogue filter designs are described in §5.2.3 following.

H IGH - PASS FILTERS

There are however applications where high pass filters are desired, although these are far more
common in the signal processing and electronic engineering fields. These filters do the opposite
of the low pass filter; namely they attenuate the low frequencies and pass the high frequencies
unchanged. This can be likened to the old saying “If I’ve already seen it, I’m not interested!”
These remove the DC component of the signal and tend to produce the derivative of the data.
You may use a high pass filter if you wanted to remove long term trends from your data, such as
seasonal effects for example.

B AND AND NOTCH - PASS FILTERS

The third type of filter is a combination of the high and low pass filters. If these two are combined,
they can form a band pass or alternatively a notch filter. Notch filters are used to remove a par-
ticular frequency such as the common 50Hz hum caused by the mains power supply. Band pass
filters are used in the opposite manner, and are used in transistor radio sets so that the listener can
tune into one particular station without hearing all the other stations. The difference between the
expensive radios and the cheap ones, is that the expensive radios have a sharp narrow pass band
so as to reduce the amount of extra frequency. Oddly enough for some tuning situations a notch
filter is also used to remove neighbouring disturbing signals. Some satellite TV stations intro-
duce into the TV signal a disturbing component to prevent non-subscribers from watching their
transmission without paying for the “secret decoder ring”. This disturbance, which essentially
makes the TV unwatchable, must of course must be within the TV station’s allowed frequency
range. The outer band pass filter will tune the TV to the station’s signal, and a subsequent notch
filter will remove the disturbing signal. Fig. 5.8 contrasts the amplitude response of these filters.

Ideal filter Low-pass High-pass Band-pass

AR AR AR AR
✻ ✻ ✻ ✻

✲ ✲ ✲ ✲ω, [rad/s]
ωc ωc ωc ωl ωh

Figure 5.8: Amplitude response for ideal, low-pass, high pass and band-pass filters.

F ILTER TRANSFORMATION

Whatever the type of filter we desire, (high-pass, band-pass notch etc), they can all be obtained
by first designing a normalised low-pass filter prototype as described by [161, p415]. This filter
is termed normalised because it has a cutoff frequency of 1 rad/s. Then the desired filter, high-
pass, band-pass or whatever, is obtained by transforming the normalised low-pass filter using the
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 201

relations given in Table 5.1 where ωu is the upper cutoff and ωl is the lower cutoff frequencies.
This is very convenient, since we need only to be able to design a low-pass filter from which all

Table 5.1: The filter transformations needed to convert from the prototype filter to other general
filters. To design a general filter, replace the s in the prototype filter with one of the following
transformed expressions.

Desired filter transformation

low pass s → s/ωu
s2 + ω u ω l
band-pass s→
s(ωu − ωl )
s(ωu − ωl )
band-stop s→ 2
s + ωu ωl
high pass s → ωu /s

the others can be obtained using a variable transformation. Low-pass filter design is described
for the common classical analogue filter families in §5.2.3. M ATLAB also uses this approach in the
S IGNAL P ROCESSING toolbox with what it terms analogue low-pass proto-type filters.

5.2.3 C LASSICAL ANALOGUE FILTER FAMILIES

Two common families of analogue filters superior to the simple cascaded first-order filter net-
work are the Butterworth and the Chebyshev filters. Both these filters require slightly more com-
putation to design, but once implemented, require no more computation time or hardware com-
ponents than any other filter of the same order. Historically these were important continuous-
time filters, and to use discrete filter designs we must convert the description into the z plane by
using the bilinear transform.

In addition to the Butterworth and Chebyshev filters, there are two slightly less common classic
analogue filters; the Chebyshev type II (sometimes called the inverse Chebyshev filter), and the
elliptic or Cauer filter. All four filters are derived by approximating the sharp ideal filter in
different ways. For the Butterworth, the filter is constructed in such a way as to maximise the
number of derivatives to zero at ω = 0 and ω = ∞. The Chebyshev filter exhibits minimum error
over the passband, while the type II version minimises the error over the stop band. Finally the
elliptic filter uses a Chebyshev mini-max approximation in both pass-band and stop-band. The
calculation of the elliptic filter requires substantially more computation than the other three. The
S IGNAL P ROCESSING toolbox contains both discrete and continuous design functions for all four
analogue filter families.

B UTTERWORTH FILTERS

The Butterworth filter, HB , is characterised by the following magnitude relation

1
|HB (ω)|2 =  2n (5.2)
ω
1+ ωc

where n is the filter order and ωc is the desired cut off frequency. If we substitute s = iω, we find
that the 2n poles of the squared magnitude function, Eqn. 5.2, are
1
sp = (−1) 2n (iωc )
202 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

That is, they are spaced equally at π/n radians apart around a circle of radius ωc . Given that
we desire a stable filter, we choose just the n stable poles of the squared magnitude function for
HB (s). The transfer function form of the general Butterworth filter in factored form is
n
Y 1
HB (s) = (5.3)
s + pk
k=1

where   
1 2k − 1
pk = ωc exp −πi + (5.4)
2 2n
Thus all the poles are equally spaced apart on the stable half-circle of radius ωc in the s plane.
The M ATLAB function buttap, (Butterworth analogue low pass filter prototype), uses Eqn. 5.4
to generate the poles.

We can plot the poles of a 5th order Butterworth filter after converting the zero-pole-gain form
to a transfer function form, using the pole-zero map function, pzmap, or of course just plot the
poles directly.

1 [z,p,k] = buttap(5) % Design a 5th order Butterworth proto-type filter

pzmap(zpk(z,p,k)) % Look at the poles & zeros

Note how the poles of the Butterworth filter are equally spread around the stable half circle in
Fig. 5.9. Compare this circle with the ellipse pole-zero map for the Chebyshev filter shown in
Fig. 5.10.

0.5
ℑ(p)

−0.5

Figure 5.9: Note how the poles of the Butterworth filter, −1

×, are equally spaced on the stable half circle. See also
−1 −0.5 0 0.5 1
Fig. 5.10. ℜ(p)

Alternatively using only real arithmetic, the n poles of the Butterworth filter are pk = ak ± bk i
where
ak = −ωc sin θ, bk = −ωc cos θ (5.5)
where
π
θ= (2k − 1) (5.6)
2n
Expanding the polynomial given in Eqn. 5.3 gives the general Butterworth filter template in the
continuous time domain,
1
HB (s) =  n  n−1 (5.7)
s s
cn ωc + cn−1 ωc + · · · + c1 ωsc + 1

where ci are the coefficients of the filter polynomial. Using Eqn 5.4, the coefficients for the general
nth order Butterworth filter can be evaluated.
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 203

Taking advantage of the complex arithmetic abilities of M ATLAB, we can generate the poles typ-
ing Eqn. 5.4 almost verbatim. Note how I use 1i, (that is I type the numeral ‘1’ followed with an
‘i’ with no intervening space) to ensure I get a complex number.

Listing 5.1: Designing Butterworth Filters using Eqn. 5.4.

>>n = 4; % Filter order
>>wc = 1.0; % Cut-off frequency, ωc
3 >>k = [1:n]';
1 2k−1


>>p = wc*exp(-pi*1i*(0.5 + (2*k-1)/2/n)); % Poles, pk = ωc exp −πi 2
+ 2n
>>c = poly(p); % should force real
>>c = real(c) % look at the coefficients
c =
8 1.0000 2.6131 3.4142 2.6131 1.0000

>>[B,A] = butter(n,wc,'s'); % compare with toolbox direct method

Once we have the poles, it is a simple matter to expand the polynomial out and find the coeffi-
cients. In this instance, owing to numerical roundoff, we are left with a small complex residual
which we can safely delete. The 4th order Butterworth filter with a normalised cut-off frequency
is
1
HB (s) = 4
s + 2.6131s3 + 3.4142s2 + 2.6131s + 1
Of course using the S IGNAL P ROCESSING toolbox, we could duplicate the above with a single
call to butter given as the last line in the above example.

We can design a low-pass filter at an arbitrary cut-off frequency by scaling the normalised ana-
logue proto-type filter using Table 5.1. Listing 5.2 gives the example of the design of a low-pass
filter with a cutoff of fc = 800 Hz.

Listing 5.2: Designing a low-pass Butterworth filter with a cut-off frequency of fc = 800 Hz.
ord = 2; % Desired filter order
[Z,P,K] = buttap(ord); Gc = tf(zpk(Z,P,K)); % 2nd-order Butterworth filter prototype

Fc = 800; % Desired cut-off frequency in [Hz]

5 wc = Fc*pi*2 ; % [rad/s]

n = [ord:-1:0]; % Scale filter by substituting s for s/ωc

d = wc.ˆn;
Gc.den{:} = Gc.den{:}./d
10

[B,A] = butter(2,wc,'s'); Gc2 = tf(B,A); % Alternative continuous design (in Matlab)

p=bodeoptions; p.FreqUnits = 'Hz'; % Set Bode frequency axis units as Hz (not rad/s)
bode(Gc,Gc2,p)
15 hline(-90); vline(Fc)

A high pass filter can be designed in much the same way, but this time substituting ωc /s for s.

Listing 5.3: Designing a high-pass Butterworth filter with a cut-off frequency of fc = 800 Hz.
ord = 2; [Z,P,K] = buttap(ord); Gc = tf(zpk(Z,P,K)); % Low-pass Butterworth proto-
type
204 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Fc = 800;wc = Fc*pi*2 ; % Frequency specifications

5 n = [0:ord]; d = wc.ˆn; % Scale filter by converting s to ωc /s

Gc.den{:} = Gc.den{:}.*d;
Gc.num{:} = [1,zeros(1,ord)];

bode(Gc,p); title('High pass')

The frequency response of the Butterworth filter has a reasonably flat pass-band, and then falls
away monotonically in the stop band. The high frequency asymptote has a slope of −n on a
log-log scale. Other filter types such as the Chebyshev (see following section) or elliptic filters
are contained in the S IGNAL P ROCESSING toolbox, and [42] details the use of them.

Algorithm 5.1 Butterworth filter design

Given a cut-off frequency, ωc , (in rad/s), and desired filter order, n,

1. Compute the n poles of the filter, HB (s), using

  
1 2k − 1
pk = ωc exp −πi + , k = 1, 2, . . . , n
2 2n

2. Construct the filter HB (s) either using the poles directly in factored form, or expanding the
polynomial
1
HB (s) =
poly(pk )

3. Convert low pass filter to high-pass, band pass etc by substituting for s given in Table 5.1.
4. Convert to a discrete filter using c2dm if desired ensuring that the cut-off frequency is much
less than the Nyquist frequency; ωc << ωN = π/T .

Algorithms 5.1 suffers from the flaw that it assumes the filter designer has already selected a
cut-off frequency, ωc , and filter order, n. However for practical cases, it is characteristics such
as band-pass ripple, stopband attenuation that are specified by the customer such as given in
Fig. 5.6, and not the filter order. We need some way to calculate the minimum filter order (to
construct the cheapest filter) from the specified magnitude response characteristics. Once that is
established, we can proceed with the filter design calculations using the above algorithm. Further
descriptions of the approximate order are given in [93, pp9305–327] and [197]. The functionally
equivalent routines, buttord and cheb1ord in the S IGNAL P ROCESSING toolbox attempt to
find a suitable order for a set of specified design constraints.

In the case for Butterworth filters, the filter order is given by

& 

 '
log10 10Rp /10 − 1 10As /10 − 1
n= (5.8)
2 log10 (ωp /ωs )

where Rp is the pass-band ripple, As is the stopband ripple, and ωp , ωs are the pass and stop
band frequencies respectively. In general, the order n given by the value inside the ⌈·⌉ brackets
(without feet) in Eqn. 5.8 will not be an exact integer, so we should select the next largest integer
to ensure that we meet or exceed the customer’s specifications. M ATLAB’s ceiling function ceil
is useful here.

Problem 5.1 1. Derive the coefficients for a sixth order Butterworth filter.
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 205

2. Draw a Bode diagram for a fourth order Butterworth filter with wc = 1 rad/s. On the same
plot, draw the Bode diagram for the filter

1
H(s) =  4
s
ωc +1

with the same wc . Which filter would you choose for a low pass filtering application?
Hints: Look at problem A-4-5 (DCS) and use the M ATLAB function bode. Plot the Bode
diagrams especially carefully around the corner frequency. You may wish to use the Matlab
commands freqs or freqz.

3. Find and plot the poles of a continuous time 5th order Butterworth filter. Also plot a circle
centered on the origin with a radius wc .

4. Plot the frequency response (Bode diagram) of both a high pass and low pass 4th order
Butterworth filter with a cutoff frequency of 2 rad/s.

C HEBYSHEV FILTERS

Related to the Butterworth filter family is the Chebyshev family. The squared magnitude function
of the Chebyshev filter is
1
|HC (ω)|2 =   (5.9)
1 + ǫ Cn2 ωωc
2

where again n is the filter order, ωc is the nominal cut-off frequency, and Cn is the nth order
Chebyshev polynomial. (For a definition of Chebyshev polynomials, see for example [205], and in
particular the cheb_pol function.) The design parameter ǫ is related to the amount of allowable
passband ripple, δ,
1
δ =1− √ (5.10)
1 + ǫ2
Alternatively the filter design specifications may be given in decibels of allowable passband rip-
ple1 , rdB ,
rdB
δ = 1 − 10− 20 (5.11)

This gives the Chebyshev filter designer two degrees of freedom, (filter order n and ǫ or equiva-
lent), rather than just order as in the Butterworth case. Similar to the Butterworth filter, the poles
of the squared magnitude function, Eqn. 5.9, are located equally spaced along an ellipse in the s
plane with minor axis radius of αωc aligned along the real axis, where

1  1/n 
α= γ − γ −1/n (5.12)
2
with
1 p
γ= + 1 + ǫ−2 (5.13)
ǫ
and a major axis radius of βωc aligned along t2he imaginary axis where

1  1/n 
β= γ + γ −1/n (5.14)
2
1 The functions in the S IGNAL P ROCESSING T OOLBOX use rdB rather than δ.
206 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

Since the poles pk of the Chebyshev filter, Hc (s) are equally spaced along the stable half of the
ellipse, (refer Fig. 5.10 for a verification of this), the real and imaginary parts are given by:
  
ℜ(pk ) = αωc cos π2 + (2k+1)π 2n

  k = 0, 1, . . . , n − 1 (5.15)
ℑ(pk ) = βωc sin π2 + (2k+1)π 2n


and the corresponding transfer function is given by

K
Hc (s) = Qn
k=0 (s − pk )
√
where the numerator K is chosen to make the steady-state equal to 1 if n is odd, or 1/ 1 + ǫ2 for
even n. Algorithm 5.2 summarises the design of a Chebyshev filter with ripple in the passband.

Algorithm 5.2 Chebyshev type I filter design

Given a cut-off frequency, ωc , passband ripple, δ, or rdB , and order filter n,

1. Calculate the ripple factor

s
p 1
ǫ = 10r/10 − 1 = −1
(1 − δ)2

2. Calculate the radius of the minor, α, and major β axis of the ellipse using Eqns 5.12–5.14.
3. Calculate the n stable poles from Eqn. 5.15 and expand out to form the denominator of
Hc (s).
4. Choose the normalising gain K such that at steady-state (DC gain)

√ 1 , n even
Hc (s = 0) = 1+ǫ2
1, n odd

We can calculate the gain of a transfer function using the final value theorem which assum-
ing a unit step gives
 a0
√
1+ǫ2
, n even
K=
a0 , n odd
where a0 is the coefficient of s0 in the denominator of Hc (s).

We can test this algorithm by designing a Chebyshev type I filter with 3dB ripple in the passband,
(rdB = 3dB or δ = 1 − 10−3/20 ), and a cross-over frequency of ωc = 2 rad/s.

Listing 5.4: Designing Chebyshev Filters

1 n = 4; wc = 2; √ n, ωc
% Order & cut-off frequency,
r = 3; % Ripple. Note −3dB = 1/ 2
rn = 1/n;
e = sqrt(10ˆ(r/10)-1); % δ = 1 − 10−r/20 , e = 1/(1 − δ)2 − 1
p

6 g = 1/e + sqrt(1 + 1/e/e); minora = (gˆrn - gˆ(-rn))/2;

majorb = (gˆrn + gˆ(-rn))/2;

k = [0:n-1]'; n2 = 2*n;
5.2. SMOOTHING MEASURED DATA USING ANALOGUE FILTERS 207

realp = minora*wc*cos(pi/2 + pi*(2*k+1)/n2); % real parts of the poles

11 imagp = majorb*wc*sin(pi/2 + pi*(2*k+1)/n2); % imaginary parts
polesc = realp + 1i*imagp; % poles of the filter
Ac = real(poly(polesc)); % force real
Bc = Ac(n+1); % overall DC gain = 1 (n =odd)
if ¬rem(n,2) % if filter order n is even
16 Bc = Bc/sqrt(1+eˆ2); % dc gain
end % if

You may like to compare this implementation with the equivalent S IGNAL P ROCESSING toolbox
version of the type 1 Chebyshev analogue prototype by typing type cheb1ap.

We can verify that the computed poles do indeed lie on the stable half of the ellipse with major
and minor axes αω and βω respectively in Fig. 5.10. I used the axis(’equal’) command for
this figure forcing the x and y axes to be equally, as opposed to conveniently, spaced to highlight
the ellipse. Compare this with the poles of a Butterworth filter plotted in Fig. 5.9.

1.5

0.5
ℑ(p)

−0.5

−1

−1.5

−2 Figure 5.10: Verifying that the poles of a fourth-order

−2 −1 0 1 2
analogue type 1 Chebyshev filter are equally spaced
ℜ(p) around the stable half of an ellipse.

As the above relations indicate, the generation of Chebyshev filter coefficients is slightly more
complex than for Butterworth filters, although they are also part of the S IGNAL P ROCESSING
toolbox. Further design relations are given in [154, p221]. We can compare our algorithm with
the equivalent toolbox function cheby1 which also designs type I Chebyshev filters. We want
the continuous description, hence the optional ‘s’ parameter. Alternatively we could use the
more specialised cheb1ap function which designs an analogue low-pass filter prototype which
is in fact called by cheby1.

Listing 5.5: Computing a Chebyshev Filter

n = 4; wc = 2;
r = 3;
3 [Bc,Ac] = cheby1(n,r,wc,'s');
[Hc,w] = freqs(Bc,Ac);
semilogx(w,abs(Hc));
√ cut-off, ωc = 2 rad/s
% Order of Chebyshev filter;
% dB of ripple, −3dB = 1/ 2
% design a continuous filter
% calculate frequency response.
% magnitude characteristics

These filter coefficients should be identical to those obtained following Algorithm 5.2. You should
exercise some care with attempting to design extremely high order IIR filters. While the algorithm
is reasonably stable from a numerical point of view, if we restrict ourselves just to the factored
form, one may see unstable poles once we have expanded the polynomial owing to numerical
round-off for orders larger than about 40.
208 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

>> [Bc,Ac] = cheby1(50,3,1,'s'); % Very high order IIR filter

>> max(real(roots(Ac)))
ans =
0.0162 % +ve, unstable RHP pole
5 >> [z,p,g] = cheb1ap(50,3); % repeat, but only calculate poles
>> max(real(p)) % are any RHP?
ans =
-5.5478e-004 % All poles stable

Of course in applications while it is common to require high order for FIR filters, it is unusual
that we would need larger than about n = 10 for IIR filters.

Fig. 5.11 compares the magnitude of both the Butterworth and Chebyshev filter and was gener-
ated using, in part, the above code. For this plot, I have broken from convention, and plotted the
magnitude on a linear rather than logarithmic scale to emphasize better the asymptote towards
zero. Note that for even order Chebyshev filters such as the 4th order presented in Fig. 5.11,
the steady-state gain is not equal to 1, but rather 1− the allowable pass-band ripple. If this is a
problem, one can compensate by increasing the filter gain. For odd order Chebyshev filters, the
steady-state gain is 1 which perhaps is preferable.

Comparison of Butterworth & Chebyshev filters

0.8
−3dB
|H(ω)|

0.6
Butterworth
Chebychev
Figure 5.11: Comparing the magni-
0.4
tude characteristics of 4th order But-
terworth and Chebyshev filters. The
0.2
Chebyshev filter is designed to have
ideal filter
a ripple equal to −3dB (dotted line)
which is approx 30%. Both filter cut- 0
10
−1
10
0 ωc 1
10
offs are at ωc = 2 rad/s. ω [rad/s]

The difference between a maximally flat passband filter such as the Butterworth and the equal
ripple filter such as the Chebyshev is clear. [133, p176] claim that most designers prefer the
Chebyshev filter owing to the sharper transition across the stopband, provided the inevitable
ripple is acceptable.

In addition to the Butterworth and Chebyshev filters, there are two more classic analogue filters;
the Chebyshev type II and the elliptic or Cauer filter. All four filters are derived by approximating
the sharp ideal filter in different ways. For the Butterworth, the filter is constructed in such a way
as to maximise the number of derivatives to zero at ω = 0 and ω = ∞. The Chebyshev filter
exhibits minimum error over the passband, while the type II version minimises the error over
the stop band. Finally the elliptic filter uses a Chebyshev mini-max approximation in both pass-
band and stop-band. The calculation of the elliptic filter requires substantially more computation
than the other three, although all four can be computed using the S IGNAL P ROCESSING toolbox.
5.3. DISCRETE FILTERS 209

5.3 D ISCRETE FILTERS

Analogue filters are fine for small applications and for where aliasing may be a problem, but
they are more expensive and more inflexible than the discrete equivalent. We would much rather
fabricate a digital filter where the only difference between filter orders is a slightly longer tapped
delay or shift register, and proportionally slower computation, albeit with a longer word length.
Discrete filters are also easy to implement in M ATLAB using the filter command.

To design a digital filter, one option is to to start the design from the classical analogue filters
and use a transformation such as the bilinear transform described in chapter 2 to the discrete
domain. In fact this is the approach that M ATLAB uses in that it first designs the filter in the
continuous domain, then employs bilinear to convert to the discrete domain. Digital filter
design is discussed further in section 5.3.2.

Traditionally workers in the digital signal processing (DSP) area have referred to two types of fil-
ters, one where the response to an impulse is finite, the other where the response is theoretically
of infinite duration. This latter recursive filter or Infinite Impulse Response (IIR), is one where
previous filtered values are used to calculate the current filtered value. The former non-recursive
filter or Finite Impulse Response, (FIR) is one where the filtered value is only dependent on past
values of the input hence A(z) = 1. FIR filters find use in some continuous signal processing
applications where the operation is to be run for very long sustained periods where the alterna-
tive IIR filters are sensitive to numerical ‘blow-up’ owing to the recursive nature of the filtering
algorithm. FIR filters will never go unstable since they have only zeros which cannot cause in-
stability by themselves. FIR filters also exhibit what is known as linear phase lag, that is each
of the signal’s frequencies are delayed by the same amount of time. This is a desirable attribute,
especially in communications and one that a causal IIR filter cannot match.

However better filter performance is generally obtained by using IIR filters for the same number
of discrete components. In applications where the FIR filter is necessary, 50 to 150 terms are com-
mon. To convert approximately from an IIR to a FIR, divide A(z) into B(z) using long division,
and truncate the series at some suitably large value. Normally, the filter would be implemented
in a computer in the digital form as a difference equation, or perhaps in hardware using a series
of delay elements and shift registers.

5.3.1 A LOW- PASS FILTERING APPLICATION

Consider the situation where you are asked to filter a measurement signal from a sensitive pres-
sure transducer in a level leg in a small tank. The level in the tank is oscillating about twice a
second, but this measurement is corrupted by the mains electrical frequency of 50Hz. Suppose
we are sampling at 1 kHz (T = 0.001s), and one notes that this sampling rate is sufficient to avoid
aliasing problems since it is above twice the highest frequency of interest, (2 × 50). We can sim-
ulate this in M ATLAB. First we create a time vector t and then the two signals, one is the true
measurement y1 , and the other is the corrupting noise from the mains y2 . The actual signal, as
received by your data logger, is the sum of these two signals, y = y1 + y2 . The purpose of filtering
is to extract the “true” signal y1 from the corrupted y as closely as possible.
210 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

1.5
t = [0:0.001:1]'; % time vector 1
2 y1 = sin(2*pi*2*t); % true signal
0.5
y2 = 0.1*sin(2*pi*50*t); % noise
y = y1+y2; % what we read 0

plot(t,y) −0.5

−1

−1.5
0 0.2 0.4 0.6 0.8 1

Clearly even from just the time domain plot the signal y is composed of two dominant frequen-
cies. If a frequency domain plot were to be created, it would be clearer still. To reconstruct the
true level reading, we need to filter the measured signal y. We could design a filter with a cut-
off frequency of fc = 30Hz. This should in theory pass the slower y1 signal, but attenuate the
corrupting y2 .

To implement such a digital filter we must:

1. select a filter type, say Butterworth, (refer page 201)

2. select a filter order, say 3, and

3. calculate the dimensionless frequency related to the specified frequency cutoff fc = 30Hz,
and the sample time (or Nyquist frequency fN ),

fc
Ω= = 2T fc
fN
= 2 × 10−3 × 30 = 0.06

We are now ready to use the S IGNAL P ROCESSING T OOLBOX to design our discrete IIR filter or
perhaps build our own starting with a continuous filter and transforming to the discrete domain
as described on page 203.

>> wc = 2*1.0e-3*30; % Dimensionless cut-off frequency Ω

>> [B,A] = butter(3,wc) % Find A(q −1 ) & B(q −1 ) polynomials
B =
0.0007 0.0021 0.0021 0.0007
10 A =
1.0000 -2.6236 2.3147 -0.6855
>> yf = filter(B,A,y); % filter the measured data
>> plot(t,y,t,yf,'--',t,y1) % any better ?

Once having established the discrete filter coefficients, we can proceed to filter our raw signal
and compare with the unfiltered data as shown in Fig. 5.12.

The filtered data Fig. 5.12 looks smoother than the original, but it really is not ‘smooth’ enough.
We should try a higher order filter, say a 6th order filter.

[B6,A6] = butter(6,wc); % 6th order, same cut-off

15 yf6 = filter(B6,A6,y); % smoother ?
plot(t,[y,yf,yf6]) % looks any better ?
5.3. DISCRETE FILTERS 211

1.4

1.2

1 raw data

0.8

0.6
6th order

0.4
3rd order
Raw data
0.2
3rd or Butterworth
Figure 5.12: Comparing a 3rd and 6th order
6th order Butterworth discrete Butterworth filters to smooth noisy
0 data.
0 0.05 0.1 0.15 0.2

This time the 6th order filtered data in Fig. 5.12 is acceptably smooth, but there is quite a notice-
able phase lag such that the filtered signal is behind the original data. This is unfortunate, but
shows that there must be a trade off between smoothness and phase lag for all causal or online
filters.

The difference in performance between the two digital filters can be shown graphically by plot-
ting the frequency response of the filter. The easiest way to do this is to convert the filter to a
transfer function and then use bode.

Ts = t(2)-t(1); % Sampling time, T , for the correct frequency scale

H3 = tf(B,A,Ts);
H6 = tf(B6,A6,Ts)
20 bode(H3,H6); % See Frequency response plot in Fig. 5.13.

Alternatively we could use freqz and construct the frequency response manually.

[h,w] = freqz(B,A,200); % discrete frequency response

[h6,w6] = freqz(B6,A6,200);
w = w/pi*500; % convert from rad/s to Hz
w6 = w6/pi*500;
25 loglog(w,abs(h), w6,abs(h6)) % See Magnitude plot in Fig. 5.13.

The sixth order filter in Fig 5.13 has a much steeper roll off after 30Hz, and this more approximates
the ideal filter. Given only this curve, we would select the higher order (6th) filter. To plot the
second half of the Bode diagram, we must ‘unwrap’ the phase angle, and for convenience we will
also convert the phase angle from radians to degrees.

ph = unwrap(angle(h))*360/2/pi;
ph6 = unwrap(angle(h6))*360/2/pi;
semilogx(w,ph,w6,ph6) % plot the 2nd half of the Bode diagram

The phase lag at high frequencies for the third order filter is 3 × 90◦ = 270◦ , while the phase lag
for the sixth order filter is 6 × 90◦ = 540◦ . The ideal filter has zero phase lag at all frequencies.
212 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

0 3rd order
10 f f
c N

Amplitude ratio
−5
10

3rd order
6th order
6th order
−10
10
0 1 2 3
10 10 10 10

Phase angle φ
−200

Figure 5.13: The frequency response for a −400

3rd and 6th order discrete Butterworth fil-
fc fN
ter. The cut-off frequency for the filters is −600
fc = 30Hz and the Nyquist frequency is 10
0
10
1
10
2 3
10
fN = 500 Hz. frequency [Hz]

The amplitude ratio plot in Fig 5.13 asymptotes to a maximum frequency of 500 Hz. This may
seem surprising since the Bode diagram of a true continuous filter does not asymptote to any
limiting frequency. What has happened, is that Fig 5.13 is the plot of a discrete filter, and that
the limiting frequency is the Nyquist frequency (2/T ). However the discrete Bode plot is a good
approximation to the continuous filter up to about half the Nyquist frequency.

A MATRIX OF FILTERS

Fig. 5.14 compares the performance of 9 filters using the 2 frequency component signal from
§5.3.1. The 9 simulations in Fig 5.14 show the performance for 3 different orders (2,5 and 8) in
the columns at 3 different cut-off frequencies (20, 50 and 100 Hz) in the rows. This figure neatly
emphasizes the trade off between smoothness and phase lag.

5.3.2 A PPROXIMATING DIGITAL FILTERS FROM CONTINUOUS FILTERS

As mentioned in the introduction, one way to design digital filters, is to first start off with a
continuous design, and then transform the filter to the digital domain. The four common classical
continuous filter families: Butterworth, Chebyshev, elliptic and Bessel, are all strictly continuous
filters, and if we are to implement them in a computer or a DSP chip, we must approximate these
filters with a discrete filter.

To convert from a continuous filter H(s), to a digital filter H(z), we could use the bilinear trans-
form as described in section 2.5.2. Since this transformation method is approximate, we will find
that some frequency distortion will be introduced. However we can minimise this distortion by
adjusting the frequency response of the filter H(s) before we transform to the discrete approxi-
mation. This procedure is called using the bilinear transformation with frequency pre-warping.
5.3. DISCRETE FILTERS 213

n=2 n=5 n=8

raw
ω = 20

filtered
ω = 50
ω = 100

Figure 5.14: The performance of various Butterworth filters of different orders 2,5, and 8 (left to
right) and different cut-off frequencies, ωc of 20, 50 and 100 Hz (top to bottom) of a two frequency
component signal.

F REQUENCY PRE - WARPING

The basic idea of frequency pre-warping is that we try to modify the transformation from contin-
uous to discrete so that H(z) better approximates H(s) at the frequency of interest, rather than
just at ω = 0. This is particularly noticeable for bandpass filtering applications.

The frequencies in the z plane (ωd ) are related to the frequencies in the s plane (ωa ) by
 
2 ωa T
ωd = tan−1 (5.16)
T 2
 
2 ωd T
ωa = tan−1 (5.17)
T 2

This means that the magnitude of H(s) at frequency ωa , |H(iωa)|, is equal to the magnitude
of H(z) at frequency ωd , |H(eiωd T )|. Suppose we wish to discretise the continuous filter H(s)
using the bilinear transform, but we also would like the digital approximation to have the same
magnitude at the corner frequency (wc = 10), then we must substitute using Eqn 6.13.

Example of the advantages of frequency pre-warping.

Suppose we desire to fabricate an electronic filter for a guitar tuner. In this application the guitar
player strikes a possibly incorrectly tuned string, and the tuner box recognises which string is
struck, and then subsequently generates the correct tone. The player can then adjust the tuning
of the guitar. Such a device can be fabricated in a DSP chip using a series of bandpass filters
tuned to the frequencies of each string (give or take a note or two).

We will compare a continuous bandpass filter centered around a frequency of 440 Hz (concert
pitch A), with digital filters using a sampling rate of 2kHz. Since we are particularly interested
in the note A, we will take special care that the digital filter approximates the continuous filter
around f = 440Hz. We can do this by frequency pre-warping. The frequency response of the
three filters are shown in Fig. 5.15.
214 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

[B,A] = butter(4,[420 450]*2*pi,'s'); % band pass from 420 to 540 Hz

2 Gc = tf(B,A) % Continuous first order filter

Ts = 1/2000; % sample rate 2kHz

wn = pi/Ts; % Nyquist freq. (rad/s)

7 Gd = c2d(Gc,Ts,'tustin') % Now sample it using bilinear transform

Gd_pw = c2d(Gc,Ts,'prewarp',440*2*pi) % Now sample it with pre-warping

Fig. 5.15 highlights that the uncompensated digital filter (dashed) actually bandpasses signals
in the range of 350 to 400Hz rather than the 420 to 450Hz required. This significant error is
addressed by designing a filter to match the continuous filter at f = 440Hz using frequency
pre-warping (dotted).

Continuous & discrete Bode plots of a band−pass filter

0
10 Continuous
Discrete
pre−warp
f
N
AR

−1
10
pre−warp

discrete continuous

−2
10
3
10
Frequency [Hz]

Figure 5.15: Comparing a continuous bandpass filter with a discrete filter and discrete filter with
frequency pre-warping (dashed). Note the error in the passband when using the bilinear trans-
form without frequency pre-warping.

For an online filter application, we should

1. Decide on the cut-off frequency, ωc .

2. Select structure of the analogue filter, (Butterworth, Chebyshev, elliptical etc). Essentially
we are selecting the optimality criteria for the filter.

3. The filter order (number of poles).

4. Transform to the discrete domain using the bilinear transform.

All these considerations have advantages and disadvantages associated with them.

5.3.3 E FFICIENT HARDWARE IMPLEMENTATION OF DISCRETE FILTERS

To implement the discrete difference equation,

yk A(z −1 ) = B(z −1 )uk

5.3. DISCRETE FILTERS 215

in hardware, we can expand out to a difference equation

na
X nb
X
yk = − ai yk−i + bi uk−i (5.18)
i=1 i=0

meaning now we can implement this using operational amplifiers and delay elements. One
scheme following Eqn. 5.18 directly given in Fig. 5.16 is known as the Direct Form I. It is easy
to verify that the Direct Form I filter has the transfer function

b0 + b1 z −1 + b2 z −2 + · · · + bnb z −nb
H(z −1 ) =
1 + a1 z −1 + a2 z −2 + · · · + ana z −na

b0
xk ✲ ✲+ ✲+ ✲ yk
✻ ✻
❄ ❄
−1 −1
z b1 −a1 z
✲ ✲+ +✛ ✛
❄ ✻ ✻ ❄
−1 −1
z .. −a2 z
.
✲ ✲+ +✛ ✛
❄ ✻ ✻ ❄
−1 −1
z .. z
.
✲ ✲+ +✛ ✛
✻ ❄
bnb z −1
−ana
| {z } +✛ ✛
numerator
| {z }
denominator

Figure 5.16: Hardware difference equation in Direct Form I

The square boxes in Fig. 5.16 with z −1 are the delay elements, and since we take off, or tap, the
signal after each delay, the lines are called tapped delays.

For the circuit designer who must fabricate Fig. 5.16, the expensive components are the delay el-
ements since they are typically specialised integrated circuits and one is economically motivated
to achieve the same result but by using fewer components. Rather than use the topology where
we need separate delay lines for both the input sequence and the output sequence separately, we
can achieve the identical result using only half the number of delays. This reduces fabrication
cost.

Starting with original IIR filter model

Y (z −1 ) B(z −1 )
−1
= (5.19)
U (z ) A(z −1 )
216 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

and introducing an intermediate state, w,

Y (z −1 ) W (z −1 ) B(z −1 )
· =
W (z −1 ) U (z −1 ) A(z −1 )
which means that we can describe Eqn. 5.19 as one pure infinite impulse response (IIR) and one
finite impulse response (FIR) process
Y (z −1 ) = B(z −1 )W (z −1 ) (5.20)
−1 −1 −1
A(z )W (z ) = U (z ) (5.21)
A block diagram following this topology is given in Fig. 5.17 which is known as Direct Form II,
DFII.

wk b0
xk ✲+ ✲ ✲+ ✲ yk
✻ ✻
❄
−a1 z −1 b1
+✛ ✛ ✲ ✲+
✻ ❄ ✻

−a2 z −1 ..
.
+✛ ✛ ✲ ✲+
✻ ❄ ✻
z −1

+✛ ✛ ✲ ✲+
✻ ❄
z −1 bnb
−ana
+✛ ✛

Figure 5.17: An IIR filter with a minimal number of delays, Direct Form II

We could have reached the same topological result by noticing that if we swapped the order of
the tapped delay lines, in the direct form 1 structure, Fig. 5.16, then we will notice that the delay
lines are separated by a unity gain branch. Thus we can eliminate one of the branches, and save
the hardware cost of half the delays.

Further forms of the digital filter are possible and are used in special circumstances such as when
we have parallel processes or we are concerned with the quantisation of the filter coefficients.
One special case is described further in section 5.3.4.

With M ATLAB we would normally use the optimised filter command to simulate IIR filters,
but this option is not available when using say C on a microprocessor or DSP chip. The following
script implements the topology given in Fig. 5.17 for the example filter
B(z −1 ) (z −1 − 0.4)(z −1 − 0.98)
=
A(z −1 ) (z −1 − 0.5)(z −1 − 0.6)(z −1 − 0.7)(z −1 − 0.8)(z −1 − 0.9)
and a random input sequence.
5.3. DISCRETE FILTERS 217

B = poly([0.4 0.98]); % Example stable process

2 A = poly([0.5:0.1:0.9]);
U = [ones(30,1); randn(50,1)]; % example input profile
Yx = dlsim(B,A,U); % Test Matlab's version

% Version with 1 shift element -----------------

7 nb = length(B); A(1) = []; na = length(A);
nw = max(na,nb); % remember A=A-1
shiftw = zeros(size(1:nw))'; % initialise with zeros
Y = []; for i=1:length(U);
w = - A*shiftw(1:na) + U(i) ;
12 shiftw = [w; shiftw(1:nw-1)];
Y(i,1) = B*shiftw(1:nb);
end % for

t = [0:length(U)-1]'; [tt,Ys] = stairs(t,Y); [tt,Yx] =

17 stairs(t,Yx);
plot(tt,Ys,tt,Yx,'--'); % plotting verification

M ATLAB’s dlsim gives a similar output but is simply shifted one unit to the right.

5.3.4 N UMERICAL AND QUANTISATION EFFECTS FOR HIGH - ORDER FILTERS

Due to commercial constraints, for many digital filtering applications, we want to be able to
realise cheap high-quality filters at high speed. Essentially this means implementing high-order
IIR filters on low-cost hardware with short word-lengths. The problem is that high-order IIR
filters are very sensitive to quantisation effects especially when implemented in the expanded
polynomial form such as DFII.

One way to reduce the effects of numerical roundoff is to implement rewrite the DFII filter in
factored form as a cascaded collection of second-order filters as shown in Fig. 5.18. Each indi-
vidual filter is a second-order filter as shown in Fig. 5.19. These are known as bi-quad filters, or
second-order sections (SOS). For an nth order filter, we will have ⌈n/2⌉ sections, that is for filters
with an odd order, we will have one second order section that is actually first order.

✲ ✲ ✲
xk ✲ ✲ ✲ ✲ yk

| {z }
SOS #1 ··· SOS # ⌈n/2⌉

Figure 5.18: Cascaded second-order sections to realise a high-order filter. See also Fig. 5.19.

We can convert higher-order filters to a collection of second-order sections using the tf2sos or
equivalent commands in M ATLAB. You will notice that the magnitude and range of the coef-
ficients of the SOS filter realisation is much smaller than the equivalent expanded polynomial
filter.
218 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

b0
xk ✲+ ✲ ✲+ ✲ yk
✻ ✻
❄
−a1 z −1 b1
+✛ ✛ ✲ ✲+
✻ ❄ ✻

−a2 z −1 b2
✛ ✲

Figure 5.19: A second-order section (SOS)

The separating of the factors into groups of second-order sections for a given high-order filter is
not unique, but one near-optimal way is to pair the poles nearest the unit circle with the zeros
nearest those poles and so on. This is is the strategy that tf2sos uses and is explained in further
detail in the help documentation. Listing 5.6 demonstrates how we can optimally decompose
a 7th order filter into four second-order sections. Of course the first of the four second-order
sections is actually a first-order section. We can see that by noticing that the coefficients b2 and a2
are equal to zero for the first filter section.

Listing 5.6: Converting a 7th-order Butterworth filter to 4 second-order sections

>> [B,A] = butter(7,0.8,'low'); H = tf(B,A,1) % Design a 7th order discrete DFII filter
2 Transfer function:

0.2363 zˆ7 + 1.654 zˆ6 + 4.963 zˆ5 + 8.271 zˆ4 + 8.271 zˆ3 + 4.963 zˆ2

+ 1.654 z + 0.2363
7

-------------------------------------------------------------------------
zˆ7 + 4.182 zˆ6 + 7.872 zˆ5 + 8.531 zˆ4 + 5.71 zˆ3 + 2.349 zˆ2

+ 0.5483 z + 0.05584
12

>> [SOS,G] = tf2sos(B,A) % Convert 7th order polynomial to 4 second-order sections

SOS =
1.0000 0.9945 0 1.0000 0.5095 0
1.0000 2.0095 1.0095 1.0000 1.0578 0.3076
17 1.0000 2.0026 1.0027 1.0000 1.1841 0.4636
1.0000 1.9934 0.9934 1.0000 1.4309 0.7687
G =
0.2363

The advantage when using second-order sections is that we can maintain reasonable accuracy
even when using relatively short word lengths. Fig. 5.20(a), which is generated by Listing 5.7,
illustrates that we cannot successfully run a 7th-order elliptic low-pass filter in single precision in
the expanded polynomial form, (DFII), one reason being that the resultant filter is actually unsta-
5.4. THE FOURIER TRANSFORM 219

zoomed region

2 0.4 Pole−Zero Map

single precision 0.35

filtered response

1.5 1
0.3
0.8
1
0.250.6

0.5 0.20.4
Double & single SOS
0.150.2
10 0
0 0.9 1 1.1

ℑ
0
−0.2
−0.4
error

−5
10
−6 −0.6
10
−0.8 double
−1 single
−10
10 −1.5 −1 −0.5 0 0.5 1
0 20 40 60 80 100
sample interval ℜ

(a) The time response of double and single precision filters (b) Pole-zero maps of single and double precision
to a unit step response. filters. Note that the single precision version has
poles outside the unit circle.

Figure 5.20: Comparing single precision second-order sections with filters in direct form II trans-
posed form. Note that the direct form II filter is actually unstable when run in single precision.

ble! However we can reliably implement the same filter, in single precision if we use 4 cascaded
second-order sections. In fact for this application, the single precision filter implemented in
SOS is indistinguishable in Fig. 5.20(a) from the double precision implementation given that the
average error is always less than 10−6 .

Listing 5.7: Comparing DFII and SOS digital filters in single precision.
[B,A] = ellip(7,0.5,20,0.08); % Design a 7th order elliptic filter

U = ones(100,1); % Consider a step response

Y = filter(B,A,U); % Compute filter in double precision
5 Ys = filter(single(B), single(A), single(U)); % Compute filter in single precision

[SOS,G] = tf2sos(B,A); % Convert to SOS (in full precision)

n = size(SOS,1); % # of sections
ysos = single(G)*U; % Compute SOS filter in single precision
10 for i=1:n
ysos = filter(single(SOS(i,1:3)), single(SOS(i,4:end)), ysos)
end
stairs([Y Ys ysos]); % Compare results in Fig. 5.20(a).

It is also clear from the pole-zero plot in Fig. 5.20(b) that the single precision DFII implementation
has poles outside the unit circle due to the quantisation of the coefficients. This explains why this
single precision filter is unstable.

5.4 T HE F OURIER TRANSFORM

The Fourier Transform (FT) is one of the most well known techniques in the mathematical and
scientific world today. However it is also one of the most confusing subjects in engineering,
partly owing to the plethora of definitions, normalising factors, and other incompatibilities found
in many standard text books on the subject. The Fourier transform is covered in many books on
220 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

digital signal processing, (DSP), such as [36, 133], and can also be found in mathematical physics
texts such as [108, 163]. The Fourier Transform is a mathematical transform equation that takes
a function in time and returns a function in frequency. The Fast Fourier Transform, or FFT, is
simply an efficient way to compute this transformation.

Chapter 2 briefly mentioned the term “spectral analysis” when discussing aliases introduced
by the sampling process. Actually a very learned authority , [178, p53], tells us that “spectral”
formally means “pertaining to a spectre or ghost”, and then goes on to make jokes about ghosts
and gremlins causing noisy problems. But nowadays spectral analysis means analysis based
around the spectrum of a something. We are going to use spectral analysis to investigate the
frequency components of a measured signal, since in many cases these frequency components
may reveal critical information hidden in the time domain description. This decomposition of
the time domain signal into its frequency components is just one application of the FFT.

Commonly in mathematics, we can approximate any function f (t) using (amongst other things)
a Taylor series expansion, or a sine and cosine series expansion. If the original signal is periodic,
then the latter technique generally gives better results with fewer terms. This latter series is called
the Fourier series expansion.

The fundamental concept on which the Fourier transform (FT) is based is that almost any periodic
wave form can be represented as a sum of sine waves of different periods and amplitudes,
∞
X ∞
X
f (t) = An sin (2πnf0 t) + Bn cos (2πnf0 t) (5.22)
n=0 n=0

This means that we can decompose any periodic wave form (PW) with a fundamental frequency2
(f0 ) into a sum of sinusoidal waves, (which is known as frequency analysis) or alternatively we
can construct complicated waves from simple sine waves (which is known as frequency synthe-
sis).

Since it is impractical to have an infinite number of terms, this series is truncated, and f (t) now
only approximates the right hand side of Eqn. 5.22. Naturally, as the number of terms tends to
infinity, the approximation to the true signal f (t) improves. All we have to do now is to determine
the constants An and Bn for a given function f (t). These coefficients can be evaluated by
Z P/2 Z P/2
2 2
An = f (t) sin (2πnf0 t) dt and Bn = f (t) cos (2πnf0 t) dt (5.23)
P −P/2 P −P/2

when n 6= 0. When n = 0, we have the special case

Z P/2
1
A0 = 0, and B0 = f (t)dt (5.24)
P −P/2

In summary, f (t) tells us how the signal develops in time, while the constants An , Bn give us a
method to generate f (t).

We can re-write Eqn. 5.22 as a sum of cosines and a phase angle

∞
X
f (t) = Cn cos (2πnfo t + ψn ) (5.25)
n=0

2 Remember the relationship between the frequency f , the period P , and the angular velocity ω.
1 ω
f= =
P 2π
5.4. THE FOURIER TRANSFORM 221

where the phase angle, ψn , and amplitude, Cn , are given by

 
An p
ψn = tan−1 − and Cn = A2n + Bn2 (5.26)
Bn

If we plot Cn vs f then this is called the frequency spectrum of f (t). Note that Cn has the same
units of An or Bn which have the same units of whatever the time series f (t) is measured in.

5.4.1 F OURIER TRANSFORM DEFINITIONS

Any function f (t) that is finite in energy can be represented in the frequency domain as F (ω) via
the transform
Z ∞
F (ω) = f (t)e−iωt dt (5.27)
−∞
Z ∞ Z ∞
= f (t) cos(ωt)dt − i f (t) sin(ωt)dt (5.28)
−∞ −∞

We can also convert back to the time domain by using the inverse Fourier transform,
Z ∞
1
f (t) = F (ω) eiωt dω
2π −∞

You may notice that the transform pair is not quite symmetrical, we have a factor of 2π in the
inverse relation. If we use frequency measured in Hertz, where ω = 2πf rather than ω, we obtain
a much more symmetrical, and easier to remember, pair of equations. [163, pp381–382] comment
on this, and [141, p145] give misleading relations.

The spectrum of a signal f (t) is defined as the square of the absolute value of the Fourier trans-
form,
Φf (ω) = |F (ω)|2
Parseval’s equation reminds us that the energy of a signal is the same whether we describe it in
the frequency domain or the time domain,
Z ∞ Z ∞
1
f 2 (t) dt = |F (ω)|2 dω
−∞ 2π −∞

We can numerically approximate the function F (ω) by evaluating Eqn 5.28 using a numerical
integration technique such as Euler’s method. Since this numerical integration is computation-
ally expensive, and we must do this for every ω of interest, the calculation procedure is quite
time consuming. I will refer to this technique as the Slow Fourier Transform (SFT). Remember
that H(ω) is a complex function of angular velocity (ω), and that the result is best displayed as a
graph with ω measured in radians/s as the dependent variable.

S OME HELPFUL PROPERTIES

Here are some helpful properties of the Fourier transform that may make analysis easier.

1. If f (t) = −f (−t) which means that f (t) is an odd function like sin t, then all the cosine
terms disappear.
222 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

2. If f (t) = f (−t) which means that f (t) is an even function like cos t, then all the sine terms
disappear.

3. If f (t) = −f (t + P/2), then the series will contain only odd components; n = 1, 3, 5, . . .

4. If f (t) = f (t + P/2), then the series will contain only even components; n = 2, 4, 6, . . .

Naturally we are not at liberty to change the shape of our measured time series f (t), but we may
be allowed to choose the position of the origin such that points 3 and 4 are satisfied.

T HE E ULER RELATIONS

We can also write the Fourier series using complex quantities with the Euler relations.

eiθ = cos θ + i sin θ, and e−iθ = cos θ − i sin θ (5.29)

√
where i = −1. This implies that

eiθ + e−iθ eiθ − e−iθ

cos θ = , and sin θ = (5.30)
2 2i
So we can re-write Eqn 5.22 in complex notation as
∞
X
f (t) = Dn ei2πnf0 t (5.31)
−∞

where Z P/2
1
Dn = f (t)e−i2πnf0 t dt (5.32)
P −P/2

Example A decomposition of a square wave

The square wave is a simple periodic function that can be written as a infinite sine and cosine
series. Fig 5.21 shows the original time series f (t). We note that the original wave is:

1. Periodic, with a period of P = 1 seconds.

2. f (t) is an odd function such as sin t so we expect all the cosine terms disappear. An odd
function is where f (t) = −f (−t), while an even function is where f (t) = f (−t).

Using Eqn. 5.23, we see that

1
An = (1 − cos πn)
πn
which collapses to An = 2/nπ for n = 1, 3, 5, . . . ∞ and An = 0 for n = 2, 4, 6, . . . ∞. This gives
the Fourier series approximation to f (t) as

2 2 2
f (t) = sin 2πt + sin 6πt + · · · + sin 2(2n − 1)πt + · · · (5.33)
π 3π (2n − 1)π

As the number of terms (n) in Eqn. 5.33 increase, the approximation improves as illustrated in Fig
5.22 which shows the Fourier approximation for 1 through 4 terms. Later in §5.4.3 we will try to
do the reverse, namely given the square wave signal, we will extract numerically the coefficients
of the sine and cosine series.
5.4. THE FOURIER TRANSFORM 223

Original signal f(t)

1

0.5

−0.5

−1

First order approximation of f(t)

1

0.5

−0.5
Figure 5.21: The original square wave sig-
−1
0 1 2 3 4 nal f (t) (top), and an approximation to f (t)
time using one sine term (bottom)

# of terms = 1
1

−1

# of terms = 2
1

−1

# of terms = 3
1

−1

# of terms = 4
1

0 Figure 5.22: The Fourier approxima-

tion to a square wave with varying
−1 number of terms.
0 1 2 3 4
224 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

5.4.2 O RTHOGONALITY AND FREQUENCY SPOTTING

Consider the function y = sin(3t) sin(5t) which is plotted for 2 full periods in Fig 5.23 (top). If
we integrate it over one period, we can see that the positive areas, cancel out the negative areas,
hence the total integral is zero. Or
Z P =π
y(t) dt = 0
0

The only time an integral of a full period will not be zero is when the entire curve is either all
positive or all negative. The only time that can happen, is when the two frequencies are the same
such as in Fig. 5.23 (bottom). Thus if

y = sin(nt) + sin(mt) (5.34)

then the integral over one full period will be one of the following two results:
Z P
y(t) dt = 0 if n 6= m (5.35)
0
Z P
y(t) dt 6= 0 if n = m (5.36)
0

sin(3t)*sin(5t)
1
0.5
0
−0.5
−1
0 1 2 3 4 5 6 7
2
sin (3t)
1
0.5
0
Figure 5.23: A plot of (top)
y = sin(3t) sin(5t) and (bottom) −0.5
y = sin(3t) sin(3t). In the top figure −1
the shaded regions cancel giving a total 0 1 2 3 4 5 6 7
integral of zero for the two periods. time

Fig 5.23 was created by the following M ATLAB commands

t=[0:0.01:2*pi]'; % create a time vector 2 Periods long

2 y3=sin(3*t); y5=sin(5*t);
plot(t,[y3.*y3, y3.*y5]) % plot both curves

We can numerically integrate these curves using the sum command. The integral of the top curve
in Fig. 5.23 is

sum(y3.*y5)
ans = -1.9699e-05
5.4. THE FOURIER TRANSFORM 225

R 2π
which is close enough to zero while 0
sin2 3t is evaluated as

sum(y3.*y3)
ans = 314.1593

and is definitely non-zero. Note that for this example we have integrated over 2 periods, so
we should divide the resultant integral by 2. This however, does not change our conclusions.
Additionally note that the simple sum command when used as an integrator assumes that the
argument is uniformly spaced 1 time unit apart. This is not the case for this example, but the
difference is only a scale factor, and again is not important for this demonstration.

P ERIOD

The period of y = sin(nt) is P = 2π/n. The period of y = sin(nt) sin(nt) can be evaluated using
an expansion for sin2 (nt) as
1
y = sin2 (nt) = (1 − cos(2nt))
2
thus the period is P = π/n.

The period of the general expression y = sin(nt) sin(mt) where n and m are potentially different
is found by using the expansion
1
y= [cos(n + m)t + cos(n − m)t]
2
which for the example above gives
1
y = sin(3t) sin(5t) = [− cos 8t + cos 2t] (5.37)
2
The period of the first term in Eqn 5.37 is P1 = π/4 and for the second term is P2 = π. The total
period is the smallest common multiple of these two periods which is for this case P = π.

The property given in Eqn. 5.36 can be exploited for spectral analysis. Suppose we have measured
a signal x(t) that is made up of a sine wave with a particular, but unknown, frequency. However
we wish to establish this unknown frequency using spectral analysis. All we need to do is to
multiply x(t) by a trial sine wave, say sin 5t, integrate over one period, and look at the result.
If the integral is close to zero, then our original signal, x(t), had no sin 5t term, if however we
obtained a significant non-zero integral, then we would conclude that the measured signal had
a sin 5t term somewhere. Note that to obtain a complete frequency description of the signal, we
must try all possible frequencies from 0 to ∞.

So in summary, the idea is to multiply the signal x(t) with different sine waves containing the
frequency of interest, integrate, and then see if the result is non-zero. We could plot the integral
result against the frequency thus producing a spectral plot of x(t).

5.4.3 U SING M ATLAB ’ S FFT FUNCTION

M ATLAB provides a convenient way to analyse the frequency component of a finite sampled
signal by using the built-in fft function. This is termed spectral analysis and is at its simplest
just a plot of the magnitude of the Fourier transform of the time series, or abs(fft(x)). The
complexities are due to the scaling of the plot, and possible windowing to improve the frequency
226 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

resolution at the expense of the magnitude value, or vice-versa and possible adjustment for non-
periodic signals.

The S IGNAL P ROCESSING toolbox includes a command psd which is an improved version of the
example given in Technical Support Note # 1702 available from www.mathworks.com/support/tech-notes/17
Other functions to aid the spectral analysis of systems are spectrum and the associated
specplot, and etfe (empirical transfer function estimate) from the S YSTEM I DENTIFICATION
toolbox. All of these routines perform basically the same task.

To look how we can perform a spectral decomposition, let us start with a simple example where
we know the answer, namely
x(t) = 4.5 sin(2π200t)
It is immediately apparent that at a frequency of 200 Hz, we should observe a single Fourier
coefficient with a magnitude of 4.5 while all others are zero. Our aim is to verify this using
M ATLAB ’ S fft routine.

The function, spsd (scaled PSD) giving in Listing 5.8 will use the Fourier transform to decompose
the signal into different frequencies and display this information. This function was adapted from
the version given in Technical Support Note # 1702 available at the M ATH W ORKS ftp server and
further details are given in the note. Since we have assumed that the input signal is real, then
we know that the Fourier transform will be conjugate symmetrical about the Nyquist frequency,
thus we can throw the upper half away. Doing this, we must multiply the magnitude by 2 to
maintain the correct size. M ATLAB also normalises the transform by dividing by the number of
data points so must also account for this in the magnitude. Since the FFT works much faster
when the series is a power of two, we automatically pad with zeros up to the next power of 2. In
the example following, I generate 1000 samples, so 24 zeros are added to make up the difference
to 210 = 1024.

Listing 5.8: Routine to compute the power spectral density plot of a time series
function [Mx,f] = spsd(x,dt);
2 % [Mx,f] = spsd(x,[dt])
% Power spectral plot of x(t) sampled at dt [s], (default dt=1).
% Output: Power of x at frequency, f [Hz]
% See also: PSD.M, SPECTRUM, SPECPLOT, ETFE

7 if nargin < 2, dt = 1.0; end % default sample time

Fs = 1/dt; Fn = Fs/2; % sampling & Nyquist frequency [Hz]
NFFT=2.ˆnextpow2(length(x)); % NFFT=2.ˆ(ceil(log(length(x))/log(2)));
FFTX=fft(x,NFFT); % Take fft, padding with zeros,
NumUniquePts = ceil((NFFT+1)/2);
12

FFTX=FFTX(1:NumUniquePts); % fft is symmetric, throw away upper half

MX=2*abs(FFTX); % Take magnitude of X & 2* since threw half away

MX(1)=MX(1)/2; % Account for endpoint uniqueness

17 MX(length(MX))=MX(length(MX))/2; % We know # of samples is even

MX=MX/length(x); % scale for length

f=(0:NumUniquePts-1)*2*Fn/NFFT; % frequency Hz

22 if nargout == 0, % do plot if requested

plot(f,MX); xlabel('frequency [Hz]');
end % if
return % SPSD.M
5.4. THE FOURIER TRANSFORM 227

We are now ready to try our test signal to verify spsd.m given in Listing 5.8.

Fs = 1000; % Sampling freq, Fs , in Hz

t=0:1/Fs:1; % Time vector sampled at Fs for 1 second
x = 4.5*sin(2*pi*t*200)'; % scaled sine wave.
5
4
3
2
1
0
0 100 200 300 400 500
spsd(x,1/Fs) % Does Fs = 200, A = 4.5? frequency [Hz]

We should expect to see a single distinct spike at f = 200Hz with a height of 4.5.

With the power spectrum density function spsd, we can also analyse the frequency components
of a square wave. We can generate a square wave using the square function in a manner similar
to that of generating a sine wave, or simply wrap the signum function around a sine wave.

1 Fs = 30; t=0:1/Fs:20; % Sampling freq, Fs , in Hz & time vector for 20 seconds

s = 0.5*square(2*pi*t); % Square wave
spsd(s,1/Fs) % See odd spikes ??
Welch Power Spectral Density Estimate
Power/frequency (dB/Hz)

−10

−20

−30

−40

−50
0 5 10 15
pwelch(s,[],[],[],Fs); % Try toolbox version Frequency (Hz)

Note that all the dominant frequency components appear at the odd frequencies (1,3,5,7. . . ), and
that the frequency components at the even frequencies are suppressed as anticipated from §5.4.1.
Of course they theoretically should be zero, but the discretisation, finite signal length and sam-
pling etc. introduce small errors here.

5.4.4 P ERIODOGRAM

There exists a parallel universe to the time domain called the frequency domain, and all the op-
erations we can do in one domain can be performed in the other. Our gateway to this frequency
domain as previously noted is the Fast Fourier Transform or known colloquially as the FFT. In-
stead of plotting the signal in time, we could plot the frequency spectrum of the signal. That is,
we plot the power of the signal as a function of frequency. The power Φ of the signal y(t) over a
228 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

frequency band is defined as the absolute square of the Fourier transform;

X 2
def
Φy (ω) = |Y (ω)|2 =  y(t)e−iωt 
 

When we plot the power as a function of frequency, it is much like a histogram where each
frequency “bin” contains some power. This plot is sometimes referred to as a periodogram.

Constructing the periodogram for real data can give rather messy, inconclusive and widely fluc-
tuating results. For a data series, you may find that different time segments of the series give
different looking spectra when you suspect that the true underlying frequency characteristics is
unchanged. One way around this is to compute the spectra or periodogram for different seg-
ments and average the results, or you could use a windowing function. The spa and etfe
functions in the System identification toolbox attempt to construct periodograms from logged
data.

Problem 5.2 Fig. 5.24 trends the monthly critical radio frequencies in Washington, D.C. from May
1934 until April 1954. These frequencies reflect the highest radio frequency that can be used for
broadcasting.

15
Critical frequency

10

0
1935 1940 1945 1950 1955
Year

Figure 5.24: Critical radio frequencies

This data is available from the collection of ‘classic’ time series data archived in: http://www-personal.buseco.
(See also Problem 6.3 for further examples using data from this collection.)

1. Find the dominant periods (in days) and amplitudes of this series.

2. Can you hypothesise any natural phenomena that could account for this periodicity?

5.4.5 F OURIER SMOOTHING

It is very easy to smooth data in the Fourier domain. You simply take all your data, FFT it,
multiply this transformed data by the filter function H(f ), and then do the inverse FFT (IFFT)
to get back into the time domain. The computationally difficult parts are the FFT and IFFT, but
this is easily achieved in M ATLAB or in custom integrated circuits. Note that this section is called
smoothing as opposed to online filtering because these smoothing filters are acausal since future
data is required to smooth present data as well as past data. Filters that only operate on past
data are referred to as causal filters. Acausal filters tend to perform better than causal filters as
they exhibit less phase lag but they cannot be used in real time applications. In process control,
physical realisation constraints mean that only causal filters are possible. However in the audio
world (compact disc and digital tapes), acausal filters are used since a time delay is not noticeable
5.4. THE FOURIER TRANSFORM 229

and hence allowed. Surprisingly time delays are allowed on telephone communications as well.
Numerical Recipes [163] has a good discussion of the advantages of Fourier smoothing.

Let us “smooth” the measurement data given in §5.3.1 and compare the smoothed signal with
the online filtered signal. This time however we will deliberately generate 210 = 1024 data pairs
rather than an even thousand so that we can take advantage of the fast Fourier Transform al-
gorithm. It is still possible to evaluate the Fourier transform of a series with a non-power of 2
number of pairs, but the calculation is much slower so its use is discouraged.

Again we will generate a measurement signal as

1
y = sin (2πf1 t) + sin (2πf2 t)
10

where f1 = 2 Hz and f2 = 50 Hz as before. We will generate 210 data pairs at a sampling frequency
of 200Hz.

1 t = 1/200*[0:pow2(10)-1]'; % 210 data pairs sampled at 200 Hz

1
y = sin(2*pi*2*t) + 0.1*sin(2*pi*50*t); % Signal: y = sin (2πf1 t) + 10 sin (2πf2 t)

Generating the Fourier transform of the data is easy. We can plot the “amount” or power in each
frequency bin Y(f ) by typing

yp = fft(y);
plot(abs(yp).ˆ2) % See periodogram in Fig. 5.25
5 semilogy(abs(yp).ˆ2) % clearer plot

In Fig. 5.25, we immediately note that the plot is mirrored about the central Nyquist frequency
meaning that we can ignore the right-hand side of the plot. The x-axis frequency scale is nor-
malised, but you can easily detect the two peaks corresponding to the two dominant frequencies
of 2 and 50 Hz. Since we are trying to eradicate the higher frequency, what we would desire in
the power spectrum is only one spike at a frequency of f = 2 Hz. We can generate a power curve
of this nature, by multiplying it with the appropriate filter function H(f ).

500 Spectrum
Filter

400 zoomed
600
300
Power

400

200 200

0
100 0 50

0 Figure 5.25: The frequency power spectrum

0 200 400 600 800 1000
for the level signal. Both the frequency axis
frequency and the power axis are normalised.

Let us generate the filter H(f ) in the frequency domain. We want to pass all frequencies less than
say 30Hz, and attenuate all frequencies above 30Hz. We do this by constructing a vector where 1
corresponds to the frequencies we wish to retain, and zero is for those we do not want. Then we
convolve the filter with the data, or elementwise multiply the two frequency series.
230 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

n = 20;
h=[ones(1,n) zeros(1,length(y)-2*n) ones(1,n)]'; % Filter H(f )
ypf = yp.*h % convolve signal & filter Y (f ) × H(f )
plot([abs(yp), h, abs(ypf)])

Now we can take the inverse FFT to obtain back the transformed the time domain signal which is
now our low pass filtered signal. Technically the inverse Fourier transform of a Fourier transform
of a real signal should return also a real time series of numbers. However owing to numerical
roundoff, the computed series may contain a small complex component. In practice we can
safely ignore these, and force the signal to be real.

10 yf = real(ifft(ypf));% force real signal, yf = ℜ{FFT −1 (Y (f ))}

plot(t,[y,yf]) % compare raw & smoothed in Fig. 5.26.

1.5
smoothed
1

0.5
output

−0.5
original

Figure 5.26: The Fourier smoothed signal, yf , −1

compared to the original noisy level signal, y.
Note the absence of any phase lag in the now −1.5
−0.2 0 0.2 0.4 0.6 0.8
offline smoothed signal. time (s)

The result shown in Fig. 5.26 is an acceptably smooth signal with no disturbing phase lag that
was unavoidable in the online filtering example.

Problem 5.3 Design a notch filter to band-stop 50Hz. Test

your filter using an input signal with a
time varying frequency content such as u(t) = sin 16t4 . Plot both the original and filtered signal
together. Sample around 1000 data pairs using a sampling frequency of 1000 Hz. Also plot the
“apparent” frequency as a function of time for u(t).

5.5 N UMERICALLY DIFFERENTIATING INDUSTRIAL DATA

In industrial applications, we often want the know the slope of a measured process variable. For
example, we may wish to know the flowrate of material into a vessel, when we only measure
the level of the vessel. In this case, we differentiate the level signal, to calculate the flowrate.
However for industrial signals, which typically have significant noise, calculating an accurate
derivative is not trivial. A crude technique such as differencing, will give intolerable error. For
these types of online applications, we must first smooth the data, then differentiate the smoothed
data. This can be accomplished using discrete filters.
5.5. NUMERICALLY DIFFERENTIATING INDUSTRIAL DATA 231

5.5.1 E STABLISHING FEEDRATES

This case study demonstrates one way to differentiate noisy data. For this project, we were trying
to implement a model based estimation scheme to a 1000 liter fed-batch chemical reactor. For our
model, we needed to know the feedrate of the material being fed to the reactor. However we
did not have a flow meter on the feed line, but instead we could measured the weight of the
reactor using load cells. As the 3 hour batch progressed, feed was admitted to the reactor, and
the weight increased from about 800 kg to 1200 kg. By differentiating this weight, we could at
any time approximate the instantaneous feed rate. This is not an optimal arrangement, since we
expect considerable error, but in this case it was all that we could do.

The weight measurement of the reactor is quite noisy. The entire reactor is mounted on load cells,
and all the lines to and from the reactor (such as cooling/heating etc) are connected with flexible
couplings. Since the reactor was very exothermic, the feed rate was very small (30 g/s) and this
is almost negligible to the combined weight of the vessel (≈ 3000 kg) and contents (≈ 1000 kg).
The raw weight signal (dotted) of the vessel over the batch is shown in the upper figures in Fig.
5.28. An enlargement is given in the upper right plot. The sample time for this application was
T = 6 seconds.

A CRUDE METHOD OF DIFFERENTIATING

The easiest (and crudest) method to establish the feedrate is to difference the weight signal. Sup-
pose the sampled weight signal is W (t), then the feedrate, Ẇ = F , is approximated by
Wi − Wi−1
F = Ẇ ≈
T
But due to signal discretisation and noise, this results in an unusable mess3 . The differencing
can be achieved elegantly in Matlab using the diff command. Clearly we must first smooth the
signal before differencing.

U SING A DISCRETE FILTER

This application is not just smoothing, but also differentiating. We can do this in one step by
combining a low pass filter (Glp ) with a differentiater, s, as
Gd (s) = Glp (s) · s (5.38)
We can design any low pass filter, (say a 3rd order Butterworth filter using the butter com-
mand), then multiply the filter polynomials by the differentiater. In the continuous domain, a
differentiater is simply s, but in the discrete domain we have a number of choices how to imple-
ment this. Ogata, [150, pp 308] gives these alternatives;

1−z −1

 T Backward difference
s≈ 1−z −1 (5.39)
−1
z T Forward difference

 2 1−z−1 Center difference
T 1+z −1

For a causal filter, we need to choose the backward or center difference variants. I will use the
backward difference Thus Eqn 5.38 using the backward difference in the discrete domain is
1 − z −1
Gd (z) = Glp (z) · (5.40)
T
3I do not even try to plot this, as it comes out a black rectangle
232 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING

with sample time T . Thus the denominator is multiplied by T , and the numerator gets convolved
with the factor (1 − z −1 ). We need to exercise some care in doing the convolution, because the
polynomials given by butter are written in decreasing powers of z (not z −1 ). Note however
that fliplr(conv(fliplr(B),[-1,1])); is equivalent to conv(B,[1,-1]);. Listing 5.9
tests this smoothing differentiator algorithm.

Listing 5.9: Smoothing and differentiating a noisy signal

dt = 0.05; t = [0:dt:20]'; % Sample time T = 0.05
U = square(t/1.5); [num,den]=ord2(2,0.45); % Input u(t) & plant dynamics
3 y = lsim(num,den,U,t)+randn(size(y))*0.005;%simulate it & add noise

dyn = gradient(y,dt); % Do crude approx of differentiation

[B,A] = butter(3,0.1); % Design smoother, ω = 0.1

8 Gd_lp = tf(B,A,dt,'variable','zˆ-1'); % Convert to low-pass filter, Glp (z −1 )
Gdd = series(Gd_lp,tf([1 -1],dt,dt,'variable','zˆ-1')) % Eqn. 5.40.
yns = filter(B,A,y); % Smooth y(t)
dyns = lsim(Gdd,y); % Smooth & differentiate y(t), See Fig. 5.27.

The results of the smoothing and the differentiating of the noisy signal are given in Fig. 5.27.
The upper plot compares the noisy measurement with the the smoothed version exhibiting the
inevitable phase lag. The lower plot in Fig. 5.27 shows the crude approximation of the deriva-
tive obtained by differencing (with little phase lag, but substantial noise), and the smooth and
differentiated version.

0.4

0.2
Measurement

−0.2

−0.4
1

0.5
Derivative

−0.5
Figure 5.27: Smoothing, (upper)
and differentiating (lower) a noisy −1
0 5 10 15 20
measurement. time [s]

Note that even though the noise on the raw measurement is insignificant, the finite differencing
amplifies the noise considerably. While the degree of smoothing can be increased by decreasing
the low pass cut-off frequency, ω, we pay the price in increasing phase lag.

Fig. 5.28 shows actual industrial results using this algorithm. The right hand plots show an
enlarged portion of the signal. The upper left plot shows the raw (dotted) and smoothed (solid)
weight signal. The smoothed signal was obtained from the normal unity gain Butterworth filter
with a ωc = 0.02 1
2T = 600 s
−1
. There is a short transient at the beginning, when the filter is first
invoked, since I made no attempt to correct for the non-zero initial conditions. The lower 2 plots
show the differentiated signal, in this case feed rate using the digital filter and differentiater.
Note that since the filter is causal, there will be a phase lag, and this is evident in the filtered
signal given in the top right plot. Naturally the differentiated signal will also lag behind the true
feedrate. In this construction, the feed rate is occasionally negative corresponding to a decrease
5.6. SUMMARY 233

in weight or filter transient. This is physically impossible, and the derived feedrate be chopped
at zero if it were to be used further in a model based control scheme for example.

1200 1140
Vessel weight [kg]

Vessel weight [kg]

1100 1120

1000 1100

900 1080

800 1060
8 10 12 10.5 11 11.5 12
time [hr] time [hr]

0.1 0.1
Feed rate [kg/s]

Feed rate [kg/s]

0.05 0.05

0 0

-0.05 -0.05
8 10 12 10.5 11 11.5 12
time [hr] time [hr]

Figure 5.28: Industrial data showing (upper) the raw & filtered weight of a fed-batch reactor
vessel [kg] during a production run, and (lower) the feedrate [kg/s]

5.6 S UMMARY

Filtering or better still, smoothing, is one way to remove unwanted noise from your data. Smooth-
ing is best done offline if possible. The easiest way to smooth data is to decide what frequencies
you wish to retain as the signal, and which frequencies you wish to discard. One then constructs
a filter function, H(f ) which is multiplied with the signal in the frequency domain to create the
frequency of the smoothed signal. To get the filtered time signal, you transform back to the time
domain. One uses the Fourier transform to convert between the time and frequency domains.

If you cannot afford the luxury of offline filtering, then one must use a causal filter. IIR filters per-
form better than FIRs, but both have phase lags at high frequencies and rounded edges that are
deleterious to the filtered signal. The Butterworth filter is better than simple cascaded first order
filters, it has only one parameter (cut-off frequency), relatively simple to design and suits most
applications. Other more complicated filters such as Chebyshev or elliptic filters, with corre-
spondingly more design parameters are easily implemented using the signal processing toolbox
in M ATLAB if desired. Usually for process engineering applications, we are most interested in
low pass filters, but in some applications where we may wish to remove slow long term trends
(adjustment for seasons which is common in reporting unemployment figures for example), or
differentiate, or remove a specific frequency such as the 50Hz mains hum, then high-pass or notch
filters are to be used. These can be designed in exactly the same way as a low pass filter, but with
a variable transformation.
234 CHAPTER 5. DIGITAL FILTERING AND SMOOTHING
C HAPTER 6

I DENTIFICATION OF PROCESS
MODELS

Fiedler’s forecasting rules:

Forecasting is very difficult, especially if it’s about the future. Consequently when presenting a forecast:
Give them a number or give them a date, but never both.

6.1 T HE IMPORTANCE OF SYSTEM IDENTIFICATION

Most advanced control schemes require a good process model to be effective. One option is to set
the control law to be simply the inverse of the plant so when the inevitable disturbances occur,
the controller applies an equal but opposite input to maintain a constant output. In practice this
desire must be relaxed somewhat to be physically realisable, but the general idea of inserting
a dynamic element (controller) into the feedback loop to obtain suitable closed loop dynamics
necessitates a process model.

When we talk about ‘models’, we mean some formal way to predict the system outputs knowing
the input. Fig. 6.1 shows the three components of interest; the input, the plant, and the output. If
we know any two components, we can back-calculate the third.

Does this match with reality?

input, u ✲ ✲ output, ŷ
Model

| {z }
Try to establish this Model

Figure 6.1: The prediction problem: Given a model and the input, u, can we predict the output,
y?

When convenient in this chapter, I will use the convention that inputs and associated parameters
such as the B polynomial are coloured blue, and the outputs and the A polynomial are coloured

235
236 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

red. As a mnemonic aid, think of a hot water heater. Disturbances will be coloured green (be-
cause sometimes they come from nature). These colour conventions may help make some of the
equations in this chapter easier to follow.

The prediction problem is given the plant and input, one must compute the output, the reconstruc-
tion problem is given the output and plant, one must propose a plausible input to explain the
observed results, and the identification problem, the one we are concerned with in this chapter, is
to propose a suitable model for the unknown plant, given input and output data.

Our aim for automatic control problems is to produce a usable model, that given typical plant
input data, will predict within a reasonable tolerance the consequences. We should however
remember the advice by the early English theologian and philosopher, William of Occam (1285–
1349) who said: “What can be accounted for by fewer assumptions is explained in vain by more.”
This is now known as the principle of “Occam’s razor”, where with a razor we are encouraged to
cut all that is not strictly necessary away from our model. This motivates us to seek simple, low
order, low complexity, succinct, elegant models.

For the purposes of controller design we are interested in dynamic models as opposed to simply
curve fitting algebraic models. In most applications, we collect discrete data, and employ digital
controllers, so we are only really interested in discrete dynamic models.

Sections 3.4 and 3.4.3 revises traditional curve and model fitting such as what you would use in
algebraic model fitting. Section 6.7 extends this to fit models to dynamic data.

Good texts for for system identification include the classic [32] for time series analysis, [191], and
[125] accompanied with the closely related M ATLAB toolbox for system identification, [126]. For
identification applications in the processing industries, [129] contains the basics for modelling
and estimation suitable for those with a chemical engineering background. A good collection
is found in [149] and a now somewhat dated detailed survey of applications, including some
interesting practical hints and observations with over 100 references is given in [80].

6.1.1 B ASIC DEFINITIONS

When we formulate and use engineering models, we typically have in mind a real physical pro-
cess or plant, that is perhaps so complex, large, or just downright inconvenient, that we want to
use a model instead. We must be careful at all times to distinguish between the plant, model,
data and predictions. The following terms are adapted from [191].

System, S This is the system that we are trying to identify. It is the physical process or plant that
generates the experimental data. In reality this would be the actual chemical reactor, plane,
submarine or whatever. In simulation studies, we will refer to this as the “truth” model.
Model, M This is the model that we wish to find such that the model predictions generated by
M are sufficiently close to the actual observed system S output. Sometimes the model may
be non-parametric where it is characterised by a curve or a Bode plot for example, or the
model might be characterised by a finite vector of numbers or parameters, M(θ), such as
say the coefficients of a polynomial transfer function or weights in a neural network model.
In either case, we can think of this as a black-box model where if we feed in the inputs, turn
the handle, we will get output predictions that hopefully match the true system outputs.
Parameters, θ. These are the unknowns in the model M that we are trying to establish such
that our output predictions ŷ are as close as possible to the actual output y. In a transfer
function model, the parameters are the coefficients of the polynomials; in a neural network,
they would be the weights of the neurons.
6.1. THE IMPORTANCE OF SYSTEM IDENTIFICATION 237

This then brings us to the fundamental aim of system identification: Given sufficient input/out-
put data, and a tentative model structure M with free parameters θ, we want to find reasonable
values for θ such that our predictions ŷ closely follow the behaviour of our plant under test, y, as
shown in Fig. 6.2.

y
✲ Plant, S ✲

❄
input ✲ + ✲ error, ǫ
✻− (hopefully small)
✲ Model, M ✲
ŷ

Figure 6.2: A good model, M, duplicates the behaviour of the true plant, S.

Models can be classified as linear or nonlinear. In the context of parameter identification, which
is the theme of this chapter, it is assumed that the linearity (or lack of) is with respect to the
parameters, and not to the shape of the actual model output. A linear model does not mean just
a straight line, but includes all models where the model parameters θ, enter linearly. If we have
the general model
y = f (θ, x)
with parameters θ and independent variables x, then if ∂y/∂θ is a constant vector independent
of θ, then the model is linear in the parameters, otherwise the model is termed nonlinear. Linear
models can be written in vector form as

y = f (x)⊤ · θ

Just as linear models are easier to use than nonlinear models, so too are discrete models compared
with continuous models.

6.1.2 B LACK , WHITE AND GREY BOX MODELS

The two extreme approaches to modelling are sometimes termed black and white box models.
The black box, or data-based model is built exclusively from experimental data, and a deliberate
boundary is cast around the model, the underlying contents inside about which we neither know
nor care. The Wood-Berry column model described in Eqn. 3.24 is a black box model and the
parameters were obtained using the technique of system identification. The white box, theoretical
or first-principles model, preferred by purists, contains already well established physical and
chemical relations ideally so fundamental, no experiments are needed to characterise any free
parameters.

However most industrial processes are such that the models are often partly empirical and partly
fundamental. This intermediate stage lying between black and white is known as a grey-box
model. Ljung in [128] somewhat whimsically distinguishes between ‘palette’s of grey’. ‘Smoke-
grey’ models are those where we may try to find a suitable nonlinear transformation such that
the model is better described by a linear model. ‘Steel-grey’ models are those where we may
locally linearise about an operating point while ‘slate-grey’ models are those where a hybrid of
models is suitable.
238 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

An example of this type of model is the Van de Waals gas law or correlations for the friction factor
in turbulent flow. Finally if the process is so complex that no fundamental model is possible, then
a black box type model may be fitted. Particularly hopeless models, based mostly on whim and
what functions happened to be lying around at the time, can be found in [78]. These are the types
of the models that should have never been regressed.

Of course many people have argued for many years about the relative merits of these two extreme
approaches. As expected, the practising engineer, tends to have the most success somewhere in
the middle, which, not unexpectedly, are termed grey box models. The survey in [187] describes
the advantages and disadvantages of these ‘colour coded’ models in further details.

6.1.3 T ECHNIQUES FOR IDENTIFICATION

There are two main ways to identify process models: offline or online in real-time. In both cases
however, it is unavoidable to collect response data from the system of interest as shown in Fig. 6.3.
Most of the more sophisticated model identification is performed in the offline mode where all
the data is collected first and only analysed later to produce a valid model. This is typically
convenient since before any regressing is performed we know all the data, and we can do this in
comfort in front of our own computer workstation. The offline developed model (hopefully) fits
the data at the time that it was collected, but there is no guarantee that it will be still applicable for
data collected sometime in the future. Process or environmental conditions may change which
may force the process to behave unexpectedly.

Computer

✻ ✻
‘Dial-up’ a signal
input output
✻

❄
signal generator PI transducer

Plant
control valve

Figure 6.3: An experimental setup for input/output identification. We log both the input and the
response data to a computer for further processing.

The second way to identify models is to start the model regression analysis while you are col-
lecting the data in real-time. Clearly this is not a preferred method when compared with offline
identification, but in some cases we will study later, such as in adaptive control, or where we
have little input/output data at our immediate disposal, but are gradually collecting more, we
must use on-line identification. As you collect more data, you continually update and correct the
model parameters that were established previously. This is necessary when we require a model
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 239

during the experiment, but that the process conditions are likely to change during the period of
interest. We will see in chapter 7 that the online least squares recursive parameter estimation is
the central core of adaptive control schemes.

6.2 G RAPHICAL AND NON - PARAMETRIC MODEL IDENTIFICA -

TION

Dynamic model identification is essentially curve fitting as described in section 3.4 but for dy-
namic systems. For model based control, (our final aim in this section), it is more useful to have
parametric dynamic models. These are models which have distinct parameters to be estimated in
a specified structure. Conversely non-parametric models are ones where the model is described
by a continuous curve for example, such as say a Bode or Nyquist plots, or collections of models
where the model structure has not been determined beforehand, but is established from the data
in addition to establishing the parameters.

In both parametric and non-parametric cases we can

make predictions, which is a necessary requirement of
any model, but it is far more convenient in a digital com-
puter to use models based on a finite number of param-
eters rather than resort to some sort of table lookup or
digitise a non-parametric curve.

How one identifies the parameters in proposed models

from experimental data is mostly a matter of preference.
There are two main environments in which to identify
models;
Perturb the process, and watch what
happens
1. Time domain analysis described in §6.2.1, and

2. frequency domain analysis described in §6.2.2.

6.2.1 T IME DOMAIN IDENTIFICATION USING GRAPHICAL TECHNIQUES

Many engineers tend to be more comfortable in the time domain, and with the introduction of
computers, identification in the time domain, while tedious, is now reliable and quite straight
forward. The underlying idea is to perturb the process somehow, and watch what happens.
Knowing both the input and output signals, one can in principle, compute the process transfer
function. The principle experimental design decision is the type of exciting input signal.

S TEP AND IMPULSE INPUTS

Historically, the most popular perturbation test for visualisation and manual identification was
the ‘step test’. The input step test itself is very simple, and the major response types such as first
order, underdamped second order, integrators and so forth are all easily recognisable from their
step tests by the plant personnel as illustrated in Fig. 6.4.

Probably the most common model structure for industrial plants is the first-order plus deadtime
240 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Simple first order 1st order with deadtime 2nd order underdamped

1.5
1 1

0.8 0.8
Amplitude

Amplitude

Amplitude
1
0.6 0.6
1 1e − 4s
0.4 s+1 0.4 s+1 1
0.5 s 2 +0.8s+1
0.2 0.2

0 0 0
0 2 4 6 8 0 5 10 15 0 5 10 15
Time (seconds) Time (seconds) Time (seconds)

Pure integrator Higher (3rd) order Inverse response

50 2
1
40 1.5
0.8
Amplitude

Amplitude

Amplitude
1
30 0.6
1 −6s+2
0.5
20
s 0.4 1 3s 2 +4s+1
6s 3 +11s 2 +6s+1 0
0.2
10
−0.5
0
0 −1
0 10 20 30 40 0 5 10 15 20 25 0 5 10 15 20 25
Time (seconds) Time (seconds) Time (seconds)

Figure 6.4: Typical open loop step tests for a variety of different standard plants.

model, (FOPDT),
K
Gp (s) = e−θs (6.1)
τs + 1
where we are to extract reasonable values for the three parameters: plant gain K, time constant
τ , and deadtime θ. There are a number of strategies to identify these parameters, but one of the
more robust is called the “Method of Areas”, [121, pp32–33].

Algorithm 6.1 FOPDT identification from a step response using the Method of Areas.

Collect some step response data (u, y) from the plant, then

1. Identify the plant gain,

y(∞) − y(0)
K=
u(∞) − u(0)
and normalise the experimental data to obtain a unit step

y − y(0)
yus =
K

2. Compute the area Z ∞

A0 = (K − yus (t)) dt (6.2)
0

3. Compute the time t1 = A0 /K, and then compute a second area

Z t1
A1 = yus (t) dt (6.3)
0

as illustrated in Fig. 6.5.

6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 241

4. Now the two remaining parameters of the proposed plant model in Eqn. 6.1 are
eA1 A0 − eA1
τ= and θ= (6.4)
K K
where e = exp(1). Note that we must constrain that the deadtime is positive for causal
plants.

1.2

0.8
A0
Output

0.6

0.4

0.2
A1

0 Figure 6.5: Areas method for a

first-order plus time delay model
0 10 20 30 40 50
time identification.

Listing 6.1 gives a simple M ATLAB routine to do this calculation and Fig. 6.5 illustrate the inte-
grals to be computed. Examples of the identification is given in Fig. 6.6 for a variety of simulated
plants (1st and 2nd order, and an integrator) with different levels of measurement noise super-
imposed. Fig. 6.7 shows the experimental results from a step test applied to the blackbox filter
with the identified model.

Listing 6.1: Identification of a first-order plant with deadtime from an openloop step response
using the Areas method from Algorithm 6.1.
Gp = tf(3, [7 2 1],'iodelay',2); % True plant: Gp (s) = 3e−2s /(7s2 + 2s + 1)
[y,t] = step(Gp); % Do step experiment to generate data, see Fig. 6.5.

4 % Calculate areas A0 & A1 as shown in Fig. 6.5.

K = y(end); % plant gain, K
A0 = trapz(t,K-y);
t0 = A0/K;
idx = find(t<t0);
9 t1 = t(idx); y1 = y(idx);
A1 = trapz(t1,y1);

% Create process model Ĝp (s) = Ke−θs /(τ s + 1)

tau = exp(1)*A1/K; % time constant, τ
14 Dest = max(0,(A0-exp(1)*A1)/K); % deadtime, θ, must be ≥ 0
Gm = tf(K,[tau 1],'iodelay',Dest);

step(Gp,Gm) % Compare model with plant, see Fig. 6.5.

Of course not all plants are adequately modelled using a first-order plus deadtime model. For
example, Fig. 6.8 shows the experimental results from a step test applied to the electromagnetic
balance arm which is clearly second order with poles very close to the imaginary axis.

However many plant operators do not appreciate the control engineer attempting to “step test”
the plant under open loop conditions since it necessarily causes periods of off-specification prod-
uct at best and an unstable plant at worst. An alternative to the step test is the impulse test where
242 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

8 8

6 6

Output

Output
4 4

2 2

0 Plant 0 Plant
Model Model
−2 −2
0 20 40 60 0 50 100 150
time time

8 6
Plant
6 4 Model

Output
Output

4
2
2
Figure 6.6: The “Areas method” for 0
0 Plant
plant identification applied to a va- Model
riety of different plants with random −2
0 5 10 15 20
−2
0 0.5 1 1.5 2
disturbances. time time

600

400
Output

200

Blackbox
0 Model

1000

Figure 6.7: Identification of the experimental 500

Blackbox using the Method of Areas. Note
that the identification strategy is reasonably 0
robust to the noise and outlier in the experi- 0 5 10 15 20 25
mental data. time

the plant will also produce off-spec product for a limited period,1 but at least it returns to the
same operating point even using limited precision equipment. This is important if the plant ex-
hibits severe nonlinearities and the choice of operating point is crucial. The impulse response is
ideal for frequency domain testing since it contains equal amounts of stimulation at all frequen-
cies, but is difficult to approximate in practice. Given that we then must accept a finite height
pulse with finite slope, we may be limited as to the amount of energy we can inject into the sys-
tem without hitting a limiting nonlinearity. The impulse test is more difficult to analyse using
manual graphical techniques (pencil and graph paper), as opposed to computerised regression
techniques, than the step test.

1 Provided the plant does not contain an integrator

6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 243

3000

2000
Output

1000

−1000
2000
Input

1000 Figure 6.8: Two step tests subjected to the bal-

ance arm. Note the slow decay of the oscilla-
0 tions indicate that the plant has at least two
0 5 10 15 complex conjugate poles very close to the sta-
time ble half of the imaginary axis.

R ANDOM INPUTS

An alternative to the step or impulse tests is actually not giving any deliberate input to the plant
at all, but just relying on the ever-present natural input disturbances. This is a popular approach
when one wants still to perform an identification, but cannot, or is unable to influence the system
themselves. Economists, astronomers and historians fall into this group of armchair-sitting, non-
interventionist identificationalists. This is sometimes referred to as time series analysis (TSA), see
[32] and [54].

Typically for this scheme to work in practice, one must additionally consciously perturb the input
(control valve, heating coils etc) in some random manner, since the natural disturbance may not
normally be sufficient in either magnitude, or even frequency content by itself. In addition, it is
more complex to check the quality of model solution from comparison with the raw time series
analysis. The plant model analysed could be anything, and from a visual inspection of the raw
data it is difficult to verify that the solution is a good one. The computation required for this type
of analysis is more complex than graphically curve fitting and the numerical algorithms used for
time series analysis are much more sensitive to the initial parameters. However using random
inputs has the advantage that they continually excite or stimulate the process and this excitation,
provided the random sequence is properly chosen, stimulates the process over a wide frequency
range. For complicated models, or for low signal to noise ratios, lengthy experiments may be
necessary, which is infeasible using one-off step or impulse tests. This property of continual
stimulation is referred to as persistent excitation and is a property of the input signal and should
be considered in conjunction with the proposed model. See [18, §2.4] for further details.

Two obvious choices for random sequences are selecting the variable from a finite width uniform
distribution, or from a normal distribution. In practice, when using a normal distribution, one
must be careful not to inject values far from the desired mean input signal, since this could cause
additional unwanted process upsets.

R ANDOM BINARY SIGNALS

Electrical engineers have traditionally used random signals for testing various components where
the signal is restricted to one of two discrete signal levels. Such a random binary signal is called
244 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

an RBS. A binary signal is sometimes preferred to the normal random signal, since one avoids
possible outliers, and dead zone nonlinearities when using signals with very small amplitudes.
We can generate an RBS in M ATLAB using the random number generator and the signum func-
tion, sign, as suggested in [125, p373]. Using the signum function could be dangerous since
technically it is possible to have three values, −1, 1 and zero. To get different power spectra of
our binary test signal, we could filter the white noise before we “sign” it.

x = randn(100,1); % white noise

rbsx = 0.0 ≤ x; % binary signal
3

y = filter(0.2,[1 -0.8],x); % coloured noise

rbsy = 0.0 ≤ y; % binary value

Compare in Fig. 6.9 the white noise (left plots) with the filtered white noise through a low-pass
filter (solid). The filter used in Fig. 6.9 is 0.2/(1 − 0.8q −1 ).

White noise Coloured noise

4 1

2 0.5

0 0

−2 −0.5

−4 −1

Random binary noise Random binary coloured noise

1 1

0.5 0.5

0 0
0 50 100 0 50 100

Figure 6.9: Comparing white and filtered (coloured) random signals. Upper: Normal signals,
lower: Two-level or binary signals.

If you fabricate in hardware one of these random signal generators using standard components
such as flip-flops or XOR gates, rather than use a process control computer, then to minimise
component cost, you want to reduce the number of these physical components. This is done in
hardware using a linear feedback shift register (LFSR), which is an efficient approximation to our
(almost random) random number generator. Fig. 6.10 shows the design of one version which
uses a 5 element binary shift register. Initially the register is filled with all ones, and at each clock
cycle, the elements all shift one position to the right, and the new element is formed by adding
the 3rd and 5th elements modulus 2.

Since we have only 5 positions in the shift register in Fig. 6.10, the maximum length the sequence
can possibly be before it starts repeating is 25 − 1 or 31 elements. Fig. 6.11 shows a S IMULINK
implementation and the resulting binary sequence which repeats every 31 samples.

Random number generators such as Fig. 6.10 are completely deterministic so are called pseudo
random binary generators since in principal anyone can regenerate the exact sequence given
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 245

binary operation
+✛
✻

✲ 1 1 1 1 1 ✲ output

| {z }
5-element shift register

Figure 6.10: A 5-element binary shift register to generate a pseudo-random binary sequence.

Math
Function Sum

mod

Constant
2

1 1 1 1 1
z z z z z
Unit Delay1 Unit Delay2 Unit Delay3 Unit Delay4 Unit Delay5 Scope

(a) S IMULINK block diagram

Pseudo random binary sequence of length 31

1.2

0.8

0.6

0.4

0.2

−0.2
0 10 20 30 40 50 60 70
sample #

(b) S IMULINK simulation output

Figure 6.11: S IMULINK simulation of a 5-element binary shift register (upper) and resulting out-
put (lower). Note the period of 31 samples. Refer also to Fig. 6.10.

the starting point since the construction in Fig. 6.10. RBS can be used as alternatives to our
random sequences for identification. These and other commonly used test and excitation signals
are compared and discussed in [204, p143].

6.2.2 E XPERIMENTAL FREQUENCY RESPONSE ANALYSIS

When first faced with an unknown plant, it is often a good idea to perform a non-parametric model
identification such as a frequency response analysis. Such information, quantified perhaps as a
Bode diagram, can then be used for subsequent controller design, although is not suitable for
direct simulation. For that we would require a transfer function or equivalent, but the frequency
response analysis will aid us in the choice of appropriate model structure if we were to regress
246 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

further a parametric model.

The frequency response of an unknown plant G(s) can be calculated by substituting s = iω and
following the definition of the Fourier transform, one gets
R∞
Y (iω)
def y(t)e−iωt dt
G(iω) = = R0∞ (6.5)
X(iω) 0 x(t)e
−iωt dt

For every value of frequency, 0 < ω < ∞, we must approximate the integrals in Eqn. 6.5. If we
choose the input to be a perfect Dirac pulse, then the denominator in Eqn. 6.5 will always equal
1. This simplifies the computation, but complicates the practical aspects of the experiment. If
we use arbitrary inputs, we must compute the integrals numerically, although they are in fact
simply the Fourier transform of y(t) and x(t) and can be computed efficiently using the Fast
Fourier Transform or FFT.

We will compare three alternative techniques to compute the frequency response of an actual
experimental laboratory plant, namely:

1. subjecting the plant to a series of sinusoids of different frequencies,

2. subjecting the plant to a single sinusoid with a time-varying frequency component (chirp
signal),

3. and subjecting a plant to a pseudo-random white noise input.

In the three alternatives, we must record (or compute) the steady-state amplitude ratio of the
output over the input and the phase lag of the output compared with the input for different
input frequencies.

Finally, to extract estimates of the model parameters from the frequency data, you could use the
M ATLAB functions invfreqs or the equivalent for the discrete domain, invfreqz. I have no
idea how “robust” this fitting works, and I would treat the results very cautiously at first. (Also
refer to problem 6.1.)

A SERIES OF INPUT SINE WAVES

The simplest way to experimentally characterise the frequency response of a plant is to input a
series of pure sine waves at differing frequencies as shown in in the S IMULINK diagram Fig. 6.12
which uses the real-time toolbox to communicate with the experimental blackbox.

Terminator uy
Simulink
Execution To Workspace
Control Sine Wave

DAC ADC

Saturation Scope
MCC DAQ DAC OUT MCC DAQ ADC IN

Maximum real time sampling rate is around 0.1s

Figure 6.12: Using the real-time toolbox in M ATLAB to subject sine waves to the blackbox for an
experimental frequency analysis.
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 247

Fig. 6.13 shows the response of the laboratory blackbox when subjected to 12 sine waves of in-
creasing frequency spanning from ω = 0.2 to 6 radians/second. The output clearly shows a re-
duction in amplitude at the higher frequencies, but also a small amplification at modest frequences.
By carefully reading the output amplitude and phase lag, we can establish 12 experimental points
given in table 6.1 suitable say for a Bode diagram.

ω = 0.2 ω = 0.3 ω = 0.5

2

−2

ω = 0.8 ω = 1.0 ω = 1.3

2

−2

ω = 1.5 ω = 2.0 ω = 2.5

2

−2
0 5 10 15 20 25

ω = 3.0 ω = 4.0 ω = 6.0

2

−2
0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25

Figure 6.13: The black box frequency response analysis using a series of sine waves of different
frequencies. This data will be subsequently processed and is tabulated in Table 6.1 and plotted
in Fig. 6.20.

Table 6.1: Experimentally determined frequency response of the blackbox derived from the labo-
ratory data shown in Fig. 6.13. For a Bode plot of this data, see Fig. 6.20.

Input frequency Amplitude ratio Phase lag

ω (rad/s) – φ (degrees)
0.2 1.19 -4.8
0.3 1.21 -5.1
0.5 1.27 -11.1
0.8 1.42 -20.8
1.0 1.59 -27.6
1.3 1.92 -46.7
1.5 2.08 -64.4
2.0 1.61 -119.2
2.5 0.88 -144.3
3.0 0.54 -156.8
4.0 0.27 -172.6
6.0 0.11 -356.1

However while conceptually simple, the sine wave approach is not efficient nor practical. Lees
and Hougen, [119], while attempting a frequency response characterisation of a steam/water
heat exchanger noted that it was difficult to produce a good quality sine wave input to the plant
248 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

when using the control valve. In addition, since the ranges of frequencies spanned over 3.5 orders
of magnitude (from 0.001 rad/s to 3 rad/s), the testing time was extensive.

Additional problems of sine-wave testing are that the gain and phase lag is very sensitive to
bias, drift and output saturation. In addition it is often difficult in practice to measure accurately
phase lag. [143, pp117-118] give further potential problems and some modifications to lessen the
effects of nonlinearities. The most important conclusion from the Lees and Hougen study was
that they obtained better frequency information, with much less work using pulse testing and
subsequently analysing the input/output data using Fourier transforms.

A CHIRP SIGNAL

If we input a signal with a time varying frequency component such as a ‘chirp signal’, u(t) =
sin(ωt2 ), to an unknown plant, we can obtain an idea of the frequency response with a single,
albeit long drawn-out experiment unlike the series of experiments required in the pure sinusoidal
case.

Fig. 6.14 shows some actual input/output data collected from the ‘blackbox’ laboratory plant
collected at T = 0.1 seconds. As the input frequency is increased, the output amplitude decreases
as expected for a plant with low-pass filtering characteristics. However the phase lag information
is difficult to determine at this scale from the plot given in Fig. 6.14. Our aim is to quantify this
amplitude reduction and phase lag as a function of frequency for this experimental plant.

Experimental Blackbox
1

0.8

0.6
output

0.4

0.2

0
0 50 100 150 200 250 300 350

−0.2

−0.4
input

−0.6

−0.8

−1
0 50 100 150 200 250 300 350
time (s)

Figure 6.14: Input (lower) /output (upper) data collected from the black box where the input is a
chirp signal.

Since the phase information is hard to visualise in Fig. 6.14, we can plot the input and output sig-
nals as a phase plot similar to a Lissajou figure. The resulting curve must be an ellipse (since the
input and output frequencies are the same), but the ratio of the height to width is the amplitude
ratio, and the eccentricity and orientation is a function of the phase lag.

If we plot the start of the phase plot using the same input/output data as Fig. 6.14, we should see
the trajectory of an ellipse gradually rotating around, and growing thinner and thinner. Fig. 6.15
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 249

shows both the two-dimensional phase plot and a three dimensional plot with the time axis
added.
Phase plot of the experimental Blackbox
1 Phase plot of the experimental Blackbox

Start
1
0.9

0.9
0.8
0.8
0.7
0.7

0.6 0.6

output (V)
output (V)

0.5 0.5

0.4
0.4

0.3
0.3
0.2

0.2
0.1
0
0.1
−0.5
350 400
250 300
150 200
0 −1 50 100
0
−1 −0.9 −0.8 −0.7 −0.6 −0.5 −0.4 −0.3 −0.2 −0.1 0 input (V)
input (V) time (s)

(a) 2D input/output phase plot (b) 2D input/output phase plot with time in the third di-
mension

Figure 6.15: Phase plot of the experimental black box when the input is a chirp signal. See also
Fig. 6.14.

Fig. 6.16 shows the same experiment performed on the flapper. This experiment indicates that the
flapper has minimal dynamics since there is little discernable amplitude attenuation and phase
lag but substantial noise. The saturation of the output is also seen in the squared edges of ellipse
in the phase plot in addition to the flattened sinusoids of the flapper response.

Experimental Flapper Phase plot of the experimental Flapper

0.5 0.5

0.4

0.3 0.4
0.2
output

0.1
0.3
0

−0.1

−0.2 0.2 Start

output (V)

0 50 100 150 200 250 300 350 400

0 0.1

−0.2
0
−0.4
input

−0.6
−0.1

−0.8

−1 −0.2
0 50 100 150 200 250 300 350 400 −1 −0.9 −0.8 −0.7 −0.6 −0.5 −0.4 −0.3 −0.2 −0.1 0
time (s) input (V)

(a) Time response (b) Phase plot of the experimental flapper

Figure 6.16: Flapper response to a chirp signal. There is little evidence of any appreciable dynam-
ics over the range of frequencies investigated.

A PSEUDO RANDOM INPUT

A more efficient experimental technique is to subject the plant to an input signal that contains
a wide spectrum of input frequencies, and then compute the frequency response directly. This
has the advantage that the experimentation is considerably shorter, potentially processes the data
more efficiently and removes some of the human bias.
250 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

The experimental setup in Fig. 6.17 shows an unknown plant subjected to a random input. If we
collect enough input/output data pairs, we should be able to estimate the frequency response of
the unknown plant.

y & spt

1
UY

Figure 6.17: Experimental setup to subject a

random input into an unknown plant. The
G
input/output data was collected, processed
through Listing 6.2 to give the frequency re- Band−Limited Unknown plant
sponse shown in Fig. 6.18. White Noise

Fig. 6.18 compares the magnitude and phase of the ratio of the Fourier transforms of the input
and output to the analytically computed Bode plot given that the ‘unknown’ plant is in fact
G(s) = e−2s /(5s2 + s + 6).

0
10

−1
10

−2
10
|G(i ω)|

−3
10

−4
10

−5
10

0
Phase angle [degrees]

−200

−400
Figure 6.18: The experimental frequency
response compared to the true analytical −600

Bode diagram. See the routine in List- 10

−3
10
−2
10
−1
10
0 1
10
2
10
fN
ing 6.2. frequency ω [rad/s]

The actual numerical computations to produce the Bode plot in Fig. 6.18 are given in Listing 6.2.
While the listing looks complex, the key calculation is the computation of the fast Fourier trans-
form, and the subsequent division of the two vectors of complex numbers. It is important to note
that S IMULINK must deliver equally spaced samples necessary for the subsequent FFT transfor-
mation, and that the frequency vector is equally spaced from 0 to half the Nyquist frequency.

Listing 6.2: Frequency response identification of an unknown plant directly from input/output
data
G = tf(1,[5 1 6],'iodelay',2); % ‘Unknown’ plant under test
Ts = 0.2; timespan = [0 2000]; % sample time, Ts , & period of interest
sl_optns = simset('RelTol',1e-5,'AbsTol',1e-5, ...
'MaxStep',1e-2,'MinStep',1e-3);
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 251

5 [t,x,UY] = sim('splant_noiseIO',timespan,sl_optns); % Refer Fig. 6.17.

n = ceil((pow2(nextpow2((length(t)))-1)+1)/2);
t = t(1:n); % discard down to a power of 2 for the FFT
u = UY(1:n,1); y = UY(1:n,2);
10

Fs = 1/Ts; Fn = Fs/2; % sampling & Nyquist frequency [Hz]

Giw = fft(y)./fft(u); % G(iω) = FFT (Y )/FFT (U )
w=2*pi*(0:n-1)*2*Fn/n; % frequency ω in [rad/s]

15 [Mag,Phase,wa] = bode(G);% Analytical Bode plot for comparison

Mag = Mag(:); Phase = Phase(:);

subplot(2,1,1); % Refer Fig. 6.18.

loglog(w,abs(Giw),wa,Mag,'r-'); ylabel('|G(i \omega)| ')
20 subplot(2,1,2);
semilogx(w,phase(Giw)*180/pi, wa,Phase,'r-');
ylim([-720,10])
ylabel('Phase angle [degrees]'); xlabel('frequency \omega [rad/s]');

We can try the same approach to find the frequency response of the blackbox experimentally.
Fig. 6.19 shows part of the 213 = 8192 samples collected at a sampling rate of 10Hz or T = 0.1 to
be used to construct the Bode diagram.

4
output

0
4

3
input

0 50 100 150
time [s]

Figure 6.19: Part of the experimental black box response data series given a pseudo-random input
sequence. This data was processed to produce the frequency response given in Fig. 6.20.

The crude frequency response obtained simply by dividing the two Fourier transforms of the
input/output data from Fig. 6.19 is given in Fig. 6.20 which, for comparison, also shows the
frequency results obtained from a series of distinct sine wave inputs and also from using the
M ATLAB toolbox command etfe which will be subsequently explained in section 6.2.3.

Since we have not employed any windows in the FFT analysis, the spectral plot is unduly noisy,
especially at the high frequencies. The 12 amplitude ratio and phase lag data points manually
read from Fig. 6.13 (repeated in Table 6.1) show good agreement across the various methods.
252 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

0
10
|G(i ω)|

−1
10

50

0
Phase angle [degrees]

−50

−100

−150

−200
FFT
−250 etfe
sine waves
−300
−2 −1 0 1 2
10 10 10 10 10
frequency ω [rad/s]

Figure 6.20: The frequency response of the black box computed using FFTs (cyan line), the Sys-
tem ID toolbox function etfe, (blue line) and the discrete sine wave input data, (✷), read from
Fig. 6.13. The FFT analysis technique is known to give poor results at high frequencies.

6.2.3 AN ALTERNATIVE EMPIRICAL TRANSFER FUNCTION ESTIMATE

The frequency response obtained from experimental input/output data shown in Fig. 6.18 and
particularly Fig. 6.20 illustrate that the simple approach of simply numerically dividing the
Fourier transforms is suspect to excessive noise. A better approach is to take more care in the
data pre-processing stage by judicious choice of windows. The empirical transfer function esti-
mate routine, etfe, from the System Identification toolbox does this and delivers more reliable
and consistent results. Compare the result given in Fig. 6.21 from Listing 6.3 with the simpler
strategy used to generate Fig. 6.18.

Listing 6.3: Non-parametric frequency response identification using etfe.

G = tf(1,[5 1 6],'iodelay',2); % Plant under test: G(s) = e−2s /(5s2 + s + 6)
2 Ts = 0.2; t = [0:Ts:2000]'; % sample time & span of interest

u = randn(size(t)); y = lsim(G,u,t);

Dat = iddata(y,u,Ts); % collect data from simulation experiment

7 Gmodel = etfe(Dat); % estimate a frequency response model
bode(Gmodel) % Refer Fig. 6.21.
6.2. GRAPHICAL AND NON-PARAMETRIC MODEL IDENTIFICATION 253

0
10
|G(i ω)|

−1
10

Estimated
Actual
−2
10
0

−200
φ [deg]

Figure 6.21: An empirical transfer function

−400 estimate from experimental input/output
data computed by etfe. The true fre-
−600
quency response is overlayed for compar-
0.3 0.5 1 2 3
ison. This is preferred over the simpler
ω [rad/s] strategy used to generate Fig. 6.18.

Problem 6.1 1. Plot the frequency response of an “unknown” process on both a Bode diagram
and the Nyquist diagram by subjecting the process to a series of sinusoidal inputs. Each
input will contribute one point on the Bode or Nyquist diagrams. The “unknown” transfer
function is
0.0521s2 + 0.4896s + 1
G(s) =
0.0047s4 + 0.0697s3 + 0.5129s2 + 1.4496s + 1
Compare your plot with one generated using bode. Note that in practice we do not know
the transfer function in question, and we construct a Bode or Nyquist plot to obtain it. This
is the opposite of typical textbook student assignments.
Hint: Do the following for a dozen or so different frequencies:
Gcn=[0.0521 0.4896 1];
(a) First specify the “unknown” continuous time Gcd=[0.0047 0.0697 0.5129 1.4496 1];
transfer function. w=2; % rad/s
(b) Choose a test frequency, say ω = 2 rad/s, and t=[0:0.1:20]’; % seconds
generate a sine wave at this frequency. u=sin(w*t); % input signal
y=lsim(Gcn,Gcd,u,t); plot(t,[u y])
(c) Simulate the output of the process given this
input. Plot input/output together and read off
the phase lag and amplitude ratio.

Note that the perfect frequency response can be obtained using the freqs command.

2. The opposite calculation, that is fitting a transfer function to an experimentally obtained

frequency response is trivial with the invfreqs command. Use this command to recon-
struct the polynomial coefficients of G(s) using only your experimentally obtained frequency
response.

Problem 6.2 Write a M ATLAB function file to return Gp (iω) given the input/output x(t), y(t) us-
ing Eqn6.5 for some specified frequency. Test your subroutine using the transfer function from
Problem 6.1 and some suitable input signal, and compare the results with the exact frequency
response.
Hint: To speed up the calculations, you should vectorise the function as much as possible. (Of
course for the purposes of this educational example, do not use an FFT)
254 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

6.3 C ONTINUOUS MODEL IDENTIFICATION

Historically graphical methods, especially for continuous time domain systems, used to be very
popular partly because one needed little more than pencil and graph paper. For first-order or for
some second-order models common in process control, we have a number of ‘graphical recipes’
which if followed, enable us to extract reasonable parameters. See [182, Chpt 7] for further details.

While we could in principle invert the transfer function given a step input to the time domain
and then directly compare the model and actual outputs, this has the drawback that the solutions
are often convoluted nonlinear functions of the parameters. For example, the solutions to a step
response for first and second order overdamped systems are:
K  
⇐⇒ y(t) = K 1 − e−t/τ
τs + 1
 
K τ1 e−t/τ1 − τ2 e−t/τ2
⇐⇒ y(t) = K 1 −
(τ1 s + 1)(τ2 s + 1) τ1 − τ2
and the problem is that while the gain appears linearly in the time domain expressions, the time
constants appear nonlinearly. This makes subsequent parameter estimation difficult, and in gen-
eral we would need to use a tool such as the O PTIMISATION toolbox to find reasonable values for
the time constants. The following section describes this approach.

6.3.1 F ITTING TRANSFER FUNCTIONS USING NONLINEAR LEAST- SQUARES

Fig. 6.22 shows some actual experimental data collected from a simple “black box” plant sub-
jected to a low-frequency square wave. The data was sampled at 10Hz or Ts = 0.1. You can
notice that in this case the noise characteristics seems to be clearly dependent on the magnitude
of y. By collecting this input/output data we will try to identify a continuous transfer function.

3.5

3
output

2.5

1.5
3

2.5

2
0 10 20 30 40 50 60 70 80 90 100
time (s)

Figure 6.22: Experimental data from a continuous plant subjected to a square wave.

We can see from a glance at the response in Fig. 6.22 that a suitable model for the plant would be
an under-damped second order transfer function
Ke−as
G(s) = (6.6)
τ 2 s2 + 2ζτ s + 1
def  T
where we are interested in estimating the four model parameters of Eqn. 6.6 as θ = K τ ζ a .
This is a nonlinear least-squares regression optimisation problem we can solve using lsqcurvefit.
Listing 6.4 will be called by the optimisation routine in order to generate the model predictions
for a given trial set of parameters.
 6.3. CONTINUOUS MODEL IDENTIFICATION 255

Listing 6.4: Function to generate output predictions given a trial model and input data.
function ypred = fmodelsim(theta, t,u)
2 % Compute the predicted output, ŷ(t), given a model & input data, u(t).
K = theta(1); tau = theta(2); zeta = theta(3); deadt = theta(4);

Ke−as
G = tf(K,[tauˆ2 2*tau*zeta, 1],'iodelay',deadt); % G(s) = τ 2 s2 +2ζτ s+1
ypred = lsim(G,u,t);
7 return

The optimisation routine in Listing 6.5 calls the function given in Listing 6.4 repeatedly in order
to search for good values for the parameters such that the predicted outputs match the actual
outputs. In parameter fitting cases such as these, it is prudent to constrain the parameters to be
non-negative. This is particularly important in the case of the deadtime where we want to ensure
that we do not inadvertently try models with acausal behaviour.

Listing 6.5: Optimising the model parameters.

K0 = 1.26; tau0 = 1.5; zeta0 = 0.5; deadt0 = 0.1; % Initial guesses for θ 0
theta0 = [K0,tau0,zeta0,deadt0]';
3 LowerBound = zeros(size(theta0)); % Avoid negative deadtime
theta_opt = lsqcurvefit(@(x,t) fmodelsim(x,t,u), theta0,t,y,LowerBound)

Once we have some optimised values, we should plot the predicted model output on top of the
actual output as shown in Fig. 6.23 which, in this case, is not too bad.

Listing 6.6: Validating the fitted model.

5 K = theta_opt(1); tau = theta_opt(2);
zeta = theta_opt(3); deadt = theta_opt(4);
Gopt = tf(K,[tauˆ2 2*tau*zeta, 1],'iodelay',deadt);

ypred = lsim(Gopt,u,t);
10 plot(t,y,t,ypred,'r-') % See results of the comparison in Fig. 6.23.

0.5

−0.5
output

−1

−1.5
Actual 1.166
fitted
G(s) = 0.5712 s2 +0.317s+1
−2
0 10 20 30 40 50 60 70 80 90 100
time (s)

Figure 6.23: A continuous-time model fitted to input/output data from Fig. 6.22.
256 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

The high degree of similarity between the model and the actual plant shown in Fig. 6.23 suggests
that, with some care, this method does work in practice using real experimental data complete
with substantial noise. However on the downside, we did use a powerful nonlinear optimiser,
we were careful to eliminate the possibility any of the parameters could be negative, we chose
sensible starting estimates, and we used nearly 1000 data points.

6.3.2 I DENTIFICATION OF CONTINUOUS MODELS USING SUCCESSIVE DIF -

FERENTIATION

While most modern system identification and model based control is done in the discrete do-
main, it is possible, as shown above, to identify the parameters in continuous models, G(s),
directly. This means that we can establish the continuous time poles and zeros of a transfer func-
tion without first identifying a discrete time model, and then converting back to the continuous
time domain.

However there are two main reasons why continuous time model identification is far less use-
ful than its discrete time equivalent. The first reason is that since discrete time models are more
useful for digital control, it makes sense to directly estimate them rather than go through a tem-
porary continuous model. The second drawback is purely practical. Either we must resort to
the delicate task of applying nonlinear optimisation techniques such as illustrated above, or if
we wish to use the robust linear least-squares approach, we need reliably to estimate high order
derivatives of both the input and output data. In the presence of measurement noise and possible
discontinuity in gradient for the input such as step changes, it is almost impossible to construct
anything higher than a second-order derivative, thus restricting our models to second order or
less. Notwithstanding these important implementation issues, [29] describes a recursive-least
squares approach for continuous system identification.

As continuous time model identification requires one to measure not only the input and output,
but also the high order derivatives of y(t) and u(t), it turns out that this construction is delicate
even for well behaved simulated data, so we would anticipate major practical problems for actual
experimental industrial data.

If we start with the continuous transfer function form,

B(s)
Y (s) = U (s)
A(s)

then by rationalising and expanding out the polynomials in s we get

an sn + an−1 sn−1 + · · · + a1 s + 1 Y (s) = bm sm + bm−1 sm−1 + · · · + b1 s + b0 U (s)

Inverting to the time domain and assuming zero initial conditions, we get a differential equation

an y (n) + an−1 y (n−1) + · · · + a1 y (1) + y = bm um + bm−1 u(m−1) + · · · + b1 u(1) + b0 u

where the notation y (n) means the n-th time derivative of y(t). This can be written with some
6.3. CONTINUOUS MODEL IDENTIFICATION 257

rearrangement as a vector dot product

 
an

 an−1 

 .. 

 . 

 a1
i
h 
.. 
y= ··· u  ··· (6.7)
 
−y (n) −y (n−1) · · · −y (1) . u(m) u(m−1) 
 bm 
 
 bm−1 
 
 . 
 .. 
b0

which is now in a form suitable for least-squares regression of the vector of unknown coefficients
of the A and B polynomials. Note that if we desire na poles, we must differentiate the output na
times. Similarly, we must differentiate the input once for each zero we want to estimate. A block
diagram of this method is given in Fig. 6.24.

parameters, θ

differentiator ✻ differentiator
✲ ✛
✲ ✛
si ✲ ✛ si
✲ regress ✛
✲ ✛
✻ ✻

u ✲ Plant ✲ y

| {z }
plant under investigation

Figure 6.24: Continuous model identification strategy

A simple simulation is given in Listing 6.7. I first generate a “truth” model with four poles and
three zeros. Because I want to differentiate the input, I need to make it sufficiently ‘smooth’ by
passing a random input through a very-low-pass filter. To differentiate the logged data I will use
the built-in gradient command which approximates the derivative using finite differences. To
obtain the higher-order differentials, I just repeatedly call gradient. With the input/output data
and its derivatives, the normal equations are formed, and solved for the unknown polynomial
coefficients.

Listing 6.7: Continuous model identification of a non-minimum phase system

Gplant = tf(3*poly([3,-3,-0.5]), ...
3(s−3)(s+3)(s+0.5)
poly([-2 -4 -7 -0.4])); % G(s) = (s+7)(s+4)(s+2)(s+0.4)

4 % Design smooth, differentiable input

dt = 5e-2; t = [0:dt:10]'; % Keep sample time T small
Ur = randn(size(t)); % white noise
[Bf,Af]= butter(4,0.2); u= filter(Bf,Af,Ur); % Input now smooth
258 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

9 y = lsim(Gplant,u,t); plot(t,[u,y]); % do input/output experiment

% Now do the identification . . .

na = 2; nb = 1; % # of poles & zeros to be estimated. (Keep small).
dy = y; % Zeroth derivative, y (0) = y
14 for i=1:na % build up higher derivatives recursively, y (1) , y (2) , . . .
dy(:,i+1) = gradient(dy(:,i),t); % should be correctly scaled
end % for
du = u; % do again for input signal
for i=1:nb % but only for nb times
19 du(:,i+1) = gradient(du(:,i),t); % could use diff()
end % for

b = y; % RHS
X = [-dy(:,2:na+1), du]; % data matrix
24 theta = X\b; % solve for parameters, θ

% re-construct linear model polynomials

Gmodel = tf(fliplr(theta(na+1:length(theta))'), ...
fliplr([1,theta(1:na)']))
29

yp = lsim(Gmodel,u,t); plot(t,[y],t,yp,'--'); % test simulation

Even though we know the truth model possessed four zeros and three poles, our fitted model
with just two poles and one zero gave a close match as shown in Fig. 6.25. In fact the plant output
(heavy line) is almost indistinguishable from the predicted output (light line) and this is due to
the lack of noise and smoothed input trajectory. However if you get a little over ambitious in
your identification, the algorithm will go unstable owing to the difficulty in constructing reliable
derivatives.

0.6
Plant
0.4 Model

0.2
output

−0.2

−0.4
1

0
input

−1

−2
0 1 2 3 4 5 6 7 8 9 10
time

Figure 6.25: A simulated example of a continuous model identification comparing the actual
plant with the model prediction showing almost no discernable error. Compare these results
with an actual experiment in Fig. 6.26.
6.3. CONTINUOUS MODEL IDENTIFICATION 259

6.3.3 P RACTICAL CONTINUOUS MODEL IDENTIFICATION

We may be tempted given the results presented in Fig. 6.25 to suppose that identifying a contin-
uous transfer function from logged data is relatively straight forward. Unfortunately however, if
you actually try this scheme on a real plant, even a well behaved plant such as the black-box, you
will find practical problems of such magnitude that it renders the scheme is almost worthless. We
should also note that if the deadtime is unknown, then the identification problem is non-linear in
the parameters which makes the subsequent identification problematic. This is partly the reason
why many common textbooks on identification and control pay only scant attention to identifica-
tion in the continuous domain, but instead concentrate on identifying discrete models. (Discrete
model identification is covered next in §6.4.)

The upper plot in Fig. 6.26 shows my attempt at identifying a continuous model of the blackbox,
which, compared with Fig. 6.25, is somewhat less convincing. Part of the problem lies in the diffi-
culty of constructing the derivatives of the logged input/output data. The lower plot in Fig. 6.26
shows the logged output combined with the first and second derivatives computed crudely us-
ing finite differences. One can clearly see the compounding effects on the higher derivatives due
to measurement noise, discretisation errors, and so forth.

Continuous model identification

0.1
output & prediction

0.05

−0.05
Blackbox
Model
−0.1
0 2 4 6 8 10 12 14 16 18 20
Sample time T=0.05 s

Measured output & derivatives

0.15

0.1

0.05
2
y, dy/dt, d y/dt

0
2

−0.05

−0.1

−0.15

−0.2
0 2 4 6 8 10 12 14 16 18 20
time (s)

Figure 6.26: A practical continuous model identification of the blackbox is not as successful as
the simulated example given in Fig. 6.25. Part of the problem is the reliable construction of the
higher derivatives. Upper: The plant (solid) and model prediction (dashed). Lower: The output
y, and derivatives ẏ, ÿ.

For specific cases of continuous transfer functions, there are a number of optimised algorithms
to directly estimates the continuous coefficients. Himmelblau, [88, p358–360] gives once such
procedure for which reduces to a linear system which is easy to solve in M ATLAB. However
again this algorithm essentially suffers the same drawbacks as the sequential differential schemes
given previously.
260 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

I MPROVING IDENTIFICATION RELIABILITY BY USING L AGUERRE FUNCTIONS

One scheme that has found to be practical is one based on approximating the continuous response
with a series of orthogonal functions known as Laguerre functions. The methodology was devel-
oped at the University of British Columbia and is available commercially from BrainWave, now
part of Andritz Automation. Further details about the identification strategy are given in [201].

Compared to the standard curve fitting techniques, this strategy using orthogonal functions is
more numerically robust, and only involves numerical integration of the raw data, (as opposed
to the troublesome differentiation). On the downside, manipulating the Laguerre polynomial
functions is cumbersome, and the user must select an appropriate scaling factor to obtain rea-
sonable results. While [201] suggest some rules of thumb, and some algorithms for choosing the
scaling factor, it is still delicate and sensitive calculation and therefore tricky in practice.

Suppose we wish to use Laguerre functions to identify a complex high-order transfer function
(−45s + 1)2 (4s + 1)(2s + 1)
G(s) = (6.8)
(20s + 1)3 (18s + 1)3 (5s
+ 1)3 (10s + 1)2 (16s + 1)(14s + 1)(12s + 1)
which was used as a challenging test example in [201, p32]. To perform the identification, the
user must decide on an appropriate order, (8 in this case), and a suitable scaling factor, p.

Fig. 6.27 compares the collected noisy step data with the identified step and impulse responses,
compares the identified and gives the error in the impulse response.

6.4 P OPULAR DISCRETE - TIME LINEAR MODELS

The previous section showed that while it was possible in theory to identify continuous models
by differentiating the raw data, in practice this scheme lacked robustness. Furthermore it makes
sense for evenly spaced sampled data systems not to estimate continuous time models, but to
estimate discrete models directly since we have sampled the plant to obtain the input/output
data anyway. This section describes the various common forms of linear discrete models that we
will find useful in modelling and control.

If we collect the input/output data at regular sampling intervals, t = kT , for most applications it
suffices to use linear difference models such as
y(k) + a1 y(k − 1) + a2 y(k − 2) + · · · + an y(k − n) = b0 u(k) + b1 u(k − 1) + · · · + bm u(k − m) (6.9)
where in the above model, the current output y(k) is dependent on m + 1 past and present inputs
u and the n immediate past outputs y. The ai s and bi s are the model parameters we seek.

A shorthand description for Eqn. 6.9 is

A(q −1 ) y(k) = B(q −1 )u(k) + e(k) (6.10)
where we have added a noise term, e(k), and where q −1 is defined as the backward shift-operator.
The polynomials A(q −1 ), B(q −1 ) are known as shift polynomials
A(q −1 ) = 1 + a1 q −1 + a2 q −2 + · · · + an q −n
B(q −1 ) = b0 + b1 q −1 + b2 q −2 + · · · + bm q −m
where the A(q −1 ) polynomial is defined by convention to be monic, that is, the leading coeffi-
cient is 1. In the following development, we will typically drop the argument of the A and B
polynomials.
6.4. POPULAR DISCRETE-TIME LINEAR MODELS 261

T =493.1, p =0.05
ss opt
1.5

1
Step test

0.5

T
ss
−0.5

Step test: Order = 8, scale factor p=0.050

0.04
Inpulse comparison

0.03

0.02

0.01

−0.01

0.05
error

−0.05
0 100 200 300 400 500 600 700 800
time [s]

Figure 6.27: Identification of the 14th order system given in Eqn. 6.8 using an 8th-order Laguerre
series and a near-optimal scaling parameter.

Even with the unmeasured noise term, we can still estimate the current y(k) given old values of
y and old and present values of u using

ŷ(k|k − 1) = −a1 y(k − 1) · · · − an y(k − n) + b0 u(k) + b1 u(k − 1) + · · · + bm u(k − m) (6.11)

| {z } | {z }
past outputs present & past inputs

= (1 − A)y(k) + Bu(k) (6.12)

Our estimate or prediction of the current y(k) based on information up to, but not including k is
ŷ(k|k − 1). If we are lucky, our estimate is close to the true value, or ŷ(k|k − 1) ≈ y(k). Note
that the prediction model of Eqn. 6.11 involves just known terms, the model parameters and past
measured data. It does not involve the unmeasured noise terms.

The model described by Eqn. 6.10 is very common and is called an Auto-Regressive with Exoge-
nous, (externally generated) input, or an ARX model. The auto-regressive refers to the fact that
the output y is dependent on old ys, (i.e. it is regressed on itself), in addition to an external input,
u. A signal flow diagram of the model is given in Fig. 6.28.

There is no reason why we could not write our models such as Eqn. 6.10 in the forward shift
operator, q as opposed to the backward shift q −1 . Programmers and those working in the digital
262 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

noise
e

known input
❄ output
u ✲ B ✲+ ✲ 1 ✲ y
A

Figure 6.28: A signal flow diagram of an auto-regressive model with exogenous input or ARX
model. Compare this structure with the similar output-error model in Fig. 6.31.

signal processing areas tended to favour the backward shift because it is easier and more natural
to write computational algorithms such as Eqn. 6.11, whereas some control engineers favoured
the forward notation because the time delay is given naturally by the difference in order of the A
and B polynomials.

6.4.1 E XTENDING THE LINEAR MODEL

Since the white noise term in Eqn. 6.10 is also filtered by the plant denominator polynomial, A(q),
this form is known as an equation error model structure. However in many practical cases we will
need more flexibility when describing the noise term. One obvious extension is to filter the noise
term with a moving average filter
y(k) + a1 y(k − 1) + · · · + an y(k − n) = b0 u(k) + b1 u(k − 1) + · · · + bm u(k − m)
| {z }
ARX model
+ e(k) white noise
+ c1 e(k − 1) + · · · + cn e(k − n)
| {z }
coloured noise

This is now known as “coloured” noise, the colouring filter being the C polynomial. The A UTO -
R EGRESSIVE M OVING -AVERAGE with E X OGENOUS input, ARMAX, model is written in compact
polynomial form as
A(q)y(k) = B(q)u(k) + C(q)e(k) (6.13)
where we have dropped the notation showing the explicit dependance on sample interval k. The
ARMAX signal flow is given in Fig. 6.29. This model now has two inputs, one deterministic u,
and one disturbing noise, e, and one output, y. If the input u is zero, then the system is known
simply as an ARMA process.

While the ARMAX model offers more flexibility than the ARX model, the regression of the pa-
rameters is now nonlinear optimisation problem. The nonlinear estimation of the parameters
in the A, B and C polynomials is demonstrated on page 307. Also note that the noise term is
still filtered by the A polynomial meaning that both the ARX and ARMAX models are suitable
if the dominating noise enters the system early in the process such as a disturbance in the input
variable.

S IMULATING AN ARMAX MODEL

We can easily simulate discrete time models in raw M ATLAB, although the handling the indices
and initialisation of the filters requires some care. Being able to simulate various discrete time
6.4. POPULAR DISCRETE-TIME LINEAR MODELS 263

noise
e

❄
C

known input
❄ output
u ✲ B ✲+ ✲ 1 ✲ y
A

Figure 6.29: A signal flow diagram of a ARMAX model. Note that the only difference between
this, and the ARX model in Fig. 6.28, is the inclusion of the C polynomial filtering the noise term.

models is useful before we start regressing the model parameters in section 6.5.

Suppose we have an ARMAX model of the form Eqn. 6.13 with coefficient matrices

A(q) = q 2 − q + 0.5, B(q) = 2q + 0.5, C(q) = q 2 − 0.2q + 0.05

Note that these polynomials are written in the forward shift operator, and the time delay given
by the difference in degrees, na − nb , is 1. A M ATLAB script to construct the model is:

A = [1 -1 0.5]; % Model A(q) = q 2 − q + 0.5

B = [2 0.5]; % No leading zeros B(q) = 2q + 0.5
C = [1 -0.2 0.05]; % Noise model C(q) = q 2 − 0.2q + 0.05
ntot = 10; % # of simulated data points
5 U = randn(ntot,1); E = randn(ntot,1);

Now to compute y given the inputs u and e, we use a for-loop and we write out the equation
explicitly as

y(k + 2) = y(k + 1) − 0.5y(k) + 2u(k + 1) + 0.5u(k) + e(k + 2) − 0.2e(k + 1) + 0.05e(k)

| {z } | {z } | {z }
old outputs old inputs noise

It is more convenient to wind the indices back two steps so we calculate the current y(k) as
opposed to the future y(k + 2) as above. We will assume zero initial conditions to get started.

n = length(A)-1; % Order na
z = zeros(n,1); % Padding: Assume ICs = zero
Uz = [z;U]; Ez = [z;E]; y = [z;NaN*U]; % Initialise
d = length(A)-length(B); % deadtime = na − nb
5 zB = [zeros(1,d),B]; % pad B(q) with leading zeros

for i=n+1:length(Uz)
y(i) = -A(2:end)*y(i-1:-1:i-n) + zB*Uz(i:-1:i-n) + C*Ez(i:-1:i-n);
end
10 y(1:n) = []; % strip off initial conditions

This strategy of writing out the difference equations explicitly is rather messy and error-prone,
so it is cleaner, but less transparent, to use the idpoly routine to create an armax model and the
264 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

idmodel/sim command in the identification toolbox to simulate it. Again we must pad with
zeros the deadtime onto the front of the B polynomial.

>>G = idpoly(A,[0,B],C,1,1) % Create polynomial model A(q)y(t) = B(q)u(t) + C(q)e(t)

Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + C(q)e(t)
A(q) = 1 - qˆ-1 + 0.5 qˆ-2

5 B(q) = 2 qˆ-1 + 0.5 qˆ-2

C(q) = 1 - 0.2 qˆ-1 + 0.05 qˆ-2

This model was not estimated from data.

10 Sampling interval: 1

>>y = sim(G,[U E]); % test simulation

Both methods (writing the equations out explicitly, or using idpoly) will give identical results.

We can replicate this simulation in S IMULINK by using discrete filters as shown in Fig. 6.30 which
directly follows Fig. 6.29. For the B and C blocks I could either use a FIR filter block or the
more general IIR discrete filter with a denominator of 1. Note that we must ensure that the B
polynomial is padded with one zero to take account of the one unit of deadtime.

Figure 6.30: A discrete time ARMAX model implemented in S IMULINK. (See also Fig. 6.29.)

6.4.2 O UTPUT ERROR MODEL STRUCTURES

Both the linear ARX model of Eqn. 6.10, and the version with the filtered noise, Eqn. 6.13, as-
sume that both the noise and the input are filtered by the plant denominator, A. A more flexible
approach is to treat the input and noise sequences separately as illustrated in Fig. 6.31. This is
known as an output error model. This model is suitable when the dominating noise term enters
late in the process, such as a disturbance in the measuring transducer for example.

Again drawback of this model structure compared to the ARX model is that the optimisation
function is a nonlinear function of the unknown parameters. This is because the estimate ŷ not
directly observable, but is itself a function of the parameters in polynomials B and F . This makes
the solution procedure more delicate and iterative. The S YSTEM I DENTIFICATION toolbox can
identify output error models with the oe command.
6.4. POPULAR DISCRETE-TIME LINEAR MODELS 265

noise
e
known input output
B ❄
u ✲ ✲+ ✲ y
F ŷ

Figure 6.31: A signal flow diagram of an output-error model. Compare this structure with the
similar ARX model in Fig. 6.28.

6.4.3 G ENERAL INPUT / OUTPUT MODELS

The most general linear input/output model with one noise input, and one known input is de-
picted in Fig. 6.32. The relation is described by

B(q) C(q)
A(q)y(k) = u(k) + e(k) (6.14)
F (q) D(q)

although it is rare that any one particular application will require all the polynomials. For exam-
ple [92, p76] point out that only the process dynamics B and F are usually identified, and D is
a design parameter. Estimating the noise C polynomial is difficult in practice because the noise
sequence is unknown and so must be approximated by the residuals.

noise C
e ✲
D

known input
❄ output
✲ B ✲+ ✲ 1 ✲ y
u
F A

Figure 6.32: A general input/output model structure

Following the block diagram in Fig. 6.32 or Eqn. 6.14, we have the following common reduced
special cases:

1. ARX: Autoregressive with eXogenous input; C = D = F = 1

2. AR: Autoregressive with no external input; C = D = 1, and B = 0

3. FIR (Finite Input Response): C = D = F = A = 1

4. ARMAX: Autoregressive, moving average with eXogenous input; D = F = 1

5. OE (Output error): C = D = A = 1

6. BJ (Box-Jenkins): A = 1
266 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

In all these above models, our aim is to estimate the model parameters which are the coefficients
of the various polynomials given the observable input/output experimental plant data. As al-
ways, we want the smallest, most efficient model that we can get away with. The next section
will describe how we can find suitable parameters to these models. Whether we have the correct
model structure is another story, and that tricky and delicate point is described in section 6.6.
Nonlinear models are considered in section 6.6.3.

6.5 R EGRESSING DISCRETE MODEL PARAMETERS

The previous section listed the various members of the discrete-time models, of which the ARX
and ARMAX are the most popular for control applications. Now we come to the identification
stage. Our aim is given a set of input and output data pairs, (uk , yk ), we wish to regress suitable
parameters for the polynomials in Fig. 6.32.

The simple auto-regressive with exogenous input (ARX) model,

A(q −1 ) y(k) = B(q −1 ) u(k) (6.15)

can be arranged to give the current output in terms of the past outputs and current and past
inputs as,

y(k) = (1 − A(q −1 ))y(k) + B(q −1 ) u(k) (6.16)

or written out explicitly as

y(k) = −a1 y(k − 1) · · · − an y(k − n) + b0 u(k) + b1 u(k − 1) + · · · + bm u(k − m) (6.17)

| {z } | {z }
past outputs current & past inputs

Our job is to estimate reasonable values for the unknown n + m + 1 parameters a1 , . . . , an and
b0 , . . . , bm such that our model is a reasonable approximation to data collected from the true plant.
Collecting all the unknown parameters into a column vector, θ, defined as
h iT
def
θ= .. (6.18)
a1 a2 · · · an . b0 b1 · · · bm

and the immediate past data vector as

h iT
def .
ϕ= −y(k − 1) −y(k − 2) · · · −y(k − n) .. u(k) u(k − 1) · · · u(k − m) (6.19)

then we could write Eqn. 6.17 compactly as

y(k) = ϕT θ (6.20)

The data vector ϕ is also known as the regression vector since we are using it to regress the values
of the parameter vector θ. Note that we place the negative sign associated with the outputs in
Eqn. 6.19 although there is nothing to stop us placing it in Eqn. 6.18.

While Eqn. 6.20 contains (n + m + 1) unknown parameters we have only one equation. Clearly
we need more information in the form of additional equations to reduce the degree of freedom
which in turn means collecting more input/output data pairs.

Suppose we collect a total of N data pairs. The first n are used to make the first prediction, and
the next N − n are used to estimate the parameters. Stacking all the N − n equations, of which
6.5. REGRESSING DISCRETE MODEL PARAMETERS 267

Eqn. 6.17 is the first, in a matrix form gives

 
  −yn−1 −yn−2 ··· −y0 un un−1 ··· un−m
yn  
 yn+1

 
  −yn −yn−1 ··· −y1 un+1 un ··· un−m+1 

 .. = θ
 .  
 .. .. .. .. .. .. .. .. 

 . . . . . . . . 
yN
−yN −1 −yN −2 ··· −yN −n−1 uN uN −1 ··· uN −m
(6.21)
or in a compact form
yN = XN θ (6.22)

where XN is known as the data matrix.

Now the optimisation problem is, as always, to choose the parameter vector θ such that the
model M is an optimal estimate to the true system S. We will choose to minimise the sum of the
squared errors, so our cost function, J , becomes

N
X
2
J = yk − ϕTk θ (6.23)
k=n

which we wish to minimise. In practice, we are rarely interested in the numerical value of J but
in the values of the elements in the parameter vector, θ, that minimises it,
( N
)
X
2
⋆
θ = argminθ yk − ϕTk θ (6.24)
k=n

Eqn. 6.24 reads as “the optimum parameter vector is given by the argument that minimises the
sum of squared errors”. Eqn. 6.24 is a standard linear regression problem which has the same
solution as in Eqn. 3.41, namely

 −1
θ = X⊤
N XN X⊤
N yN (6.25)

For the parameters to be unique, the inverse of the matrix X⊤

N XN must exist. In order to ensure
this, the N × (n + m + 1) data matrix, XN , must at least have more rows than columns, or in
other words it should be taller than it is wide. Admittedly that does not appear to be the case
at first glance in Eqn. 6.21. Secondly the matrix X⊤N XN , which has dimensions of the number
of unknowns, must have full rank. We can ensure this second requirement by constructing an
appropriate input sequence to the unknown plant. This is known as experimental design.

Finally we should take care in numerically computing the unknown vector by using the im-
proved strategies given in section 3.4.2 or at least using Matlab’s backslash as opposed to at-
tempting to follow Eqn. 6.25 verbatim.

E XAMPLE OF OFFLINE DISCRETE MODEL IDENTIFICATION .

Suppose we have collected some input/output data from a plant

268 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Figure 6.33: I have always wondered what the ‘regression gun’ does.

100

50
k
y

time, k input output 0

0 1 1 −50
1 4 10 5
2 −3 −29
k

0
3 2 64
u

−5
0 1 2 3 4
time, k

to which we would like to fit an ARX model of the form A(q)yk = B(q)uk . We propose a model
structure of
A(q) = 1 + a1 q −1 , and B(q) = b0
def  T
which has two unknown parameters, θ = a1 b0 which we are to estimate such that our
predictions adequately match the measured output above. We can write our model as a difference
equation at time k,
yk = −a1 yk−1 + b0 uk

which if we write out in full, at time k = 3 using the 4 previously collected input/output data,
gives
   
y1 −y0 u1  
 y2  =  −y1 a1
u2 
b0
y3 −y2 u3

or by inserting the numerical values from the table above

   
10 −1 4  
 −29  =  −10 −3  a1
b0
64 29 2

The solution for the parameter vector θ is over-constrained as we have 3 equations and only 2
unknowns, so we will use Eqn. 6.25 to solve for the optimum θ.
 −1
θ N = X⊤
N XN X⊤
N yN
6.5. REGRESSING DISCRETE MODEL PARAMETERS 269

and substituting N = 3 gives

  −1  
  −1 4   10
−1 −10 29  −1 −10 29 
θ3 =  −10 −3  −29 
4 −3 2 4 −3 2
29 2 64
  
1 29 −84 2136
=
20262 −84 942 255
 
2
=
3
This means the parameters are a1 = 2 and b0 = 3 giving a model
3
M(θ) =
1 + 2q −1
This then should reproduce the measured output data, and if we are lucky, be able to predict
future outputs too. It is always a good idea to compare our predictions with what was actually
measured.

time input output prediction

0 1 1 —
1 4 10 10
2 −3 −29 −29
3 2 64 64

Well, surprise surprise, our predictions look unrealistically perfect! (This problem is continued
on page 288 by demonstrating how by using a recursive technique we can efficiently process new
incoming data pairs.)

6.5.1 S IMPLE OFFLINE SYSTEM IDENTIFICATION ROUTINES

The following routines automate the offline identification procedure given in the previous sec-
tion. However it should be noted that if you are serious about system identification, then a better
option is to use the more robust routines from the the S YSTEM I DENTIFICATION T OOLBOX. These
routines will be described in section 6.5.3.

In Listing 6.8 below, we will use the true ARX process with polynomials
A(q) = 1 − 1.9q −1 + 1.5q −2 − 0.5q −3 , and B(q) = 1 + 0.2q −1 (6.26)
to generate 50 input/output data pairs.

Listing 6.8: Generate some input/output data for model identification

1+0.2q −1
G = tf([1,0.2,0,0],[1 -1.9 1.5 -0.5],1); % ‘Unknown’ plant 1−1.9q −1 +1.5q −2 −0.5q −3
G.variable = 'zˆ-1';
3 u = randn(50,1); % generate random input, uk
y = lsim(G,u); % Generate output with no noise, yk = Guk

Now that we have some trial input/output data, we can write some simple routines to calculate
using least-squares the coefficients of the underlying model by solving Eqn. 6.21. Once we have
performed the regression, we can in this simulated example, compare our estimated coefficients
with the true coefficients given in Eqn. 6.26.
270 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Listing 6.9: Estimate an ARX model from an input/output data series using least-squares
1 na = 3; nb = 2; % # of unknown parameters in the denominator, na , & numerator, nb .

N = length(y); % length of data series

nmax = max(na,nb)+1; % max order

6 X = y(nmax-1:N-1); % Construct Eqn. 6.21

for i=2:na
X = [X, y(nmax-i:N-i)];
end
for i=0:nb-1
11 X = [X, u(nmax-i:N-i)];
end
y_lhs = y(nmax:N); % left hand side of y = Xθ
theta = X\y_lhs % θ = X+ y

16 G_est = tf(theta(na+1:end)',[1 -theta(1:na)'],-1,'Variable','zˆ-1')

If you run the estimation procedure in Listing 6.9, your estimated model, G_est, should be
identical to the model used to generate the data, G, in Listing 6.8. This is because in this simulation
example we have no model/plant mismatch, no noise and we use a rich, persistently exciting
identification signal.

As an aside, it is possible to construct the Vandemonde matrices (data matrix X, in Eqn. 6.22)
using Toeplitz (or the closely related Hankel matrices) as shown in Listing 6.10.

Listing 6.10: An alternative way to construct the data matrix for ARX estimation using Toeplitz
matrices. See also Listing 6.9.
na = length(A)-1; nb = length(B); % # of parameters to be estimated
nmax = max(na,nb);

4 Y = toeplitz(y(nmax:end-1),y(nmax:-1:nmax-na+1));
U = toeplitz(u(nmax+1:end),u(nmax+1:-1:nmax-nb+2));
X = [Y,U]; % Form data matrix, X.

theta = X\y(nmax+1:end) % θ = X+ y

Note that in this implementation, na is both the order of the A(q −1 ) polynomial and the number
of unknown coefficients to be estimated given that A is monic, while nb is the order of B plus
one.

6.5.2 B IAS IN THE PARAMETER ESTIMATES

The strategy to identify ARX systems has the advantage that the regression problem is linear, and
the estimated parameters will be consistent, that is, they can be shown to converge to the true pa-
rameters as the number of samples increases provided the disturbing noise is white. If however,
the disturbances are non-white as in the case where we have a non-trivial C(q) polynomial, then
the estimated A(q) and B(q) parameters will exhibit a bias.

To illustrate this problem, suppose we try to estimate the A and B polynomial coefficients in the
 6.5. REGRESSING DISCRETE MODEL PARAMETERS 271

the ARMAX process

(1 + 0.4q −1 − 0.5q −2 )yk = q −1 (1.2 − q −1 )uk + (1 + 0.7q −1 + 0.3q −2 ) ek (6.27)
| {z }
noise filter

where we have a non-trivial colouring C polynomial disturbing the process. However the results
shown in Fig. 6.34 illustrate the failure for the ARX estimation routine to properly identify the
A(q) and B(q) parameters. Note that the estimate of a1 after 300 iterations is around 0.2 which
is some distance for the true result of 0.4. Similar errors are evident for a2 and b1 and it does not
look like they will converge even if we collect mode data. This inconsistency of the estimated
parameters is a direct result of the C(q) polynomial which filters the white noise, ek . As the
disturbing signal is no longer white, the ARX estimation routine is now no longer consistent.

input
20
output
Input/output

−20
2

1.5
b
0
1
Paramaters, θ

0.5 a1

−0.5 a2
Figure 6.34: In this example, the
−1 b
1 ARX estimation routine fails to con-
verge to the true A(q) and B(q) pa-
−1.5 rameters. This bias is due to the pres-
0 100 200 300 ence of the C(q) polynomial which
Sample No. [k] ‘colours’ the noise.

To obtain the correct parameters without the bias, we have no option other than to estimate the
C(q) polynomial as well. This ARMAX estimation strategy is unfortunately a nonlinear estima-
tion problem and appropriate identification strategies are available in the S YSTEM I DENTIFICA -
TION T OOLBOX described next.

6.5.3 U SING THE S YSTEM I DENTIFICATION TOOLBOX

As an alternative to using the rather simple and somewhat ‘home grown’ identification routines
developed in section 6.5.1 above, we could use the collection in the S YSTEM I DENTIFICATION
T OOLBOX. This comprehensive set of routines for dynamic model identification that closely fol-
low [125] and is far more reliable and robust than the purely illustrative routines presented so
far.
272 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

The simplest model considered by the S YS ID toolbox is the auto-regressive with exogeneous
input, or ARX model, described in Eqn. 6.10. However for computational reasons, the toolbox
separates the deadtime from the B polynomial so that the model is now written

A(q −1 )y = q −d B(q −1 )u + e (6.28)

where the integer d is the assumed number of delays. The arx routine will, given input/output
data, try to establish reasonable values for the coefficients of the polynomials A and B using
least-squares functionally similar to the procedure given in Listing 6.9.

Of course in practical identification cases we do not know the structure of the model (i.e. the
order of B, A and the actual sample delay), so we must guess a trial structure. The arx command
returns both the estimated parameters and associated uncertainties in an object which we will
call testmodel and we can view with present. Once we fitted out model, we can compare
the model predictions with the actual output data using the compare command. The script in
Listing 6.11 demonstrates the load, fitting and comparing, but in this case we deliberately use a
structurally deficient model.

Listing 6.11: Offline system identification using arx from the System Identification Toolbox
>>plantio = iddata(y,u,1); % Use the data generated from Listing 6.8.
2

>>testmodel = arx(plantio,[2 2 0]) % estimate model coefficients

Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + e(t)
A(q) = 1 - 1.368 qˆ-1 + 0.5856 qˆ-2

7 B(q) = 1.108 + 0.8209 qˆ-1

Estimated using ARX from data set plantio

Loss function 0.182791 and FPE 0.214581
Sampling interval: 1
12

>>compare(plantio,testmodel) % See Fig. 6.35.

Measured Output and Simulated Model Output

10
Measured Output
8 testmodel Fit: 71.34%

2
y1

−2
Figure 6.35: Offline system
−4
identification using the Sys-
−6
tem Identification Toolbox
−8
and a structurally deficient
model. −10
0 10 20 30 40 50

Note however that with no structural mismatch, the arx routine in Listing 6.12 should manage
to find the exact values for the model coefficients,

Listing 6.12: Offline system identification with no model/plant mismatch

>> testmodel = arx(plantio,[3 2 0]) % Data from Listing 6.11 with correct model structure.
2 Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + e(t)
A(q) = 1 - 1.9 qˆ-1 + 1.5 qˆ-2 - 0.5 qˆ-3
6.5. REGRESSING DISCRETE MODEL PARAMETERS 273

B(q) = 1 + 0.2 qˆ-1

7 Estimated using ARX from data set plantio

Loss function 1.99961e-029 and FPE 2.44396e-029
Sampling interval: 1

which it does.

Note that in this case the identification routine ‘perfectly’ identified our plant and that the loss
function given in the result summary is practically zero (≈ 10−30 ). This indicates an extraordi-
nary goodness of fit. This is not unexpected, since we have used a perfect linear process, which
we knew perfectly in structure although naturally we could not expect this in practice.

The arx command essentially solves a linear regression problem using the special M ATLAB com-
mand \ (backslash), as shown in the simple version given in Listing 6.9. Look at the source of
arx for further details.

P URELY AUTO - REGRESSIVE MODELS

As the name suggests, purely auto-regressive models have no measured input,

A(q −1 ) yk = ek (6.29)

and are used when we have a single time series of data which we suspect is simply disturbed
by random noise. In Listing 6.13 we attempt to fit a 3rd order AR model and we get reasonable
results with the coefficients within 1% of the true values. (We should of course compare the
pole-zero plots of the model and plant because the coefficients of the polynomials are typically
numerically ill-conditioned.) Note however that we used 1000 data points for this 3 parameter
model, far more than we would need for an ARX model.

Listing 6.13: Demonstrate the fitting of an AR model.

1 >> N = 1e3; e = randn(N,1); % Take 1000 data points
>> B=1; A = [1 0.4 0.1 0.2]; % True AR plant is: A(q −1 ) = 1 − 0.4q −1 + 0.1q −2 + 0.2q −3
>> y = filter(B,A,e);

>> Gm = ar(iddata(y),3); % Fit AR model with no model/plant mismatch

6 Discrete-time IDPOLY model: A(q)y(t) = e(t)
A(q) = 1 + 0.419 qˆ-1 + 0.1008 qˆ-2 + 0.1959 qˆ-3

Estimated using AR ('fb'/'now') from data set Dat

Loss function 1.00981 and FPE 1.01587
11 Sampling interval: 1

E STIMATING NOISE MODELS

Estimating noise models is complicated since some of the regressors are now functions them-
selves of the unknown parameters. This means the optimisation problem is now non-linear com-
pared to the relatively straight forward linear arx case. However, as illustrated in section 6.5.2,
we need to estimate the noise models to eliminate the bias in the parameter estimation.
274 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

In Listing 6.14 we generate some input/output data from an output-error process,

B
y(k) = u(k − d) + e(k)
F
1 + 0.5q −1
= u(k − 1) + e(k)
1 − 0.5q −1 + 0.2q −2
whose parameters are, of course, in reality unknown. The easiest way to simulate an output-error
process is to use the idmodel/sim command from the System ID toolbox.

Listing 6.14: Create an input/output sequence from an output-error plant.

>> Ts = 1.0; % Sample time T
>> NoiseVariance = 0.1;

4 >> F = [1 -0.5 0.2]; % F (q −1 ) = 1 − 0.5q −1 + 0.2q −2

>> B = [0 1 0.5]; % Note one unit of delay: B(q −1 ) = q −1 + 0.5q −2
>> A = 1;C=1;D=1; % OE model: y = (B/F )u + e

>> G = idpoly(A,B,C,D,F,NoiseVariance,Ts)
9 Discrete-time IDPOLY model: y(t) = [B(q)/F(q)]u(t) + e(t)
B(q) = qˆ-1 + 0.5 qˆ-2

F(q) = 1 - 0.5 qˆ-1 + 0.2 qˆ-2

14 This model was not estimated from data.

Sampling interval: 1

>> N = 10000; % # of samples

>> U = randn(N,1); E = 0.1*randn(N,1); % Input & noise
19 >> Y = sim(G,[U E]);

Using the data generated in Listing 6.14, we can attempt to fit both an output-error and an arx
model. Listing 6.15 shows that in fact the known incorrect arx model is almost as good as the
structurally correct output-error model.

Listing 6.15: Parameter identification of an output error process using oe and arx.
1 >> Gest_oe = oe(Z,'nb',2,'nf',2,'nk',1,'trace','on');
>> present(Gest_oe);

Discrete-time IDPOLY model: y(t) = [B(q)/F(q)]u(t) + e(t)

B(q) = 1.002 (+-0.0009309) qˆ-1 + 0.5009 (+-0.002302) qˆ-2
6

F(q) = 1 - 0.4987 (+-0.001718) qˆ-1 + 0.1993 (+-0.00107) qˆ-2

Estimated using OE from data set Z

Loss function 0.00889772 and FPE 0.00890484
11 Sampling interval: 1
Created: 02-Jul-2007 11:35:15
Last modified: 02-Jul-2007 11:35:16

>> Gest_arx = arx(Z,'na',2,'nb',2,'nk',1);

16 >> present(Gest_arx);
Discrete-time IDPOLY model: A(q)y(t) = B(q)u(t) + e(t)
A(q) = 1 - 0.4816 (+-0.001879) qˆ-1 + 0.1881 (+-0.001366) qˆ-2
6.5. REGRESSING DISCRETE MODEL PARAMETERS 275

B(q) = 1.001 (+-0.001061) qˆ-1 + 0.518 (+-0.002154) qˆ-2

21

Estimated using ARX from data set Z

Loss function 0.0111952 and FPE 0.0112041
Sampling interval: 1
Created: 02-Jul-2007 11:35:16
26 Last modified: 02-Jul-2007 11:35:16

The System Identification toolbox has optimal parameters for the fitting of the nonlinear models.
One of the more useful options is to be able to fix certain parameters to known values. This can
be achieved using the ’FixParameter’ option.

6.5.4 F ITTING PARAMETERS TO STATE SPACE MODELS

So far we have concentrated on finding parameters for transfer function models, which means
we are searching for the coefficients of just two polynomials. However in the multivariable case
we need to identify state-space models which means estimating the elements in the Φ, ∆ and C
matrices. If we are able to reliably measure the states, then we can use the same least-squares
approach (using the pseudo inverse) that we used in the transfer function identification. If how-
ever, and more likely, we cannot directly measure the states, x, but only the output variables, y
then we need to use a more complex subspace approach described on page 277.

The state-space identification problem starts with a multivariable discrete model

xk+1 = Φxk + ∆uk , x ∈ ℜn , u ∈ ℜm (6.30)

where we want to establish elements in the model matrices Φ, ∆ given the input/state sequences
uk and xk . To ensure we have enough degrees of freedom, we would expect to need to collect at
least (n + m) × m data pairs, although typically we would collect many more if we could.

Transposing Eqn. 6.30 gives xTk+1 = xTk ΦT + uTk ∆T , or expanded out

 
φ1,1 φ2,1 ··· φn,1
   φ1,2 φ2,2 ··· φn,2 
xTk+1 =
 
x1 x2 · · · xn 
k .. .. .. 
. . . 
φ1,n φ2,n · · · φn,n
| {z }
n×n
 
∆1,1 ∆2,1 · · · ∆n,1
   . .. .. 
+ u1 u2 · · · um k
.. . . 
∆1,m ∆2,n · · · ∆m,n
| {z }
m×n
276 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

we can see that it is simple to rearrange this equation into

 
φ1,1 φ2,1 ··· φn,1
 
 
 φ1,2 φ2,2 · · · φn,2 
 
 . .. .. 
 . . 
 . . 
 
xTk+1 = [ x1 x2 · · · xn u1 u2 · · · um ] k 
 φ1,n φ2,n · · · φn,n 
 (6.31)
 
 ∆1,1 ∆2,1 · · · ∆n,1 
 
 
 . 
 .. 
 
∆1,m ∆2,n · · · ∆m,n
| {z }
Θ matrix (n+m)×n

where we have stacked the unknown model matrices Φ and ∆ together into one large parameter
matrix Θ. If we can estimate this matrix, we have achieved our model fitting.

Currently Eqn. 6.31 comprises of n equations, but far more, (n + m)n, unknowns; the elements
in the Φ and ∆ matrices. To address this degree of freedom problem, we need to collect more
input/state data. If we collect say N more input/states and stack them underneath Eqn. 6.31, we
get
     
xTk+1  x1 x2 · · · xn | u1 · · · um k
 xTk+2   x1 x2 · · · xn | u1 · · · um 
   k+1 
 .. = .. Θ (6.32)
 .  
 . 

xTk+N +1 x1 x2 · · · xn | u1 · · · um k+N
| {z } | {z }
N ×n N ×(n+m)

or Y = XΘ. Inverting this, using the pseudo inverse of the data matrix X, gives the least-squares
parameters
 T 
Φ
Θ= = X+ Y (6.33)
∆T
and hence the model coefficient Φ and ∆ matrices as desired.

The computation of the model matrices Φ and ∆ is a linear least-squares regression which is easy
to program, and robust to execute. However we are still constrained to the normal restrictions
on the invertibility and conditioning of the data matrix X apply in order to obtain meaningful
solutions. The largest drawback of this method is that we must measure states x, as opposed to
only outputs. (If we only have outputs then we need to use the more sophisticated technique
described on 277). If we had measured the outputs in addition to the states, it is easy to calculate
the C matrix subsequently using another least-squares regression.

In M ATLAB we could use the fragment below to compute Φ and ∆ given a sequence of x(k), and
u(k).

[y,t,X] = lsim(G,U,[],x0); % Perform expriment to generate the I/O data

Xdata = [X,U]; Xdata(end,:) = []; % data matrix

4 y = X; y(1,:) = []; % delete first row

theta = Xdata\y; % solve for parameters Θ = X+ Y

Phiest = theta(1:n,:)'; Delest = theta(n+1:end,:)'; % Reconstruct Φ & ∆.
6.6. MODEL STRUCTURE DETERMINATION AND VALIDATION 277

If any of the parameters are known a priori, they can be removed from the parameter matrix,
and the constants substituted. If any dependencies between the parameters are known (such as
a = 3b), then this may be incorporated as a constraint, and a nonlinear optimiser used to search
for the resulting best-fit parameter vector. The S YSTEM I DENTIFICATION T OOLBOX incorporates
both these extensions.

Estimating the confidence limits for parameters in dynamic systems uses the same procedure
as outlined in §3.4.4. The difficult calculation is to extract the data matrix, X, (as defined by
Eqn. 3.62) accurately. [23, pp226–228] describes an efficient method which develops and subse-
quently integrates sensitivity equations, which is in general, the preferred technique to that using
finite differences.

I DENTIFICATION OF STATE - SPACE MODELS WITHOUT STATE MEASUREMENTS

The previous method required state measurements, but it would be advantageous to be able still
to estimate the model matrices, but only be required to measure input, u, and output y data. This
is addressed in a method known as a subspace identification described in [72, p536] and in more
detail by the developers in [155]. The state-space subspace algorithm for identification, known
as n4sid, is available in the System Identification toolbox.

S UMMARY OF IDENTIFICATION ALTERNATIVES IN STATE - SPACE

Given the underlying system

xk+1 = Φxk + ∆uk
yk = Cxk
then Table 6.2 summarises the identification possibilities.

Table 6.2: Identification alternatives in state-space

We measure Wish to calculate Algorithm

page #
uk , yk , Φ, ∆, C xk Kalman filter 439
uk , yk , xk Φ, ∆, C Normal linear regression 275
uk , y k n, Φ, ∆, C, xk Subspace identification 277

6.6 M ODEL STRUCTURE DETERMINATION AND VALIDATION

The basic linear ARX model using in Eqn. 6.28 has three structural parameters: the number of
poles, na , the number of zeros nb and the number of samples of delay, d. ARMAX models have
four structural parameters. Before regressing suitable parameters given a tentative model struc-
ture, we need some method to establish what a tentative structure is. Typically this involves some
iteration in the model fitting procedure.

If we inadvertently choose too high an order for the model, we are in danger of overfitting the
data, which will probably mean that our model will perform poorly on new data. Furthermore,
for reasons of efficiency and elegance, we would like the simplest model, with the smallest num-
ber of parameters that still adequately predicts the output. This is known as the principle of
278 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

parsimony which aims to reward simpler, rather than complex models. This is the rational for
dividing the collected experimental data into two sets, one estimation set used for model fitting,
and one validation set used to distinguish between models structures.

One way to penalise overly complex models is to use the Akaike Information Criteria (AIC)
which for normally distributed errors is

2p
AIC = + ln (V ) (6.34)
N
where p is the number of parameters, N is the number of observations, and V is the loss function
or sum of squared errors. If we increase the number of parameters to be estimated, the sum of
squared errors decreases, but this is partially offset by the 2p term which increases. Hence by
using the AIC we try to find a balance between models that fit the data, but are parsimonious
in parameters. The following example illustrates using the AIC for determining the appropri-
ate order of an interpolating polynomial to some (x, y) data. Since this is a linear optimisation
problem, it is mathematically equivalent to the fitting of an ARX model.

D ETERMINING THE ORDER OF AN INTERPOLATING POLYNOMIAL

Suppose we have collected some experimental data as shown in Fig. 6.36 and we wish to fit a
polynomial through the data cloud. The key question is: what is an appropriate polynomial
order to use? Fig. 6.36 shows three alternatives, a quadratic with 3 parameters, a cubic (with 4)
and an extreme 10th order polynomial. Which polynomial is best?

−5

−10
y

−15

−20 data
Figure 6.36: Fitting various polynomi- n=3
n=4
als to some noisy experimental data. −25
n=11
Our aim is to establish an appropriate −30
−2 −1 0 1 2
order. x

Now clearly our fit is going to improve as we increase the interpolation order, but that is probably
not what we want since we end up fitting overly complex models that tend to essentially just fit
the noise. Instead we should by plot the AIC from Eqn. 6.34 as a function of order as shown in
Fig. 6.37.

6.5

5.5

5
AIC

4.5

3.5

Figure 6.37: Using the AIC to establish 3

0 5 10 15 20
model structure based on fitted data. # of parameters
6.6. MODEL STRUCTURE DETERMINATION AND VALIDATION 279

When using the AIC criteria we are looking for a distinct minimum to tell us the appropriate
polynomial order. However looking at Fig. 6.37 we can see that the AIC improves rapidly to
n = 5 meaning that a quartic polynomial is appropriate, but then the curve flattens out. There
is a distinct ‘knee’ in the curve, but the AIC does not convincingly increase for the high order
polynomials.

The oversight that we have made in Fig. 6.37 is that we have used the same data for the validation
as that used for the fitting. In situations where we have scant experimental data, we may have
to resort to this, but it is far preferable to collect some separate data that we put aside and only
use for validation. Fig. 6.38 shows the improvement this separation of fitted and validation data
makes. Now we can see a distinct minimum in the AIC plot which clearly shows the penalty of
overfitting.

20

−20

−40
y

−60

−80 Data for fitting

Data for validation
−100
−3 −2 −1 0 1 2
x

30

25

20
AIC

Figure 6.38: Using the AIC to estab-

15
lish model structure based on validation
10 data. Compare the AIC plot in this case
when using the validation data with
5 that where we used only the fitted data
0 5 10 15 20
# of parameters in Fig. 6.37.

As it turns out, the ‘experimental’ data using in this example was synthesised from a simple
quartic polynomial with some added noise. Consequently in this artificial case we know the true
number of parameters is 5. Note however that the AIC criteria suggests that optimum number
of parameters is 4, which in my opinion is acceptably close.

The S YSTEM I DENTIFICATION toolbox computes the AIC with the aic command or when using
the automated structure selection, selstruc. Alternatively we could for a given model esti-
mated say with armax, we can extract the following parameters

p = nparams(Model,'free') % # of free parameters

N = Model.es.DataLength % # of data pairs, N
3 V = Model.es.LossFcn % Sum of squared errors

and then compute the information criteria with

AIC = 2*p/N + log(V) % Akaike Information Criteria from Eqn. 6.34

280 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Given a collection of potential models, we should pick the one that minimises the AIC.

6.6.1 E STIMATING MODEL ORDER AND DEADTIME

If we restrict our attention to linear models, the most crucial decision to make is that of model
order. We can estimate the model order from a spectral analysis, perhaps by looking at the high-
frequency roll-off of the experimentally derived Bode diagram.

An alternative strategy is to monitor the rank of the matrix XT X. If we over-estimate the order,
then the matrix XT X will become singular. Of course with additive noise, the matrix may not
be exactly singular, but it will certainly become illconditioned. Further details are given in [127,
pp496–497].

I DENTIFICATION OF DEADTIME

The number of delay samples, d in Eqn. 6.28 is a key structural parameter to be established prior
to any model fitting. In some cases, an approximate value of the deadtime is known from the
physics of the system, but this is not always a reliable indicator. For example, the blackbox from
Fig. 1.4 should not possess any deadtime since it is composed only of passive resistive/capacitor
network. However the long time constant, the high system order, or the enforced one-sample
delay in the A/D sampler may make it appear if there is some deadtime.

If we suspect some deadtime, then we have the following options:

1. Do a step test and look for a period of inactivity.

2. Use the S YSTEM I DENTIFICATION T OOLBOX (SITB) and optimise the fit for deadtime. This
will probably require a trial & error search. The delayest routine may help us with this
identification.

3. Use some statistical tests (such as correlation) to estimate the deadtime.

Note: Obviously the deadtime will be an integer number of the sampling time. So if you want
an accurate deadtime, you will need a small sample time T . However overly fast sampling will
cause problems in the numerical coefficients of the discrete model.

Method #1 Apply a step response & look for the ‘first sign of life’.

1. First we do an open loop step test using a sample time T = 0.5 seconds.

2. We should repeat the experiment at a different sampling rate to verify the deadtime. Fig. 6.40
compares both T = 1 and T = 0.1 seconds.

Method #2: Use ‘trial & error’ with the S YSTEM IDENTIFICATION T OOLBOX. We can collect some
input/output data and identify a model M(θ) with different deadtimes, namely d = 0 to 3.
Clearly the middle case in Fig. 6.41 where d = 2 has the least error.

This shows, at least for the model M(θ) with na = nb = 2, then the optimum number of delays
is d = 2.
6.6. MODEL STRUCTURE DETERMINATION AND VALIDATION 281

Blackbox step response

0.5 zoomed region input
0.06 output
0.04
0.4
0.02

0
0.3
input/output

−0.02
19 20 21 22 23
zoomed region
0.2
0.28
0.26
0.1 0.24
0.22
0.2
0 0.18
40 41 42 43 44

15 20 25 30 35 40 45 50
time (s)

Figure 6.39: Identification of deadtime from the step response of the blackbox. Note that the
deadtime is approximately 2T ≈ 1 second.

0.2

0.15
i/o

0.1

0.05

0
20 21 22 23 24 25
Figure 6.40: Deadtime estimation at both T =
time [s] 1 and T = 0.1s.

0.5
Deadtime=0

−0.5

0.5
Deadtime=1

−0.5

0.5
Deadtime=2

−0.5

actual
0.5
predicted
Deadtime=3

0
Figure 6.41: Model prediction and
−0.5
20 40 60 80 100 120 140 160
actual using a variety of tentative
time (s) deadtimes, d = 0, 1, 2, 3.
282 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

uy
To Workspace2
0
In1 Out1
initialise
Manual Switch Saturation Scope1
bb_pcl711
Mux1

Band−Limited
White Noise

0.002129s2 +0.02813s+0.5788
s2+2.25s+0.5362
BB Delay
Blackbox dyn

(a) Blackbox experimental setup in S IMULINK

0.6

0.4

0.2

−0.2

−0.4
u
−0.6 y
y predict
−0.8
25 30 35 40 45 50 55 60 65 70 75
time (seconds)
(b) Blackbox model predictions compared to actual data

Figure 6.42: A dynamic model for the blackbox compared to actual output data.

M ODELLING THE BLACKBOX

A model of the experimental blackbox is useful to test control schemes before you get to the lab.
Based on the resistor/capcitor internals of the black-box, we expect an overdamped plant with
time constants in the range of 2 to 5 seconds.

Identification tests on my blackbox gave the following continuous transfer function model
0.002129s2 + 0.02813s + 0.5788 −0.5s
Ĝbb (s) = e
s2 + 2.25s + 0.5362
or in factored form
0.002129(s2 + 13.21s + 271.9) −0.5s
= e
(s + 1.9791)(s + 0.2709)
Fig. 6.42 illustrates the S IMULINK experimental setup and compares the model predictions to the
actual measured data.

6.6.2 R OBUST MODEL FITTING

So far we have chosen to fit models by selecting parameters that minimise the sum of squared
residuals. This is known as the principle of the least squares prediction error and is equivalent
6.7. ONLINE MODEL IDENTIFICATION 283

to the maximum likelihood method when the uncertainty distribution is gaussian. This strategy
has some very nice properties first identified by Gauss in 1809 such as the fact that the estimates
are consistent, that is they eventually converge to the true parameters, the estimate is efficient in
that no other unbiased estimator has a smaller variance, and the numerical strategies are simple
and robust as described in [10].

However by squaring the error, we run into problems in the presence of outliers in the data. In
these cases which are clearly non-gaussian, but actually quite common, we must modify the form
of the loss function. One strategy due to [91] is to use a quadratic form when the errors are small,
but a linear form for large errors, another suggested in [10] is to use something like

ǫ2
f (ǫ) = (6.35)
1 + aǫ2
and there are many others. Numerical implementations of these robust fitting routines are given
in [205].

6.6.3 C OMMON NONLINEAR MODEL STRUCTURES

Nonlinear models, or more precisely, dynamic models that are nonlinear in the parameters are
considerably more complex to identify than linear models such as ARX, or even pseudo-linear
models such as ARMAX as outlined in the unified survey presented in [187]. For this reason fully
general nonlinear models are far less commonly used for control and identification than linear
models. However there are cases where we will need to account for some process nonlinearities,
and one popular way is to only consider static nonlinearities that can be separated from the linear
dynamics.

Fig. 6.43 shows one way how these block-orientated static nonlinear models can be structured.
These are known as Hammerstein-Wiener models, or block-orientated static nonlinear models.
The only difference between the two structures is the order of the nonlinear block element.
In the Hammerstein model, it precedes the linear dynamics such as in the case for when we
are modelling input nonlinearities such as modelling equal percentage control valves, and for
the Wiener models, the nonlinearity follows the linear dynamics such as in the case where we
are modelling nonlinear thermocouples. Note of course that unlike linear systems, the order of
nonlinear blocks does make a difference since in general nonlinear systems do not commute.

The S YSTEM I DENTIFICATION TOOLBOX can identify a selection of commonly used nonlinear
models in Hammerstein and Wiener structures using the nlhw routine.

6.7 O NLINE MODEL IDENTIFICATION

In many practical cases such as adaptive control, or where the system may change from day to
day, we will estimate the parameters of our models online. Other motivations for online identifi-
cation given by [125, p303] include optimal control with model following, using matched filters,
failure prediction etc. Because the identification is online, it must also be reasonably automatic.
The algorithm must pick a suitable model form (number of dead times, order of the difference
equation etc.), guess initial parameters and then calculate residuals and measures of fit.

As time passes, we should expect that owing to the increase in number of data points that we are
continually collecting, we are able to build better and better models. The problem with using the
offline identification scheme as outlined in §6.4 is that the data matrix, X in Eqn. 6.22 will grow
284 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Hammerstein Model

..
.. vt
..
✲ .....................
.. ✲ B(q −1 ) ✲ yt
ut ..
.. A(q −1 )
.
Static nonlinearity Linear dynamics
vt = N (ut )

Wiener Model
..
vt ..
✲ B(q −1 ) ..
✲ .....................
.. ✲ yt
ut ..
A(q −1 ) ..
.
Linear dynamics Static nonlinearity
yt = N (vt )

Figure 6.43: Hammerstein and Wiener model structures

and grow as we add more input/output rows to it. Eventually this matrix will grow too large to
store or manipulate in our controller. There are two obvious solutions to this problem:

1. Use a sliding window where we retain only the last say 50 input/output data pairs

2. Use a recursive scheme where we achieve the same result but without the waste of throwing
old data away.

The following section develops this second, and more attractive alternative known as RLS or
recursive least-squares.

6.7.1 R ECURSIVE LEAST SQUARES

As more data is collected, we can update the current estimate of the model parameters. If we
make the update at every sample time, our estimation procedure is now no longer off-line, but
now online. One approach to take advantage of the new data pair would be to add another
row to the X matrix in Eqn. 6.22 as each new data pair is collected. Then a new θ estimate can
be obtained using Eqn 6.25 with the now augmented X matrix. This equation would be solved
every sample time. However this scheme has the rather big disadvantage that the C matrix grows
as each new data point is collected so consequently one will run out of storage memory, to say
nothing of the growing impracticality of the matrix inversion required. The solution is to use a
recursive formula for the estimation of the new θk+1 given the previous θk . Using this scheme,
we have both constant sized matrices to store and process, but without the need to sacrifice old
data.
6.7. ONLINE MODEL IDENTIFICATION 285

M EAN AND VARIANCE CALCULATED RECURSIVELY

Digressing somewhat, let us briefly look at the power of a recursive calculation scheme, and the
possible advantages that has for online applications. Consider the problem of calculating the
mean or average, x̄, of a collection of N samples, xi . The well known formula for the mean is

N
1 X
x̄ = xi
N i=1

Now what happens, (as it often does when I collect and grade student assignments) is that after
I have laboriously summed and averaged the class assignments, I receive a late assignment or
value xN +1 . Do I need to start my calculations all over again, or can I can use a recursive formula
making use of the old mean x̄N based on N samples to calculate the new mean based on N + 1
samples? Starting with the standard formula for the new mean, we can develop an equivalent
recursive equation

N +1 N
!
1 X 1 X
x̄N +1 = xi = xN +1 + xi
N + 1 i=1 N +1 i=1
1
= (xN +1 + N x̄N )
N +1
1
= x̄N + (x − x̄ )
|{z} N + 1 | N +1{z N }
old | {z } error
gain

which is in the form of new estimate = old estimate + gain × error. Note that now the computation
of the new mean, assuming we already have the old mean, is much faster and more numerically
sound.

In a similar manner, the variance can also be calculated in a recursive manner. The variance at
sample number N + 1 based on the mean and variance using samples 1 to N is
 
2 2 1 1 2 2
σN +1 = σN + (xN +1 − x̄N ) − σN (6.36)
N +1 N +1

which once again is the linear combination of the previous estimate and a residual. For further
details, see [122, p29]. This recursive way to calculate the variance published in by B.P. Welford
goes back to 1962 and has been shown to have good numerical properties.

A RECURSIVE ALGORITHM FOR LEAST- SQUARES

Suppose we have an estimate at time t = N of the model parameters, θN , perhaps calculated

offline, and we subsequently collect a new data pair (uN +1 , yN +1 ). To find the new improved
parameters, θN +1 , Eqn. 6.22 is augmented by adding a new row, where the row to be added is
h i
ϕTN +1 = .. (6.37)
−yN −yN −1 · · · −yN −n+1 . uN +1 uN · · · uN −m+1

to the old data matrix XN to obtain the new, now slightly taller, system
   
yN XN
 · · ·  =  · · ·  θN +1 (6.38)
yN +1 ϕTN +1
286 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

or yN +1 = XN +1 θN +1 . This we can solve in exactly the same way as in Eqn. 6.25,

 −1
θ N +1 = XTN +1 XN +1 XTN +1 yN +1 (6.39)
  −1  
h i XN h i yN
=  X⊤ ... ϕ  · · ·  .
XTN .. ϕN +1
 ···  (6.40)
N N +1 T
ϕN +1 yN +1
except that now our problem has grown in the vertical dimension. Note however, the number of
unknowns in the parameter vector θ remains the same.

However we can simplify the matrix to invert by noting

  −1
h i XN  −1
 T
..  · · ·  = XTN XN + ϕN +1 ϕTN +1 (6.41)
XN . ϕN +1
ϕTN +1

The trick is to realise that we have already computed the inverse of XTN XN previously in Eqn. 6.25
and we would like to exploit this in the calculation of Eqn. 6.41. We can exploit this using the
Matrix inversion lemma which states
−1
−1
(A + BDC) = A−1 − A−1 B D−1 + CA−1 B CA−1 (6.42)
In the special case where B = b and C = cT are both n × 1 vectors, then Eqn. 6.42 simplifies to

−1 A−1 bcT A−1
A + bcT = A−1 − (6.43)
1 + cT A−1 b
If we compare Eqn. 6.43 with Eqn. 6.41, then with
def def
A = X⊤ X, b = c = ϕN +1
we can rewrite Eqn. 6.41 as
 −1  −1
 −1  −1 XTN XN ϕN +1 ϕTN +1 XTN XN
XTN XN + ϕN +1 ϕTN +1 = XTN XN −  −1 (6.44)
1 + ϕTN +1 XTN XN ϕN +1

and we can also rewrite Eqn. 6.40 as

  −1  −1 

 −1 XTN XN ϕN +1 ϕTN +1 XTN XN 
 
T
θN +1 = XN XN −  −1 XTN yN + ϕTN +1 yN +1
1 + ϕTN +1 XTN XN
 
 ϕN +1 
(6.45)
and recalling that the old parameter value, θ N , was given by Eqn. 6.39, we start to develop the
new parameter vector in terms of the old,
 −1
XTN XN ϕTN +1 ϕN +1  −1
θN +1 = θN −  −1 θN + XTN XN ϕTN +1 yN +1
T
1 + ϕN +1 XN XN ϕTN +1
 −1  −1
XTN XN ϕTN +1 ϕN +1 XTN XN
−  −1 ϕTN +1 yN +1 (6.46)
T T
1 + ϕN +1 XN XN ϕN +1
 −1
XTN XN ϕTN +1

= θN +  −1 yN +1 − ϕN +1 θ N (6.47)
1 + ϕN +1 XTN XN ϕTN +1
6.7. ONLINE MODEL IDENTIFICATION 287

If we define P as the covariance matrix,

 −1
def
PN = X⊤
N XN (6.48)

then the parameter update equation, Eqn. 6.47 is more concisely written as


θN +1 = θN + KN +1 yN +1 − ϕTN +1 θN (6.49)

with the gain K in Eqn. 6.49 and new covariance given by Eqn. 6.44 as

PN ϕN +1
KN +1 = (6.50)
1 + ϕTN +1 PN ϕN +1
" #
ϕN +1 ϕTN +1 PN h
T
i
PN +1 = PN I − = PN I − ϕ N +1 KN +1 (6.51)
1 + ϕTN +1 PN ϕN +1

Note that the form of the parameter update equation, Eqn. 6.49, is such that the new value of
θN +1 is the old value θN with an added correction term which is a recursive form. As expected,
the correction term is proportional to the observed error. A summary of the recursive least-square
(RLS) estimation scheme is given in Algorithm 6.2.

Algorithm 6.2 Recursive least-squares estimation

Initialise parameter vector, θ0 , to something sensible (say random values), and set co-variance to
a large value, P0 = 106 I, to reflect the initial uncertainty in the trial guesses.

1. At sample N , collect a new input/output data pair (yN +1 and uN +1 )

2. Form the new ϕN +1 vector by inserting uN +1 and yN into Eqn. 6.37.

3. Evaluate the new gain matrix KN +1 , Eqn. 6.50

4. Update the parameter vector θ N +1 , Eqn. 6.49

5. Update the covariance matrix PN +1 , Eqn. 6.51 which is required for the next iteration.

6. Wait out the remainder of one sample time T , increment sample counter, N ← N + 1, then
go back to step 1.

The routine in listing 6.16 implements the basic recursive least-squares (RLS) algorithm 6.2 in
M ATLAB. This routine will be used in subsequent identification applications, although later in
section 6.8 we will improve this basic algorithm to incorporate a forgetting factor.

Listing 6.16: A basic recursive least-squares (RLS) update (without forgetting factor)
function [thetaest,P] = rls0(y,phi,thetaest,P)
% Basic Recursive Least-Squares (RLS) parameter update
% Inputs: yk ,ϕk : current output & past input/outputs column vector
4 % Input/Outputs: θ, P: parameter estimates & covariance matrix

K = P*phi/(1 + phi'*P*phi); % Gain K, Eqn. 6.50.

P = P-(P*phi*phi'*P)/(1 + phi'*P*phi); % Covariance update, P, Eqn. 6.51.
thetaest = thetaest + K*(y - phi'*thetaest);% θN+1 = θN + KN+1 yN+1 − ϕTN+1 θ N

9 return % end rls.m

288 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

A RECURSIVE ESTIMATION EXAMPLE

On page 267 we solved for the parameters of an unknown model using an offline technique by
collecting four data pairs. Now we will try the recursive scheme given that we have just collected
a new fifth input/output data pair at sample time k = 4,

time, k uk yk θT P
..
.
1 4 10
2 −3 −29
3 2 64 2 3 0.0014 −0.0041 0.0465
4 6 10 ? ? ? ? ?

and we wish to update our parameter estimates.

Previously we had computed that θ3 = [2, 3]T , but now with the new data we can possibly
improve on that estimate using the update


θ4 = θ3 + K4 y4 − ϕT4 θ3
where ϕT4 = [−y3 , u4 ] = [−64, 6].

Now the residual  

  2
y4 − ϕT4 θ3 = 10 − −64 6 · = 120
3
is clearly non-zero indicating that something has changed, so we should continue to update the
parameter estimates. The covariance matrix at sample time k = 3 has previously been computed
and (repeated here) is
 −1    
def ⊤ 1 29 −84 0.0014 −0.0041
P3 = X3 X3 = =
20262 −84 942 −0.0041 0.0465
so the new gain matrix K4 from Eqn. 6.50 is now
 
P3 ϕ4 −0.0099
K4 = = ,
1 + ϕT4 P3 ϕ4 0.0464

the new parameter estimate, θ4 , is corrected to

     
2 −0.0099 0.8074
θ4 = + · 120 = ,
3 0.0464 8.5727
and the updated covariance matrix P4 from Eqn. 6.51 is also updated to
 
P3 ϕ4 ϕT4 P3 0.0003 0.0013
P4 = P3 − = .
1 + ϕT4 P3 ϕ4 0.0013 0.0212

Note that P4 is still symmetric, and less immediately apparent, is that it is also still positive
definite which we could verify by computing the eigenvalues (which are incidentally 0.0002 and
0.0213; both positive). When we subsequently obtain another input/output data pair (u5 , y5 ), we
can just continue with this recursive estimation scheme.

We can compare the new estimates θ4 , with the values obtained if we used to nonrecursive
method of Eqn 6.25 on the full data set. At this level, both methods show much the same com-
plexity in terms of computation count and numerical round-off problems, but as N tends to ∞,
the non-recursive scheme is disadvantaged.
6.7. ONLINE MODEL IDENTIFICATION 289

Plant under test

white noise ✲ 1+1.2q−1 ✲ output

1−1.9q−1 +1.5q−2 −0.5q−3

❄ ❄
)
Initial covariance, P0 = 103 I ✲ RLS
θ 0 = random
❄
parameter estimate, θ

Figure 6.44: Ideal RLS parameter estimation. (See also Fig. 6.46(a).)

S TARTING THE RECURSIVE SCHEME

In practice there is a small difficulty in starting these recursive style estimation schemes, since ini-
tially we do not know the covariance PN . However we have two alternatives: we could calculate
the first parameter vector using a non-recursive scheme, and the switch to a recursive scheme
only later once we know PN , or we could just assume that at the beginning, since we have no
estimates, our expected error will be large, say PN = 104 I. The latter method is more common
in practice, although it takes a few iterations to converge to the “correct” estimates if they exist.

6.7.2 R ECURSIVE LEAST- SQUARES IN M ATLAB

M ATLAB can automate the estimation procedure so that it is suitable for online applications. To
test this scheme we will try to estimate the same system S from Eqn. 6.26 used previously on
page 269 repeated here

1 + 0.2q −1
S: G(q −1 ) =
1−1.9q −1 + 1.5q −2 − 0.5q −3
where the unknown vector of A and B polynomial coefficients consist of the five values
h iT
.
θ= −1.9 1.5 −0.5 .. 1 0.2

which are to be estimated. Note of course we do not need to estimate the leading 1 in the monic
A polynomial.

The estimation is performed in open loop using a white noise input simply to ensure persistence
excitability. We will select a model structure the same as our true system. Our initial parameter
estimate is a random vector, and the initial covariance matrix is set to the relatively large value
of 106 I as shown in Fig. 6.44. Under these conditions, we expect near perfect estimation.

Listing 6.17 calls the rls routine from listing 6.16 to do the updating of estimated parameters θ
and associated statistics using Eqns 6.50 and 6.51.
290 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Listing 6.17: Tests the RLS identification scheme using Listing 6.16.
1 G = tf([1,0.2,0,0], ...
1+0.2q −1
[1 -1.9 1.5 -0.5],1); % ‘unknown’ plant G = 1−1.9q −1 +1.5q −2 −0.5q −3
G.variable = 'zˆ-1';

N = 12; % # of samples
6 U = randn(N,1); % design random input u(k) = N (0, 1)
Y = lsim(G,U); % Compute output

n = 5; P = 1e6*eye(n); % Identification initialisation, P0 = 106 I

thetaest = randn(n,1); Param = [];
11

for i=4:N % start estimation loop

phi = [Y(i-1:-1:i-3); U(i:-1:i-1)]; % shift X register (column) ϕ
[thetaest, P] = rls0(Y(i),phi,thetaest,P); % RLS update of θ
Param = [Param; thetaest']; % collect data
16 end

If you run the script in Listing 6.17 above you should expect something like Fig. 6.45 giving
the online estimated parameters (lower) which quickly converge to the true values after about 5
iterations. The upper plot in Fig. 6.45 shows the input/output sequence I used for this open loop
identification scheme. Remember for this identification application, we are not trying to control
the process, and that provided the input is sufficiently exciting for the identification, we do not
really care precisely what particular values we are using.

input
5
output
Input/output

−5
2

1.5 a
2

1 b
0

0.5
b
Paramaters, θ

1
0

−0.5 a
3

Figure 6.45: Recursive least squares param- −1

eter estimation.
Upper: The input/output trend for an un- −1.5
known process. a
1
−2
Lower: The online estimated parameters
quickly converge to the true parameters.
−2.5
Note that no estimation is performed until 0 2 4 6 8 10 12
after we have collected 4 data pairs. Sample No.
6.7. ONLINE MODEL IDENTIFICATION 291

The near perfect estimation in Fig. 6.45 is to be expected, and is due to the fact that we have no
systematic error or random noise and we have sufficient excitation in the input to stimulate a re-
sponse in the output. This means that after 4 iterations (we have five parameters), we are solving
an over constrained system of linear equations for the exact parameter vector. Of course if we
superimpose some random noise to y, which is more realistic, then our estimation performance
will suffer.

R ECURSIVE LEAST- SQUARES IN S IMULINK

The easiest way to perform recursive estimation in S IMULINK is to use the the identification
routines from the S YSTEM I DENTIFICATION T OOLBOX. Fig. 6.46 shows both the S IMULINK block
diagram for the example given in Fig. 6.44, and the resultant trend of the prediction error.

1+1.2z−1
1−1.9z−1+1.5z−2−0.5z−3
Scope
Band −Limited Plant under test
White Noise
y
u

AutoRegressive
with eXternal input ARX
model estimator

(a) Identifying a simple plant using the RLS block in S IMULINK .

Actual Output (Red Line) vs. The Simulated Predicted Model output (Blue Line)
5

−5
40 45 50 55 60 65 70 75 80 85
Time (secs)
Error In Simulated Model

0.5

40 45 50 55 60 65 70 75 80 85
Time (secs)
(b) The output of the RLS block in S IMULINK

Figure 6.46: RLS under S IMULINK. In this case, with no model/plant mismatch, and no distur-
bances, we rapidly obtain perfect estimation.

The ARX block also stores the resultant models in a cell array which can be accessed from the
workspace once the simulation has finished. The structure of the model to be estimated, and the
frequency at which it is updated is specified by double clicking on the ARX block.
292 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

If we don’t have the System Identification toolbox, we can still reproduce the functionality of
the ARX block by writing out the three recursive equations for the update in raw S IMULINK as
shown in Fig. 6.47. The phi block in Fig. 6.47(a) generates the shifted input/output data vector
using a simple discrete state-space system, while the bulk of the RLS update is contained in the
RLS block shown in Fig. 6.47 which simply implements the thee equations given in Algorithm 6.2
on page 287.

1+1.2z−1
u 1−1.9z−1+1.5z−2−0.5z−3 y
Band −Limited u y
"Unknown" plant
White Noise

theta
y
u

Subsystem
5
phi

5
y (k) theta theta
5
5
[5x5] To Workspace1
phi P P
5
RLS To Workspace3

(a) Implementing RLS in raw S IMULINK .

5
y(k)
1 1
Unit Delay
z
phi 5
5 5 5
2 5 5
phi 5
5
1
phi ’*theta 5 theta
theta
K*err

[5x5] Matrix 5
Multiply P*phi 5 5
5 K
P*phi Divide

5
u+1
5
phi ’*P*phi Bias

2
[5x5]
P
Unit Delay 1 P*(I−phi *K’)
[5x5] phi *K’
P(k) 1 P(k+1) Matrix 5 Transpose1
[5x5] [5x5] Multiply phi*K’ Matrix
z phi
[5x5] [5x5] Multiply T 5
[1x5] u
[5x5]

Constant

eye(na+nb)
[5x5]

(b) The internals of the RLS block in S IMULINK . The inputs are the current plant output and ϕ,
and the outputs are the parameter estimate, θ, and the covariance matrix.

Figure 6.47: Implementing RLS under S IMULINK without using the System Identification toolbox.

An elegant way to generate the past values of input and output vector required for the ϕ vector is
to implement a shift register in discrete state-space. If we have a stream of data, and we want to
keep a rolling collection of the most recent three values, then we could run a discrete state-space
system as
      
yk 0 0 0 yk−1 1
 yk−1  =  1 0 0   yk−2  +  0  yk (6.52)
yk−2 0 1 0 yk−3 0

where the state transition matrix has ones on the first sub-diagonal.
6.7. ONLINE MODEL IDENTIFICATION 293

In general, if we want to construct a shift register of multiple old values of input and output, we
can construct a discrete state-space system as
   
   
yk  0 0 ··· 0  yk−1  1 
      
 yk−1   1 0 · · · 0   yk−2   0 
      
   0    0
 .   0 1 ... 0   ...
 .     ..   
 
 .    .   
      
 yk−na   0 0 · · · 1   yk−na−1   0  
       yk
 =  +  (6.53)
 uk
0 0 · · · 0   uk−1  
 uk     
   1 
      
 uk−1     uk−2   


 
  1 0 · · · 0 
 
 
  0 

 .   0  .  0 . 
 .   ..  .   .. 
 .   0 1 . 0  .   
   
uk−nb 0 0 ··· 1 u k−nb−1 0

where we are collecting na shifted values of y, and nb values of u.

6.7.3 T RACKING THE PRECISION OF THE ESTIMATES

One must be careful not to place too much confidence on the computed or ‘inherited’ covariance
matrix P. Ideally this matrix gives an indication of the errors surrounding the parameters, but
since we started with an arbitrary value for P, we should not expect to believe these values.
Even if we did start with the non-recursive scheme, and then later switched, the now supposedly
correct covariance matrix will be affected by the ever present nonlinearities and the underly-
ing assumptions of the least square regression algorithm (such as perfect independent variable
knowledge) which are rarely satisfied in practice.

Suppose we attempt to estimate the two parameters of the plant

b0
G= (6.54)
1 + a1 q −1
where the exact values for the parameters are b0 = 1 and a1 = −0.5. Fig. 6.48 shows the parameter
estimates slowly converging to the true values, (starting with an unreasonably low P0 = 2I to
better illustrate the trend) and the elements of the covariance matrix P decreasing for the first 10
samples.

Now even though in this simulated example we have the luxury of knowing the true parameter
values, the RLS scheme also delivers an uncertainty estimate of the estimated parameters via
the covariance P matrix. Fig. 6.49, which uses the same data from Fig. 6.48, superimposes the
confidence ellipses in parameter space around the the current best-estimate. (The darker points
and associated ellipses correspond to more recent data.) It is interesting to note that after the 10 or
so samples, the estimates in Fig. 6.48 have nearly converged to the true values, but the confidence
ellipses according to Fig. 6.49, show surprisingly large uncertainties.

A MORE REALISTIC ESTIMATION EXAMPLE

The impressive results of the fully deterministic estimation example shown in Fig. 6.45 is some-
what misleading since we have a no process noise, good input excitation, no model plant/mis-
match in structure and we only attempt to identify the model once. All of these issues are crucial
294 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Recursive least−squares parameter estimation

2

Input/output
0

−1 Output
Input
−2
4

3
parameters, θ 2

1 b
0
−a
1
0

−1

−2

−3

Figure 6.48: Estimation of a two parameter

trace(P)

plant. Note that the parameter estimates 10

0

converge to their correct values, and that

the trace of the covariance matrix decreases
with time. See also Fig. 6.49 for the associ-
0 2 4 6 8 10
ated confidence ellipses. sample No.

4
b0

1
b*0

*
0 a1
Figure 6.49: The trend of the 95% con-
fidence regions for the first 10 estimates −6 −5 −4 −3 −2 −1 0 1
shown in Fig. 6.48. a
1
6.8. THE FORGETTING FACTOR AND COVARIANCE WINDUP 295

for the practical implementation of an online estimator. The following simulation examples will
investigate some of these problems.

Suppose we want to identify a plant where there is an abrupt change in dynamics at sample time
k = 50,

 1.2 q −1

 −1 − 0.4 q −2
, for k < 50,
S(θ) = 1 − 0.2 q −1
 1.5 q

 , for k ≥ 50
1 − 0.3 q −1 − 0.7 q −2
A real example of a such an abrupt change in dynamics might arise in the case of a gantry crane
when model 1 is when the crane is unloaded, and model 2 is when the crane has picked up a
heavy shipping container for example.

In this case shown, as in Fig. 6.45, the parameters quickly converge to the correct values as shown
in Fig. 6.50 for the first plant, but after the abrupt model change, they only reluctantly converge
to the new values.

Identifying abrupt plant changes

40
Input/output

20

−20
1
error

−1
1.6
b0
1.4

1.2

0.8
−a2
θ

0.6

0.4
Figure 6.50: Identifying an abrupt
−a1 plant change at k = 50. The estima-
0.2
tor’s parameters initially converges
0 quickly to the first plant (lower
−0.2
0 50 100 150 200 250 300
trend), but find struggle to converge
sample No. to the second plant.

This problem of an increasing tendency to resist further updating is addressed by incorporating

a forgetting factor into the standard RLS algorithm and is described next in §6.8.

6.8 T HE FORGETTING FACTOR AND COVARIANCE WINDUP

Sometimes it is a good idea to forget about the past. This is especially true when the process pa-
rameters have changed in the interim. As we saw in Fig. 6.50, if we simply implement Eqns 6.50
and 6.49, or alternatively run the routine in Listing 6.17, we will find the estimation works well
296 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

initially since the estimates converge to the true values, and the trace of the covariance matrix de-
creases as our confidence improves. However if the process subsequently changes again, we find
that the parameters do not converge as quickly to the new parameters. This may be surprising
since the convergence worked fine at first, so why didn’t it work a second time?

The reason for this failure to converge a second time is due to the fact that the now minuscule
covariance matrix, reflecting our large, but misplaced confidence, is inhibiting any further large
changes in θ.

One solution to this problem, is to reset the covariance matrix to some large value, say 103 I for
example when the system change occurs. While this works well, it is not generally feasible, since
we do not know when the change is likely to take place. If we did, then gain scheduling may be
more appropriate.

An alternative scheme is to introduce a forgetting factor, λ, to decrease the influence of old sam-
ples. With the inclusion of the forgetting factor, the objective function of the fitting exercise is
modified from Eqn. 6.23 to
N
X
2
J = λN −k yk − ϕTk θ , λ<1 (6.55)
k=1

where data n samples past is weighted by λn . The smaller λ is, the quicker the identification
scheme discounts old data. Actually it would make more sense if the forgetting factor λ were
called the “remembering” factor, since a higher value means more memory!

Incorporating the forgetting factor into the recursive least squares scheme modifies the gain and
covariance matrix updates very slightly to
PN ϕN +1
KN +1 = (6.56)
λ + ϕTN +1 PN ϕN +1
!
1 PN ϕN +1 ϕTN +1 PN
PN +1 = PN −
λ λ + ϕTN +1 PN ϕN +1
PN  
= I − ϕN +1 KTN +1 (6.57)
λ

To choose an appropriate forgetting factor, we should note that the information dies away with a
time constant of N sample units where N is given by
1
N= (6.58)
1−λ
As evident from Fig. 6.51, a value of λ is 0.99 (which is typical) gives a time constant of 100
samples.

Consequently most textbooks recommend a forgetting factor of between 0.95 and 0.999, but lower
values may be suitable if you expect to track fast plant changes. However if we drop λ too much,
we will run into covariance windup problems which are further described later in section 6.8.2.

We can demonstrate the usefulness of the forgetting factor by creating and identifying a time
varying model, but first we need to make some small changes to the recursive least-squares
update function rls given previously in Listing 6.16, to incorporate the forgetting factor. as
shown in Listing 6.18 which we should use from now on. Note that if λ is not specified explicitly,
a value of 0.995 is used by default.
6.8. THE FORGETTING FACTOR AND COVARIANCE WINDUP 297

200
λ=1
1

150
0.8
weighting

0.6

N
100

0.4 λ=0.995
λ=0.99 50
0.2 λ=0.98
λ=0.95
0 0
0 50 100 150 200 250 300 0.6 0.7 0.8 0.9 1
historic time λ
(a) The weighting as a function of the forgetting factor, (b) The ‘memory’, N when using a forgetting factor λ
λ. from Eqn. 6.58. Typically λ = 0.99 which corresponds
to a time constant of 100 samples.

Figure 6.51: As the forgetting factor is reduced, older samples are deemed less important.

Listing 6.18: A recursive least-squares (RLS) update with a forgetting factor. (See also List-
ing 6.16.)
function [thetaest,P] = rls(y,phi,thetaest,P,lambda)
% Basic Recursive Least-Squares (RLS) parameter update
% Inputs: yk ,ϕk , λ: current output & past input/outputs column vector & forgetting factor
4 % Input/Outputs: θ, P: parameter estimates & covariance matrix

if nargin < 5
lambda = 0.995; % Set default forgetting factor, λ
end % if
9

K=P*phi/(lambda + phi'*P*phi); % Gain K, Eqn. 6.56.

P=(P-(P*phi*phi'*P)/(lambda+phi'*P*phi))/lambda;% Covariance update, P, Eqn. 6.57.

thetaest = thetaest + K*(y - phi'*thetaest);% θN+1 = θN + KN+1 yN+1 − ϕTN+1 θ N
return % end rls.m

6.8.1 T HE INFLUENCE OF THE FORGETTING FACTOR

Figure 6.52 shows how varying the forgetting factor λ alters the estimation performance when
faced with a time varying system. The true time varying parameters S(θ) are the dotted lines,
and the estimated parameters M(θ) for different choices of λ from 0.75 to 1.1 are the solid lines.

When λ = 1, (top trend in Fig. 6.52), effectively no forgetting factor is used and the simulation
shows the quick convergence of the estimated parameters to the true parameters initially, but
when the true plant S(θ) abruptly changes at t = 200, the estimates, M(θ), follows only very
sluggishly, and indeed never really converges convincingly to the new S(θ). However by incor-
porating a forgetting factor of λ = 0.95, (second trend in Fig. 6.52), better convergence is obtained
for the second and subsequent plant changes.

Reducing the forgetting factor still further to λ = 0.75, (third trend in Fig. 6.52), increases the con-
vergence speed, although this scheme will exhibit less robustness to noisy data. We also should
note that there are larger errors in the parameter estimates (S(θ) − M(θ)) during the transients
298 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

1 b1

λ=1
−a
2
0 −a
3
−1

1 b1

λ = 0.95
−a
2
0 −a
3
−1

1 b1
λ = 0.75

−a2
0 −a3
−1

2
Figure 6.52: Comparing the estimation per-
formance of an abruptly time-varying un- 1 b1
λ = 1.1

known plant with different forgetting factors, −a

2
λ. 0 −a
3
Estimated parameters (solid) and true param-
−1
eters (dashed) for (a) λ = 1, (b) λ = 0.95, (c)
λ = 0.75, (d) λ = 1.1. Note that normally one 0 200 400 600
would use λ ≈ 0.95, and never λ > 1. sample time [k]

when λ = 0.75 than in the case where λ = 1. This is a trade-off that should be considered in the
design of these estimators. We can further decrease the forgetting factor, and we would expect
faster convergence, although with larger errors in the transients. In this simplified simulation
we have a no model/plant mismatch, no noise and abrupt true process changes. Consequently
in this unrealistic environment, a very small forgetting factor is optimal. Clearly in practice the
above conditions are not met, and values just shy of 1.0, say 0.995 are recommended.

One may speculate what would happen if the forgetting factor were set greater than unity (λ > 1).
Here the estimator is heavily influenced by past data, the more distant past—the more influence.
In fact, it essentially disregards all recent data. How recent is recent? Well actually it will disre-
gard all data except perhaps the first few data points collected. This will not make an effective
estimator. A simulation where λ = 1.1 is shown in the bottom trend of Fig. 6.52), and demon-
strates that the estimated parameters converge to the correct parameters initially, but fail to budge
from then on. The conclusion is not to let λ assume values > 1.0.

6.8.2 C OVARIANCE WIND - UP

Online estimation in practice tends to work well at first, but if the estimator is left running for
extended periods of time, such as a few days or even weeks, the algorithm often eventually fails.
One of the reasons leading to a breakdown is that during periods of little interest such as when
6.8. THE FORGETTING FACTOR AND COVARIANCE WINDUP 299

the system is a steady-state, there is no information arriving into the estimator to reduce further
the uncertainty of the estimated parameters. In fact the reverse occurs, the uncertainty grows,
and this is seen as a slow but sure exponential growth in the elements of the covariance matrix
P. Soon the matrix will overflow, and this is referred to as covariance wind-up. Any practical
algorithm must take steps to monitor for the problem and take steps to avoid it.

During periods of little interest, the covariance update, Eqn. 6.57, at sample time k, reduces to
effectively
 
Pk  T 
Pk+1 = I − ϕk+1 Kk+1 
λ | {z }
almost zero
Pk
≈ (6.59)
λ
which means that the P matrix increases each sample by a factor of 1/λ which under normal
circumstances is say around 2%. It will continue this exponential increase until either there is
sufficient excitation in input and output to start to drop the covariance again or it overflows.

Consequently it is prudent to watch the size of P by monitoring the size of the individual ele-
ments. A suitable scalar size measure is given by the trace of the P matrix, which is simply the
sum of the diagonal elements
n
X Xn
def
tr(P) = pi,i = λi
i=1 i=1

The trace of a matrix has the nice property in that it also equals the sum of the n eigenvalues, λi .
Since P is positive definite, the trace will always be positive. If the trace gets too large, then some
corrective action is required.

Suppose we try to identify the same system as used in section 6.7.2,

1.2q −1
S=
1 − 0.25q −1 − 0.50q −2

using a recursive least-squares algorithm with a forgetting factor of 0.98. Rather than using the
optimal white noise as an input excitation signal, we will use a poor input excitation, such as a
square wave with an excessively long period. While the parameters do converge to the correct
values, Fig. 6.53 shows also that the trace of P increases during the periods of no input excitation
to astronomically large values such as 1015 . Clearly the covariance matrix would grows without
bound and will soon exceed the numerical limits of our control computer. If instead we had used
the ideal white noise input signal, the continual excitation keeps the trace of P small. As evident
from Eqn. 6.59, the smaller our forgetting factor λ is, the faster the covariance matrix will windup.

M ODIFICATIONS ON THE ‘ FORGETTING ’ THEME

Introducing a forgetting factor is just one attempt to control the behaviour of the covariance ma-
trix. As you might anticipate, there are many more modifications to the RLS scheme along these
lines such as variable forgetting factors, directional forgetting factors, constant trace algorithms,
adaptive forgetting factors, startup forgetting factors etc. See [204] for further details.

The variable forgetting factor introduced in [68] is particularly easy to implement. A similar
algorithm was developed at around the same time in [203] and the two algorithms compared in
[177].
300 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Input/output
0

−5
2

θ 1

0
20
10

λ=0.98
Figure 6.53: Covariance wind-up. In
tr(P)

this case with λ = 0.98, the param-

eters converge to their correct val-
ues, (middle trend), but the trace of
0
the covariance matrix grows during 10
the periods of little input excitation, 0 1000 2000 3000 4000 5000
(lower plot). sample No.

The time-varying forgetting factor is now no longer a constant value, but adjusted according to

(1 − ϕT K)ǫ2
λ(t) = 1 − (6.60)
Σ0
where ǫ is the residual error, and Σ0 is chosen as N σ 2 and where σ 2 is the anticipated measure-
ment noise variance of the system, and N adjusts the rate of adaption. One way to interpret N is
from Eqn. 6.58 thus giving
σ2
Σ0 = N σ 2 =
1 − λ⋆
⋆
using a nominal value of λ = 0.995. It may be also prudent to place some safety limits on λ(t)
such as
0.3 ≤ λ(t) ≤ 0.9999
Fig. 6.54 shows the result of using this simple improvement to the estimation algorithm when
we have multiple abrupt plant changes. Note that in this case, (as opposed to the situation given
in Fig. 6.50), we do manage to both re-identify the second and subsequent changes, and that
the trace of the covariance matrix does not ever get too large. During the changes however, the
varying forgetting factor does drop significantly at times reaching the lower limit of 0.3.

6.9 I DENTIFICATION BY PARAMETER OPTIMISATION

Model identification can also be achieved using local parametric optimisation theory. This has
the advantage that it is conceptually simple, allows for general, possibly nonlinear functions,
and closely follows classical optimisation techniques. The basic idea is to update the parameter
vector so that a performance index is minimised. In this manner, the problem is reduced to one
of a normal minimisation problem where many computer algorithms already exist. However
one typically assumes that we have good starting guesses for the parameters, and that one must
6.9. IDENTIFICATION BY PARAMETER OPTIMISATION 301

output & setpoint

4
2
0
−2

0.01
input

−0.01

2 a
2
1
b
1
0
θ

a3
−1

−2 a
1
tr(P)

4
10

0.8

0.6
λ(t)

0.4

0 100 200 300 400

sample time

Figure 6.54: Using a variable forgetting factor to improve the identification of abrupt plant
changes. Note the improvement over the problems exhibited in Fig. 6.50.

live with the resulting slow rate of convergence. Also these simple optimisation schemes can
become unstable, and the resulting stability analysis is typically quite involved. This section
closely follows [114, pp66-73].

Suppose we are given a system, S,

B(s)
S : y(t) = u(t) (6.61)
A(s)
where we know the denominator, A(s) = (1 + a1 s + a2 s2 ), and wish to estimate the steady-state
gain, B(s) = b0 . Therefore our model M of the process is
M : (1 + a1 s + a2 s2 )ŷ(t) = b̂0 u(t) (6.62)
where we wish b̂0 → b0 . We will take the squared error as our performance index,
Z
1
J = ǫ2 dt (6.63)
2
302 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

where the error between the actual and predicted is ǫ = y − ŷ.

The gradient optimisation technique is an intuitive method that tells us how to adjust our un-
known parameter vector to search for the minimum. We adjust our parameters in such a manner
as to decrease the residual at a maximum rate. We will use the steepest descent method,

∂J
∆b̂0 = −k (6.64)
∂ b̂0
where k > 0 is the adaption gain which can be adjusted to vary the speed of convergence. The
rate of change of our parameter is
 
db̂0 ∂ ∂J
= −k (6.65)
dt ∂t ∂ b̂0
Now we can reverse the order of differentiation in Eqn. 6.65 provided the system does not change
too fast,
 
db̂0 ∂ ∂j k ∂ǫ2
= −k =−
dt ∂ b̂0 ∂t 2 ∂ b̂0
∂ǫ
= −kǫ (6.66)
∂ b̂0
Eqn. 6.66 is referred to as the MIT rule. To use the MIT rule in an adaptive scheme, we must
evaluate the sensitivity function, ∂ǫ/∂ b̂0,

∂ǫ ∂y ∂ ŷ
= − (6.67)
∂ b̂0 ∂ b̂0 ∂ b̂0
|{z}
=0

so therefore the adaption rule is

db̂0 ∂ ŷ
= kǫ (6.68)
dt ∂ b̂0
The sensitivity function of the predicted output ŷ with respect to the parameter is obtained by
differentiating the original model, Eqn. 6.62. Note that

(1 + a1 s + a2 s2 )ŷ = b̂0 u
⇒ y + a1 ẏ + a2 ÿ = b̂0 u

so that the partial derivative of model M with respect to b̂0 is

∂ ŷ ∂ ŷ˙ ∂ ŷ¨
+ a1 + a2 =u (6.69)
∂ b̂0 ∂ b̂0 ∂ b̂0
Again we can change the order of differentiation so
   
∂ ŷ ∂ ∂ ŷ ∂2 ∂ ŷ
+ a1 + a2 2 =u (6.70)
∂ b̂0 ∂t ∂ b̂0 ∂t ∂ b̂0
∂ ŷ
(1 + a1 s + a2 s2 ) =u (6.71)
∂ b̂0
Dividing Eqn. 6.71 by Eqn. 6.61 gives
∂ ŷ y
= (6.72)
∂ b̂0 b0
6.9. IDENTIFICATION BY PARAMETER OPTIMISATION 303

thus giving the adaption law (from Eqn. 6.68)

˙ ∂ ŷ y
b̂0 = kǫ = kǫ (6.73)
∂ b̂0 b 0

which would all work very nicely except that we do not know b0 ! (It is after all, what we are
trying to estimate.) So we will substitute the estimate in the adaption law giving,
˙ y
b̂0 = kǫ (6.74)
b̂0

Algorithm 6.3 Searching for the plant gain Given an unknown system S producing measurable
outputs y(t) and a model M with an initial parameter estimate b̂0 producing outputs ŷ(t) sub-
jected to known inputs u(t),

1. Using the model M with the current best estimate of b̂0 predict ŷ and compare with the
actual system output, ǫ(t) = y(t) − ŷ(t).
2. Update the parameter estimate using the adaption law, Eqn. 6.74
3. Go back to step 1.

E XAMPLE OF PARAMETRIC ADAPTION

We will demonstrate this parametric adaption where our true, but only partially known, system
and corresponding model are

0.45 b̂0
S= , and model M=
s2 + 4.2s + 4.4 s2 + 4.2s + 4.4

where the initial estimate of the unknown gain b0 has the wrong sign at b̂0 = −0.3.

We will do most of the calculations in the discrete domain, so we will approximate the time
derivative in the adaption law, db̂0 /dt, with an Euler approximation which when discretised at
sample time T gives,  
y
b̂0 (t) = b̂0 (t − 1) + T kǫ
b̂0
The upper plot in Fig. 6.55 calculated from Listing 6.19 compares the result of using the model
M with no adaption (dash-dot) with one where the model is adapted according to the MIT rule
(dotted) line. Note that eventually the model with adaption closely follows the true process (thick
solid line). The lower plot shows that the parameter estimate b̂0 slowly converging to the true
value of 0.45. Note that in this case the adaptation gain k = 8 and that the adapted model has a
much reduced integrated error than the model with no adaption.

Listing 6.19: Adaption of the plant gain using steepest descent

S = tf(0.45,[1 4.2 4.4]); % True plant S = 0.45/(s2 + 4.2s + 4.4)
2

b0_est = -0.3; % Estimated of parameter b̂0 = −0.3, while actual b0 = 0.45

G = tf(b0_est,[1 4.2 4.4]); % Estimated model b̂0 /(s2 + 4.2s + 4.4)

dt = 0.175; t = [0:dt:60]'; % Simulate truth & openloop with a random input

7 U = 0.6*randn(size(t))+0.5*sin(t/4);
304 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Adaption gain = 8
True
0.1
Open loop
Adapted

Plant & Model output

0.05

−0.05

Figure 6.55: Parameter adaption us- −0.1

ing the steepest descent method. The 0.5
Gain estimate, b̂0

b
0
upper plot compares the true sys-
tem (solid) with the openloop model 0
(dotted) and the adapted model
(dashed). The lower plot shows the
−0.5
parameter adapting gradually to the 0 10 20 30 40 50 60
correct value of b0 = 0.45. Time T =0.175 [s]
s

Yx = lsim(S,U,t); % truth output, S

Yol = lsim(G,U,t); % open loop estimate, M

Y = Yx; B=NaN*Y; k = 8; % Adaption gain, k = 8

12

for i=3:length(t)-1 % Now do the estimation . . .

G.num = b0_est; Gd = c2d(G,dt);
[num,den] = tfdata(Gd,'v');
Y(i) = -den(2:3)*Y(i-1:-1:i-2) + num*U(i:-1:i-2);
17

e = Yx(i)-Y(i) ; % model/plant error

b0_est = b0_est + dt*k*e*Y(i)/b0_est; % Update b̂0 ← b̂0 + T kǫy/b̂0 , Eqn. 6.74
B(i,:) = b0_est;
end % for
22

d = [Y-Yx, Yx-Yol]; jise = sum(abs(d)); % Performance analysis, See Fig. 6.55.

Choosing an appropriate adaptation gain is somewhat of an art. Too small a value and the adap-
tation will be too slow. Too large, and the adaptation may become unstable. Fig. 6.56 trends the
integrated absolute error (ISE) for the adaption scheme as a function of adaptation gain which
can be compared with the openloop scheme over the time horizon used in Fig. 6.55. In this ap-
plication, the best setting for k is around 50, but this is very dependent on starting conditions,
problem type etc. Above this value the performance rapidly deteriorates, and one would often
be better off using only the uncorrected openloop model. As k → 0, then there effectively no
adaption, and this is equivalent to the openloop model.

6.10 O NLINE ESTIMATING OF NOISE MODELS

So far we have assumed that the noise acting on our system has been random white noise. How-
ever in practice, coloured noise is far more common, and blindly applying the RLS scheme to a
process disturbed by coloured noise will in general result in biased estimates. The text [204, p102]
6.10. ONLINE ESTIMATING OF NOISE MODELS 305

3
10

2
10
ISE

1
10

0
10

Figure 6.56: Optimising the adaptation

optimal gain
−1
gain. The IAE for the for different adap-
10
−1
10 10
0
10
1
10
2 3
10
tation gains, k. The optimum gain, k ⋆ , is
Adpation gain, k around 50.

contains further details. Recall that the ARMAX model of Eqn. 6.13, repeated here,

A(q)yk = B(q)uk−d + C(q)ek (6.75)

| {z }
coloured noise

is comprised of three polynomials, A, B and C,

A = 1 + a1 q −1 + · · · + ana q −na , B = b0 + b1 q −1 + · · · + bnb q −nb

C = 1 + c1 q −1 + · · · + cnc q −nc

where the A and C polynomials are assumed monic. A schematic of Eqn. 6.75 is given in Fig. 6.57
which is a special case of the general linear model of Fig. 6.32.

white noise, e(t)

❄
input, u(t) ✲ q −d B ✲+ ✲ 1 ✲ output, y(t)
A

Figure 6.57: Addition of coloured noise to a dynamic process. See also Fig. 6.32.

The noise model polynomial, C, is typically never larger than first or second order. We wish to
estimate the na + nb + nc + 1 unknown coefficients of A, B and C. The problem we face when
trying to estimate the coefficients of C is that we can never directly measure the noise term ek .
So the best we can do is replace this unknown error term with an estimate such as the prediction
error,
ǫk = yk − ϕk θ̂k−1 (6.76)
and augment the data vector with the estimates of the C polynomial. This scheme is referred to as
recursive extended least-squares, or RELS. Note carefully the difference in nomenclature between
the actual, but unmeasured error, e and the estimate of that error, ǫ.
306 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

The recursive extended least-squares algorithm given in Algorithm 6.4 is an extension of the
standard recursive least-squares algorithm.

Algorithm 6.4 Recursive extended least-squares

We aim to recursively estimate the parameters of the A, B and C polynomials in the dynamic
process
A(q −1 )yk = B(q −1 )uk−d + C(q −1 )ek
by measuring uk and yk . We note that A and C are assumed monic, so the parameter vector is
h iT
def
θ= .. ..
a1 · · · ana . b0 · · · bnb . c1 · · · cnc

where na , nb and nc are the orders of the A, B and C polynomials respectively.

1. Construct the augmented data row vector using past input/output data and past prediction
error data,
 
.. ..
ϕk =  |−yk−1 , . .{z. , −yk−na}, . |uk−d , . . . ,{z
uk−d−1−nb , . ǫk−1 , . . . , ǫk−nc 
} | {z }
past outputs inputs predicted errors

2. Collect the current input/output data, yk , uk , and calculate the present prediction error
using the previously obtained model parameter estimates

ǫk = yk − ϕk θ̂ k−1

3. Update the parameter covariance matrix, gain, and parameter estimate vector using the
standard RLS update equations, (Eqns 6.49, 6.50, and 6.51), from Algorithm 6.2.
4. Wait out the remaining of the sample time, and then return to step (1).

Of course we could modify these update equations to incorporate additional features such as
forgetting factors in the same manner as in the normal recursive least squares case.

An alternative algorithm is the approximate maximum likelihood, where instead of using the
prediction error, one uses the residual error,

ηk = yk − ϕk θ̂k (6.77)

and minor changes to the covariance and parameter update equations. (Note the difference be-
tween this and Eqn. 6.76.) Further implementation details are given in [204, p105].

6.10.1 A RECURSIVE EXTENDED LEAST- SQUARES EXAMPLE

For this simulated example, we will compare the parameter estimates of an unknown plant dis-
turbed by coloured noise using both the normal recursive least-squares (where we anticipate
biased estimates), and the recursive extended least-squares algorithm to additionally estimate
the C polynomial and to remove the bias estimates of the A and B polynomials.

Our true plant is

(1 − 0.4q −1 − 0.5q −2 ) yk = (1.2q −1 + 0.3q −2 ) uk + (1 − 0.8q −1 − 0.1q −1 ) ek (6.78)

| {z } | {z } | {z }
A(q−1 ) B(q−1 ) C(q−1 )
6.10. ONLINE ESTIMATING OF NOISE MODELS 307

where ek ∼ N (0, 2.5), and the input uk ∼ N (0, 1). Note that we have coloured noise as C 6= 1
and that this noise variance is relatively large.

If we just ignore the noise model, and attempt to estimate just the four unknown A and B polyno-
mial coefficients using the normal RLS algorithm, then we get large biases especially as evident
in the first half of the Fig. 6.58.

λ=0.9990
2

RLS RELS
1.5

b
1
Parameter estimates

1
c
2

0.5 a
3
b2

0
c
3

a
2
−0.5

−1
0 1000 2000 3000 4000 5000 6000
sample time [k]

Figure 6.58: Estimating model parameters using recursive least squares in the presence of
coloured noise. The first period shows the A and B estimates (solid) and true values (dashed)
using normal RLS. The second half of the trend shows the A, B and C estimates (solid) and true
values (dashed) using extended RLS. Note the eventual lack of bias in the latter RELS estimates.

If however, we swap over to use the extended recursive least-squares algorithm at time k = 2000,
then the estimation performance is much improved, exhibits much less bias, and even manages
(eventually) to estimate the noise polynomial correctly as shown in the latter section of Fig. 6.58.
The downside of course is that now we must estimate the noise polynomial coefficients as well
increasing the number of unknown parameters from 4 to 6.

Example Estimating parameters in an ARMAX model.

For the dynamic system described by Eqn. 6.78, we will create an ARMAX model using the
idpoly object, create some suitable random input, and use the model to generate and output as
shown in Listing 6.20. Note how I must pad the B polynomial with a leading zero to take into
account the deadtime.

Listing 6.20: Create an ARMAX process and generate some input/output data suitable for sub-
sequent identification.
nt= 100; u= randn(nt,1); e= randn(nt,1); % # of samples, input & disturbance
2

A = [1 -0.4 0.5]; B = [0, 1.2, 0.3]; C = [1, 0.8 -0.1]; % True coefficients
G = idpoly(A,B,C) % Create true ARMAX plant
y = sim(G,[u e]); % Simulate Ay = Bu + Ce
Zdata = iddata(y,u,1); % collect output/input data, sample time T = 1.

To estimate the parameters we could either use the general pem or the more specific armax com-
mands. We will estimate a structurally perfect model, that is 2 parameters in each of the A, B and
C polynomials and 1 unit of dead time.
308 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

Listing 6.21: Identify an ARMAX process from the data generated in Listing 6.20.
>>G_est= pem(Zdata,'na',2,'nb',2,'nc',2,'nk',1);% Estimate A(q −1 ), B(q −1 ) & C(q −1 )
>>G_est= armax(Zdata,'na',2,'nb',2,'nc',2,'nk',1)% Alternative to using pem.m

4 >>present(G_est) % present results

G_est =
Discrete-time ARMAX model: A(z)y(t) = B(z)u(t) + C(z)e(t)
A(z) = 1 - 1.416 zˆ-1 + 0.514 zˆ-2

9 B(z) = 1.209 zˆ-1 + 0.2935 zˆ-2

C(z) = 1 + 0.7703 zˆ-1 - 0.1181 zˆ-2

>>yp = sim(G_est,[u e]); % form prediction

14 >>plot([y yp]) % check prediction performance in Fig. 6.59.

Fig. 6.59 compares the true and predicted output data. In the left-hand case, the fit is not perfect
even though we used the correct model structure which is partly due to the iterative nature of the
nonlinear regression, but mostly the poor fit is due to the fact that we used only 100 input/output
samples.

On the other hand, if we used 10,000 samples for the fitting as in the right-hand case, we will
estimate a far superior model. It is interesting to note just how many input/output samples we
need to estimate a relatively modest sized system of only 6 parameters.

Using only 100 points for estimation Using 10000 points for estimation
15 −2

−4
10
−6
Output & estimated output

−8
5
−10

0 −12

−14
−5
−16

−18
−10
Actual
Estimated −20
Estimated
−15 −22
0 20 40 60 80 100 9900 9920 9940 9960 9980 10000
sample time sample time

Figure 6.59: A validation plot showing the actual plant output compared to the model predictions
of an ARMAX model using series of different lengths. Note that to obtain good results, we need
a long data series.

Listing 6.22 repeats this estimation, but this time does it recursively in M ATLAB.

Listing 6.22: Recursively identify an ARMAX process.

1 A = [1 -0.4 0.5]; B = [1.2, 0.3]; C = [1, 0.8 -0.1]; % True plant to be estimated

na = length(A); nb = length(B); nc = length(C); % Order of the polynomials+1

d = na-nb; % deadtime

6 Npts = 5e4;
u = randn(Npts,1); a = randn(Npts,1);
y = zeros(Npts,1); err = y; thetaAll = []; % dummy
6.10. ONLINE ESTIMATING OF NOISE MODELS 309

np = na+nb+nc - 2; % # of parameters to be estimated

11 theta = randn(np,1); P = 1e6*eye(np); % initial guess of θ
lambda = 1; % forgetting factor, λ ≤ 1

for i=na:Npts;
y(i)= -A(2:end)*y(i-1:-1:i-na+1)+ B*u(i-d:-1:i-nb+1-d)+ C*a(i:-1:i-nc+1);
16

phi = [-y(i-1:-1:i-na+1)', u(i-d:-1:i-nb+1-d)', err(i-1:-1:i-nc+1)']';

err(i,1) = y(i) - phi'*theta; % estimated noise sequence

K = P*phi/(lambda + phi'*P*phi); % update gain

21 P = (P-(P*phi*phi'*P)/(lambda + phi'*P*phi))/lambda; % covariance update
theta = theta + K*err(i,1); % update θ
thetaAll(i,:) = theta';
end

26 Aest = [1, thetaAll(end, 1:na-1)]; Best = thetaAll(end, na:na+nb-1);

Cest = [1, thetaAll(end,na+nb:end)]; % See Fig. 6.60.

As shown in Fig. 6.60, the estimates do eventually converge to the correct values without bias,
and furthermore, the step response of the final estimate compares favourably with the true step
response.

RELS: λ=0.9990 Model at k=4e4

2 5

0
1.5
Estimated
Actual
−5
0 5 10 15 20 25
1 (seconds)
c
b2
2
Parameter estimates

0.5
a3
b
3
0 b1
c
3

−0.5

Model at k=2e4

−1 3
a2
2

−1.5 1
Estimated
Actual
0
0 5 10 15 20 25
−2 (seconds)
0 1 2 3 4 5
sample time [k] 4
x 10

Figure 6.60: Recursive extended least-squares estimation where there is an abrupt model change
at time k = 2 · 104. The inserted figures compare a step response of the true plant with the current
estimate.

6.10.2 R ECURSIVE IDENTIFICATION USING THE SI TOOLBOX

The System Identification toolbox can also calculate the parameter estimates recursively. This
essentially duplicates the material presented in §6.7.1. However in this case there are many more
310 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

options to choose from regarding type of method, numerical control, starting point etc.

The toolbox has a number of recursive algorithms (See [126, p1–64]) but they all work in the same
way. For most demonstration purposes, we will recursively estimate the data pseudo online. This
means that we will first generate all the data, and then estimate the parameters. However we
will make sure that when we are processing element i, the data ahead in time, elements i + 1 . . .,
are not available. This pseudo online approach is quite common in simulation applications. The
Toolbox command that duplicates the recursive least squares estimation of a ARMAX process
with a forgetting factor is rarmax(z,nn,’ff’,0.95).

6.10.3 S IMPLIFIED RLS ALGORITHMS

The standard recursive least squares (RLS) algorithm such as presented in §6.7.1 requires two
update equations;– one for the update gain, and one for the covariance matrix. But for imple-
mentation in a small computer, there is some motivation to simplify the algorithm, even at the
expense of the quality of parameter estimates. Åström and Wittenmark, [17, p69-70] describe one
such method known as Kaczmarz’s projection algorithm. The input/output relation is assumed

yk = ϕk θk

where θ is the parameter vector and ϕ is the row data vector, and the correction to the estimated
parameter vector is
θ̂k = θ̂k−1 + α ϕ⊤
k

where α is chosen such that yk = ϕk θ̂ k which gives the full updating formula in our now familiar
form as
ϕT  
θ̂k = θ̂k−1 + y k − ϕk θ̂ k−1 (6.79)
ϕk ϕT
Note that in [17], there is some nomenclature change, and that the data vector is defined as a
column vector, where here I have assumed it is a row vector. This update scheme (Eqn 6.79) is
sometimes modified to avoid potential problems when the parameter vector equals zero.

The script in listing 6.23 first generates data from an ARX plant, and then the data is processed
pseudo-online to obtain the estimated parameters.

Listing 6.23: Kaczmarz’s algorithm for identification

u = randn(50,1); % random input

3 A = [1 -1.25 0.8]; B = 0.3; % G = 0.3/(1 − 1.25q −1 + 0.8q −2 )

G = idpoly(A,B,1); y = sim(G,u);

theta = randn(3,1);% Initial parameter estimate, θ 0 , is random

Param = []; % estimated parameters so far
8 for i=4:length(u);
phi= [-y(i-1:-1:i-2)', u(i)]'; % regressor vector, ϕk 
ϕ

theta=theta+phi*(y(i)-phi'*theta)/(phi'*phi); % θ̂k ← θ̂k−1 + ϕTkϕ yt − ϕTk θ̂ k−1
k
Param= [Param; theta']; % collect parameters
end %
13

Param_x = ones(size(Param(:,1)))*([A(2:3) ,B(1)]); % true parameters

plot([Param_x, Param]) % have they converged ?
6.10. ONLINE ESTIMATING OF NOISE MODELS 311

The input/output data (top) and parameters from this data are shown in Fig. 6.61. The estimated
parameters converge relatively quickly to the true parameters, although not as quickly as in the
full recursive least-squares algorithm.

5
Input & output

−5
1
a2
Estimated & true parameters

0.5
b
0

−0.5 Figure 6.61: The performance of a simpli-

fied recursive least squares algorithm, Kacz-
marz’s algorithm.
−1
Upper: The input/output data.
a
1 Lower: The estimated parameters (solid)
−1.5 eventually converge to the true parameters
0 10 20 30 40 50
sample time (dashed).

Problem 6.3 The following questions use input/output data from the collection of ‘classic’ time
series data available from the time series data library maintained by Rob Hyndman at:
www-personal.buseco.monash.edu.au/˜hyndman/TSDL/ or alternatively from the DAISY:
D ATABASE FOR THE I DENTIFICATION OF S YSTEMS collection maintained by De Moor, Department
of Electrical Engineering, ESAT/SISTA, K.U.Leuven, Belgium at
www.esat.kuleuven.ac.be/sista/daisy/.

These collections are useful to test and verify new identification algorithms. Original sources for
this data include [32] amongst others.

1. The data given in /industry/sfbw2.dat shows the relationship in a paper making ma-
chine between the input stock flow (gal/min) and basis weight (lb/3300 ft2 ), a quality control
parameter of the finished paper. Construct a suitable model of this time series.
Ref: [158, pp500–501].

2. This problem is detailed and suitable for a semester group project.

An alternative suite of programs to aid in model development and parameter identification is
described in [32, Part V]. The collection of seven programmes closely follows the method-
ology in the text, and are outlined in pseudo-code. Construct a T IME S ERIES A NALYSIS
toolbox with graphics based around these programs.
Test your toolbox on one of the time series tabulated in the text. A particularly relevant series
is the furnace data that shows the percentage of carbon dioxide output from gas furnace
at 9 seconds intervals as a function of methane input, (ft3 /min), [32, Series J, pp532–533].
This input/output data is reproduced in Fig. 6.62, although the input methane flowrate is
occasionally given as negative indicating that the series is probably scaled. The original
reference footnoted that the unscaled data lay between 0.6 and 0.04 ft3 /min.
312 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

65

60

% CO2
55

50

45
4

Methane flow
0
Figure 6.62: The percent CO2 in the
−2
exhaust gas from a furnace as a func-
tion of the methane input (scaled). −4
0 500 1000 1500 2000 2500
Data from [32, p532–533]. time [s]

6.11 C LOSED LOOP IDENTIFICATION

The identification techniques described this far assume that the input signal can be chosen freely
and that this signal has persistent excitation. However if you are trying to identify while con-
trolling, you have lost the freedom to choose an arbitrary input since naturally, the input is now
constrained to give good control rather than good estimation. Under good control, the input will
probably be no longer exciting nor inject enough energy into the process so that there is sufficient
information available for the identification. Closed loop identification occurs when the process is
so important, or hazardous, that one does not have the luxury of opening the control loop simply
for identification. Closed loop identification is also necessary, (by definition), in many adaptive
control algorithms.

If we use a low order controller, then the columns of the data matrix X in Eqn. 6.22 become
linearly dependent. This is easy to see if we use a proportional controller, ut = kyt , then this in
turn creates columns in X that only differ by the constant controller gain k. In reality, any noise
will destroy this exact dependency, but the numerical inversion may still be a computationally
difficult task.

One way to ensure that u(t) is still persistently exciting the process is to add a small dither or
perturbation signal on to the input u. This noisy signal is usually of such a small amplitude,
that the control is not seriously effected, but the identification can still proceed. In some cases
there may be enough natural plant disturbances to make it unnecessary to add noise to u. With
this added noise, added either deliberately or otherwise, the estimation procedure is the same as
described in § 6.7.

The above arguments seems to suggest that closed loop identification is suboptimal, and that if
possible, one should always try to do the identification in open loop, design a suitable controller,
then close the loop. This avoids the possible ill-conditioning, and the biasing in the estimated
parameters. However Landau in a series of publications culminating in [113] argues that in fact
closed loop control, with suitable algorithms, actually develops better models since the identi-
fication is constrained to the frequencies of interest for good closed loop control. It all depends
in the final analysis on whether we want a good model of the plant for say design in which
case we do the identification in open loop, or if we want good control in which case closed loop
identification may be better.

A very readable summary of closed loop identification is given in [94, pp517–518], [17, p82] and
especially [204]. Identification in the closed loop is further discussed in chapter 7.
6.12. SUMMARY 313

6.11.1 C LOSED LOOP RLS IN S IMULINK

Implementing recursive identification using S IMULINK is not as straight forward as writing raw
M ATLAB code because of the difficulty in updating the parameters inside a transfer function
block. Under normal operation these are considered constant parameters within the filter block,
so it is not a problem. Fig. 6.63 shows the closed loop estimation using the same RLS blocks as
those used in Fig. 6.47 on page 292. In this example the identified parameters are not used in the
controller in any sort of adaptive manner, for that we need an adaptive controller which is the
subject of the following chapter.

theta

5 5
5 phi1 theta theta
5 5
Parameters
phi

[5x5]
y(k) P P
uy Plant i/o data phi generator
Covariance
u

RLS
3

PID Gd
u y
Signal PID Controller Unknown plant y
Generator

Figure 6.63: Closed loop estimation using RLS in S IMULINK. See also Fig. 6.47.

6.12 S UMMARY

System identification is where we try to regress or fit parameters of a given model skeleton or
structure to fit input/output data from a dynamic process. Ideally we would obtain not only
the values of the parameters in question, but also an indication of the “goodness of fit”, and
the appropriateness of the model structure to which the parameters are fitted. Identification
methodologies usually rely on a ‘trial & error’ approach, where different model structures are
fitted, and the results compared. The S YSTEM I DENTIFICATION T OOLBOX within M ATLAB, [126],
is a good source of tools for experimentation in this area, because the model fitting and analysis
can be done so quickly and competently.

Identification can also be attempted online, where the current parameter estimate is improved
or updated as new process data becomes available in an efficient way without the need to store
all the previous historical data. We do this by re-writing the least-squares update in a recursive
algorithm. Updating the parameters in this way enables one to follow time-varying models or
even nonlinear models more accurately. This philosophy of online estimation of the process
model is a central component of an adaptive control scheme which is introduced in chapter 7.

However there are two key problems with the vanilla recursive least-squares algorithm which
become especially apparent when we start to combine identification with adaptive control. The
first is the need to prevent the covariance matrix P from being illconditioned. This can, in part be
solved by ensuring persistent excitability in input. The second problem is that given non-white
noise or coloured noise, the RLS scheme will deliver biased estimates. However, there are many
extensions to the standard schemes that address these and other identification issues. Succinct
314 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

summaries of system identification with some good examples are given in [22, pp422–431] and
[150, p856], while dedicated texts include [125, 191, 204].
6.12. SUMMARY 315

P ROBLEMS

Problem 6.4 An identification benchmark problem from [110].

1. Simulate the second-order model with external disturbance vt ,

2
X 2
X 2
X
yt = ai yt−i + bi ut−i + di vt−i + e(t), e(t) ∼ N (0, σ 2 ) (6.80)
i=1 i=0 i=1

where the parameter values are:

a1 a2 b0 b1 b2 d1 d2 σ
0.98 −0.9 0.5 −0.25 0.1 0.8 0.2 0.1

The input ut is normally distributed discrete white noise, and the external disturbance is
to be simulated as a rectangular signal alternating periodically between the values +1 and
−1 at t = 100, 200, . . . Run the simulation for 600 time steps, and at t = 300, change a1 to
−0.98.

2. This is a challenging identification problem, because the rarely varying external disturbance
signal gives little information about the parameters d. Identify the parameters in Eqn. 6.80
perhaps starting with θ̂1|0 = 0, P1|0 = 50I and using an exponential forgetting factor, λ =
0.8.

Problem 6.5 1. Run through the demos contained in the S YSTEM I DENTIFICATION toolbox for
M ATLAB.

2. Investigate the different ways to simulate a discrete plant with a disturbing input. Use the
model structure
A(q −1 )yt = B(q −1 )ut + C(q −1 )et
Choose a representative stable system, say something like

A(q −1 ) = 1 − 0.8q −1 , B(q −1 ) = 0.5, C(q −1 ) = 1 + 0.2q −1

and choose some sensible inputs for ut and et . Simulate the following and explain any
differences you find.

(a) Write your own finite difference equation in M ATLAB, and compare this using filter.
(b) Use the transfer function object, tf, in MISO (multiple input/single output) mode.
Hint: Use cell arrays as
>>G = tf( {B ,C} , {A,A},T); % discrete version
>>G.variable = ’q’;
(c) Use idpoly and idmodel/sim from the S YSTEM I DENTIFICATION toolbox.
(d) Build a discrete S IMULINK model

Hint: You may find problems due to the differing lengths of the polynomials. Either ensure
all polynomials are the same length, or swap from z −1 to z, or introduce delays.

3. Repeat problem 2 with an unstable model (A is unstable.)

4. What is the difference in S IMULINK when you place the A polynomial together with the B
and C polynomials? Particularly pay attention to the case where A is unstable.
316 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

5. Identification of plants that include coloured noise inputs is considerably more complex than
just using arx. The following illustrates this.

(a) Create a ‘truth’ plant with polynomials, A, B, C. (Use idpoly).

(b) Simulate this plant using idmodel/sim for the following two cases:
i. A noise-free version: i.e. random u(t) and no e(t)
ii. A version with coloured noise: i.e. random u(t), e(t).
(c) Estimate an ARX model (i.e. only A, B polynomials) using both data sets. ‘Present’
the model with uncertainty bounds on the estimated parameters using present. What
was the problem with the coloured noise version and how can we avoid it?
(d) Estimate the A, B and C polynomials using an extended-LS scheme such as armax
or pem. Has the identification improved significantly? How did the computational time
change from using to arx to armax?

6. The D AISY (Data Base for the Identification of Systems), [137], located at www.esat.kuleuven.ac.be/sist
and contains a number of simulated and experimental multivariable data sets. Down load
one of these data sets, preferably one of the real industrial data trends, and make an iden-
tification.
7. RLS with modified forgetting factors
Construct a simulation to estimate the parameters in a dynamic model (as in tident.m) to
test various extensions to the forgetting factor idea. Create a simulation with output noise,
and step change the model parameters part way through the run. (For more information on
these various extensions, see [204, pp140–160].)
(a) Use a start-up forgetting factor of the form
  
−t
λ1 (t) = λ0 + (1 − λ0 ) 1 − exp (6.81)
τf
where λ0 is the initial forgetting factor, (say 0.9–0.95), and τf is the time constant that
determines how fast λ(t) approaches 1.0.
Note that Eqn. 6.81 can be re-written in recursive form as

λ1 (t) = αλ1 (t − 1) + (1 − α)

where α = e−1/τf and λ1 (0) = λ0 .

(b) Combine with the start-up forgetting factor, an adaptive forgetting factor of the form
 
τf ǫ2 (t)
λ2 (t) = 1− (6.82)
τf − 1 τf sf (t)

where sf (t) is a weighted average of past values of the squared error ǫ2 .

[204, p159] suggest using the filter
τf − 1 ǫ2
sf (t) = sf (t − 1) +
τf τf

Combine both the start-up and the adaptive forgetting factor to get a varying forgetting
factor λ(t) = λ1 (t)λ2 (t).
(c) A directional forgetting factor tries to update in only those directions in parameter space
where there is information available. The covariance update with directional forgetting
is " #
xt x⊤
t Pt−1
Pt = Pt−1 I − −1 (6.83)
rt−1 + x⊤ t Pt−1 xt
6.12. SUMMARY 317

where the directional forgetting factor r(t) is

1 − λ′
rt = λ′ − (6.84)
x⊤
t+1 Pt xt+1

and the scalar λ′ is like the original fixed forgetting factor.

What ever happened to elegant solutions?

8. Compare the operation count of the RLS scheme with square root updates with the nor-
mal update. Construct a sufficiently ill-conditioned example where you can demonstrate a
significant difference.
Summary of the RLS square root update scheme for y = ϕT θ.
Define the square root of P as
def
P = SST

Update St by following

f t = STt−1 ϕt
βt = 1 + f T f
1
αt = √
βt + βt
Lt = St−1 f t
St = St−1 − αt Lt f Tt

Update the parameter estimates using

Lt

θ t+1 = θt + yt − ϕT θt
βt

9. Modify the square root RLS scheme to incorporate a forgetting factor.

318 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

10. Construct an RLS update algorithm in S IMULINK, perhaps using the matrix/vector blocks in
the DSP blockset. Use the following update equations:
" #
Pk ϕk+1 ϕk+1 ϕTk+1 Pk
Kk+1 = , Pk+1 = Pk I −
1 + ϕTk+1 Pk ϕk+1 1 + ϕTk+1 Pk ϕk+1


θk+1 = θk + Kk+1 yk+1 − ϕTk+1 θ k
Note that as of Matlab R12, there is considerable support for vector/matrix manipulations in
raw S IMULINK.
11. Construct a square root update algorithm (from Problem 8) in S IMULINK, perhaps using the
matrix/vector blocks in the DSP blockset.
12. Wellstead & Zarrop, [204, p103], develop an expression for the bias as a function of the
model parameters for a simple 1-parameter model
yt = ayt−1 + et + cet−1 , |a| < 1, |c| < 1
where et is white noise with variance σe2 , as
c(1 − a2 )
â − a ≈
1 + c2 + 2ac
(a) Validate with simulation the expression for the size of the bias, a − â.
(b) The size of the bias should not be a function of the size of the variance of et . Is this a
surprising result? Verify this by simulation. What happens if you drop the variance of
the noise, σe2 , to zero? Do you still observe any bias? Should you?
13. Starting with the S IMULINK demo rlsest which demonstrates an adaptive pole-placement
of a discrete plant do the following tasks.

(a) Modify the rlsest simulation to use a continuous plant of your choice and, for the mo-
ment, a fixed (non-adaptive) PID controller. (I.e. disconnect the closed-loop adaption
part of the controller.) Use Fig. 6.64 as a guide to what you should construct.
Explain in detail how the state generator works and how the RLS estimator works.
Modify the blocks so that you can identify a different number of A terms from the B
terms, (such as say two B(q −1 ) and three A(q −1 ) polynomials terms).
(b) Run the estimation routine under closed-loop conditions. What do you observe espe-
cially as you change the noise level? (To be realistic, you should add at least one more
noise source, where ?)
(c) Modify the RLS estimator s-function to export the covariance matrix as well as the
updated parameters. Do these confirm your findings from part 13b?
(d) Modify your RLS scheme to better handle noise and the closed loop conditions. You
may like to use some simple heuristics based on the trace of the covariance matrix, or
use a directional and/or variable forgetting factor.

14. Read the M ATLAB Technical support note #1811 How Do I Create Time Varying Matrix
Gain and State-space Blocks as C MEX S-functions?. Download the relevant files and
implement an adaptive controller.
15. Model identification in state-space with state information.
The English mathematician Richardson has proposed the following simple model for an
arms race between two countries:
xk+1 = axk + byk + f (6.85)
yk+1 = cxk + dyk + g (6.86)
6.12. SUMMARY 319

theta
To Workspace1

RLS
Estimator

parameters Recursive least squares

Parameter Estimator
State
phi Generator
t
Clock1 time
yr
To Workspace
ZOH ZOH1

Mux

spt/plant
+ u[n] 3(s+0.3) y[n]
− PID +
(s+2)(s+1)(s+0.4) +
Setpoint
PID Controller Sum1
plant

distrurbance

Figure 6.64: A modified version of rlsest in Simulink.

where xk and yk are the expenditures on arms of the two nations and a, b, c, d, f and g
are constants. The following data have been compiled by the Swedish International Peace
Research Institute, (SIPRI yearbook 1982).

Millions of U.S. dollars at 1979 prices and 1979 exchange rates

year Iran Iraq NATO WTO
1972 2891 909 216478 112893
1973 3982 1123 211146 115020
1974 8801 2210 212267 117169
1975 11230 2247 210525 119612
1976 12178 2204 205717 121461
1977 9867 2303 212009 123561
1978 9165 2179 215988 125498
1979 5080 2675 218561 127185
1980 4040 225411 129000
1981 233957 131595

(a) Determine the parameters of the model and investigate the stability of this model.
(b) Determine the estimate of the parameters based on 3 consecutive years. Look at the
variability of the estimates.
(c) Create a recursive estimate of the parameters. Start with 1975 with the initial values
determined from 1972–74.
320 CHAPTER 6. IDENTIFICATION OF PROCESS MODELS
C HAPTER 7

A DAPTIVE C ONTROL

Adopt, adapt and improve . . . Motto of the round table.

John Cleese whilst robbing a lingerie boutique.

7.1 W HY ADAPT ?

Since no one yet has found the “holy grail” of controllers that will work for all conceivable oc-
casions, we must not only select an appropriate class of controller, but also be faced with the
unpleasant task of tuning it to operate satisfactorily in our particular environment. This job is
typically done only sporadically in the life of any given control loop, but in many cases owing to
equipment or operating modifications, the dynamics of the plant have changed so as to render to
controller sub-optimal. Ideally the necessary re-tuning could be automated, and this is the idea
behind an adaptive controller.

In a control context, adaption is where we adjust the controller as the process conditions change.
The problem is that for tight control we essentially want to have a high feedback gain, giving
a rapid response and good correction with minimal offset. However if the process dynamics
subsequently change due to environmental changes for example, the high feedback gain may
create an unstable controller. To prevent this problem, we adapt the controller design as the
process conditions change, to achieve a consistently high performance.

Adaptive control was first used in the aerospace industry around the late 1950s where automatic
pilots adapted the control scheme depending on external factors such as altitude, speed, payload
and so forth. One classic example was the adaptive flight control system (AFCS) for the X-15 in
the late 1960s described in [7]. The adaptive control system was developed by Honeywell and
was fundamentally a gain scheduler that adapted the control loop gains during the flights that
ranged from sea level to space at 108 km.

Unfortunately the crash of X-15-3 and loss of pilot Michael Adams in 1967 somewhat terminated
the enthusiasm of adaptive flight control, and the research program was terminated shortly af-
terwards.

In the past couple of decades however, adaptive control has spread to the processing industries
with several commercial industrial adaptive controllers now available. Some applications where
adaptive control is now used are:

321
322 CHAPTER 7. ADAPTIVE CONTROL

Aeronautical control As the behaviour of high performance fighter aircraft depends on altitude,
payload, speed etc, an adaptive control is vital for the flight to be feasible.
Tanker steering The dynamics of oil supertankers change drastically depending on the depth of
water, especially in relatively shallow and constricted water-ways.
Chemical reactions Chemical processes that involve catalytic reactions will vary as the catalyst
denatures over time. Often coupled to this, is that the heat transfer will drop as the tubes
in the heat exchanger gradually foul.
Medical The anesthetist must control the drug delivery to the patient undergoing an operation,
avoiding the onsets of either death or consciousness.1

Some further examples of adaptive control are given in a survey paper in [183].

Adaptive control is suitable: Adaptive control is not suitable if:

• if the process has slowly varying parameters
◦ if clean measurements are available
• for batch processes or startup/shut down of
continuous processes
• for widely changing operating conditions (such
as in pH controlled processes)
• PID control is already satisfactory
• The system output is essentially constant
• The system is essentially linear and time invari-
ant
• If the system is nonlinear, but easy to model

7.1.1 T HE ADAPTION SCHEME

When faced with tuning a new control loop, the human operator would probably do the follow-
ing steps:

1. Disturb the plant and collect the input/output data.

2. Use this data to estimate the plant dynamics.
3. Using the fitted model, an appropriate controller can be designed.
4. Update the controller with the new parameters and/or structure and test it.

If the scheme outlined above is automated, (or you find an operator who is cheap and does
not easily get bored), we can return to step 1, and repeat the entire process at regular intervals
as shown in Fig. 7.1. This is particularly advantageous in situations where process conditions
can change over time. The process and feedback controller are as normal, but we have added
a new module in software (dashed box) consisting of an on-line identification stage to extract a
process model M(θ), and given this model information, a controller design stage to find suitable
controller tuning constants.

Of course there will be some constraints if we were to regularly tune our controllers. For example,
it is advisable to do the the estimation continuously in a recursive manner, and not perturb the
plant very much. We could adapt the three tuning constants of a standard PID controller, or we
could choose a controller structure that is more flexible such as pole-placement. The adaption
module could operate continuously or it could be configured to adapt only “on demand”. In the
latter case, once the tuning parameters have converged, the adaption component of the controller
turns itself off automatically, and the process is now controlled by a constant-tuning parameter
controller such as a standard PID controller.
1 One particularly ambitious application is described in [52, p429].
7.2. GAIN SCHEDULING 323

Adaptive module

Control design ✛ Identify M(θ)

✯ ❨

Update controller if necessary

❄ Measure input & output
+
r(t) ✲+ ✲ controller ✲ Plant, S ✲ y(t)
setpoint ✻− u(t) output

Figure 7.1: The structure of an indirect adaptive controller

7.1.2 C LASSIFICATION OF ADAPTIVE CONTROLLERS

As one may expect, there are many different types of adaptive control, all possessing the same
general components, and an almost equal number of classification schemes. To get an overview
of all these similar, but subtly different controllers, we can classify them according to what we
adapt;

Parametric adaption Here we continually update the parameters of the process model, or even
controller, or

Signal adaption where we vary a supplementary signal to be added directly to the output of the
static controller.

Alternatively we can classify them as to how we choose a performance measure,

Static A static performance measure is one such as overall efficiency, or maximising production.

Dynamic A dynamic performance measure is to optimise the shape of a response to a step test.

Functional of states and inputs This more general performance index is discussed more gener-
ally in chapter 9.

7.2 G AIN SCHEDULING

The simplest type of adaptive control is gain scheduling. In this scheme, one tries to maintain
a constant overall loop gain, K by changing the controller gain, Kc in response to a changing
process gain Kp . Thus the product Kc Kp is always constant.
324 CHAPTER 7. ADAPTIVE CONTROL

An example where gain scheduling is used is in the control of a liquid level tank that has a vary-
ing cross sectional area. Suppose we are trying to control the level in a spherical tank by adjusting
a valve on the outlet. The exiting flow is proportional to the square root of height of liquid above
the valve, which in turn is a nonlinear function of the volume in the tank. Fig. 7.2 shows how
this gain varies as a function of material in the tank, illustrating why this is a challenging control
problem. No single constant controller setting is optimal for both operating in the middle and
the extremes of the tank.

1.5

Plant gain K ∝ h1/2

1

p
0.5

Vmax
0
8
Controller gain Kc

4
Figure 7.2: The plant gain, Kp is pro-
portional to the square toot of the hy-
2
draulic head which in turn is a non-
linear function of volume in the tank.
0
Consequently the controller gain, Kc , 0 1 2 3 4 5
should be the inverse of the plant gain. Volume Vmax

A way around this problem is to make the controller parameter settings a function of the level
(the measured variable, y). Here the controller first measures the level, then it decides what
controller constants (Kc , τi , τd ) to use based on the level, and then finally takes corrective action
if necessary. Because the controller constants change with position, the system is nonlinear and
theoretical analysis of the control system is more complicated.

To do this in practice one must:

1. Determine the open loop gain at the desired operating point and several points either side
of the desired operating point.

2. Establish good control settings at each of these three operating points. (This can be done by
any of the usual tuning methods such as Ziegler-Nichols, Cohen-Coon etc)

3. Generate a simple function (usually piecewise linear) that relates the controller settings to
the measured variable (level).

The gain scheduling described above is a form of open-loop or feedforward adaption. The pa-
rameters are changed according to a pre-set programme. If the tank is in fact different from the
tank that was used to calculate the parameter adaption functions such as having a △ cross section,
then the controller will not function properly. This is because the controller is not comparing its
own system model with the true system model at any time. However despite this drawback, gain
scheduling is an appropriate way to control things like tanks of varying cross sectional area. It is
unlikely that the tank’s cross sectional area will change significantly with time, so the instrument
engineer can be quite confident that the process once tuned, will remain in tune.
7.3. THE IMPORTANCE OF IDENTIFICATION 325

7.3 T HE IMPORTANCE OF IDENTIFICATION

The adaptive controller (with feedback adaption) must be aware of the process, or more accu-
rately it must be aware of changes in process behaviour — hence the importance of system iden-
tification covered in the previous chapter. Central to the concept of controllers that adapt, is an
identification algorithm that constantly monitors the process trying to establish if any significant
changes have occurred. In the gain schedular given in §7.2, the identification component was
trivial, it was simply the level reading. The purists will argue that the gain schedular is not a true
adaptive controller, because if the dynamics of the system change (such as the cross section of
the tank suddenly varies), the controller is unaware of this change, and cannot take appropriate
corrective action.

There are two popular adaptive control techniques used currently in industry. The first approach
is to use a controller with auto-tuning. Normally these controllers are operating in just the same
manner as a traditional PID controller, but if the operator requests, the controller can be tuned
automatically. This is sometimes called “tuning on demand”. Once the auto-tuning part is com-
plete, the tuning procedure switches itself off and the controller reverts back to the traditional
PID mode. These types of “adaptive controllers” are used where adaptive control is required,
but the loop may be too sensitive or critical to have constant adaptation.

The second approach to adaptive control is called true adaptive control. This is where the con-
troller will adapt itself to the changing process without operator demand. Unlike the auto-tuners,
the “true adaptive” control remains continuously identifying the process and updating the con-
troller parameters. Many use variants of the least-squares identification that was described in
§6.7.

Industrially auto-tuners have been successful and are now accepted in the processing industries.
Most of the major distributed computer control companies provide an adaptive control in the
form of a self-tuner as part of their product (Satt Control, Bailey, ABB, Leeds-Northrup, Foxboro).
Åström and Hägglund describe some of these auto-tuning commercial products in chapter 5 of
[15, 105-132].

However the true adaptive control has not been accepted as much as the self-tuner. This is due to
the fact that true adaptive control requires a higher level of expertise than self-tuning regulators.
In addition, because the adaptive control is continuous, it requires a much larger safety network
to guarantee correct operation. Even though the adaptive controller does not require “tuning”,
the correct operation still requires estimates of values before the estimator can correctly func-
tion. These types of parameters are things such as expected model order, sampling period and
expected dead time. If these values are significantly different from the true values, then poor
adaption and hence control will result. Section 7.4 will consider self-tuning regulators.

7.3.1 P OLYNOMIAL MANIPULATIONS

The controller design strategies in this section will use the so-called polynomial approach. In this
design technique, we will need to be able to add and multiply polynomials, of possibly differing
lengths. The two routines given in Appendix B on page 519 add and multiply polynomials and
are used in the following controller design scripts in this chapter. An alternative programming
methodology is to create a polynomial object and then overload the plus and times operators.
326 CHAPTER 7. ADAPTIVE CONTROL

7.4 S ELF TUNING REGULATORS (STR S )

A better adaptive controller than a gain schedular is one where the controller parameters are
continually updated as the process characteristics change with time in an undetermined man-
ner. These are called self tuning regulators and an early application was in the pulp and paper
industry controlling a paper machine cited in [9]. Further development culminating in the self
tuning regulator is described in the classic reference, [12]. Self tuning regulators are more flexible
than gain scheduling controllers, which to be effective, the relation between the input and output
must be known beforehand and is assumed constant. A textbook covering many aspects of the
design and use of self tuning regulators is [204].

The basic elements of a self tuning regulator are:

1. Identify the model using input-output data. Usually the model is estimated online by using
a recursive least squares parameter estimation.

2. A relation between the model parameters estimated in step 1 and the controller parameters.

3. The control algorithm. Many different types of control algorithms exist, but many are func-
tions that try to adjust the controller parameters, given the continually changing process
parameters in such a way that the closed loop response is constant. Because the controller
is being continuously updated to the new varying conditions, these control algorithms can
be quite vigorous.

Step 1, the identification phase, was covered in chapter 6. Steps 2 and 3 are the controller tuning
and the controller structure sections and are discussed further in this chapter.

Suppose we have previously identified our plant under test using an ARX model of the form

A(q)yk = B(q)uk + ek (7.1)

then it is natural to ask what should the control law be? The simplest control law is a direct model
inversion. We choose u such that y = ek which is the best possible result given the unknown
noise. This results in the control law
A(q)
uk = yk (7.2)
B(q)
which is called minimum variance control, [9, 12, 14]. Expanding this control law gives the ma-
nipulated input at sample time k as

1
uk = (y ∗ + a1 yk−1 + a2 yk−2 + · · · + an yk−n − b1 uk−1 − · · · − bm uk−m ) (7.3)
b0 k

where yk∗ is the setpoint.

Clearly for this type of controller to be realisable, the b0 term in Eqn. 7.3 must be non-zero, or
in other words, the plant must have no dead time. With that constraint, this means that we can
calculate the present control action u that will drive y to the desired y ∗ .

Even without deadtime however, this type of control law will exhibit extremely vigorous action
in an attempt to bring the process back to the setpoint in a single sample time. While the simu-
lated results look very impressive, (see for example Fig. 7.3), in practice we will need to alter the
structure of the control law, relax it somewhat, to account for imperfect modelling and manipu-
lated variable constraints. These problems are partly circumvented in §7.6.4 and summarised in
§7.7.
7.4. SELF TUNING REGULATORS (STRS) 327

7.4.1 S IMPLE MINIMUM VARIANCE CONTROL

The algorithm to construct an adaptive minimum variance controller consists of two parts: the
identification of the model parameters, θ, in this instance using recursive least-squares, and the
implementation of the subsequent minimum variance control law.

Algorithm 7.1 Simple minimum variance control for plants with no delay

1. Measure the current output yt .

2. Form the new input/output data matrix X.
3. Calculate the prediction ŷt from ϕ and θt−1 .
4. Apply the control law, Eqn. 7.3, to obtain the current input, ut , using the estimated param-
eters.
5. Update the estimated parameters θk using a recursive least-squares algorithm such as Al-
gorithm 6.2 or equivalent.

A simple minimum variance controller is demonstrated in Listing 7.1 To test the simulation, we
will construct a siumulation where we abruptly swap between two different plants:

0.2 + 0.05q −1
G1 (q −1 ) = , for 0 ≤ t < 100 (7.4)
1 − 0.78q −1 − 0.65q −2 + 0.2q −3
0.3 + 0.15q −1
G2 (q −1 ) = , for t > 100 (7.5)
1 − 0.68q −1 − 0.55q −2 + 0.1q −3

For this simulation we will assume no model/plant structural mismatch and we will use the
recursive estimation with the forgetting factor option.

Listing 7.1: Simple minimum variance control where the plant has no time delay
t = [0:200]'; yspt = square(t/18); % Create time & setpoint
y=zeros(size(yspt)); u=zeros(size(y));
nr=0.02; % noise or dither sigma

5 n1 = [0.2 0.05]; d1 = [1 -0.78 -0.65 0.2]; % system #1, Eqn. 7.4

n2 = [0.3 0.15]; d2 = [1 -0.68 -0.55 0.1]; % system #2, Eqn. 7.5, for t > 100
thetatrue = [ones(t(1:100))*[d1 n1]; ones(t(101:length(t)))*[d2 n2]];

na = 4;nb = 2; dead=0; nt=na+nb; % Estimated model parameter orders, na , nb & nk .

10 lam = 0.95; P=1000*eye(nt); % Identification parameters: λ, P0
thetaest=rand(nt,1); % random start guess for θ 0
Param=rand(na,nt);Pv = ones(na,1)*diag(P)';

for i=na+1:length(yspt)
15 theta = thetatrue(i,:)'; % Actual plant, G
thetar=[thetaest(1:na); thetaest(na+2:nt)]; % Estimated plant, Ĝ
b0 = thetaest(na+1);
yv=y(i-1:-1:i-na); uv=u(i-1:-1:i-nb+1); % old output & input data
u(i)=(yspt(i)-[yv; uv]'*thetar)/b0; % Min. Var. control law
20

c = [yv', u(i:-1:i-nb+1)']; % Now do identification

328 CHAPTER 7. ADAPTIVE CONTROL

y(i) = c*theta(2:end) + 0.01*randn(1)*nr; % true process with noise

[thetaest,P] = rls(y(i),c',thetaest,P,lam); % RLS with forgetting
Param = [Param; thetaest']; Pv = [Pv;diag(P)'];
25 end % for loop

Initially we will set the forgetting factor λ to 0.99, and we will introduce a small amount of noise
so the estimation remains reasonable even under closed loop conditions, nr=0.02. We will aim
to follow the trajectory given as the dashed line in the upper figure of Fig. 7.3.

Our minimum variance controller will both try to estimate the true plant and control the process
to the setpoint, y ⋆ . The lower trend in Fig. 7.3 shows the parameter values for the estimated
model.

Minimum variance control

4
output & setpoint

G1 (q −1 ) G2 (q −1 )
2

−2
20

10
input

−10

−20
λ = 0.99
1

0
θ

−1
0 50 100 150 200 250 300
time

Figure 7.3: Simple minimum variance control with an abrupt plant change at t = 100. (See also
Fig. 7.4.)

In this unrealistically ideal situation with no model/plant mismatch, the controlled response
looks practically perfect, despite the plant change at t = 100 (after of course the converging
period). Fig. 7.4 shows an enlarged portion of Fig. 7.3 around the time of a setpoint change. Note
how the response reaches the setpoint in one sample time, and stays there. This is as expected
when using minimum variance control since we have calculated the input to force y exactly equal
to the desired output y ⋆ in one shot. In practice this is only possible for delay-free plants with no
model-plant mismatch.

Of course under industrial conditions (as opposed to simulation), we can only view the con-
trolled result, and perhaps view the trends of the estimated parameters, we don’t know if they
have converged to the true plant parameters. However if the estimates converge at all to steady
values, and the diagonal elements of the covariance matrix are at reasonable values, (these are ap-
plication dependent), then one may be confident, but never totally sure, that a reasonable model
has been identified.
7.5. ADAPTIVE POLE-PLACEMENT 329

Minimum variance control

1
output & setpoint

0.5

−0.5

−1

10

0
Figure 7.4: A zoomed portion of Fig. 7.3
input

showing the near-perfect response around

−10
a setpoint change when using simple min-
−20
35 40 45
imum variance control on a plant with no
time delay.

7.5 A DAPTIVE POLE - PLACEMENT

Clearly the problem with the simple minimum variance control is the excessive manipulated
variable demands imposed by the controller in an attempt to reach the setpoint in one sample
time. While our aim is to increase the speed of the open-loop response, we also need to reduce
the sensitivity to external disturbances. Both these aims should be addressed despite changes in
the process.

Fig. 7.5 shows a generalised control structure given in shift polynomial form. In this section we
are going to modify our colour scheme slightly. As shown in the figure, the controller polyno-
mials will be couloured blue hues, and the plant to be controlled, red hues. The B polynomial,
because it operates on the blue input is coloured purple, i.e. between blue and red.

Controller

pre-compensator Process
1 u(t) B(q −1 )
r(t) ✲ H(q −1 ) ✲+ ✲ ✲ ✲ y(t)
F (q −1 ) A(q −1 )
setpoint ✻−

G(q −1 ) ✛

Figure 7.5: Adaptive pole-placement control structure

If the controller polynomials G = H = 1, then we have the classic controlled feedback loop,
otherwise we have a slightly more general configuration known as a two-degree of freedom
configuration. Our aim as controller designers is to select polynomials F , G, H given process
polynomials B, A such that our performance aims are met. The closed loop transfer function of
Fig. 7.5 is
HB
y(t) = r(t)
F A + GB
330 CHAPTER 7. ADAPTIVE CONTROL

where clearly the stability is given by the denominator, F A + GB. Note that this polynomial
equation consists of two terms, each with a blue term multiplied by a red term. This fact will be
important later when we derive an algorithm to solve for the unknown blue polynomials.

Rather than demand the one-step transient response used in the minimum variance case, we can
relax this criterion, and ask only that the controlled response follow some pre-defined trajectory.

Suppose that we restrict the class of dynamic systems to the form,

Ay(t) = Bu(t − 1) (7.6)

where we have assumed a one sample time delay between u and y. This is to leave us enough
time to perform any controller calculations necessary. We will use the controller structure given
in Fig. 7.5,
F u(t) = Hr(t) − Gy(t) (7.7)
where the polynomials F, G, H are of the form:

F = 1 + f1 q −1 + f2 q −2 + · · · + fnf q −nf
G = g0 + g1 q −1 + g2 q −2 + · · · + gng q −ng
H = h0 + h1 q −1 + h2 q −2 + · · · + hnh q −nh

With the computational delay, the closed-loop is

(F A + q −1 BG)y(t) = q −1 BHr(t) (7.8)

and we desire the closed-loop poles to be assigned to specified locations specified in the polyno-
mial T ,
T = 1 + t1 q −1 + t2 q −2 + · · · + tnt q −nt (7.9)
Assigning the closed-loop poles to given locations is called pole-placement or pole assignment.
(In practice, the closed loop polynomial T will be a design requirement given to us perhaps by
the customer.)

Note that some authors such as [21] use a slightly different standard nomenclature convention
for the controller polynomials. Rather than use the variable names F, G, H in the control law
Eqn. 7.7, the variable names R, S, T are used giving a control law of Ru(t) = T r(t) − Sy(t).

7.5.1 T HE D IOPHANTINE EQUATION AND THE CLOSED LOOP

To assign our poles specified in T , we must select our controller design polynomials F, G such
that the denominator of the actual closed-loop is what we desire,

F A + q −1 BG = T (7.10)

where I have coloured the design polynomial red indicating that it is to drive the output to the
setpoint. The polynomial equation Eqn. 7.10 is called the Diophantine equation, and may have
many solutions. To obtain a unique solution, A, B must be coprime (no common factors), and we
should select the order of the polynomials such that

nf = nb
ng = na − 1, (na 6= 0)
nt ≤ na + nb − nc
7.5. ADAPTIVE POLE-PLACEMENT 331

One way to establish the coefficients of the design polynomials F, G and thus solve the Diophan-
tine equation is by expanding Eqn. 7.10 and equating coefficients in q −i giving a system of linear
equations.  
 1 0 ··· 0 b0 0 ··· 0 
 
..  f1
 
 .. ..  
 a1
 1 . b1 b0 . . 
 
 t1 − a 1
 .. .. .. ..   f2   t2 − a 2 
 a
 2 a1 . . . b1 .  
 .
 
  ..


  .   . 
 .. . .. ..   .
a2 . .
  
 .
 bnb. .  
  fnf
 
  tnt − ant 

 .. . .   
= −ant+1 
(7.11)
a
 na . . 1 0 bnb 0  
  g0   ..



.. .    . 
ana . a1 ..
    
 0 0 ··· b0   g1   −ana 
    
 . .. ..    0 
 .. . a2 . · · · · · ·
0 b1    ..
   

.
 ..
 .
 . .. 
..    .
 . . .  gng | {z }
  b
0 · · · · · · ana 0 · · · · · · bnb | {z }
| {z } θc
A
The matrix A in Eqn. 7.11 has a special structure, and is termed a Sylvester matrix and will be
invertible if polynomials A, B are co-prime. Solving Eqn. 7.11
θ c = A−1 b
gives us the coefficients of our controller polynomials, F , G. Now the dynamics of the closed-
loop requirement are satisfied,
BH
y(t) = r(t − 1)
T
but not the steady-state gain, which ideally should be 1. We could either set H to be the scalar,

T
H =  (7.12)
B q=1
which forces the steady-state gain to 1, or preferably cancel the zeros
1
H= (7.13)
B
which improves the transient response in addition to keeping the steady-state gain of unity. Can-
celling the zeros should only be attempted if all the zeros of B are well inside the unit circle.
Section 7.6.1 describes what to do if the polynomial B does has poorly damped or unstable zeros.

This scheme has now achieved our aim of obtaining a consistent dynamic response irrespective
of the actual process under consideration. Since we cannot expect to know the exact model B, A
we will use an estimated model, B̂, Â in the controller design.

7.5.2 S OLVING THE D IOPHANTINE EQUATION IN M ATLAB

Since solving the Diophantine equation is an important part of the adaptive pole-placement reg-
ulator, the following functions will do just that. Listing 7.2 solves for the polynomials F and G
such that F A + BG = T . If a single time delay is introduced into the B polynomial such as
in Eqn. 7.10, then one needs simply to pad the B polynomial with a single zero. Note that the
expression AF + BG is optionally computed using the polyadd routine given in Listing B.1, and
this polynomial should be close to the T polynomial specified.
332 CHAPTER 7. ADAPTIVE CONTROL

Listing 7.2: A Diophantine routine to solve F A + BG = T for the polynomials F and G.

function [F,G,Tcheck] = dioph(A,B,T);
% [F,G] = dioph(A,B,T);
% Solves the Diophantine equation by setting up the Sylvester matrix,
% AF + BG = T and subsequently solves for polynomials F & G.
5 % See [18, p175] (follows nomenclature of except R=F, S=G).

if abs(A(1)-1) > 1.0e-8;

error('Error: Polynomial A not monic A(1) = %g\n',A(1));
end % if
10

na = length(A)-1; nb = length(B)-1; % Order of polynomials A(q −1 ), B(q −1 )

nf = nb; ng = na-1; n = nf+ng+1; n = nf+ng+1; % Required orders for F & G
D = zeros(n,n); % construct Sylvester matrix
for i=1:nf
15 D(i:i+na,i) = A'; % fill in columns
end % for i
for i=1:ng+1
D(i:i+nb,nf+i) = B'; % fill in columns
end % for i
20 rhs = [A(2:na+1), zeros(size(na+1:n))]'; % right-hand side
rhs = [T(2:length(T)), zeros(size(length(T):n))]' - rhs;

FG= D\rhs; % Find solution for polynomials F & G

F= FG(1:nf)'; G= FG(nf+1:n)';% Monic polynomial F has leading 1 dropped.
25

if nargout >2 % then check that AF + BG = T as required

Tcheck = polyadd(conv(A,[1,F]), conv(B,G));
end
return

A N ALTERNATIVE D IOPHANTINE SCHEME

An alternative, and somewhat more elegant, scheme for solving the Diophantine equation is
given in Listing 7.3 which uses matrix convolution to construct the Sylvester matrix.

Listing 7.3: Alternative Diophantine routine to solve F A + BG = T for the polynomials F and
G. Compare with Listing 7.2.
1 function [F,G,Tcheck] = dioph_mtx(A,B,T);
% Alternative version for polynomial Diophantine equation solving
% Solves AF + BG = T for F & G.

da = length(A)-1; db=length(B)-1; % da = deg(A) etc.

6 T = [T, zeros(1,da+db-length(T)+1)]; % pad with zeros
dt=length(T)-1; % Convert to full length
dg =da-1; df=dt-da ; % assuming for RAB to be square
B = [ zeros(1, df-db+1), B]; % pad with leading zeros

11 Rac = [convmtx(A,df+1);convmtx(B, dg+1)]; % Construct RAB = [RA ; RB ]

FG = T/Rac; % [F, G] = T R−1

AB
F = FG(1:df+1); G = FG(df+2:df+dg+2);% Note F & G are row vectors.

16 Tcheck = polyadd(conv(A,F), conv(B,G)); % Verify solution

7.5. ADAPTIVE POLE-PLACEMENT 333

return

The alternative scheme in Listing 7.3 is slightly different from the one presented earlier in that
this time the F polynomial contains the leading unity coefficient. A third method to construct
the Sylvester matrix is to use the lower triangle of a Toeplitz matrix in M ATLAB say with the
command tril(toeplitz(1:5)).

E XAMPLE OF A D IOPHANTINE PROBLEM

Suppose we only want to place one pole, T = 1 + t1 q −1 , given a na = 3, nb = 2 model. The

recommended orders of the controller polynomials are
nf = nb = 2
ng = na − 1 = 2
nt = 1 ≤ na + nb − nc = 5
and expanding the closed-loop equation Eqn. 7.10,
(1 + f1 q −1 + f2 q −2 )(1 + a1 q −1 + a2 q −2 + a3 q −3 )+
q −1 (b0 + b1 q −1 + b2 q −2 )(g0 + g1 q −1 + g2 q −2 ) = 1 + t1 q −1
which by equating the coefficients in q −i , i = 1, . . . , 5 gives the linear system following Eqn. 7.11
    
1 0 b0 0 0 f1 t1 − a 1
 a1 1 b1 b0 0   f2   −a2 
    
 a2 a1 b2 b1 b0   g0  =  −a3 
    
 a3 a2 0 b 2 b 1   g 1   0 
0 a3 0 0 b 2 g2 0

A N EXAMPLE OF SOLVING THE D IOPHANTINE EQUATION

In this example we start with some known polynomials, and then try to reconstruct them using
the Diophantine routines presented previously.

Suppose we start with some given plant polynomials, say

A(q −1 ) = 1 + 2q −1 + 3q −2 + 3q −3 , B(q −1 ) = 5 + 6q −1 + 7q −2
where the orders are na = 3 and nb = 2. This means we should choose orders nf = nb = 2, and
ng = na − 1 = 2, so let us invent some controller polynomials
F (q −1 ) = 1 + 8q −1 + 9q −2 , G(q −1 ) = 10 + 11q −1 + 12q −2
noting the A and F are monic. We can now compute the closed loop, AF + BG using convolution
and polynomial addition as shown in Listing 7.4

Listing 7.4: Constructing polynomials for the Diophantine equation example

>> A = [1 2 3 4]; B = [5 6 7];
>> F = [1 8 9]; G = [10 11 12]; % Note nf = nb & ng = na − 1.
3

>> T = polyadd(conv(A,F),conv(B,G)) % Compute T (q −1 ) = A(q −1 )F (q −1 ) + B(q −1 )G(q −1 )

T =
1 60 143 242 208 120
334 CHAPTER 7. ADAPTIVE CONTROL

which gives T (q −1 ) = 1 + 60q −1 + 143q −2 + 242q −3 + 208q −4 + 120q −5 .

We are now ready to essentially do the reverse. Given the polynomials A(q −1 ), B(q −1 ), and
T (q −1 ), we wish to reconstruct F (q −1 ) and G(q −1 ) using the Diophantine routines given in List-
ings 7.2 and 7.3.

Listing 7.5: Solving the Diophantine equation using polynomials generated from Listing 7.4.
>> [F1,G1,Tcheck1] = dioph(A,B,T) % Use Listing 7.2
F1 =
8.0000 9.0000
4 G1 =
10.0000 11.0000 12.0000
Tcheck1 =
1 60 143 242 208 120

9 >> [F2,G2,Tcheck2] = dioph_mtx(A,B,T) % Use Listing 7.3

F2 =
1 8 9
G2 =
10.0000 11.0000 12.0000
14 Tcheck2 =
1 60 143 242 208 120

Note that in both cases we manage to compute the polynomials F (q −1 ) and G(q −1 ) which recon-
struct back to the correct T (q −1 ), although the routine in Listing 7.2 drops the leading 1 in the
monic F polynomial.

7.5.3 A DAPTIVE POLE - PLACEMENT WITH IDENTIFICATION

We have now all the components required for an adaptive controller. We will use an RLS mod-
ule for the plant identification, and then subsequently solve the Diophantine equation to obtain
a required closed loop response. Algorithm 7.2 describes the algorithm behind this adaptive
controller.

Algorithm 7.2 Adapting pole-placement algorithm

We will follow Fig. 7.6 which is a specific case of the general adaption scheme given in Fig. 7.1.

1. At each sample time t, collect the new system output, y(t).

2. Update the polynomial model estimates A, B using recursive least-squares on

Ay(t) = Bu(t − 1) + e(t)

3. Synthesize the controller polynomials, F, G by solving the identity,

F A + q −1 BG = T

where T is the desired closed-loop response. (You could use the dioph routine given in
Listing 7.2.)

4. Construct H using either Eqn. 7.12 or Eqn. 7.13.

7.5. ADAPTIVE POLE-PLACEMENT 335

Pole-placement identify
controller design plant

Performance Â, B̂
requirements ✲ Diophantine ✛ RLS ✛ Structure
T ✶ ❨
F, G & H
❄❄❄

1 −1
r(t) ✲ H ✲+ ✲ u(t) ✲ B(q ) ✲ y(t)
F (q −1 ) −1
A(q )
setpoint ✻− output

Unknown plant
G(q −1
)✛

2 DOF controller

Figure 7.6: Adaptive pole-placement control structure with RLS identification

5. Apply control step

F u(t) = −Gy(t) + Hr(t)

6. Wait out the remainder of the sample time & return to step 1.

A N ADAPTIVE CONTROLLER WITH RLS IDENTIFICATION EXAMPLE

In this example will attempt to control three different plants in the configuration as shown in
Fig. 7.7. The open loop step tests of the plants are compared in Fig. 7.8. Of the three plants, G1 is
stable and well behaved, G2 has open-loop unstable poles, and G3 has unstable zeros.

In practice we must estimate the process model and if the plant changes, we redesign our con-
troller such that we obtain the same closed loop response. The customer for this application
demands a slight overshoot for the controlled response as shown in Fig. 7.9. We can translate this
requirement for the denominator of the closed-loop transfer function to be

T = 1 − 1.4685q −1 + 0.6703q −2

Listing 7.6 and Fig. 7.10 show the impressive results for this combined identification and adaptive
control simulation when using the three plants from Fig. 7.8. Note that the controller design
computation is done in the dioph.m routine given in Listing 7.2 while the identification is done
using the rls routine.

Here I assume a model structure where na = 5, nb = 3, and I add a small amount of noise,
σ = 0.005, to keep the estimation quality good. A forgetting factor of λ = 0.95 was used since the
model changes were known a priori to be rapid.
336 CHAPTER 7. ADAPTIVE CONTROL

Plant alternatives
z }| {

✲ current active plant

G1
setpoint ❄ switch
✲
r Gc ✲ G2 ✲ ✲ y
✲
✻
Adaptive ✲ G3
controller

current inactive plant models

Figure 7.7: Control of multiple plants with an adapting controller. We desire the same closed
loop response irrespective of the choice of plant.

Open−loop plant step responses

6000 Pole−Zero Map

1.5
5000 G
1
G2
1

4000
G2 (q−1 ) G
3
Imaginary Axis

−1
G1 (q ) 0.5
output

3000
0

2000
−0.5

1000
G3 (q−1 )
−1

0 −1.5
0 20 40 60 80 100 120 −1 −0.5 0 0.5 1 1.5
time Real Axis

Figure 7.8: Step responses of the 3 discrete open-loop plants (one of which is unstable), and the
corresponding pole-zero maps. Note that G3 has zeros outside the unit circle.

Desired closed loop response

1.4

1.2

0.8
output

1
T(q −1 )
0.6

0.4

0.2

Figure 7.9: The desired closed loop response, 0

0 5 10 15 20 25 30 35
1/T . time
7.5. ADAPTIVE POLE-PLACEMENT 337

Listing 7.6: Adaptive pole-placement control with 3 different plants

Gx=struct('b',[3 0.2 0.5],'a',poly([0.95 0.9 0.85])); % 3 different plants
Gx(2) = struct('b',[2,2,1],'a',poly([1.1 0.7 0.65]));
Gx(3) = struct('b',[0.5 0.2 1],'a',poly([0.6 0.95 0.75]));

5 na = 3; nb = 3; % order of polynomials, na , nb
a = randn(1,na); b = randn(1,nb); % model estimates
np = length([a,b]); % # of parameters to be estimated
P = 1.0e4*eye(np); % Large initial co-variance, P0 = 104 I
lambda = 0.95; % Forgetting factor, 0 < λ < 1
10 param_est = [a,b]';

% Design a reasonable CLTF response with τ = 2 and ζ = 0.4

tau=2; zeta = 0.4; Grc = tf(1,[tauˆ2 2*tau*zeta 1]);
Grd = c2d(Grc,1); T = cell2mat(Grd.den);
15

dt = 1; t = dt*[0:700]'; r = square(t/10); % sample time & setpoint

y = zeros(size(t)); u = y; noise = 0.005*randn(size(t));
Param = zeros(length(t),np); trP = zeros(size(t));% plot holder
tG2 = 150; tG3 = tG2+200; k=1; % Swap over times for the different plants
20

for i=10:length(t)
if t(i) == tG2, k=2; end % Swap over to model #2
if t(i) == tG3, k=3; end % Swap over to model #3

25 % Plant response
y(i) = Gx(k).b*u(i-1:-1:i-length(Gx(k).b)) - ...
Gx(k).a(2:end)*y(i-1:-1:i-length(Gx(k).a)+1);

% Model identification
30 x = [-y(i-1:-1:i-na)', u(i-1:-1:i-length(b))']; % shift register
[param_est,P] = rls(y(i),x',param_est,P,lambda); % update parameters, Listing 6.18.
a=[1 param_est(1:na)']; b=param_est(na+1:na+nb)';% extract estimates: Â, B̂
Param(i,:) = param_est'; trP(i) = trace(P);

35 % Controller design
[F,G] = dioph(a,b,T); % Solve Diophantine polynms, Listing 7.2.
%[F,G] = dioph_mtx(a,b,T); F(1)=[]; % Alternative Diophantine, Listing 7.3.
H = sum(T)/sum(b); % Get steady-state correct, H = T /B|q=1 .

40 % Control law, F u = −Gy + Hr

u(i) = H*r(i) - G*y(i:-1:i-length(G)+1) - F*u(i-1:-1:i-length(F));
end % for i

%% Check the results

45 G = tf(sum(T),T,1); % desired closed loop
yref = lsim(G,r);

Fig. 7.10 shows the impressive results of this adaptive controller. When the plant changes, un-
beknown to the controller, the system identification updates the new model, and the controller
consequently updates to deliver a consistent response.

If the pole-placement controller is doing its job, then the desired closed loop response should
match the actual output of the plant. These two trajectories are compared in Fig. 7.11 which
shows an enlarged portion of Fig. 7.10. Here it can be seen that apart from the short transients
when the plant does change, and the identification routine is struggling to keep up, the adaptive
pole-placement controller is satisfying the requirement to follow the reference trajectory.
338 CHAPTER 7. ADAPTIVE CONTROL
output & setpoint

2
1
0
−1
−2
−3
−4

0.5
input

−0.5

1 b
2
b
a0
3
0 b
a1
θ

2
−1 a
1
−2
trace(P)

0 50 100 150 200 250 300 350 400 450

sample time

Figure 7.10: Adaptive pole-placement with RLS identification. After a period for the estimated
parameters to converge (lower trends), the controlled output response (upper trend) is again
reasonably consistent despite model changes. Results from Listing 7.6.

1.8 y

1.6 Reference, y*

1.4

1.2

0.8

0.6
0 50 100 150 200 250 300 350 400 450 500
time

Figure 7.11: An enlarged portion of Fig. 7.10 showing the difference between the actual plant
output y(t), and the reference output, y ∗ (t).
7.5. ADAPTIVE POLE-PLACEMENT 339

7.5.4 A DAPTIVE POLE - PLACEMENT IN S IMULINK

The time-varying filter blocks make it relatively easy to implement an adaptive pole-placement
controller in S IMULINK as showed in Fig. 7.12. The controller design is executed in the orange
‘RLS & APP’ sub-block which is shown in Fig. 7.13. This sub-block contains two m-file scripts
that first do the RLS model identification update, and then do the pole-placement design based
on the currently identified model. The various memory blocks and the initial condition blocks
are needed to ensure that we do not have any algebraic loops when we start the simulation.

Figure 7.12: Adaptive pole-placement with identification in S IMULINK. See also Fig. 7.13.

Figure 7.13: Internals of the Adaptive pole-placement design routine from Fig. 7.12.
340 CHAPTER 7. ADAPTIVE CONTROL

7.6 P RACTICAL ADAPTIVE POLE - PLACEMENT

Some problems still remain before the adaptive pole-placement algorithm is a useful and practical
control alternative. These problems include:

• Poor estimation performance while the system is in good control, and vice-versa, (poor
control during the periods suitable for good estimation). (This is known as the dichotomy
of adaptive control, or dual control).

• Non-minimum phase systems possessing unstable zeros in B which must not be cancelled
by the controller.

• Dead time where at least b0 = 0.

We must try to solve all these practical problems before we can recommend this type of controller.

A N EXAMPLE OF BURSTING

If our controller does its job perfectly well, then the output remains at the setpoint. Under these
circumstances we do not have any identification potential left, and the estimated model starts to
drift. The model will continue to drift until the controller deteriorates so much that there is a
period of good excitation (poor control), that the model re-converges again. This

Fig. 7.14 shows the experimental results from using an adaptive controller, (in this case an adap-
tive LQR regulator) on the blackbox over a substantial period of time. The controller sampling at
T = 1 second was left to run over night for 10 hours (36,000 data points), and for a 3 hour period
in the middle we had no setpoint changes, and presumably no significant disturbances. During
this time, part of which is shown in Fig. 7.14, the adaptive controller oscillated into, and out of,
good control. This is sometimes known as “bursting”.

7.6.1 D EALING WITH NON - MINIMUM PHASE SYSTEMS

If we use the steady-state version for H,


def T 
H =
B q=1

as proposed in Eqn. 7.12, then we do not cancel any zeros, and we avoid the problem of inadver-
tently inverting unstable model zeros. However we can achieve a better transient response if we
cancel the zeros with the pre-compensator H,

1
H=
B
giving the closed loop transfer function,

HB 1
y(t) = r(t − 1) = r(t − 1)
T T
which looks fine until we investigate closer what is likely to happen if B has zeros outside the
unit circle. Remember that the polynomial B is a function of the plant that we are trying to
7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT 341

Adaptive LQR of the blackbox ∆ t=1.00 (sec)

0.4

0.3
output & setpoint

0.2

0.1

−0.1

−0.2

−0.3
2.5 3 3.5 4

RLS: λ=0.995
1

0.5
Input

−0.5

−1
2.5 3 3.5 4
time (hours)

Figure 7.14: Bursting phenomena exhibited by the black box when using an adaptive controller.
With no setpoint changes or external disturbances, the adaptive controller tends to oscillate be-
tween periods of good control and bad estimation and the reverse.

control and hence an output of the estimation scheme. We have no influence on the properties of
B; we get what we are given.

In fact, we will have problems not only if the zeros are unstable, but even if they are stable but
poorly damped. Fig. 7.15 shows the adaptive pole-placement response of a plant with stable, but
◦
poorly damped zeros when using H = 1/B. In fact the zeros are 0.95e±j140 indicating that they
are close to, but just inside the unit circle. While the output response looks fine, it is the ringing
behaviour of the input that should cause concern.

The results in Fig. 7.15 were obtained by using the modified controller given in Listing 7.7 in
Listing 7.6.

Listing 7.7: The pole-placement control law when H = 1/B

if any(abs(roots(b))>1) % Use H = T /B|q=1 if any zeros of B are unstable.

H = polyval(T,1)/polyval(b,1); % Simple scalar pre-filter, H = T /B|q=1 .

4 u(i) = H*r(i) - G*y(i:-1:i-ng) - F*u(i-1:-1:i-nf);
else % Alternatively, use dynamic compensation, H = 1/B.
v(i) = 1/b(1)*(-b(2:end)*v(i-1:-1:i-nb+1)+sum(T)*r(i));
u(i) = v(i) - G*y(i:-1:i-ng) - F*u(i-1:-1:i-nf); % control action
end

Suppose we have estimated a B polynomial possessing an unstable zero at say q −1 = +2/3, or

B = 1 − 1.5q −1
342 CHAPTER 7. ADAPTIVE CONTROL

3
yref
2 y*
output & setpoint

y
1

−1

−2
2

1.5

0.5
input

−0.5

−1

−1.5

−2
0 50 100 150 200 250 300 350 400 450 500
sample time

Figure 7.15: Adaptive pole-placement with a stable, but poorly damped, B polynomial. The
poorly damped zeros cause the ringing behaviour in the input.

and we desire T = 1 − 0.7q −1 . Then the closed-loop transfer function is

BH q −1 (1 − 1.5q −1 )
y(t) = −1
r(t − 1) = Hr(t)
F A + q BH 1 − 0.7q −1

Now suppose we use a dynamic pre-compensator, H = 1/B,

1
H=
q(1 − 1.5q −1 )

which has a time-advance term, q +1 , meaning that the control law is a non-causal function of the
setpoint. In other words, we need to know r(t + 1) at time t. In fact since we need to know only
future values of setpoint, and not output, this type of control law is easily implemented.

The main drawback stems, however, not from the acausality, but from the ill-advised cancel-
lation. In all practical cases, we will experience some small numerical round-off error such as
where the pole is represented in a finite-word length computer as 1.50001 rather than exactly 1.5.
Consequently the pole will not completely cancel the zero,

q −1 (1 − 1.5q −1 ) q
y(t) = × r(t)
1 − 0.7q −1 (1 − 1.50001q −1)
| {z }
small error

leaving an unstable controller.

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT 343

The remedy is that if we discover unstable modes in B, we simply do not cancel them, although
we continue to cancel the stable modes. We achieve this by separating the B polynomial into two
groups of factors: the stable roots, B + , (which we will colour ‘green for good’) and the unstable
roots, B − , (which we will colour ‘red for bad’)
def
B = B+B− (7.14)

and then set H = 1/B + only. A simple M ATLAB routine to do this factorisation is given in
Listing 7.8 in section 7.6.2.

So given the model and control law,

Ay(t) = Bu(t − 1)
F u(t) = −Gy(t) + Hr(t)

we modify the pole assignment to

AF + q −1 BG = T B + (7.15)

so closed-loop transfer function is

HB HB −
y(t) = +
r(t − 1) = r(t − 1) (7.16)
TB T

We solve the modified pole-assignment problem, Eqn. 7.15, by noting that B + must be a factor of
F,
F = B + F1
so we solve the now reduced Diophantine equation

AF1 + q −1 B − G = T

in the normal manner.

We should always check for unstable zeros in the B polynomial when using online estimation
because we have no control over what the RLS routine will deliver. Even in a situation where we
know, or reasonably suspect a stable B, we may stumble across an unstable B while getting to
the “correct” values.

As an example, we can apply the adaptive pole-placement controller where we cancel the stable
B polynomial zeros, but not the unstable ones, using the three plants from page 335. Fig. 7.16
shows the result of of this controller where we note that the performance of plant #3 is also
acceptable exhibiting no ringing.

For diagnostic purposes, we can also plot the instantaneous poles and zeros of the identified
model, and perhaps display this information as a movie, or perhaps as given in Fig. 7.16, shown
with decreasing intensity of colour for the older data. This will show how the poles and zeros
migrate, and when zeros, or even poles, leap outside the unit circle. Fig. 7.16 also shows the
magnitude of the estimated poles and zeros which clearly indicate when they exceed 1 and jump
outside the unit circle.

7.6.2 S EPARATING STABLE AND UNSTABLE FACTORS

When applying pole-placement in the case with unstable zeros, we need to be able to factor
B(q) into stable and unstable factors. Listing 7.8 does this factorisation, and additionally it also
344 CHAPTER 7. ADAPTIVE CONTROL

2 G1 (q −1 ) G2 (q −1 ) Unstable zeros: G3 (q −1 )
output & setpoint

−1

−2
0.2

0.1
input

−0.1

−0.2
4
λ=0.99
2 a
2
b
b 2
0
0 b
θ

1
a
3

−2 a1

−4
5
10
tr(P)

0
10
0 500 1000 1500
sample time

def
Figure 7.16: Adaptive pole-placement control with H = 1/B and an unstable B polynomial in
Plant #3. Note that the controller does not exhibit any ‘ringing’, even with a plant with unstable
zeros.
7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT 345

Pole−Zero Map
1.5
1.5 z1, z2

1
0.5 p1

|poles & zeros|

Imaginary Axis

p2
0

p3

−0.5
0.5

−1

−1.5 0
−1 −0.5 0 0.5 1 1.5 0 500 1000 1500
Real Axis sample time

Figure 7.17: The migration of poles and zeros in the complex plane (left hand plot) and the time
trend of the magnitude of the poles and zeros (right-hand plot) from the simulation presented in
Fig. 7.16.

rejects the stable, but poorly damped factors given a user-specified damping ratio, ζ. (If you want
simply stable/unstable, use ζ = 0.) Fig. 7.18 illustrates the region for discrete well-damped poles
where ‘well-damped’ is taken to mean damping ratios ζ > 0.15. This algorithm is adapted from
[150, p238].

Listing 7.8: Factorising an arbitrary polynomial B(q) into stable, B + (q), and unstable and poorly
damped, B − (q), factors such that B = B + B − and B + is defined as monic.
function [Bp,Bm] = factorBz(B,zeta);
2 % Factorise B(q) = B + (q)B − (q) where B + are factors inside the unit circle and B − (q) are outside.
% The optional parameter, ζ defines the cutoff. Use ζ = 0 for inside/outside circle.
% Note B + (q) or Bp is defined as monic.

if ¬(exist('zeta') == 1)
7 zeta = 0.15; % default reasonable cut-off is ζ = 0.15.
end
zeta = max(min(zeta,1),0); % Ensure ζ ∈ [0, 1].
tol = 1e-6; r = roots(B);

12 % Now separate into poorly damped & good poles

mag_r = abs(r); rw = angle(r)/2/pi; % See equations in [150, p238].
mz = exp(-2*pi*zeta/sqrt(1-zetaˆ2)*rw); % magnitude of z
idx = mag_r < mz;

17 Bp = poly(r(idx)); % Set B + (q) to contain all the zeros inside unit circle
Bm = poly(r(¬idx))*B(1); % All the others

if norm(conv(Bp,Bm)-B) > tol % Check B = B + B −

warning('Problem with factorisation')
22 end % if
return
346 CHAPTER 7. ADAPTIVE CONTROL

Figure 7.18: The boundary of the shaded area

is the loci of constant damping ratio with
ζ = 0.15. Inside this region, the poles are con-
sidered well damped. (Figure plotted with
zgrid.)

7.6.3 E XPERIMENTAL ADAPTIVE POLE - PLACEMENT

Actually running an adaptive pole-placement scheme on a real plant, even a well behaved one
in the laboratory is considerably more difficult than the simulation examples presented so far.
Our desire is to test an adaptive pole-placement controller on the black-box to demonstrate the
following:

1. To follow a specified desired trajectory (as quantified by the T polynomial),

2. under both the ‘NORMAL’ and ‘SLOW’ positions on the black box, and
3. to run stably for at least 10 minutes following both disturbances and servo changes.

The first design decision is to choose a desired response (closed loop poles) and suitable sampling
interval. We shall select a second-order prototype response with a time constant of τ = 2.1
seconds and some overshoot, ζ = 0.6. With a sampling time of T = 0.6 seconds, the desired
discrete closed loop transfer function is
1 0.03628 + 0.03236q −1
G⋆cl (s) = 2
→ G⋆cl (q −1 ) =
4.41s + 2.52s + 1 1 − 1.641q −1 + 0.7097q −2
from which follows that T = 1 − 1.641q −1 + 0.7097q −2.

Fig. 7.19 shows the results from one experiment where the characteristics of the plant are changed
at t = 128 by manually switching over from 7 cascaded low-pass filters to 9. This increases the
effective time constant of the plant.

It is important to verify that the output from the blackbox did in fact follow the intended trajec-
tory once the model parameters have converged. An enlarged portion of Fig. 7.19 is repeated in
Fig. 7.20 comparing the intended trajectory, y ⋆ , (dashed line) with the actual trajectory, y. This
shows that the response converged to the desired trajectory even after the plant subsequently
changed. Fig. 7.19 shows clearly that the shape of the input must vary considerably to force the
same output if the plant itself changes.

7.6.4 M INIMUM VARIANCE CONTROL WITH DEAD TIME

The minimum variance control formulation of §7.4.1 is not very useful not the least due to its in-
ability to handle any dead time and unstable process zeros. Even without the dead time, practical
7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT 347

Adaptive pole−placement with RLS

0.4
output & setpoint

0.3 Normal Slow

0.2
0.1
0
−0.1
−0.2

∆ t=0.6,
1

0.5
input

−0.5

−1

λ=0.99, na=4, nb=3

2

1
parameters

−1

−2

5
10
tr(P)

4
10

3
10
0 50 100 150 200 250 300 350 400
time (sec)

Figure 7.19: Adaptive pole-placement of the black-box showing consistent output response be-
haviour despite the plant change when the switch was manually activated at t = 128 seconds.
Plot (A): black-box output (solid), intended trajectory (dashed) and setpoint point (dotted). (Re-
fer also to Fig. 7.20 for an enlarged version.)
Plot (B): Controller input to the plant, Plot (C): Plant parameter estimates, and Plot (D): Trace of
the covariance matrix.
348 CHAPTER 7. ADAPTIVE CONTROL

0.28

actual y
setpoint
0.26 Normal Slow intended y*

0.24

output & setpoint

0.22

Figure 7.20: A comparison of the in- 0.2

tended trajectory, y ⋆ (dashed line)
with the actual trajectory y (solid
line) for the black box from Fig. 7.19. 0.18

Even after the plant change, the ac-

tual response gradually converges to 0.16
60 80 100 120 140 160 180 200 220 240 260 280
the desired trajectory. time (sec)

experience has shown that this control is too vigorous and tends to be over-demanding causing
excessive manipulator variable fatigue. Moreover when using discrete time models, delay be-
tween input and output is much more common, and typically the deadtime is at least one sample
period. The best we can do in these circumstances is to make the best prediction of the output
we can for d samples times ahead, and use this prediction in our control law as done above. This
introduces the general concept of minimum variance control which combines prediction with
control. Further details are given in [22, pp367–379].

Suppose we start with the ARMAX canonical model written in the forward shift operator q,

A(q)yk = B(q)uk + C(q)ǫk (7.17)

and we assume that A and C are monic. We can always divide out the coefficients such that A
is monic, and we can choose the variance of the white noise term so that C is monic. For this
simplified development we will also need to assume that all the zeros of both C and B are inside
the unit circle. The pole excess of the system d, is the difference in the order of the A and B
polynomials and is equivalent to the system time delay. The output d steps ahead in the future is
dependent on the current and past control actions, u, which are known, and the current, past and
future disturbances, the latter which are not known. To avoid using the unknown disturbance
values we need to make an optimal prediction.

O PTIMAL PREDICTION

If we have a process driven by white noise ǫ,

C(q)
zk = ǫk (7.18)
A(q)

then the minimum variance predictor over d steps is

qG(q)
ẑk+d|k = zk (7.19)
C(q)

where G(q) is defined from

q d−1 C(q) = A(q)F (q) + G(q) (7.20)
7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT 349

We can calculate the quotient F and remainder G polynomials in Eqn. 7.20 from polynomial
division in M ATLAB using the deconv function, or alternatively, one could calculate them by
equating the coefficients in Eqn. 7.20.

M INIMUM VARIANCE CONTROL LAW

Once we can make an optimal prediction using the formulation of Eqn. 7.20, we can predict the
output d samples times into the future and then calculate the control action required to equate
this with the setpoint. This will give the desired minimum variance control law as,
G(q)
uk = − yk (7.21)
B(q)F (q)
The closed loop formulation by combining Eqn. 7.17 and Eqn. 7.21 is
G
Ayk = −B yk + C ǫk
BF
AF yk = −Gyk + CF ǫk
CF
yk = ǫk = F ǫk (7.22)
AF + G
where the last equation is simplified using Eqn. 7.20. Eqn. 7.22 indicates that the closed loop
response is a moving average process of order d − 1. This provides us with a useful check in that
the covariance of the error will vanish for delays larger than d − 1.

Listing 7.9 designs the F (q) and G(q) polynomials required in the control law, Eqn. 7.21, by using
deconvolution. Note this routine uses stripleadzeros to strip any leading zeros from the
B(q) polynomial from Listing B.3 given in Appendix B.

Listing 7.9: Minimum variance control design

function [F,G,BF] = minvar_design(A,B,C)
2 % Design a minimum variance controller for plant B(q)/A(q) with noise polynomial C(q).
% Control law, Eqn. 7.21, is uk = −G(q)/(B(q)F (q))yk
B = stripleadzeros(B); % Remove any leading zeros from the B polynomial
n = length(A); % order of plant

7 if length(C) 6= n
C = [C,zeros(1,1:n-length(C))]; % Force C(q) the same length as A(q)
end % if

d0 = length(A)- length(B); % Time delay is na − nb = d0

12

qd0 = [1,zeros(1,d0-1)]; % Construct q d0 −1

qd0C = conv(qd0,C); % Construct q d0 −1 C(q)
[F,G] = deconv(qd0C,A);

17 BF = conv(B,F); % Construct B(q)F (q)

return

U NSTABLE PROCESS ZEROS

As in the adaptive pole-placement case outlined in section 7.6.1, if we have a non-minimum

phase plant with unstable zeros, then the resulting controller given by Eqn. 7.21 will be unstable
350 CHAPTER 7. ADAPTIVE CONTROL

since essentially we are attempting to cancel right-hand plane zeros with right-hand plane poles.
Again the solution is to factor B into two factors, one containing all the stable modes, B + , and
one containing all the unstable modes, B − , and using only B + in the control law. (See Listing 7.8
to do this.) The F and G polynomials are obtained from solving the Diophantine equation which
is a generalisation of Eqn. 7.20. Refer to [22, pp380-385] for further details.

Moving average example

Consider the following plant system with a noise term,

(q − 0.9)(q − 0.8)(q − 0.2)yk = −2(q − 1.25)(q − 0.4)uk + (q 3 − 0.7q 2 + 0.14q − 0.2) ek

noting the unstable plant zero at q = 1.25.

Plant = zpk([1.25 0.4],[0.9 0.8 0.2],-2,1); % Plant poles & zeros

2 Ptf = tf(Plant);
B = stripleadzeros(Ptf.num{:});
A = Ptf.den{:};
C = [1 -0.7 0.14 -0.2]; % Noise polynomial
var_e = 1; % Variance of noise

The step response and pole-zero plot in Fig. 7.21 illustrates the potential difficulties with this
plant.

Pole−Zero Map Step Response

20
1

15

0.5
Imaginary Axis

Amplitude

10

−0.5

Figure 7.21: A non-minimum phase 0

plant with an unstable zero which −1

−5
causes an inverse step response. −1 −0.5 0
Real Axis
0.5 1 1.5 0 20 40
Time (seconds)
60 80

We can design a moving average controller for this NMP plant as follows.

B = stripleadzeros(B); % Remove the leading zeros, (if any)

[Bp,Bm] = factorBz(B); % Factorise B(q) into well-damped and unstable zeros

4 n = length(A); % order of plant

C = [C,zeros(1,n-length(C))]; % Back-pad with zeros to get same length as A(q) (if neces-
sary)

d = length(A)- length(Bp); % # of process zeros to cancel

9 % Now solve Diophantine eqn AR + BS = q d−1 CB + to compute control law

qd = [1,zeros(1,d-1)]; % Construct q d−1
qdCBp = mconv(qd,C,Bp); % Evaluate q d−1 CB + using (multiple) convolution
[R,S] = dioph_mtx(A,B,qdCBp) % Listing 7.3.

14 [R1, R1rem] = deconv(R,Bp) % Check that R/B + has no remainder

G = S; BF = R; F=R1; % relabel
B = [zeros(1,length(A)-length(B)), B]; % Make B(q) same length as A to prevent AE loops
7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT 351

sim('sminvarctrl',[0 1e5]) % Now run the Simulink simulation

19 variance = var(ey);
varR1 = var_e*sum(R1.*R1);
disp([variance, varR1]) % Note that var{y} ≈ var{(F e)}

Note that this controller does not attempt to cancel all two process zeros since that would result
in an unstable controller giving rise to unbounded internal signals. However instead we cancel
the one stable process zero giving rise to a moving average controller.

In this example shown in Fig. 7.22, F (q) = q+5.0567 which indicates that the expected covariance
of the output is σy2 = (12 + 5.062 ) = 26.6 which, provided we take enough samples as shown in
Fig. 7.22(b), it is.

Random
Number

F(z)
ey
1
F To Workspace

C(z)
C
1

B(z) 1
1 A(z)
Add
B A Scope

Gain
G(z)
−1
den(z)
Min var ctrlr
(a) A minimum variance moving average controller implementation in S IMULINK .

30

25

20 variance of output
Variance

15

10
variance of noise

0
3 4 5 6
10 10 10 10
Experiment duration

(b) The variance of the output should converge to

σy2 = 26.6.

Figure 7.22: Minimum variance moving average control. If the controller is designed correctly,
then the output variance should be approximately equal to the variance of the noise signal filtered
through F , which in this case it is, provided we take enough samples.
352 CHAPTER 7. ADAPTIVE CONTROL

7.7 S UMMARY OF ADAPTIVE CONTROL

Adaptive control is where the controller adjusts in response to changing external conditions.
This is suitable for plants that are time varying, severely nonlinear or perhaps just unknown at
the controller design stage. The general adaptive controller has two parts; an identification part
and a controller design part although in some implementations these two components are joined
together. The main styles of adaptive control are:

◦ Where the controller parameters are some static function of the output. The simplest exam-
ple is gain scheduling. This has the advantage that it is easy to implement, but it is really
only open loop adaption, and may not perform as expected if the conditions change in an
unpredictable manner.
◦ Where the classical PID controller is augmented with a “self-tuning” option. This can be
based on the Ziegler-Nichols design, using a relay feedback to extract the ultimate fre-
quency and gain.
• Where the parameters of the plant transfer function model are estimated online using the
techniques described in chapter 6, and the controller is designed to control this model. If
the controller is approximately the inverse of the process model, this leads to minimum
variance control, but is usually too severe, and even tends to instability for many actual
applications. A better alternative is adaptive pole-placement, where reasonable closed loop
poles are specified by the control engineer, and the controller is varied to provide this per-
formance as the identified model changes.

As with all high performance control schemes, there are some open issues with adaptive con-
trollers. For the simple schemes presented here we should be aware of the following issues:

1. The structure (order and amount of deadtime) of the plant must be known beforehand, and
typically is assumed constant.
2. Minimum variance control, while simple to design, attempts to force the process back to
the desired operating point in one sample time. This results in overly severe control action.
3. There are potential numerical problems when implementing recursive estimators, particu-
larly in fixed point or with short word lengths.
4. If dead time is present, and usually “dead time” of at least one sample time is always
present, then the controller will be unrealisable since future values of y are needed to calcu-
late current values of u. In this case, a sub-optimal controller is used where the dead time
is simply ignored, or the future output is predicted.

However the real problem of adaptive control is that when the plant is in control, there is lit-
tle or no information going to the estimator. Hence the controller parameters begin to oscillate
wildly. This is not normally noticed by the operator, because the process is in control. However
as soon as a disturbance enters the plant, the controller will take extra action attempting to cor-
rect for the upset, but now based on the wildly inaccurate plant model. During this time of major
transients however, the widely changing process conditions provide good information to the es-
timator which soon converges to good process parameters, which in turn subsequently generate
appropriate controller parameters. The adaptive controller then brings the process back under
control and the loop starts all over again. This results in alternating periods of very good control
and short periods of terrible control. One solution to this problem is to turn off the estimator
whenever the system is at the setpoint.
7.7. SUMMARY OF ADAPTIVE CONTROL 353

In summary when applying adaptive control, one wants:

• A constant output for good control, but

• a varying input and output for good estimation (which is required for good control)!

This conflict of interest is sometimes called the dichotomy of adaptive control.

354 CHAPTER 7. ADAPTIVE CONTROL

P ROBLEMS

Problem 7.1 1. Construct a S IMULINK diagram to simulate the general linear controller with
2-degrees of freedom controller. That is, with the R, S and T controller polynomials. Follow
[18, Fig3.2, p93].
Hint: Use the discrete filter blocks in z −1 , since these are the only blocks that you can
construct S(z − )/1 in.
2. Simulate the controlled response in problem 1 using the polynomials from [18, Ex3.1], re-
peated here
B(q) 0.1065q + 0.0902
= 2
A(q) q − 1.6065q + 0.6065
and
R(q) = q + 0.8467, S(q) = 2.6850q − 1.0319, T (q) = 1.6530q

3. Verify that the closed loop is the same as the expected model,
BT Bm
=
AR + BS Am
in the example 3.1, [18, Ex3.1, p97–98].
Hint: You may like to use the M ATLAB symbolic toolbox.
4. Construct an m-file to do the minimum degree pole-placement design.
(a) Do an ‘All zeros cancelled’ version
(b) Do an ‘No zeros cancelled’ version
(c) Do an general version, perhaps with some automated decider which zeros to cancel.
5. For the minimum variance controller we need to be able to compute the variance of the
output of a FIR filter, refer [18, chapter 4].

(a) What is the theoretical output variance of a FIR filter with input variance of σe2 ?
i. In S IMULINK you have the choice of the DSP format z −1 or the transfer function
version z. What would be the natural choice for q?
ii. Test your answer to part (1) by constructing a simulation in S IMULINK.
(b) Approximately how many samples do you need to obtain 2, 3 and 4 decimals accuracy
in your simulation?
(c) Using the hist function look at the distribution of both the input, e, and output, y. Is
the output Gaussian and how could you validate that statistically?

6. For stochastic control and for regulatory control topics we want to be able to elegantly
simulate models of the form

A(q)y(t) = B(q)u(t) + C(q)e(t) (7.23)

where A, B, C are polynomials in the forward shift operator q. Note that the deadtime is
defined as deg(A)-deg(B).

(a) For the standard model in Eqn. 7.23 what are the constraints on the polynomials
A, B, C?
i. Choose sensible polynomials for A, B, C with at least one sample of deadtime.
Create a model object from the System Identification toolbox using the idpoly
command.
7.7. SUMMARY OF ADAPTIVE CONTROL 355

ii. Simulate this model either using idmodel/sim or the idsim command. What is
the difference between these two commands? Choose a square wave for u(t) and
say random noise for e(t).
(b) Create a S IMULINK model and compare the simulated results with above. Use the
same random input.
(c) Write a raw M ATLAB script file to run the simulation and compare with the two alter-
natives above. (They should give identical results.)
356 CHAPTER 7. ADAPTIVE CONTROL
C HAPTER 8

M ULTIVARIABLE CONTROLLER
DESIGN

As our engineering endeavors became more sophisticated, so did our control problems. The
oil crises of the mid 1970s gave impetus for process engineers to find ways to recycle energy
and materials. Consequently the plants became much harder to control because they were now
tightly coupled to other plants. As the plants became multivariable, it made sense to consider
multivariable controllers.

If we have a dynamic system with say 3 states and 3 inputs, then assuming that our system is
controllable, we can feedback the three states directly to the three inputs, state 1 to input 1, state 2
to input 2 etc. This is termed de-centralised control, and our first dilemma is which state should
be paired with which input. We then have the problem of tuning the three loops, especially if
there are some interactions.

Alternatively we could feedback each state to all the inputs, which is termed centralised control.
In matrix form, the two alternative control laws look like
   
⋆ ⋆ • •
u= ⋆ x or u =  • ⋆ •  x
⋆ • • ⋆
| {z } | {z }
centralised de-centralised

Exploiting the off-diagonals of the gain matrix K tends to be more efficient and is often what is
meant by multivariable control. While centralised or multivariable control structures typically
give better results for similar complexity, they are harder to design and tune since we have more
controller parameters (9 as opposed to 3) to select. Not surprisingly, decentralised control re-
mains popular in industry, none the least for the following reasons given by [90]:

1. Decentralised control is easier to design and implement than a full multivariable controller,
and also requires fewer tuning parameters.
2. They are easier for the operators to understand and possibly retune.
3. Decentralised controllers are more tolerant of manipulated or measurement variable fail-
ure.
4. Startup and shutdown operating policies are easier with decentralised controllers, since the
control loops can be gradually brought into service one by one.

357
358 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Despite these application style advantages of decentralised controllers, it is useful to study what
is possible with a full multivariable controller. This is the purpose of this chapter.

Finally a one-step optimal controller that is applicable for nonlinear processes called GMC (Generic
Model Control) is presented in §8.5. Exact feedback nonlinearisation is a generalisation of the
GMC controller follows in §8.6.

8.1 C ONTROLLABILITY AND OBSERVABILITY

This section will introduce in a descriptive manner the concept of controllability, observability
and other “abilities”. For a more formal treatment see for example [152, §9–4, p699-713] or [150,
p627 §6–2]. The criteria of controllability is given first in §8.1.1 followed by the related property
of observability (which is the dual of controllability) in §8.1.2. These conditions are necessary for
some of the multi-variable controller and estimator design that follow. Methods to compute the
controllability matrix, and establishing the condition are given in §8.1.3.

8.1.1 C ONTROLLABILITY

When designing control systems, we implicitly assume that we can, at least in principle influence
the system to respond in some desired manner. However, it is conceivable, perhaps due to some
structural deficiency, that we cannot manipulated independently all the states of the system. The
controllability criteria tells us if this is the case.

Formally defined, a system is controllable if it is possible to transfer from any arbitrary initial
state x0 to any other desired state x⋆ in a finite time by using only an uncompensated control
signal u.

Note that while the transformation is allowed to take place over more than one sample time, it
must be achieved in a finite time. In addition, even if the above controllability requirement is
satisfied, you may not be able to keep the state x at the desired position, the guarantee is only
that x could pass through x⋆ at some time in the future. Nevertheless, it is instructive to know
if a given system is controllable or not. Some care is needed with these terms. Åström and
Wittenmark, [22, p127], define reachability as I have defined controllability above, and then define
controllability as a restricted version where x⋆ = 0. These two definitions are equivalent if A or
Φ is invertible.

First consider a continuous dynamic system that has a diagonal A matrix and an arbitrary B
matrix,
 
∗ 0 ··· 0
 0 ∗ ··· 0 
 
ẋ =  ..  x + Bu
 0 0 . 0 
0 0 ··· ∗

where an ∗ indicates the presence of a non-zero value. Now suppose if u(t) = 0, then the steady
state is xss = 0. Since A is a diagonal matrix, there is no interaction between the states. This
means that if one particular state xi , is 0 at some time t, then it will always remain at zero since
u= 0. Now consider the full system (u 6= 0), but suppose that the ith row of B are all zeros such
8.1. CONTROLLABILITY AND OBSERVABILITY 359

as the second row in the following control matrix,

 
∗ ∗ 0 0
 0 0 0 0 
B=  
0 ∗ 0 ∗ 
∗ ∗ 0 ∗
Then if xi happens to be zero, it will remain at zero for all time irrespective of the value of the
manipulated variable u. Quantitatively this is due to the following two points:

1. No manipulated variable, uj , can modify the value of xi because all the coefficients in the
B matrix for that particular state are zero.
2. No other states can influence xi because in this special case the system matrix A is diagonal.

Hence if xi is zero, it will stay at zero. The fact that at least one particular state is independent of
the manipulated vector means that the system is uncontrollable. We cannot influence (and hence
control) all the states by only changing the manipulated variables.

In summary, a system will be uncontrollable if the system matrix A is diagonal, and any one or
more rows of B are all zero. This is sometimes known as Gilbert’s criteria for controllability.

If the matrix A is not diagonal, then it may be transformed to a diagonal form (or Jordon block
form), and the above criteria applied. We can easily transform the system to a block diagonal
form using the canon function in M ATLAB. However a more general way to establish the con-
trollability of the n × n system is to first construct the n × (m ∗ n) controllability matrix C,
h i
C = B ... AB ... A2 B ... · · · ... An−1 B
def
(8.1)

and then compute the rank of this matrix. If the rank of C is n, then all n system modes are
controllable. This is proved in problem [150, A-6-6 p753]. The discrete controllability matrix is
defined in a similar manner;
h i
C = ∆ ... Φ∆ ... Φ2 ∆ ... · · · ... Φn−1 ∆
def
(8.2)

and also must be of rank n for the discrete system to be controllable.

An often less restrictive controllability requirement is output controllability. This is where we

wish only to control the outputs y, rather than the full state vector x. The output controllability
matrix C y is defined as
h i
C y = CB ... CAB ... CA2 B ... · · · ... CAn−1 B
def
(8.3)

Output controllability is given in Example A-9-11 Ogata, [152, p756], and [150, p636].

The rank of a matrix is defined as the number of independent rows (or columns) in a matrix, and
is a delicate numerical computation using a finite precision computer. The presence of a non-zero
determinant is not a good indication of rank because the value of the determinant scale so poorly.
By the same argument, a very small determinant such as say 10−6 is also not a reliable indication
of dependence. Numerically the rank is computed reliably by counting the number of non-zero
singular values, or the number of singular values above some reasonable threshold value. See
also §8.1.3.

The controllability matrix for the submarine system in §2.8.2 is

 
h i 0 −0.095 0.0192
C = B ... AB ... A2 B =  −0.095 0.0192 −0.0048 
0.072 −0.0283 0.0098
360 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

which has a rank of 3. Hence the submarine system is controllable.

8.1.2 O BSERVABILITY

Observability is a related property to controllability.

A system is observable if one can evaluate x0 by recording only the output yt

over a finite time interval. In other words, the system is observable if all the
states eventually “effect” the outputs.

Note that the observation is allowed to take place over more than one sample time. Once we have
back calculated x0 , we can use the history of the manipulated variable ut to forward calculate
what the current state vector xt is. It is clear that this condition of observability is important in
the field of estimation.

As a trivial example, suppose we had a 3 state system and we have three outputs which are
related to the states by
 
1 0 0
y = Cx =  0 1 0  x
0 0 1

Here the outputs (measurements) are identically equal to the states (C= I). It is clear that we can
evaluate all the states simply by measuring the outputs and inverting the measurement matrix
x = C−1 y = Iy = y since C−1 = I in this trivial example. This system is therefore observable.
Suppose, however, we measure only two outputs where C is the non-square matrix
 
1 0 0
y = Cx = x
0 0 1

Now, obviously, if we recorded only one output vector (2 measurements), we could not hope to
reconstruct the full 3 state vector. But the observability property does not require us to estimate x
in one sample time, only in a finite number of sample times. Hence it may be possible for a system
with fewer outputs than states to be still observable.

To get a feel for the observability of a system, we will use a similar line of argument to that used
for the controllability. Suppose we have a diagonal system and an arbitrary output matrix
 
∗ 0 ··· 0
 0
 ∗ ··· 0 

ẋ =  . x (8.4)
 0 0 .. 0 
0 0 ··· ∗
y = Cx (8.5)

Note that any system can be converted into the above form, and for observability criteria we are
not interested in the manipulated variable effects. Now if any of the columns of the C matrix are
all zeros, then this is equivalent to saying that that a particular state will never influence any of
the outputs, and since the system matrix is diagonal, that same state is never influenced by any
of the other states. Therefore none of the outputs will ever influence that state variable. Note the
symmetry of this criterion to the controllability criteria.
8.1. CONTROLLABILITY AND OBSERVABILITY 361

Formally for a continuous time system (Eqn 2.37), the n-state system is observable if the np × n
observability matrix O is of rank n where
   
C C

 CA 


 CΦ 

def

O=  CA2 
, or in the discrete case
def

O=  CΦ2 
 (8.6)
 ..   .. 
 .   . 
CAn−1 CΦn−1

Some authors, perhaps to save space, define O as a row concatenation of matrices rather than
stacking them in a column as done above.

Problem 8.1 1. What are the dimensions of the observability and controllability matrices?
What is the maximum rank they can achieve?

2. Establish C and O for the Newell & Lee evaporator.

3. Examples B-9-13, B-9-10, B-9-11, [152, p771]

4. Examples B-6-6, B-6-7, [150, p805]

5. [157] gives some alternative mathematical definitions for controllability and observability:
A system is controllable if and only if

rank(B, A − λi I) = n, for i = 1, · · · , n (8.7)

and observable if and only if

rank(C⊤ , A⊤ − λi I) = n, for i = 1, · · · , n (8.8)

where λ are the eigenvalues of A.

Construct a M ATLAB m-file implementation using these algorithms. Test your routines on
various systems, perhaps using the rmodel command.

8.1.3 C OMPUTING CONTROLLABILITY AND OBSERVABILITY

The M ATLAB functions ctrb and obsv create the controllability and observability matrices re-
spectively, and the rank function can be used to establish the rank of them. While the concept
of observability and controllability applies to all types of dynamic systems, Eqns 8.1 and 8.6 are
restricted to linear time invariant systems. Read problem A-6-1 [150, p745] for a discussion on
the equivalent term “reachable”.

The algorithms given above for computing controllability or observability, (equations 8.1 and 8.6
or the discrete equivalents), seem quite straight forward, but are in fact often beset by subtle
numerical errors. There exist other equivalent algorithms, but many of these are also susceptible
to similar numerical problems. Issues in reliable numerical techniques relevant to control are
discussed in a series of articles collected by [159].

An example quoted in [157] illustrates common problems to which these simple algorithms can
fall prey. Part of the problem lies in the reliable computation of the rank of a matrix which is best
done by computing singular values.
362 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Problem 8.2 Write a M ATLAB m-file to compute the exponential of a matrix using the Taylor
series formula,
∞
X 1 k
eAt = A
k!
k=0

Test your algorithm on  

−49 24
A=
−64 31
and comment on the expected accuracy of your result.

Problem 8.3 1. When using good (in the numerical reliability sense) software, we tend to for-
get one fundamental folklore of numerical computation. This folklore, repeated in [159, p2],
states
If an algorithm is amenable to “easy” hand calculation, then it is probably a poor
method if implemented in the finite floating point arithmetic of a digital com-
puter.
With that warning in mind, calculate the singular values of
 
1 1
A= µ 0 
0 µ
√
where µ is a real number such that |µ| < ǫ where ǫ is the machine precision. (Note that
this effectively means that 1 + µ2 = 1 where as 1 + µ > 1 when executed on the computer.)
Calculate the singular values both analytically, and using the relationship between singular
values and the eigenvalues of A⊤ A.
2. Calculate the controllability of the system
   
1 µ 1
A= , B=
0 1 µ
√
with |µ| < ǫ. Solve this both analytically and numerically.

8.1.4 S TATE RECONSTRUCTION

If a system is observable, we should in principle, be able to reconstruct the states based only
on the current, and perhaps historical outputs. So, supposing we have collected the following
input/output data,

time (k) u(k) y(k) x1 (k) x2 (k)

0 4 −4 ? ?
1 5 4 ? ?
2 – 36 ? ?

we should be able to reconstruct the states assuming a model of the form

   
0 1 0
xk+1 = xk + uk
−3 2 1
 
yk = 4 0 x k
8.1. CONTROLLABILITY AND OBSERVABILITY 363

Note that we cannot simply invert the measurements back to the states, partly because the mea-
surement matrix is not square.

However we can see that y(k) = 4x1 (k) from the measurement relation, so we can immediately
fill in some of the blank spaces in the above table, in the x1 (k) column; x1 (0) = −1, x1 (1) =
1, x1 (2) = 9. From the model relation, we can see that x1 (k + 1) = x2 (k), so we can set x2 (0) =
x1 (1) = 1. Now we have established the initial condition
 ⊤
x(0) = −1 1
and with the subsequent manipulated variable history u(t), we can calculate x(t) simply using
the recursive formula, Eqn. 2.38. This technique of estimation is sometimes called state recon-
struction and assumes a perfect model and no measurement noise. Furthermore, the estimation
reconstruction takes, at most, n input-output sample pairs.

As a check, we can construct the observability matrix,

   
  4 0  
C = 4 0

O= =   0 1
CΦ 4 0 × 0 4
−3 2
which clearly has a rank of 2 indicating that the state reconstruction is in fact possible.

We can generalise this state reconstruction scheme outlined above, following [19, p315], for any
observable discrete plant
xk+1 = Φxk + ∆uk
yk = Cxk
where we wish to reconstruct the current state, xk , given past values of input and output, uk , uk−1 , . . .
and yk , yk−1 , . . .

Our basic approach will be to use the outputs from the immediate past n samples, to develop
linear equations for the n unknown states at the point n samples in the past. We will then use the
knowledge of the past inputs to roll this state forwards to the current sample k. For the present
development, we will just consider SISO systems.

Listing the immediate previous n values finishing at the current sample k gives
yk−n+1 = Cxk−n+1
yk−n+2 = CΦxk−n+1 + C∆uk−n+1
yk−n+3 = CΦ2 xk−n+1 + CΦ∆uk−n+1 + C∆uk−n+2
..
.
yk = CΦn−1 xk−n+1 + CΦn−2 ∆uk−n+1 + · · · + C∆uk−1 (8.9)
To make things clearer, we can stack the collected input and output values in a vector,
 
yk−n+1  
 yk−n+2  uk−n+1
   uk−n+2 
Yk = 
 .. 
, U =
 
(8.10)
.  k−1  .. 

 yk−1 
  . 
uk−1
yk
and we note that the input vector collection Uk−1 is one element shorter than the output vector
collection Yk . Now we can write Eqn. 8.9 more succinctly as
Y k = Oxk−n+1 + Wu Uk−1 (8.11)
364 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

with matrices defined as

   
C 0 0 ··· 0
 CΦ




 C∆ 0 ··· 0 

2
O =  CΦ
 
,

Wu =  CΦ∆ C∆ ··· 0 
 (8.12)
 ..   .. .. .. .. 
 .   . . . . 
CΦn−1 CΦn−2 ∆ CΦn−3 ∆ · · · C∆

and we may recognise O is the observability matrix defined in Eqn. 8.6.

Solving Eqn. 8.11 for the states (in the past at sample time k − n + 1) gives

xk−n+1 = O −1 Y k − O −1 Wu Uk−1 (8.13)

provided O is non-singular. This is why we require the observability condition to be met for state
reconstruction.

The minor problem with Eqn. 8.13 is that it delivers the state n sample times in the past, rather
than the current state, xk , which is of more immediate interest. However, it is trivial to use
the plant model and knowledge of the past inputs to ‘roll the states onwards’ to current time k.
Repeated forward use of the model gives

xk = Φn−1 xk−n+1 + Φn−2 ∆uk−n+1 + Φn−3 ∆uk−n+2 + · · · + ∆uk−1

The entire procedure for state reconstruction is summarised in the following algorithm.

Algorithm 8.1 State reconstruction assuming a perfect observable model and no noise. Given

xk+1 = Φxk + ∆uk , yk = Cxk

then the current state, xk , is given by the linear equation

xk = Ay Yk + Bu Uk−1

where the matrices are

Ay = Φn−1 O +
 
Bu = Φn−2 ∆ Φn−3 ∆ · · · ∆ − Φn−1 O+ Wu

and Y k , Uk−1 are defined in Eqn. 8.10 and O, Wu are defined in Eqn. 8.12. We should note:

• To reconstruct the state (at the current time k), we need to log past input and output data,
and the minimum number of data pairs required is n, where n is the number of states. Any
more will gives a least-squares fit.

• Since we invert O, this matrix must not be singular. (Alternatively we require that the
system must be observable.) For the multivariable case we should use the pseudo-inverse,
O+ , as written above.

• We use in the algorithm, but do not invert, the controllability matrix.

A simple state reconstructor following Algorithm 8.1 is given in Listing 8.1.

8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 365

Listing 8.1: A simple state reconstructor following Algorithm 8.1.

function x_est = state_extr(G,Yk,Uk)
% Back extract past state from outputs & inputs (assuming observable)

4 Phi = G.a; Del = G.b; C = G.c;

error(ABCDCHK(Phi,Del,C)) % bullet proofing
[p,n] = size(C); [n,m] = size(Del);

Wo = obsv(G); % same as: O = obsv(Phi,C)

9 Ay = Phiˆ(n-1)/Wo; % similar to Φn−1 O −1

S = C*Del;
for i=1:n-2
S = [S; C*Phiˆi*Del]; % Building Wu
14 end % for

zC = zeros(p,m); % or zeros(size(C*Del));
Wu1 = [zC;S]; Wu = Wu1;
for i=n-3:-1:0
19 Wu1 = [zC; Wu1]; Wu1(end-p+1:end,:) = [];
Wu = [Wu, Wu1];
end % for

Co = Del; % Reversed controllability matrix

24 for i=1:n-2
Co = [Phiˆi*Del,Co];
end % for

Bu = Co - Ay*Wu; x_est = Ay*Yk + Bu*Uk;

29 return

Fig. 8.1 illustrates the results of the state reconstruction algorithm in the case where there is no
model/plant mismatch, but there is noise on the input and output. Given that the algorithm has
the wrong u(t) and y(t) trends, then the reconstructed states (⋆ in Fig. 8.1) are slightly different
from the true states.

8.2 S TATE SPACE POLE - PLACEMENT CONTROLLER DESIGN

This section describes how to design a controller to control a given process, using state space
techniques in the discrete domain.

When a process engineer asks the control engineer to design a control system, the first question
that the control engineer should ask back, is ‘what sort of performance is required?’ In this way,
we start with some sort of closed loop specification, which, for example, may be that we demand
that the system return to setpoint after a disturbance with no offset, or rise to a new setpoint in a
reasonable time, perhaps one open-loop time constant. If the loop is a critical one, then we may
wish to tighten this requirement by specifying a smaller time to reach control, or conversely if
we have a non-critical loop we could be satisfied with a more relaxed settling time requirement.
It is our job as control engineers to translate these vague verbal requirements into mathematical
constraints, and if possible, then design a controller that meets these constraints.

When we talk about a response, we are essentially selecting the poles (or eigenvalues) of the closed
loop transfer function, and to some lesser degree, also the zeros. Specifying these eigenvalues by
366 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

0.3
y
0.2 actual y

Output
0.1

−0.1
0 1 2 3 4 5 6 7

1
x
1
0.5
estimate
1

0
x

−0.5

−1
0 1 2 3 4 5 6 7

2
x
2
Figure 8.1: Reconstructing states. estimate
1
Since the state reconstructor is given
2
x

the undisturbed input and a noisy

0
output, then it is no surprise that the
estimated states ⋆ are slightly differ-
−1
ent from the true states (solid lines in 0 1 2 3 4 5 6 7
the lower two trends). time [s]

changing the controller parameters is called controller design by pole placement. In order to
design a controller by pole placement, we must know two things:

1. A dynamic model of the process, and

2. what type of response we want (i.e. what the closed loop eigenvalues should be).

However choosing appropriate closed loop poles is non-trivial. Morari in [138] reminds us that
“poles are a highly inadequate measure of closed loop performance”, and he continues to assert
that because of the lack of guidance to the control designer, “pole placement techniques have been
abandoned by almost everybody today”. Notwithstanding this opinion, it is still instructive for
us to understand how our closed loop response is affected by different choices of eigenvalues, and
how this forms the basis for more acceptable tuning methods such as linear quadratic controllers
discussed in the following chapter.

Suppose we have a single input–multiple output (SIMO) system such as

xk+1 = Φxk + ∆uk (8.14)

where u is a scalar input and we wish to control all n states by using a proportional feedback
control law such as
u = −Kx (8.15)
where K is called the state feedback gain matrix. (Actually in this SIMO case, the gain “matrix”
collapses to a row vector). Substituting Eqn 8.15 into Eqn 8.14 gives the closed loop response as

xk+1 = (Φ − ∆K) xk (8.16)

8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 367

Now we desire (Φ − ∆K) to have the n eigenvalues at arbitrary positions µ1 , µ2 , · · · , µn To

achieve this, we must check that arbitrary pole placement is achievable. In other words, the
system must be controllable. This means that the controllability matrix Co , must be of rank n.
If this condition is satisfied, we can design the feedback gain matrix K, by using Ackermann’s
formula i−1
 h .. .. ..
K = 0 0 ··· 0 1 ∆ . Φ∆ . · · · . Φ n−1
∆ α(Φ) (8.17)

where α is the matrix characteristic equation of the desired closed loop transfer function. Note
that the matrix to be inverted in Eqn 8.17 is the discrete controllability matrix Co , hence the need
for full rank.

A quick way to calculate α(Φ) is to use poly and polyvalm. For instance we can verify the
Cayley-Hamilton theorem, (any matrix satisfies its own characteristic equation), elegantly using

1 >> A = rand(3) % take any test matrix

A =
0.0535 0.0077 0.4175
0.5297 0.3834 0.6868
0.6711 0.0668 0.5890
6

>> polyvalm(poly(A),A) % Cayley-Hamilton theorem: α(A) should = 0

ans =
1.0e-016 *
0.1388 0.0119 0.1407
11 0.1325 0 0.7412
-0.0496 0.0073 0.5551

Algorithm 8.2 SIMO state-space pole-placement

Given a continuous system model, ẋ = Ax + Bu, or discrete version, xk+1 = Φxk + ∆uk , and
design requirements for the closed loop poles, µ, then

1. Check the controllability of the system, ctrb(A, B), or ctrb(Φ, ∆). If not all the states are
controllable, then terminate the algorithm, otherwise carry on.

2. Compute the characteristic polynomial, α, and substitute either matrix A or Φ.

3. Compute the row-vector state feedback gain using Eqn. 8.17.

Example Pole placement of the double integrator system.

In this example, we wish to design a controller for the discrete double integrator system sampled
at T = 1 that has closed loop eigenvalues at µ = 0.5 ± 0.5i.

In this case we will assume a zeroth-order hold on

the discretisation. giving matrices
   
2 −0.5 1
Gc = tf(1,[1 0 0]); % Continuous system Φ= , ∆=
2 0 0
G = ss(c2d(Gc,1)); % Sampled at Ts = 1

To use Ackermann formula, Eqn. 8.17, we need to both compute the controllability matrix and
the matrix characteristic equation α.
368 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

def
Note that as α(z) = (z − µ1 )(z − µ2 ) = z 2 − z + 0.5, then
 
2 1 −0.5
α(Φ) = Φ − Φ + 0.5I =
2 −0.5
and that the controllability matrix  
1 2
C=
0 2
has full rank.

>> mu = [0.5+0.5i; 0.5-0.5i]; % Design requirements for µ

>> alpha = polyvalm(poly(mu),G.a)
3 >> Cb = ctrb(G.a,G.b)

Finally the feedback gain is computed by

 
>> K = [0 1]/Cb*alpha % giving K = 1 −0.25

We can easily check that the closed loop poles are as required with

>> eig(G.a-G.b*K) % Check that µ = eig(Φ − ∆K)

ans =
0.5000 + 0.5000i
4 0.5000 - 0.5000i

Rather than constructing Ackermann’s equation by hand, M ATLAB can design a pole placement
controller using the acker routine. At heart this routine simply computes Eqn. 8.17 as

1 k = ctrb(a,b)\polyvalm(real(poly(p)),a); % Form gains using Ackerman’s formula

k = k(n,:);

or, as noted in the next section, acker is very ill-conditioned, so we could use the preferred
place function. The latter function gives an estimate of the precision of the closed loop eigen-
values, and an optional error message if necessary. (See also Fig. 8.3.)

[K,prec] = place(G.a,G.b,mu)
K =
3 1.0000 -0.2500
prec =
15

Eqn 8.15 assumes that the setpoint is at x=0. (We can always transform our state variables by
subtracting the setpoint so that this condition is satisfied, but this becomes inefficient if we are
likely to change the setpoint regularly.) In these cases, it is better to modify our control law from
Eqn 8.15 to
u = K(r − x) (8.18)
where r is a vector of the state setpoints or reference states, now assumed non-zero. Consequently
the closed loop response is now modified from Eqn 8.16 to
xk+1 = (Φ − ∆K) xk + ∆Krk (8.19)
8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 369

8.2.1 P OLES AND WHERE TO PLACE THEM

While pole-placement seems an attractive controller design strategy, the problem lies in deciding
where the closed loop poles out to be.

Listing 8.2 designs a pole-placement controller for the blackbox, G(s) = 0.98/(2s + 1)(s + 1), with
discrete closed loop poles specified, somewhat arbitrarily, at µT = [0.9, 0.8].

Listing 8.2: Pole-placement control of a well-behaved system

>> Gc = tf(0.98,conv([2 1],[1,1])) % Blackbox model
>> G = ss(c2d(Gc,0.1)) % Convert to discrete state-space with sample time Ts = 0.1
>> [Phi,Delta,C,D] = ssdata(G);

5 >> lambda = [0.9 0.8]'; % Desired closed loop discrete poles µT = [0.9, 0.8]
>> K = place(Phi,Delta,lambda)% Compute controller gain K
K =
2.4971 -2.2513

Now we can check the closed loop performance from a non-zero initial condition with

Gcl = ss(Phi-Delta*K,zeros(size(Delta)),eye(size(Phi)),0,Ts);
2

x0 = [1; -2]; % start position

initial(Gcl,x0) % See Fig. 8.2

The results of a controlled disturbance recovery is given in the left hand plot of Fig. 8.2. However
an obvious question at this point is where exactly should I place the poles. The three trends in
Fig. 8.2 illustrate the difference. A conservative choice is as given above is where we place the
poles in right-hand side of the origin, on on the real axis close to the +1 point. In this case I chose
µT = [0.9, 0.8]. The second choice is the mirror image of those poles µT = [−0.9, −0.8], and the
third choice is a pair of complex conjugates at 0.66 ±80◦ .

Clearly there is a trade off between the speed of the disturbance rejection, the amount of input
energy, and the degree of oscillation. If we can quantify these various competing effects, then
we can design controllers systematically. This is the subject of optimal control given in the next
chapter.

T HE POLE - PLACEMENT DESIGN PROBLEM IS ILL - CONDITIONED

The design of controllers using pole-placement is inherently numerically illconditioned, espe-

cially for large systems. Ackermann’s algorithm requires double precision for systems larger
than about 10th order, and outright fails for systems larger than about 15th order. Fig. 8.3 illus-
trates this degradation in accuracy by plotting an estimate of the number of accurate decimal
places in the solution using place for 500 randomly generated discrete models from orders 2
to 15. Note that once the order is above 6 or so, we should only expect about 6 decimal places
accuracy.
370 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

°
T T T ± j80
µ = [0.9, 0.8] µ = [−0.9, −0.8] µ = 0.6e

6
States x1 & x2

−2

100
input

−100
0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5

1 1 1
1.71.2 1.71.2 1.71.2
2.2 0.2 2.2 0.2 2.2 0.2
0.5 0.40.7 0.5 0.40.7 0.5 0.40.7
2.7 0.6 2.7 0.6 2.7 0.6
0.8 0.8 0.8
0 0.2 0 0.2 0 0.2
ℑ

0.2 0.2 0.2

2.7 2.7 2.7
−0.5 0.7 −0.5 0.7 −0.5 0.7
2.2 2.2 2.2
1.71.2 1.71.2 1.71.2
−1 −1 −1
−1 0 1 −1 0 1 −1 0 1
ℜ

Figure 8.2: Pole-placement of the black box for three different pole locations. From left to right:
◦
(a) µT = [0.9, 0.8], (b) µT = [−0.9, −0.8], and (c) µ = 0.6e±j80 .

place

15

10
Precision

−5

Acker

15

10
Precision

−5

−10

2 3 4 5 6 7 8 9 10 11 12 13 14 15
System size

Figure 8.3: The loss of accuracy using pole-placement algorithms place (upper) and acker
(lower) as a function of system order. It is clear that both routines struggle for systems of order
above about 6, but acker fails more often.
8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN 371

8.2.2 D EADBEAT CONTROL

If the system is controllable such that any arbitrary poles are allowable, one is naturally inclined
to extrapolate the selection to poles that will give a near perfect response? Clearly a perfect
response is one when we are always at the setpoint. If a step change in reference value occurs,
then we instantaneously apply a corrective measure to instantaneously push the system to the
new desired point. A system that rises immediately to the new setpoint has a closed loop time
constant of τ = 0 or alternatively a continuous pole at s = −∞. This corresponds to a discrete
pole at e−∞T or the origin. If we decide to design this admittedly rather “drastic” controller, we
would select all the desired closed loop poles at µ = 0. Such a controller is feasible in the discrete
domain and is called a deadbeat controller.

The design of a deadbeat controller is the same as for an arbitrary pole placement controller.
However it is better to use the acker function rather than the place function, because the place
function will not allow you to specify more multiple eigenvalues than inputs. This restriction
does not apply for the Ackermann algorithm.

Suppose we design a deadbeat controller for the continuous plant

   
−0.3 0 0 3
ẋ =  0 −0.5 2  x +  −2  u (8.20)
0 −2 −0.5 5

To design a deadbeat controller, we specify the desired discrete closed loop eigenvalues as µ = 0
and select a sample time of T = 4, we obtain
 
K = 3.23 · 10−2 −3.69 · 10−4 1.98 · 10−2

The closed loop transition matrix is

 
0.075 0.003 −0.139
Φcl = Φ − ∆K =  −0.063 −0.019 0.095 
−0.060 −0.133 −0.056

and we could also compute that all the elements in Φ3cl are in the order of 10−19 and all the
eigenvalues of eig(Φ − ∆K) are in the order of 10−7 as required.

Such a matrix where An−1 6= 0, but An = 0 is called a Nilpotent matrix.

Fig. 8.4 shows the impressive controlled response of the plant (Eqn. 8.20) under deadbeat control.
The top half of the diagram shows the states which converge from the initial condition, ( x0 =
 ⊤
−2 −3 1 ), to the setpoint x=0 in only three sample times (3T = 12 s). It takes three
sample times for the process to be at the setpoint, because it is a third order system.

The deadbeat controlled response looks almost perfect in Fig. 8.4, at least after the 3 samples
required to get top the setpoint. We can investigate this by repeating the simulation at a smaller
time interval (say T = 0.05 s) but with the same input as before, to see what happens between
the sample intervals. (This is how I evaluated the smooth state curve in Fig 8.4 between the states
at the controller sample time denoted with a • in Listing 8.3.) Note that the states are oscillating
between the samples, but quickly die out after 10 seconds.

Listing 8.3: A deadbeat controller simulation

1 x0= [-2 -3 1]'; dt=4; tf=dt*5; t=[0:dt:tf]'; % initial condition & time vector

A= [-0.3,0,0;0,-0.5,2;0,-2,-0.5];B= [3;-2;5]; % Continuous plant, Eqn. 8.20

372 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Deadbeat controller, Ts = 4
4

states
0

−1

−2

−3
Figure 8.4: The states (top) and the
manipulated variable (bottom) for −4
0.06
the deadbeat controlled response. 0.04
The states at the controller sample
Input

0.02
times are denoted with a • and af-
ter the third sample time, t ≥ 12, the 0

states remain exactly at the setpoint −0.02

0 2 4 6 8 10 12 14 16 18 20
at the sample times. time

Gc= ss(A,B,eye(3),0); Gd= c2d(Gc,dt,'zoh'); % Convert to a discrete model

6 mu= [0;0;0]; % Desired closed loop eigenvalues, µ = 0

K= acker(Gd.a,Gd.b,mu); % construct deadbeat controller

Gd_cl = Gd; Gd_cl.a = Gd.a-Gd.b*K; % closed loop

[y,t,x]= lsim(Gd_cl,zeros(size(t)),[],x0);
11 U = [-K*x']'; % back extract input

% Reduce sample time at same conditions to see inter-sample ripple

dt = dt/100; t2 = [0:dt:tf]'; % new time scale, Ts⋆ = Ts /100
[ts,Us] = stairs(t,U); % create new input
16 ts = ts + [1:length(ts)]'/1e10; ts(1) = 0;
U2 = interp1(ts,Us,t2); % approx new input
[y2,t2,x2]= lsim(Gc,U2,t2,x0);% continuous plant simulation

subplot(2,1,1);plot(t,x,'o',t2,x2); % Refer Fig. 8.4

21 subplot(2,1,2);plot(ts,Us);

Naturally under practical industrial conditions, the plant will not be brought under control so
rapidly owing to either model–plant mismatch, or to manipulated variable constraints. This type
of controller can be made more robust by specifying a larger sample time, which is in fact the
only tuning constant possible in a deadbeat controller. Ogata gives some comments and cautions
about deadbeat control in [150, pp670-71].

Problem 8.4 Create a system with eigenvalues of your choice; say

 T
v= −0.2 −0.3 −0.1

and design a deadbeat controller. Simulate this using the dlsim function. (Try a sample time
of ≈ 7.) Investigate the same controller, but at a higher sampling rate and plot the intersample
behaviour of the states.

Investigate the effect on the gain when using different sample rates.
8.3. ESTIMATING THE UNMEASURED STATES 373

8.3 E STIMATING THE UNMEASURED STATES

The pole-placement control design strategies as developed in § 8.2 utilises all the states in the
control law u = −Kx so implicitly assumes that all the states are measurable. In many cases this
is not practically possible, so the only alternative then is to estimate the states.

The states can be reconstructed using the outputs, the inputs, and the model equations. In fact we
need not restrict ourselves to just estimating states, we could also estimate derived quality control
variables, or even estimate the outputs themselves. The quantities that we estimate are known as
observers, and offer cost and reliability advantages over actual sensors in many applications.

The simplest state estimator is where we use a model and subject it to the same input u that is
applied to the true process. The resultant x̂ is the estimate for the true state x.

x̂k+1 = Φx̂k + ∆uk (8.21)

If we know the true model (Φ,∆) and the manipulated variable history u(k), then this estimator
will work provided we select the correct initial condition x̂0 . Since we have no feedback infor-
mation as to how well we are doing, this estimator is called an “open loop” estimator. If the
conditions stated above are not met, and they will rarely be satisfied in practice, then the esti-
mator will provided estimates that will probably get worse and worse. The error of the estimate
is
ǫk = xk − x̂k (8.22)
Ideally, we would like to force this error vector to zero. This is actually still a control problem, al-
though while we are not controlling the plant, we are controlling the error of the estimate scheme.
We introduce a feed back error scheme such as

x̂k+1 = Φx̂k + ∆uk + L (yk − Cx̂k ) (8.23)

which is called a prediction error observer since the prediction is one sample period ahead of the
observation. L is a (n × 1) column vector and is called the observer gain matrix and is a design
parameter. Now the error model is

ǫk+1 = (Φ − LC) ǫk (8.24)

We must select the matrix L such that the observer system Φ − LC is stable and has eigenvalues
(observer poles) that are suitably fast. This means that even if we were to get the initial condition
wrong for x0 , we can still use the estimator since any initial error will eventually be forced to
zero. Typically we would choose the poles of the observer system (Eqn 8.24) much faster than
the openloop system. To be able to solve for L, the system must be observable.

For many chemical process applications, the sampling rate is relatively slow, since the processes
are generally quite slow, so the basic estimator of Eqn 8.23 can be modified so that the prediction
of xk+1 uses the current measurement yk+1 rather than the previous measurement yk as is the
case in Eqn 8.23. This is only possible if the computational time is short, compared with the sam-
ple time, but if this is the case, then the sample period is better utilised. This type of formulation
is termed a current estimator and both are described in [71, pp147–8] and in terms of the Kalman
filter in [150, pp873–75]. The current estimator is

x̄k+1 = Φx̂k + ∆uk (8.25)

x̂k+1 = x̄k+1 + L yk+1 − Cx̄k+1 (8.26)

In practice for most systems, the two forms (current and prediction estimators) are almost equiv-
alent.
374 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

The design of the observer gain L by pole-placement is similar to the controller gain K given in
§8.2, but in the observer case we have the closed loop expression

eig(Φ − LC) rather than eig(Φ − ∆K)

However we note that

eig(Φ − LC) = eig(ΦT − CT LT )
since eig(A) = eig(AT ), so we can use the M ATLAB functions place or acker for the observer
problem by using the dual property

L = place(ΦT , CT , λ)T

Table 8.1 shows the equivalent relationships between the matrices used for observation and reg-
ulation and make the relevant substitution.

Table 8.1: The relationship between regulation and estimation

Control Estimation
Φ Φ⊤
B C⊤
K L⊤
Co Ob
eig(Φ − ∆K) eig(Φ − LC)

8.4 C OMBINING ESTIMATION AND STATE FEEDBACK

We can combine the state feedback control from §8.2 with the state estimation from §8.3 into
a single algorithm. This is useful because now we can use the estimator to provide the states
for our state feedback controller. For linear plants, the design of the state feedback controller
is independent from the design of the observer. The block diagram shown in Fig. 8.5, which is
adapted from [19, p438], shows the configuration of the estimator and controller.

Remember that in this example, we have no model plant mis-match, the plant is linear, but we do
not know the initial conditions of the states x0 . The equations are summarised below. The true
process is
xk+1 = Φxk + ∆uk (8.27)
and the measurement is
yk = Cxk (8.28)
but we do not know the true state vector x, so we estimate it with

x̂k+1 = Φx̂k + ∆uk + L (yk − Cx̂k ) (8.29)

and the scalar control law is

uk = −Kx̂k (8.30)
Combining Eqns 8.30 and 8.27 gives

xk+1 = Φxk − ∆Kx̂k (8.31)

and combining Eqns 8.29 to 8.30 gives

x̂k+1 = (Φ − LC − ∆K) x̂k + LCxk (8.32)

8.4. COMBINING ESTIMATION AND STATE FEEDBACK 375

✲ ∆ ✲+ ✲ q −1 ✲ C ✲+ output, y
✻

Φ ✛
Plant

ǫ
L ✛

❄
input, u ✲ ∆ ✲ + ✲ q −1 x̂ ✲ −C ✲+✛
✻

Φ ✛
Estimator

−K ✛
State feedback

Figure 8.5: Simultaneous control and state estimation

Concatenating Eqns 8.27 and 8.32 together gives the homogeneous equation
    
xk+1 Φ −∆K xk
zk+1 = = (8.33)
x̂k+1 LC Φ − ∆K − LC x̂k
| {z }
Φcl-ob

I will call the transition matrix for Eqn 8.33 the closed-loop/observer transition matrix (Φcl-ob ).
Ogata gives a slightly different formulation of the combined observer–state feedback control sys-
tem as Eqn 6-161 in [150, p714]. Ogata also recommends choosing the observer poles 4–5 times
faster than the system response.

Example: Combined state-feedback and observer

Suppose we are given a continuous plant

0.98
G(s) = , T = 0.1
(3s + 1)(2s + 1)
which we will sample at T = 0.1. The design specifications are that the estimator poles are to be
relatively fast compared to the controller poles
   
def 0.8 def 0.92
λest = and λctrl =
0.85 0.93

We can design the controller and estimator gains as follows

Listing 8.4: Pole placement for controllers and estimators

376 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

G = tf(0.98,conv([2 1],[3 1])); % Blackbox plant

Gss = ss(G); % convert to state-space

4 Gss1 = Gss; % Now create a special version with full outputs

Gss1.c = eye(2); Gss1.d = [0;0];

T = 0.1; Gssz = c2d(Gss,T); % Convert to discrete state-space at Ts = 0.1

9 p = [0.92 0.93]'; % Design controller gain, K, such that . . .

K = place(Gssz.a, Gssz.b, p); % λctrl = eig(Φ − ∆K)

pe = [0.80 0.85]'; % Design estimator gain, L, such that . . .

L = place(Gssz.a', Gssz.c', pe)'; % λest = eig(Φ − LC)
14

sort(eig(Gssz.a-L*Gssz.c)) - sort(pe) % Test results should be zero

which gives controller, K, and observer, L, gains as

 
  0.2146
K= 1.3848 1.6683 and L =
0.4109
Fig. 8.6(a) gives the S IMULINK configuration for the controller which closely resembles the block
diagram given in Fig. 8.5. To test the estimation and control, we will simulate a trend where the
actual and estimated starting conditions are different, namely
   
1 −1.6
x0 = , which is not the same as x̂0 =
1 2
The trends shown in Fig. 8.6(b) show the (discrete) estimated states converging rapidly to the
true continuous states, and that both converge, although slightly less rapidly, to the setpoint.

8.5 G ENERIC MODEL CONTROL

Control of multivariable nonlinear processes such as

ẋ = f (x, u, d, θ, t) (8.34)
is in general, much harder than that for linear processes. Previously we discussed the simple
state feedback control law; u = −K(x∗ − x) where x∗ is the desired state vector or setpoint. We
showed that we could augment this simple proportional only law to incorporate integral action
to ensure zero offset to get a control law such as 9.71. Now suppose that we wish to control not x,
but the derivative of x, dx/dt, to follow some desired derivative trajectory (ẋ)⋆ . We now wish to
find u(t) such that ẋ = r(t) where r(t) is some reference function that we can specify and where
dx∗ /dt is given by Eqn 8.34.

For any controller, when the process is away from the desired state, x∗ , we want to force the rate
of change of the states, ẋ to be such that it is moving in the direction of the desired setpoint, and
in addition, we would like the closed loop to have zero offset. The control law
Z t
(ẋ)∗ = K1 (x∗ − x) + K2 (x∗ − x) dτ (8.35)
0

combines these two weighted ideas. Ignoring any model-plant mistmatch, (ẋ)∗ = ẋ, so substi-
tuting in the plant dynamics Eqn. 8.34 to Eqn. 8.35, we get the control law
Z t
f (x, u, t) − K1 (x∗ − x) − K2 (x∗ − x) dτ = 0 (8.36)
0
8.5. GENERIC MODEL CONTROL 377

states yv To Workspace

4 4

4
Mux1

2 2

2
states
K*u input
2 output
ctrl
Gain plant BB output
C model
2
K*u

K* u
estimator Gain1

2 2 1 2
K*u
z
2
Delta
Unit Delay
Phi
K*u

(a) S IMULINK configuration

1 x1 (t)
0.5
0
x1

−0.5
−1
−1.5 x̂1 (t)
actual
−2
estimated
2 x̂2 (t)

1.5
x2

1 x2 (t)

0.5

0
0 1 2 3 4 5 6 7
time

(b) Estimated and true states

Figure 8.6: Simultaneous control and estimation

378 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Solving Eqn. 8.36 for u is the control law and this scheme is called “Generic Model Control” or
GMC and was developed by Lee and Sullivan in [117]. Note that solving Eqn 8.36 for u is in
effect solving a system of m nonlinear algebraic equations. This must be done at every time step.
Actually, the solution procedure is not so difficult since the previous solution ut−1 can be used as
the initial estimate for ut .

8.5.1 T HE TUNING PARAMETERS

The GMC control law (Eqn 8.36) imbeds a model of the process, and two matrix tuning parame-
ters, K1 and K2 . Selecting different tuning parameters changes the type of response. Assuming
we have a perfect model, the control law (Eqn 8.36) is
Z t
ẋ = K1 (x∗ − x) + K2 (x∗ − x) dτ (8.37)
0

If we differentiate once the control law Eqn 8.37, we get

ẍ = K1 (ẋ∗ − ẋ) + K2 (x∗ − x)

ẍ + K1 ẋ + K2 x = K1 ẋ∗ + K2 x∗ (8.38)
and taking Laplace transforms of 8.38 gives

s2 x + sK1 x + K2 x = sK1 x∗ + K2 x∗
x(s) K1 s + K2
∗
= 2
x (s) s + K1 s + K2
2τ ξs + 1
= 2 2 (8.39)
τ s + 2τ ξs + 1
where
1 k1
τ=√ , ξ= √ (8.40)
k2 2 k2
or alternatively
2ξ 1
k1 = , k2 = 2 (8.41)
τ τ
Note that Eqn 8.39 is a modified second-order transfer function. The time domain solution to Eqn
8.39 for a step change in setpoint, X ∗ (s), is given by
 √  √ 
−ξt
1−ξ 2 t p 1−ξ 2 t
1 − ξ2 − e τ 1 + ξ 2 cos τ + ξ 1 − ξ 2 sin
τ
x(t) = 2
, ξ 6= 1 (8.42)
1−ξ
For the limiting case where ξ = 1, the solution to Eqn. 8.39 simplifies to just
 
−t t
x(t) = 1 + e τ −1 (8.43)
τ

Fig. 8.7 shows the response for a perfectly GMC controlled process for different ξ. Note that as
ξ ⇒ ∞, the output response approaches a step response at t = 0, and as ξ ⇒ 0, the output
tends to x(t) = sin(2πt) + 1. This improvement comes, naturally, at the expense of decreased
robustness.

Fig. 8.8 shows the comparison between the overshoot, decay ratio and the ξ tuning value. These
values are easily obtained from Fig. 8.7 or analytically starting with Eqn 8.42. Note that for ξ > 1,
there is no oscillation in the response, and that a ξ ≈ 0.22 gives a quarter decay ratio.
8.5. GENERIC MODEL CONTROL 379

1.6

1.4

1.2

1
response

0.8

0.6
ξ=0.2
0.4
ξ=0.5
0.2 ξ=1
ξ=3
0
0 2 4 6 8 10
Normalised time, τ=1
(a) Various choices for shape factor. ξ = 0.2, 0.5, 1 and (b) A 3-dimensional plot of the same relation.
ξ = 3.

Figure 8.7: The normalised (τ = 1) response curve for a perfect GMC controller for various
choices of shape factor ξ.

overshoot
1
decay ratio
overshoot & decay ratio

0.8

0.6

0.4

0.2

Figure 8.8: The relation between the shape

0 factor ξ, the maximum overshoot (solid)
0 0.2 0.4 0.6 0.8 1 1.2 and decay ratio (dashed) for the perfectly
shape factor ξ controlled GMC response when τ = 1

Algorithm 8.3 Designing the GMC controller

To choose appropriate values for K1 and K2 , do the following:

1. For each state to be controlled, choose an appropriate shape from Fig. 8.7 for the response. In
other words, choose a ξ value. For tight control with little overshoot, choose a shape corre-
sponding to ξ ≈ 3 or higher , or if the loop is not so important, choose a more conservative
response such as ξ ≈ 1.
2. Specify the time scale by choosing an appropriate value for τ . Often it is convenient to
specify the rise time. For distillation column control, you may choose a cross over time of
30 minutes, or for a flow controller you may wish to cross the setpoint after only 5 seconds.
The time scale is application dependent.
3. From the specified ξ and τ , evaluate the scalars k1 and k2 using Eqn. 8.41.
380 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

4. The diagonal matrices K1 and K2 are formed by collecting the k scalars into diagonal n × n
matrices.

8.5.2 GMC CONTROL OF A LINEAR MODEL

Since GMC can be applied to a nonlinear process, one would expect it to work especially well
given a linear process ẋ = Ax + Bu. Now the GMC control law simplifies from the general
Eqn. 8.36 to
 Z t 
−1 ∗ ∗
u=B K1 x − (K1 + A)x + K2 (x − x) dτ (8.44)
0

noting that B−1 must exist and hence at least should be square. This means that we must have
the same number of manipulated variables as state variables.

It is easy to test such a controller using a stable random model using rmodel. Hopefully our
randomly generated control matrix B is inevitable. (If it is not, just re-run the script file.) To tune
the response, I will select a slightly different shape factor and time constant for each state, namely

ξ1 = 0.7, τ1 = 1.5
ξ2 = 0.8, τ2 = 2.6

Running the script file in Listing 8.5 gives results much like Fig. 8.9.

Listing 8.5: GMC on a Linear Process

dt = 0.5; n = 2;% sample time & # of states
m = n; p = 1; % ensure square system & 1 output

[A,B,C,D]=rmodel(n,p,m); % generate random continuous model

5 D = zeros(size(D)); C = eye(size(C)); % not interesting
[Phi,Del] = c2dm(A,B,C,D,dt,'zoh'); % discretise

x0 = round(3*randn(n,1)); % nice starting position

t = [0:dt:80]'; % simulation lifetime
10 Xspt = [square(t/8), 0.75*square(t/8*0.7)+3]; % setpoint

% GMC tuning rules

zeta=[0.7 0.8]'; tau=[1.5 2.6]'; % Shape factor ζ, & time constant τ , for each state.
K1 = 2*diag(zeta./tau); K2 = diag(1.0./tau./tau); % Diagonal K1 , K2 .
15 Ie = zeros(size(x0)); % reset integrator

x= x0; X = []; Uc = [];

for i=1:length(t) % start simulation
err = (Xspt(i,:)-x); % current error (x∗ − x).
20 Ie = Ie + err*dt; % update integral
Rt
rhs = K1*err + K2*Ie - A*x; % K1 x∗ − (K1 + A)x + K2 0 (x∗ − x) dτ
u = B\rhs; % Solve (linear) control law, Eqn. 8.44.

x = Phi*x + Del*u; % Do 1-step integration

25

X = [X; x']; Uc = [Uc;u']; % collect input for plotting

end % for

plot(t,[Xspt, X],t,Uc,'--') % Refer Fig. 8.9.

8.5. GENERIC MODEL CONTROL 381

Linear GMC
6
Outputs & setpoints

−2
0 20 40 60 80

20

15

10
inputs

5
Figure 8.9: The response of a randomly
0 generated linear process controlled by
GMC. Top: state (—) and setpoint (dot-
−5
0 20 40 60 80 ted) trajectories; Bottom=manipulated
time variables). See also Fig. 8.10.

We would expect the GMC controller to perform well, but what should really convince us is if
the actual response y(t) follows the analytical & desired trajectory, y ⋆ (t) since GMC is really a
model reference controller. These two curves are compared in Fig. 8.10 which shows an enlarged
portion of Fig. 8.9. The differences between the actual response and the specified trajectory are
due to sampling discretisation.

1.5
output & reference output

0.5

−0.5 Figure 8.10: Comparing the actual re-

Actual y
* sponse, y, (solid line) to a setpoint change
Reference y
−1
setpoint with the expected response, y ⋆ , (– · –). This
48 50 52 54 56 58 60 62
response is a zoomed part of the response
time given in Fig. 8.9.

This demonstration of the GMC controlling a linear process does not do the algorithm justice,
since the controller really is intended for nonlinear processes. This is demonstrated in the follow-
ing section.
382 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Table 8.2: The state, manipulated and disturbance variables and parameter values for the Litch-
field nonlinear CSTR, [124].
(a) State, input and disturbance variables
variable description units
T reactor temperature K
Ca concentration of A kgmol m−3
Tc temperature of jacket K
Tci temperature of cooling inlet K
Cai inlet concentration of A kgmol m−3
Fb inlet flow of stream B m3 s−1
Ta inlet temp. of stream A 50◦ C (333K)
Fa inlet flow of stream A 3.335 · 10−6 m3 s−1
Tb inlet temp. of stream B 50◦ C (333K)
Fc cooling flow 3 · 10−4 m3 s−1
(b) Model parameters
parameter value units parameter value units
V 0.0045 m3 K 16.6 · 106 m3 /kmol/s
Vc 0.0074 m3 E/R 5.6452 · 103 K
ρ 1000 kg/m3 ∆H −74.106 · 103 kJ/kg
Cp 4.187 kJ/kg/K Uar 0.175 kW/K

8.5.3 GMC APPLIED TO A NONLINEAR PLANT

Section 8.5.2 did not show the true power of the GMC algorithm since the plant was linear and
any linear controller such as an LMR would work. However this section introduces a nonlinear
model of a continuously stirred tank reactor, CSTR, from [124]. The CSTR model equations are

Fa Ta Fb Tb (Fa + Fb )T ∆H Uar
Ṫ = + − − r− (T − Tc ) (8.45)
V V V ρCp ρV Cp
Fa Cai (Fa + Fb )Ca
C˙a = − −r (8.46)
V V
Fc Uar
T˙c = (Tci − Tc ) + (T − Tc ) (8.47)
Vc ρVc Cp

where the reaction kinetics are second order

E
r = Ke− RT Ca2 (8.48)

The state, manipulated and disturbance variables as explained in Table 8.2 are defined as1

def  T def  T def  T

x= T Ca Tc , u= Tci Cai Fb , d= Ta Fa Tb Fc (8.49)

The model parameters are also given in Table 8.2.

The plant dynamics defined by Eqns 8.45 to 8.47 are nonlinear owing to the reaction rate r, being
an exponential function of the temperature T . This type of nonlinearity is generally considered
significant.
1 The original reference had only one manipulated variable T . This was changed for this simulation so the GMC is
ci
now a true multivariable controller.
8.5. GENERIC MODEL CONTROL 383

D ESIGNING THE GMC CONTROLLER

First we note that have n states and n manipulated variables. Thus the B matrix, or at least a
linearised version of the nonlinear control function will be square. We also note that the GMC
algorithm assumes that the process states (x) are measured, hence the measurement relation must
be C = I. The GMC control law, Eqn. 8.36, for this case is
    Fa ∆H Uar

0 0 TbV−T Tci V (Ta − T ) − ρCp r − ρC
p
(T − Tc )
 0 Fa − Ca   Cai  +   − FaVCa + r

−
V V
Fc Fc Tc Uar
Vc 0 0 F b − Vc + ρVc Cp (T − Tc )
Z t
K1 (x∗ − x) − K2 (x∗ − x) dτ = 0
0

Now you will notice that the control law is a system of algebraic equations unknown in u. For
this case, the control law is linear in u because I had the freedom of choosing the manipulated
variables carefully. If, for example, I had chosen Fa rather than Fb as one of the manipulated
variables, then the system would be control nonlinear, and hence more difficult to solve. But
in this case, the system is control linear, the GMC control law is a linear system of algebraic
equations, and that the 3 × 3 matrix to be inverted is in a lower diagonal form already making
the computation straight forward.

S IMULATING GMC WITH A NONLINEAR PROCESS

We will use a GMC controller to drive the states to follow a setpoint as follows:
  ⊤
  300 0.03 290 
 0 ≤ t < 5 min.
∗ ⊤
x = 310 0.03 290 5 ≤ t < 10 min.

  ⊤
310 0.025 295 t > 10 min.
 ⊤
starting from an initial condition of x0 = 290 0.035 280 and with a relatively coarse
sample time of T = 10 seconds. We set the shape factor ξ = 0.5, and the time constant for the
1 1
response to τ = 20 seconds. This gives k1 = 20 and k2 = 400 . The disturbance variables (d) are
assumed constant for this simulation and are taken from Table 8.2.

The state derivative ẋ as a function of x, u, d, θ, t is evaluated using Eqns 8.45 to 8.47 and is given
in listing 8.7 which is called by the integrator ode45 in listing 8.6.

Listing 8.6: GMC for a batch reactor

1 % GMC CSTR simulation
% Ref: Litchfield, Campbell & Locke Trans I Chem E Vol 57 1979

x0 = [290, 0.035, 280]'; % initial condition

dt = 10; t = [0:dt:15*60]'; % sample time (seconds)
6 Xspt = [300+10*square(t/150,70), 0.03+0.05*square(t/150), ...
295-5*square(t/150,30)]; % setpoint
d = [333 3.335e-6 333 3e-4]'; % disturbance vector constant
Ta = d(1); Fa = d(2); Tb = d(3); Fc = d(4);

11 k1 = 1/20; k2=1/400; % GMC tuning constants

K1 = eye(3)*k1; K2 = eye(3)*k2;
384 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

%Parameters for the CSTR

V = 0.0045; Vc = 0.0074; rho = 1000.0;
16 K = 16.6e6; ER = 5.6452e3; dH = -74.106e3;
Cp = 4.187; %kJ/Kg.K
Uar = 0.175; % kW/K
rCp = rho*Cp ;

21 x=x0;Ie=zeros(size(x)); X=[]; U=[];

for i=1:length(t) % start simulation
xspt = Xspt(i,:)'; % current setpoint
T = x(1); Ca = x(2); Tc = x(3); % state variables
Ie = Ie+(xspt-x)*dt; % update integral error
26 r = K*exp(-ER/T)*Ca*Ca; % reaction rate
R = [Fa/V*(Ta-T)-dH*r/rCp-Uar*(T-Tc)/rCp;-(Fa*Ca/V+r); ...
-Fc*Tc/Vc+Uar/rCp/Vc*(T-Tc)];
A = [0 0 (Tb-T)/V; 0 Fa/V -Ca/V; Fc/Vc 0 0];
u = A\(K1*(xspt-x)+K2*Ie - R); % solve for u GMC control law
31 % u = max(zeros(size(u)),u); % ensure feasible

[ts,xv] = ode45('fcstr',[0,dt/2,dt],x,[],u, ...

V, Vc, K, ER, dH, rCp, Uar, Ta, Fa, Tb ,Fc); % integrate nl ODEs
x = xv(3,:)'; % keep only last state
36

X(i,:) = x'; U(i,:) = u';

end %for
plot(t,[X,U]); % will need to rescale

Listing 8.7: The dynamic equations of a batch reactor

1 function x_dot = fcstr(t,x,flag,u, ...
V, Vc, K, ER, dH, rCp, Uar, Ta, Fa, Tb ,Fc);
% CSTR dynamic model, called from gmccstr.m

T = x(1); Ca = x(2); Tc = x(3); % states

6 Tci = u(1); % Inputs, Coolant temp in Kelvin
Cai = u(2); Fb = u(3);

r = K*exp(-ER/T)*Caˆ2; % reaction rate kmol/m3/s

x_dot(1,1)=(Fa*Ta+Fb*Tb-(Fa+Fb)*T)/V - dH*r/rCp - Uar/V/rCp*(T-Tc);
11 x_dot(2,1)=(Fa*Cai - (Fa+Fb)*Ca)/V - r ;
x_dot(3,1)=Fc*(Tci-Tc)/Vc+Uar*(T-Tc)/rCp/Vc;
return % fcstr.m

Physically the manipulated variables are constrained to be positive, and unless these constraints
are added explicitly to the GMC optimisation control law, negative values for some of the inputs
will occur. In the above example I simply clamp the input when these violations occur. If you
removed the u=max(zeros(size(u)),u); line from the control law, you will have the uncon-
strained version that ignores the physically realistic constraints which is the case shown in the
simulations given in Fig. 8.11.

Fig 8.11 shows the response of the CSTR controlled by the GMC. The upper plots show the state
trajectories and the lower plots give the manipulated variables used. Note that the controlled
response is good, but not perfect as we can still see some interaction between the states. The re-
action concentration Ca particularly seems strongly coupled to the other states, and this coupling
is not completely removed by the GMC controller. We could improve on this response both by
decreasing the sample time, or tightening the controller design specifications.
8.5. GENERIC MODEL CONTROL 385

40
Temp T,T (deg C)

30
C

20

10
0 5 10 15

0.4
A
Conc C

0.2

0
0 5 10 15

40
Temp T (deg C)

30
Ci

20

10
0 5 10 15
Figure 8.11: The nonlinear CSTR controlled
Conc C , Flow F * 5e4

40 by a GMC controller where ξ = 0.5 and

b

τ = 20 seconds and sample time T = 10

20
seconds.
0 Upper: The state trajectories (temperatures
Ai

T and Tc , and reaction concentration Ca ).

−20 Lower: The manipulated variable trajecto-
0 5 10 15
time (min) ries Tci and Cai and 5 · 104 Fb .

Just for fun, we can plot a three dimensional plot of the trajectory of the state variables from
the CSTR example under GMC control. Fig. 8.12 shows the actual state trajectory (solid line)
converging to the 4 distinct setpoints, (∗), in state space. Note that the independent variable time
is removed in this illustration.

A LTERNATIVE MANIPULATED VARIABLES CHOICES

Manipulating the inlet concentration of A or the inlet temperature of the cooling fluid is not very
practical. It is much easier to manipulate flow rates rather than temperatures of concentrations.
Therefore in this situation, a better manipulated variable vector would be u = [Fa , Fb , Fc ]⊤ rather
than that used above. Note however that if we had chosen Fa , Fc and Tci as u, we would have
a control problem that is not only nonlinear, but one that is also degenerate. If you construct the
GMC control law of Eqn. 8.36, you will get a structure such as
  
⋆ 0 0 Fa Z
 ⋆ 0 0   Fc  ⋆
= g(x) + K1 (x − x) + K2 (x⋆ − x) (8.50)
0 ⋆ ⋆ Tci

where we know everything on the right hand side of Eqn. 8.50, (it is a constant for the purposes
of this calculation) and wish to solve for u. Unfortunately, however, we can see from the struc-
ture matrix on the left hand-side of Eqn. 8.50, that Fa appears alone in two equations, and the
remaining two manipulated variables appear together in one equation. Thus this nonlinear al-
gebraic system is degenerate since in general we cannot expect a single unknown Fa to satisfy
386 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

35

30

c
Temperature T
25

20

15

10
0.5

0.4
40
0.3 35
Figure 8.12: A phase plot of T, Tc and
30
Ca for the GMC controlled CSTR ex- 0.2 25
ample. The state trajectory is the Conc CA 20
solid line, and the setpoints are ∗. 0.1 15 Temperature T

two independent conditions. Note that even though we do not have an overall degree of freedom
problem, (we have three unknowns and three equations), we do have a sub-structure that has a
degree of freedom problem.

Problem 8.5 1. Re-run this simulation and test the disturbance rejection properties of the
GMC controller. You will need to choose some suitable disturbance profile.

2. Re-run the simulation for different choices of GMC tuning constants. For each simulation,
verify that the response is the linear response that you actually specified in the GMC design.
(You may need to decrease the sampling interval to T ≈ 2 seconds to get good results.) Try
for example;

(a) τ = 20, ξ = 1.5

(b) τ = 5, ξ = 0.5
(c) any other suitable choice.

8.6 E XACT FEEDBACK LINEARISATION

For certain nonlinear systems, we can derive controllers such that the closed loop input/output
behaviour is exactly linear. This is quite different from our traditional approach of approximating
the linearities with a Taylor series approximation and then designing a linear controller based
on the approximate model. Designing exact feedback linearisation uses nonlinear results from
the field of differential geometry, which, to say the least, is somewhat abstract to most engineers.
Lie algebra for nonlinear systems will replace matrix algebra for linear systems, and Lie brackets
and Lie derivatives will be extensions to normal matrix operations. A good tutorial of this topic
with applications of interest to the process industry, and one from where most of this material
was taken, is [106] and the follow up, [107]. A review of nonlinear control for chemical processes
which includes a section on exact feedback linearisation is [28] and the classic text book for this
and other topics in nonlinear control is [190]. Extensions for when the process is not affine in
input, and for systems where the relative degree is greater than unity are discussed in [84].
8.6. EXACT FEEDBACK LINEARISATION 387

8.6.1 T HE NONLINEAR SYSTEM

We will only deal with SISO nonlinear systems of the form:

ẋ = f (x) + g(x)u (8.51)

y = h(x) (8.52)

where f , g are vector functions, or vector fields, h(x) is a scalar field and the input u, and output
y are scalars. Note that the system described by Eqn 8.51 is written in affine form, that is the ma-
nipulated variable enters linearly into the system. It is always possible to transform an arbitray
system into affine form by introducing a new variable. For example, given the non-affine system

ẋ = f (x, u)

we can define a new input, ν, which is the derivative of our original input, u̇ = ν, so now our
augmented system becomes
     
ẋ f (x, u) 0
= + ν (8.53)
u̇ 0 1
which is now in affine form. Whether creating a new variable ν which requires differentiating
the input is practical, or even feasible under industrial conditions is a different matter.

8.6.2 T HE INPUT / OUTPUT FEEDBACK LINEARISATION CONTROL LAW

Given the nonlinear system in Eqn 8.51 and 8.52, we will try to construct a control law of the form

u = p(x) + q(x)v (8.54)

where v is the command signal or setpoint, such that the closed loop system (obtained by substi-
tuting Eqn 8.54 into Eqns 8.51 and 8.52) is

ẋ = f (x) + g(x)p(x) + g(x)q(x)v (8.55)

y = h(x) (8.56)

and is linear from reference variable v to output y. The dashed box in Fig. 8.13 contains the
nonlinearity, but as far as we are concerned, viewing the system from the outside, it may as well
be a linear black box.

Note that for a controller in this configuration, the internal states will still be nonlinear. Notwith-
standing, once we have a linear relation from reference, v(t), to outputs, y(t), it is much easier to
design output-based controllers to control this now linear system, rather than the original non-
linear system. It is also clear from Fig. 8.13, that the controller requires state information to be
implemented.

The control design problem is to select a desired linear response, perhaps by specifying the time
constants of the desired closed loop, and then to construct the p(x) and q(x) functions in the
control law, Eqn. 8.54. These are given by the following relations:

−(Lrf h(x) + β1 Lr−1

f h(x) + · · · + βr−1 Lf h(x) + βr h(x))
p(x) = (8.57)
Lg Lr−1
f h(x)
1
q(x) = (8.58)
Lg Lr−1
f h(x)
388 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

u(t) ẋ x(t) output

command ✲Π ✲+ ✲Π ✲+ ✲ 1 ✲ h(·) ✲
v(t) s y(t)
✻ ✻ ✻ ✻

f (·) ✛
q(x) p(x)
g(·) ✛
✻ ✻

Nonlinear
plant

Figure 8.13: The configuration of an input/output feedback linearisation control law

where the Lie derivative, Lf h(x), is simply the directional derivative of the scalar function h(x)
in the direction of the vector f (x). Here, the Lie derivative is calculated by
n
⊤ ∂h(x) X ∂h(x)
Lf h(x) = f (x) · = fi (x) · (8.59)
∂x i=1
∂xi

One can apply the Lie derivative operator repeatedly, L2f = Lf Lf either to the same function, or
to different functions as in Lg L2f .

The relative order of a system such as Eqns 8.51–8.52 is the smallest integer r such that
Lg Lr−1
f h(x) 6= 0 (8.60)
An alternative interpretation of relative order is that r is the smallest order of derivatives of y that
depends explicitly on u. For linear systems, the relative order is simply the difference between
the number of poles and the number of zeros of the system, i.e. na − nb .

Finally, the βi in Eqn 8.57 are the user-chosen tuning constants that determine how our output
will respond to the command v. They are defined by
dr y dr−1 y dy
r
+ β 1 r−1
+ · · · + βr−1 + βr y = v (8.61)
dt dt dt
Typically it is convenient to choose Eqn. 8.61 as some sort of stable low-pass filter, such as a
Butterworth filter.

Algorithm 8.4 Exact feedback linearisation design procedure:

1. We are given a single-input/single-output nonlinear model (which may need to be trans-

formed into affine form), Eqn 8.51–8.52.
2. Find the relative order, r, of the system using Eqn. 8.60. This will govern the order of the
desired response differential equation chosen in the next step. (If r = 1, perhaps a GMC
controller is easier to develop.)
3. We choose a desired linear response, which we consider reasonable, in the form of Eqn 8.61,
that is, we need to choose r time constants or equivalent.
8.6. EXACT FEEDBACK LINEARISATION 389

4. We create our exact linearising nonlinear controller, Eqn 8.54 using the relations Eqns 8.57
and 8.58 by constructing Lie derivatives. (A symbolic manipulator such as M APLE may be
useful here.)

5. Closing the loop creates a linear input/output system, which can be controlled by a further
controller sitting around this system, designed by any standard linear controller design
technique such as LMR or pole placement.(However this is hardly necessary since we have
the freedom to choose any reasonable response in step 3.)

C AUTIONS AND RESTRICTIONS

This technique is not applicable if the “zero dynamics” are unstable. This is analogous to the
linear case where we are not allowed to cancel right-hand plane zeros with unstable poles. Tech-
nically, it works, but in practice, the slightest model mis-match will cause an unstable pole which
in turn will create an unstable controller. This then excludes process systems with dead time.
Clearly also owing to the nature of the cancellation of the process nonlinearities, exact feedback
linearisation is very sensitive to modelling errors. The technique demands state feedback, but
only delivers output linearisation. Full state linearisation techniques do exist, but exist only un-
der very restrictive conditions.

8.6.3 E XACT FEEDBACK EXAMPLE

Building an exact feedback controller is relatively easy for well behaved smooth nonlinear sys-
tems. The only calculus operation required is differentiation, and this poses few problems for
automated computer tools. This example shows how to build such a controller using a simple
program written for a symbolic manipulator. With this program, we can subsequently change
the system, and regenerate the new nonlinear control law. This is a departure from all the other
simulations presented so far, since here we can change the entire structure of the process in our
program, not just the order or the values of the parameters, and subsequently re-run the simula-
tion.

Suppose we have a simple 3-state nonlinear SISO system,

     
x˙1 sin(x2 ) + (x2 + 1)x3 0
 x˙2  =  x51 + x3 + 0 u (8.62)
x˙3 x21 1
| {z } | {z }
f (x) g(x)
 
y= 1 0 0 x (8.63)
| {z }
h(x)

which is already in the form of Eqn. 8.51–Eqn. 8.52 or affine form, and we wish to construct an
exact feedback linearised system following Eqn. 8.54. To do this we will follow Algorithm 8.4.

Since we will need to compute some Lie derivatives analytically,

∂h(x)
Lf h(x) = f (x)⊤ ·
∂x
to establish the relative order r, it may help to write a small procedure to do this automatically.
Listing 8.8 gives an example using the Symbolic toolbox.
390 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Listing 8.8: Find the Lie derivative for a symbolic system

function L = lie_d(F,h,x)
2 % Find the Lie derivative Lf h(x)
% Note jacobian returns a row vector

∂h(x)
L = F'*jacobian(h,x)'; % f (x)⊤ · ∂x
return

First we define the actual nonlinear functions f (x), h(x), and g(x) for our particular problem.

% define : must be real or will have conjugate problems

syms x1 x2 x3 real
x = [x1 x2 x3]';
4

F = [sin(x2)+(x2+1)*x3; x1ˆ5+x3; x1ˆ2]; % Eqn. 8.62

h = [1 0 0]*x
g = [0 0 1]'

The Lie derivatives are easy to construct,

∂h(x)
Lf h(x) = f (x)⊤ ·
∂x  
  1
= sin(x2 ) + (x2 + 1)x3 , x51 + x3 , x21 × 0 
0
= sin(x2 ) + (x2 + 1)x3

and check in M ATLAB using the routine giving in Listing 8.8

>> lie_d(F,h,x)
ans =
3 sin(x2)+(x2+1)*x3

To find the relative degree, we simply repeatedly call the function in Listing 8.8 starting at r = 1
and incrementing r until Lg Lr−1
f h(x) 6= 0 at which point we have established the relative order.

Listing 8.9: Establish relative degree, r (ignore degree 0 possibility)

Lfn = h % Dummy start
2 LgLfnh = lie_d(g,h,x) % Lg L1f h(x) case.
r = 1; % # relative degree counter

while LgLfnh == 0
r =r+1; % increment relative degree counter
7 Lfn = lie_d(F,Lfn,x) % need to build-up
LgLfnh = lie_d(g,Lfn,x) % stop when non-zero
end % while

Now that we know the relative degree r, we can design our desired response filter

Listing 8.10: Design Butterworth filter of order r.

8.6. EXACT FEEDBACK LINEARISATION 391

1 [z,p,K] = buttap(r) % Low-pass Butterworth filter analogue prototype

beta = real(poly(p));
%[B,beta] = butter(r,2,'s') % Alternative: but must scale with B(end).

and now construct the controller components p(x) and q(x) to form the closed loop.

Listing 8.11: Symbolically create the closed loop expression

qx = 1/LgLfnh; % Controller components q(x), p(x) , Eqn. 8.57, Eqn. 8.58.
2 px = beta(r+1)*h;
Lfn = h;
for i=r-1:-1:0
Lfn = lie_deriv(f,Lfn,x);
px = px+ beta(i+1)*Lfn;
7 end
px = -px*qx;

xdot = f+g*(px+qx*v) % Closed loop, Eqn. 8.55, hopefully linear.

In practice, we are only interested in the control law

>> (px+qx*v) % Control law, u = p(x) + q(x)v

ans =
(-4*x1-2*2ˆ(1/2)*(sin(x2)+(x2+1)*x3)-(x1ˆ5+x3)
*(cos(x2)+x3)-x1ˆ2*(x2+1))/(x2+1)+1/(x2+1)*v

which could, in principle, now be implemented on the plant.

A good test for this controller is to see if the closed loop response is indeed indistinguishable
from the specified linear filter. As demonstrated in Fig. 8.14, there is no discernable difference
indicating that the nonlinear controller design worked as intended in this instance.

Problem 8.6 Create an exact nonlinear feedback controller for a penicillin fermentation process.
The system dynamics are:

dX
= µ(S, X)X − DX (8.64)
dt
dS
= −σ(S, X)X + D(Sf − S) (8.65)
dt
y=X (8.66)

where X, S are the cell mass and glucose concentration, D is the dilution rate and manipulated
variable, and Sf is the feed glucose concentration, considered constant in this example. The
empirical growth rate expressions are:

µx S
µ(S, X) = (8.67)
Kx X + S
σ(S, X) = µ(S, X)/Yxs + π(S)/Yps + m (8.68)
µp S
π(S) = (8.69)
Kp S(1 + S/Ki )

and for simulation purposes, use parameters values as

392 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

3 4 MATLAB 3 1 3
u(1) 3
Function s 3
xdot Integrator h(x)
Signal Scope
Generator
butter

Analog
Filter Design

(a) S IMULINK configuration

1.5

1 output
linear filter
0.5
1
y=x

−0.5

−1

−1.5
0 10 20 30 40 50 60 70 80 90 100
time

(b) The closed loop response of the controlled nonlinear system compared
to the desired linear response

Figure 8.14: This example of exact feedback linearisation shows that there is no practical differ-
ence between the output of the nonlinear system and the desired linear filter.

µx 1.5 Kx 2.34
µp 3.45 Kp 1.56
Yxs 0.45 Yps 0.78
Ki 0.89 m 0.34

and manipulated, disturbance and initial conditions as

manipulated D 0<D<1
disturbance Sf 1.25 could also be varied
initial conditions X0 0.05
S0 0.6

Choose an appropriate β and design a linearising controller using Eqn 8.54. Test the closed loop
response by simulation and verify that your input/output response is actually linear.

Problem 8.7 1. What is the relative order of the following “nasty” system from §3.5.1?

ẋ1 = x2
ẋ2 = 6x1 − 7x2 + 24x3 + 10x22 + u
ẋ3 = x3 − x2 − x52
y = x1
8.7. SUMMARY 393

2. Implement and simulate an exact linearising controller for this system.

3. What problems do you face when you try to construct an input/output GMC controller for
this system? What requirement therefore must be satisfied to apply GMC?
4. Choose a suitable alternative measurement function such that GMC will work, and imple-
ment this as a M ATLAB simulation. Compare the performance of this controller with one
developed using geometric methods.

8.7 S UMMARY

This chapter introduced the theory and technique of the design of state space control systems.
Writing the dynamic equations in a discrete matrix form is both convenient for complex systems,
and intuitive since the design is done in the time domain. The major disadvantage is the fact that
the computation is more complex, and is the reason why the design is typically carried out on a
computer. The important conclusions of this chapter are:

• Controller design by pole placement means that we choose a gain matrix K, such that the
closed loop system (Φ − ∆K) has a desired dynamic response or alternatively has the de-
sired eigenvalues. To do this, the system must be controllable.
• We can design observers L in much the same way as controllers.

However, in this chapter we only covered the design of single input–multiple output controllers
or single measurement - multiple state observers. The true multivariable case is under-constrained,
and is covered in chapter 9.

Control of a nonlinear process may require a different approach and typically more computation.
Generic model control or GMC is a simple technique that can be used to control a nonlinear plant
in an optimal manner. Many of the so-called “difficult” processes in chemical process control
are due to nonlinearities and these are well suited to techniques such as GMC. An alternative,
more general nonlinear controller, particularly suitable for SISO nonlinear systems in control
affine form is one where a controller is sought that transforms the nonlinear system into a linear
openloop system that can subsequently be controlled using the far more extensive library of
linear controllers.
394 CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN
C HAPTER 9

C LASSICAL OPTIMAL CONTROL

God does not throw dice ? Nor is it our business to prescribe to God how He should run the
world.
Attributed to Niels Bohr replying to Einstein.

9.1 I NTRODUCTION

This, and the following chapter, is concerned with designing optimal controllers. In practice opti-
mal controllers are complicated owing to issues such as plant nonlinearities, constraints on state
and manipulated variables for example, but for some applications the improvement in controlled
response is well worth the implementation effort. Following a small aside, demonstrating opti-
mal tuning of a known controller, we will begin with the general optimal control problem in §9.3,
develop a solution strategy and test it on a batch reactor example in §9.2.4. Building on the gen-
eral nonlinear optimisation material, we will make some simplifying assumptions, and develop
an optimal linear controller in §9.4

Chapter 8 described the pole placement method of controller design for single input/single out-
put (SIMO) systems. It was also briefly mentioned that the evaluation of the gain matrix K
for MIMO systems is now under-constrained and an infinite number of possible controller con-
figurations satisfy the original design specifications. However, by using these extra degrees of
freedom, optimal controllers are possible. Section 9.4 will describe an optimal multivariable con-
troller termed a Linear Quadratic Regulator (LQR). Section 9.5 will present the dual problem of
an optimal estimator, the Kalman filter.

Model predictive control, which is the subject of chapter 10, lies somewhere between the full
generic optimal control problem, and the linear quadratic control. This chapter introduces the
concept of a moving horizon for the optimisation and compares a computationally intensive but
very general nonlinear optimal scheme with a much faster linear optimal controller based on a
FIR model.

395
396 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

9.2 PARAMETRIC OPTIMISATION

The simplest type of optimisation problem is known as parametric optimisation where we at-
tempt to find the ‘best’ values of a finite number of parameters. Finding the best three PID tuning
constants to minimise say the integral of square error is an example of parametric optimisation
described in §9.2.1.

Alternatively we could try to find the best shape of a trajectory for a missile to hit a target whilst
say minimising energy. In principle the shape may require an infinite number of discrete pa-
rameters to completely describe it, perhaps using a Taylor series. This then becomes an infinite
dimensional parametric optimisation problem, so we must tackle it with other techniques such
as variational calculus.

9.2.1 C HOOSING A PERFORMANCE INDICATOR

Optimal control is where you try to extremetise something to get the best controlled result. To
optimise, we must at least be able to quantify the controlled performance, that is to put numbers
to the quality of control. Typically however, this assessment is, in practice, a tricky function of
many variables such as process economics, safety customer satisfaction etc, which are all very
difficult to quantify.

We call the value that quantifies the performance the ‘objective function’, and typically use the
symbol J . Some authors use the term performance index or perhaps profit function, and natu-
rally we would like to maximise this quantity. However by convention most numerical software
packages are designed to minimise the objective function, and we certainly do not want to min-
imise the profit function, but rather minimise the cost function. Consequently the term ‘cost
function’ is often used in optimisation where it is assumed that we are interested in minimising
the objective function. For our purposes, where we are typically interested in minimising the
error residuals, we will be using ‘cost function’ for our objective function.

A simple example of optimal control is the optimal tuning of a PID controller for a given process
responding to a setpoint change. First we must choose an objective function that we wish to
optimise, often some sort of cost term (which we want to minimise), and then we search for the
tuning variables that minimise this cost. Since it is sometimes difficult to correlate cost to per-
formance in a simple manner, we often choose a performance objective more easily quantifiable
such as the integral of the difference between the actual process and the setpoint. This is termed
the integral of the absolute error or IAE performance index,
Z ∞
JIAE = |ǫt | dt (9.1)
0

where ǫt is defined as the difference between setpoint and actual, rt − yt at time t. Classically
control engineers have tried to design closed-loop control systems that behave similarly to a
second-order, slightly under-damped process. From experience this is found to be a reasonable
trade-off between speed of response, overshoot and ease of design. We can plot both the pro-
cess response, and visualise the area given by Eqn. 9.1 we wish to minimise using the floodfill
command fill.

1 t = linspace(0,10)';
[num,den] = ord2(1,0.54); % Choose ζ = 0.54
y = step(num,den,t); % do simulation
fill([t;0],[y;1],'c') % floodfill area as shown in Fig. 9.1.
9.2. PARAMETRIC OPTIMISATION 397

A damping factor of ζ = 0.54 is near the recommended value.

1.4

1.2
controlled response

0.8

0.6

0.4

0.2 Figure 9.1: The area of the shaded fig-

ure is the approximate value of the per-
0 formance integral. You could try other
0 2 4 6 8 10
time values for ζ, say 0.1.

Other common performance index definitions are given in Table 9.1. Classically the least-squares
index, ISE, was popular because for simple problems the solution for the optimum is analyti-
cal. However the so called robust techniques such as IAE or ITAE are now popular, although a
nonlinear optimiser is required to find the optimum tuning parameters.

Fig. 9.2 shows the steps to numerically compute the ITAE performance index for a given con-
trolled response. In this case the ITAE is 3.85 cost units. Notice how the time weighting empha-
sizes deviations late in the response.

We can compare the performance of a second order process with different damping ratios to find
the optimum, ζ ⋆ , for a given criteria.

1 t = linspace(0,10)'; dt = t(2)-t(1);
zeta = linspace(0.01,1.5,100)';% trial ζ values
Perf = [];
for i=1:length(zeta); % For each ζ . . .
[num,den] = ord2(1,zeta(i)); % 2nd order
6 y = step(num,den,t); % do simulation

err = 1-y; % error

Perf(i,:) = trapz(t,[abs(err) err.ˆ2 t.*abs(err) t.*err.ˆ2]);
end % for
11 plot(zeta,Perf) % Find optimum ζ for each index as shown in Fig. 9.3.

These curves are given in Fig. 9.3 from which we can extract the optimum damping ratios which
are summarised in Table 9.1. Note that the optimum ζ are all in the 0.5 to 0.75 region. Ogata, [152,
pp295-303], and especially Fig 4–38 on page 298 compares these alternatives in a similar manner.

9.2.2 O PTIMAL TUNING OF A PID REGULATOR

For example, say we wish to tune a PID controller to minimise the integral of the absolute error
times time of a response to a step change in setpoint. For this example I will choose the ITAE
performance index, which is a slight modification on the IAE performance index given previously
in Eqn. 9.1,
Z ∞
J = t |ǫt | dt (9.2)
0
398 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

ITAE performance

y and setpoint
2

1
error=(y−r)
0

−1

1
|(y−r)|

0.5

1
|(y−r)|*t

0.5

Figure 9.2: A breakdown of the ITAE 0

performance index. The top plot
shows the controlled response, and 4
ITAE=3.85
∫ |(y−r)|*t

the second plot the error. The third

and fourth plots show the difference 2
between the IAE and the ITAE. The
cumulative integral under the curve 0
in the fourth plot (shaded portion) 0 2 4 6 8 10
gives the overall performance. time

Optimum tuning of a 2nd order response

ise
ITAE iae
itae
ITSE itse

1
10
Cost factor

0
10 ISE

IAE
Figure 9.3: Various performance criteria as a 0 0.5 1 1.5 2
function of damping ratio ζ. Damping factor, ζ
9.2. PARAMETRIC OPTIMISATION 399

Table 9.1: Common integral performance indices and the optimum shape factor, ζ ⋆ , assuming a
second order prototype response.

Name Z ∞J ζ⋆ comment
Integral Squared Error, ISE ǫ2t dt 0.507 analytical, Gauss noise
Z 0∞
Integral Time Squared Error, ITSE tǫ2t dt 0.597 encourages ISE to stabilise rapidly
Z 0∞
Integral Absolute Error, IAE |ǫt | dt 0.657 recommended index
Z 0∞
Integral Time Absolute Error, ITAE t|ǫt | dt 0.748 similar to ITSE
0

The parameters, θ, we can adjust in this example are the PID tuning constants Kc , τi and τd ,
while the controller structure is fixed as a three term PID controller. The optimisation problem is
to choose suitable parameters θ such that the scalar J is minimised subject to the constraint that

Y (s) Gc Gp
= (9.3)
R(s) 1 + Gc Gp

where R(s) = 1/s. Here Gc is assumed a PID controller, and we will also assume that the ma-
nipulated variable is unbounded. Since we chose the ITAE performance index, (Eqn 9.2), no
analytical solution will exist for the optimisation. The ITAE is a good performance index, but
difficult to analytically manipulate hence the solution is best achieved numerically.

First the performance index, J , must be written in terms of the parameter vector θ. To do this,
we must insert our particular process model into Eqn. 9.3 and also substitute the PID controller
transfer function with the unknown parameter vector θ. One can then algebraically invert Eqn
9.3 to obtain a time domain solution yt if the model is sufficiently simple, or we could simulate
the closed loop system for a given θ. The error ǫt is yt − yt⋆ where yt⋆ is the reference setpoint,
in this case 1. One then substitutes this equation into Eqn 9.2 and numerically integrates to
calculate the performance index J . Finally any numerical optimisation routine can be used to
locate the minimum J as a function of the PID tuning parameters. Many such general purpose
optimisers exist such as Powell’s conjugate directions, Nelder-Mead, simplex etc. (See also [205]
for optimisation routines in M ATLAB.)

The hard part in attempting to optimally tuned PID controllers is locating the optimal solution.
In the above description, it was done via simulation using an iterative numerical technique. This
solution technique is completely general and can be applied to systems with nonlinear process
models, bounds on the manipulated variables and complex performance indices. It is however
often slow and computationally expensive, and prone to fall into local minima etc, as any numer-
ical technique.

Example: Optimal PID tuning Suppose we wish to optimally tune a PID controller given the
non-minimum phase plant,

−3s + 1
Gp (s) = (9.4)
(s2 + 0.8s + 0.52)(s + 4)(s + 0.4)

We can see the non-minimum phase from the right-hand zero s = +1/3 from the numerator term.
Gp (s) has two complex and two real poles.

Simulating the openloop model gives us some information about the severity of the inverse re-
sponse and the dominant time constants.
400 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

>>Gp = zpk(1/3,[-0.4, -4, -0.4+0.6i, -0.4-0.6i],-3)

Gp =

4 -3 (s-0.3333)
---------------------------------
(s+0.4) (s+4) (sˆ2 + 0.8s + 0.52)

Step Response

1.5

1
Amplitude

0.5

−0.5
>>step(Gp) % 0 5 10
Time (seconds)
15 20

We can use fminsearch (which uses Nelder-Mead’s simplex method), or one of the optimisation
routines from the O PTIMISATION toolbox to do the searching for parameters. Listing 9.1 returns
the IAE performance for a given plant and PID controller tuning constants subjected to a step-
test.

Listing 9.1: Returns the IAE performance for a given tuning.

function [j,y] = foptpidt(x,Gp);
2 % Computes the IAE for a given PID tuning
% Called via a numerical optimiser via optpidt.m
Kc = x(1); taui=1/x(2); taud=x(3); % PID controller parameters
Gc = tf(Kc*[taui*taud, taui, 1],[taui, 0]); % PID controller
Gcl = feedback(Gc*Gp,1); % closed loop Gc Gp /(1 + Gc Gp )
7

tsim = 50; dt=0.1; t = [0:dt:tsim]';

y = step(Gcl,t); % Now do simulation
R 50
j = sum(abs(y-1))*dt; % J (x) = 0 |ǫ| dt
return

The main script file that calls the optimiser to establish the best PID tuning constants is given in
Listing 9.2. Here we specify the plant of interest which we would like to tune, Eqn. 9.4, some trial
PID constants as a first estimate, and then we call the numerical search routine to refine these
estimates.

Listing 9.2: Optimal tuning of a PID controller for a non-minimum phase plant. This script file
uses the objective function given in Listing 9.1.
Kc = 0.5; taui=0.2; taud=1.; % trial start PID constants
x0 = [Kc,taui,taud]'; % trial tuning vector
3s−1
Gp = zpk(1/3,[-0.4,-4,-0.4+0.6i,-0.4-0.6i],3);% Gp (s) = (s2 +0.8s+0.52)(s+4)(s+0.4)
4
R 50
Perf = @(x) foptpidt(x,Gp); % J (x) = 0
|ǫ| dt

optns = optimset('TolFun',1e-2,'MaxIter',200); % optimisation tolerances

x = fminsearch(Perf,x0,optns) % Return optimal Kc , 1/τI , τD
9

[jbest,y] = foptpidt(x,Gp,jtype); % best-shot

t = [0:0.1:50]';
plot(t,y,t,ones(size(y)),'--'); % Refer Fig. 9.4.

In the above example we used the IAE performance objective, but we could just as well have
9.2. PARAMETRIC OPTIMISATION 401

chosen other alternatives such as ISE or ITAE, and we would see slightly different optimal tuning
parameters. The following table gives the values to which I converged for the three commonly
used performance indices.

Performance Kc⋆ reset⋆ , 1/τi τd⋆

IAE 0.407 0.382 1.43
ISE 0.359 0.347 1.93
ITAE 0.388 0.317 1.17

A comparison of the closed-loop response with a PID controller using these tuning constants is
given in Fig. 9.4. You can see that all responses are similar, although I personally favour the ITAE
alternative.

1.2
IAE
1

0.8
ITAE
Output & setpoint

0.6
ISE
0.4

0.2

0 Figure 9.4: A comparison of the op-

∫ ε2 = 7.100
timal PID tuning for a step response
−0.2 ∫ |ε| = 7.267
∫ t|ε| = 7.352 of a non-minimum phase system us-
−0.4
0 5 10 15 20 25 30 35 40
ing IAE, ISE and ITAE performance
Time objectives.

I MPLEMENTATION DETAILS

To be honest, I have cheated somewhat in the above example. Optimisation, even for small
dimensional problems such as this is always difficult and typically needs some ‘expertise’ to
get working properly. For the above case, even though I chose good starting guesses for the
tuning constants, the optimiser still had some numerical conditioning problems while searching
for the solution. We can avoid these problems partly by encouraging the optimiser to search in
locations that we think the solution will be, and also by telling the optimiser explicitly parameter
spaces to avoid such as negative values. If the optimiser tentatively tries a negative controller
gain, the closed loop system will be unstable, and the objective function meaningless. However
the computer program does not know this, and will still try to extract an improved estimate
from this information. Typically in these types of examples, the objective function loses any real
dependance on the parameters, and the response surface becomes flat, or worse, vertical! This is
the principle cause behind the poor conditioning warnings M ATLAB gives sometimes.

The final point to watch out for in an optimisation problem such as this is that the optimum does
not lie far outside any reasonable tuning values. For first or second order systems, the optimum
controller gain is infinity, and the optimiser will naturally tend in this direction. In this case
either constraints are needed, or the objective function can be modified to take into account the
vigorous controller action, or a new more complex model can be developed.

Optimal PID tuning #2 Finding optimal PID tuning constants for actual plant equipment is a
good example of practical optimisation. In this example, we wish to find the best Kc and τi
402 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

values for a step response of the black box,

0.002129s2 + 0.02813s + 0.5788 −0.5s

Gbb (s) = e ,
s2 + 2.25s + 0.5362

presented in Fig. 1.4 using an ISE performance criteria.

A contour of the performance surface is given in Fig. 9.5 from which we can directly identify the
minimum corresponding to the tightest control is obtained when using Kc ≈ 4.4 and τi ≈ 9.8. For
comparison, you can see from the insert plots of the error, that for small gains and large integral
times, the response is too sluggish, while for large gains and small integral times (large integral
actions), the response is overly oscillatory.

30
0.2

J=1.41
0.4
0.5

0.2

0.3
0.5
0.6

25
0.3

J=5.63
20 0.4
Integral time, τi

0.1
0.2

0.1

0.3
15
0.4
0.5
0.6

0.2

0.3
0.3

0.1

0.2
10 J=1.14
0.1
0.2

J=1.98 0.1
5
0.4 0.5
0.1 0.2 0.3 0.6
0.4
0.6

J=2.09
0.5

0.2 0.3 0.4 0.5

0.3 0.4 0.5 0.6
0
0 1 2 3 4 5 6 7 8
Controller gain, Kc

Figure 9.5: Optimum PI tuning of the blackbox plant. The contour plot shows the ISE perfor-
mance for a range of gains and integral times while the inserts show the actual error given a
step change in reference at those specific Kc , τi combinations. The optimum tuning constants are
Kc = 4.4 and τi = 9.8.

Naturally the optimum is only for a step test using our specific model. However the results for
other similar input scenarios should be similar. To construct the contour plot of Fig. 9.5 we used
an exhaustive grid search which requires lots of trials, hence the use of a model. An alternative,
and more efficient strategy is to use a nonlinear numeric optimiser which is the subject of the
next section.
9.2. PARAMETRIC OPTIMISATION 403

9.2.3 U SING S IMULINK INSIDE AN OPTIMISER

In cases where we have a complex model with nonlinearities or time delays (such as the black box
model of Fig. 9.5), it may be more convenient to use S IMULINK inside the objective function of
an optimiser. In this case we can use the sim command to execute a S IMULINK simulation from
within the objective function, but we must take care to tell S IMULINK to obtain its parameters
from the current workspace as opposed to the global workspace which it will do by default.
Further implementation details for these types of optimisation problems using S IMULINK are
given in [58, §8.6].

Example: Find the optimum PI tuning constants for controlling the plant
2
G(s) =
0.25s2 + 0.6s + 1
The S IMULINK model (named in this example sopt_pi.mdl) is given in Fig. 9.6. The objective

1
error

K*[taui 1](s) 2
taui.s 0.25s2 +0.6s+1
Step PI ctrl Scope
Plant

Figure 9.6: S IMULINK model where the PI tuning constants are to be optimised. Note that we
export the error in an output block at the highest level and that the PI controller has parameters
K and taui which are to be optimised.

function is called via the optimiser, (fminunc in this case), which in turn executes the S IMULINK
model from Fig. 9.6 and returns the time-weighted integral of the square error as a performance
index is given in listing 9.3.

Listing 9.3: Returns the ITSE using a S IMULINK model.

function j = foptslink_pi(x)
% Optimisation objective function that will call a simulink model
3

K = x(1); taui = x(2); % parameters to be optimised

t0 = 0; tf = 20; h = 0.1; % start & finish times

% System parameters used by simulink

8 opts = simset('SrcWorkspace','current','DstWorkspace','current');

[t,states,error] = sim('sopt_pi',[t0:h:tf],opts); % run simulink model

j = trapz(t,t.*error.ˆ2); % compute ITSE
return

From the M ATLAB command line we call the general unconstrained optimiser to compute the
two optimum PI tuning constants.
404 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

>> x0 = [0.2, 0.1]'; % Start guess for Kc and τi .

>> x = fminunc(@foptslink_pi,x0) % See function in Listing 9.3.
3

Optimization terminated successfully:

Current search direction is a descent direction, and magnitude of
directional derivative in search direction less than 2*options.TolFun
x =
8 66.5402
25.6549

The optimisation routine found that a gain of Kc = 66.5 and an integral time of τi = 25.7 min-
imised our integral-time squared error performance index.

9.2.4 AN OPTIMAL BATCH REACTOR TEMPERATURE POLICY

This example illustrates how to establish an optimal input profile u(t) for a given system and
initial conditions. In this section we are going to approximate the smooth profile with a small
number of constant variables and we are going to use parametric optimisation. Later in sec-
tion 9.3.3 we will repeat this optimisation problem, but solve for the continuous profile using
functional optimisation.

Suppose we have a batch reactor as shown in Fig. 9.7 in which two chemical reactions are carried
out in series,
k1 k2
A → B → C (9.5)

The first reaction produces the desired product (B) which is what we wish to maximise after two
hours reaction duration, but the second reaction consumes our valuable product, and produces
a byproduct C, which naturally we wish to minimise.

A Open reactor after 2 hours

✲ product (at end)

steam (hot) ✲ condensate (cold)

heating coil

Figure 9.7: Production of a valuable chemical in a batch reactor.

def
In this case, the only manipulated variable is the jacket temperature, u = T and we will consider
def
the concentrations of A and B to be the two states, x = [ca , cb ]T . As the jacket temperature varies,
so does the reaction temperature which in turn affects the Arrhenius rate constants (k1 , k2 ), which
changes the ratio of desired product to by-product. The plant dynamics are given by the reaction
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 405

kinetics and initial conditions are

dx1
= −2e−6/u x1 , x1 (t = 0) = 0.9
dt
dx2
= 2e−6/u x1 − 5e−11/u x2 , x2 (t = 0) = 0.1
dt
Our objective for the operation of this reactor is to maximise the amount of B after a given re-
action time, in this example t = 2 hours, by adjusting the temperature over time as the reaction
progresses, or
Z t=2
J = x2 (t = 2) + 0 dt (9.6)
0

If we decide to operate at a constant temperature, then Fig. 9.8(a) shows us which temperature is
the one to maximise the amount of B at the end of the reaction. Alternatively, if we decide to run
the batch reactor for 1 hour at one temperature, and the second hour for another then we would
expect the optimum to improve. Fig. 9.8(b) shows the three dimensional response surface for this
case.

0.5 2 Temperatures

0.45 0.5

0.4
0.4
0.35
b
final C
C final

0.3
0.3
0.2
0.25

0.2 0.1
0
0.15 10
5
5
0.1 0
10
0 1 2 3 4 5 6 7
Temperature T T
1 2

(a) Optimisation of a single constant temperature (b) The response surface of the objective function when
using two temperatures

Figure 9.8: By using two temperatures, we can improve (slightly) the final product concentration.

We could in principle continue this discretisation of the input profile. Fig. 9.9 shows the situation
when we decide to use three temperatures equally spaced across the 2 hour duration of the batch.
In this case the objective function becomes a response volume so we must use volume visualisa-
tion techniques to see where the optimum lies. Fig. 9.10 compares the optimum trajectories for
the case where we select 1,2,3,5 and 7 equally spaced temperatures divisions. Also given is the
limiting case where the number of temperature divisions tend to ∞. The analytical computation
for this latter case is given in section 9.3.3.

9.3 T HE GENERAL OPTIMAL CONTROL PROBLEM

The optimal control problem has the following characteristics:

1. A scalar performance index or cost function, J (u) that is to be minimised (or maximised)
by adjusting the manipulated variables u(t).
406 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

0.42
6 0.4
5 0.38
4 0.36
3
T

3 0.34
2 0.32
1 0.3
8
10 0.28
6
5 0.26
T 4 0 T1
2

(a) The response volume when using 3 temperatures (b) Isosurfaces of the response volume better showing
where the optimum point in 3D space is.

Figure 9.9: Temperature profile optimisation using 3 temperatures. Compare with Fig. 9.8.

Figure 9.10: The optimum temperature

profiles for cases where we have different
numbers of temperature divisions. Note
how as the number of allowable tempera-
ture divisions increase, the discretised tra-
jectory converges to the smooth analytical
optimum profile.
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 407

2. Constraints that must be either:

(a) continuously satisfied, that is the constraint must be satisfied for all time, or
(b) satisfied only at the end of the optimisation problem, a terminal constraint.

These constraints can either be hard (they must never be violated under any conditions, or
soft (they can be violated under special conditions).

3. The optimisation horizon is the length of time over which the optimisation is allowed to
take place. Sometimes the horizon is infinite, but in practical cases the horizon has some
upper limit over which an optimal solution is sought to minimise the objective function.

The optimal control problems can be written as a standard mathematical optimisation problem,
thus many algorithms and therefore computer programmes exist to give a solution. Two com-
mon solution procedures are Pontryagin’s maximum principle and dynamic programming. [64,
pp16–20] or [51] covers these aspects in more detail and a simple introduction to Pontryagin’s
maximum principle is given in [174, Chpt 13] and [170, pp 84–105].

The optimal control problem as described above is quite general, and the solution will be different
for each case. Section 9.3.1 develops the equations to solve the general optimal control problem
using variational calculus and Section 9.2.4 demonstrates this general approach for a simple non-
linear batch reactor application. The accompanying M ATLAB simulation clearly shows that the
computation required is involved, and motivates the development of less general linear optimal
control designs. This results in the Linear Quadratic Regulator (LQR) described in subsequently
Section 9.4.

There are many other texts that describe the development of optimal controllers. Ogata has al-
ready been mentioned, and you should also consult [101, p226] for general linear optimal control,
[165, 169, 170] for chemical engineering applications and [38] is considered a classic text in the
optimisation field. A short summary available on the web is [45]. A survey really highlighting
the paucity of applications of linear quadratic control, (which is a simplified optimal controller
for linear systems), in the process industries is [98].

9.3.1 T HE OPTIMAL CONTROL FORMULATION

To develop our optimal controller, we need to define, as listed in §9.3, an objective function,
a feasible search space, and list our constraints. The Optimal Control Problem, or OCP, is when
given a general nonlinear dynamic model,

ẋ = f (x, u, t), x(t = 0) = x0 (9.7)

we wish to select u(t) to minimise the scalar performance criteria

Z tf
J = φ (x(tf )) + L(x, u, t) dt (9.8)
| {z }
Mayer problem
|0 {z }
Lagrange problem
| {z }
Bolza problem

This performance criteria J (u) to be minimised is quite general and is usually related to process
economic considerations and is often called a cost function. Technically Eqn. 9.8 is termed a
functional since it maps the function u(t) to the scalar J . The first term of Eqn. 9.8 is called
the termination criteria, and is only concerned about the penalty of the final state. The second
408 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

integral term is concerned with the cost incurred getting to the final state. This final term is
typically some sort of ISE or IAE type term. For example if

L(x, u, t) = x⊤ Qx

where the matrix Q is a positive definite matrix, then this term computes the squared deviations
in state variables and is used when we are concerned with the cost of deviations in state variables.
Alternatively if
L(x, u, t) = 1
then we are concerned sorely with the total time for the optimisation (which we want to min-
imise). If only the terminal criterion, φ, is present, we have a Mayer problem, if we are only
concerned with the integral term, we have a Lagrange problem, and if we have both terms, then
we have what is known as a Bolza problem. We can convert between all three types of problems
by introducing extra state variables, see [51, p71] for more hints how to do this.

We solve this constrained optimisation problem by converting it to an unconstrained optimisa-

tion problem using generalised Lagrange multipliers. These are a vector of introduced variables
λ, that are pre-multiplied to each of the constraint equations and then added to the original ob-
jective function. Now converting Eqns 9.8 and 9.7 to an unconstrained optimisation problem
gives
Z tf  
J =φ+ L(x, u, t) + λ⊤ (f − ẋ) dt (9.9)
0
Z tf  
=φ+ H − λ⊤ ẋ dt (9.10)
0

where the scalar H is termed the Hamiltonian and is defined

def
H = L + λ⊤ f (9.11)

We can eliminate ẋ in Eqn 9.10 by integrating by parts giving

0 Z tf  
⊤  ⊤
J = φ + λ x + H + λ̇ x dt (9.12)
tf 0

Using variational calculus, we get the Euler-Lagrange equations which define the dynamics of
the co-state variables

∂H ∂φ 
λ̇ = − with final condition λ(tf ) = (9.13)
∂x ∂x tf

where we emphasise that the co-state dynamics are specified by terminal boundary conditions as
opposed to the initial conditions given for the state dynamics in Eqn. 9.7.

Finally, the optimum control input profile is given by extremitising the Hamiltonian, or equiva-
lently solving for u such that
∂H
=0 (9.14)
∂u
Solving these two equations, and applying the manipulated variable to our dynamic system
(Eqn. 9.7) will result in optimal operation.

In many practical cases, the manipulated variable may be constrained between an upper and
lower limit. In this case, we would not blindly use the stationary point of the Hamiltonian
(Eqn. 9.14) to extract the manipulated variable, but we would simply minimise H, now conscious
of the constrained manipulated variable.

In summary, the Optimal Control Problem, or OCP, is given in Algorithm 9.1.

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 409

Algorithm 9.1 The Optimal Control Problem

The solution procedure of the optimum control problem (OCP) is as follows:

1. Given a dynamic model with initial conditions

ẋ = f (x, u, t), x(t = 0) = x0 (9.15)

and a scalar performance index which we wish to extremetise over a finite time from t = 0
to t = tf ,
Z tf
J (u) = φ (x(tf )) + L(x, u, t) dt (9.16)
0

2. then we can form the Hamiltonian

def
H = L + λT f (9.17)

where the n introduced co-state dynamics are given by


∂H ∂φ 
λ̇ = − with final condition λ(tf ) = (9.18)
∂x ∂x tf

3. and the optimum input is given by extremetising H, or by solving

∂H
=0 (9.19)
∂u
for u(t).

Section 9.3.3 solves the optimal control problem for some simple examples. including the optimal
temperature profile of a batch reactor from section 9.2.4. For the batch reactor we have only a
termination criteria in our objective function, and no state or manipulated variable energy cost.
This makes the calculation somewhat easier.

9.3.2 T HE TWO - POINT BOUNDARY PROBLEM

Equations 9.7, 9.13–9.14 need to be solved in order to find the desired u(t), the optimal control
policy at every sampling instance. This, however, is not a trivial task. Normally we would expect
to know the initial conditions for the states so in principle we could integrate Eqn. 9.7 from the
known x(0) for one sample interval, excepting that we do not know the correct control variable
(u) to use. However, we can solve for the instantaneous optimum u by solving Eqn. 9.14. The
only problem now is that we do not know the correct co-state (λ) variables to use in Eqn. 9.14.
So far the only things we know about the co-state variables are the dynamics (Eqn 9.13), and the
final conditions, but unfortunately not the initial conditions.

These types of ODE problems are known as Two Point Boundary Value problems. If we have n state
dynamic equations, then the number of state and co-state dynamic equations is 2n. We know n
initial state conditions, but that still leaves us n initial conditions short for the costates. (To solve
the system of ODEs, we must know as many initial conditions as DEs). However we do know
the final values of the costates, and this information supplies the missing n conditions.

Methods to solve two point boundary value problems are described in [163, Chp16] and [199,
p176] and are in general much harder to solve than initial value ODE problems, and often require
410 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

an iterative approach. The most intuitive way to solve two point boundary value ODEs is by the
shooting method or as it is sometimes termed, the boundary condition iteration, or BCI [169,
p237]. This technique is analogous to the artillery gunner who knows both his own and the
enemy’s position, but does not know the correct angle to point the gun so that the shell lands on
the enemy. Typically an iterative approach is used where the gun is pointed at different angles
with an observer who watches where the shells land, and then communicates with the gunner
what corrections to make. However, the iteration loop outside the integration is not the only
complication. In many physical cases, numerically integrating the state equations backwards or
the co-state equations forwards is unstable, where integrating in the opposite direction causes no
problems.

An alternative method known as Control Vector Iteration (CVI) avoids some of the failings of the
BCI method. A comparison of these two techniques is given in [96]. Perhaps the most robust
scheme is to use collocation which is the strategy used by the M ATLAB routine bvp4c designed
for boundary value problems.

9.3.3 O PTIMAL CONTROL EXAMPLES

The following examples illustrate two optimal control problems. In both cases we follow Algo-
rithm 9.1 and use the S YMBOLIC toolbox to assist in developing the necessary equations.

R AYLEIGH ’ S OPTIMAL CONTROL PROBLEM

We wish to minimise Z 2.5

J (u) = x21 + u2 dt (9.20)
0

subject to the state dynamics and state initial conditions

ẋ1 = x2 , x1 (t = 0) = −5


ẋ2 = −x1 + 2 − 0.1x22 x2 + 4u, x2 (t = 0) = −5

For this problem it is convenient to use the Symbolic toolbox to develop the necessary equations.
First we define the state dynamics, the objective function, and form the Hamiltonian.

1 >> syms u lam1 lam2 x1 x2 real % define symbolic variables

def def
>> x = [x1 x2]'; lambda = [lam1 lam2]'; % x = [x1 , x2 ]T , λ = [λ1 , λ2 ]T
>> f = [x(2); ...
-x(1)+(2-0.1*x(2)ˆ2)*x(2) + 4*u];% State dynamics ẋ = f (x, u) R
>> phi = 0; L = x(1)ˆ2 + uˆ2; % Terms in the objective function, φ + L dt
6

>> H = L + lambda'*f % Hamiltonian, H = L + λT f

H =
lam1*x2 + uˆ2 + x1ˆ2 - lam2*(x1 - 4*u + x2*(x2ˆ2/10 - 2))

Given the Hamiltonian, we can construct the co-state dynamics, and the terminal conditions from
Eqn. 9.18.

1 >> lam_dot = -jacobian(H,x)' % Eqn. 9.18. Note: Jacobian is a row vector

lam_dot =
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 411

lam2 - 2*x1
lam2*((3*x2ˆ2)/10 - 2) - lam1
>> lam_final = jacobian(phi*x(1),x)'
6 lam_final =
0
0

The optimal control input is given by solving Eqn. 9.19 for u(t), or

>> uopt = solve(diff(H,u),u) % Eqn. 9.19

2 uopt =
(-2)*lam2

Now all that is left to do is to form and solve our two-point boundary value problem. We can
substitute the expression for the optimum u(t) in the state dynamics, and append the co-state
dynamics

>> x_dot = subs(f,u,uopt);

2 >> [x_dot; lam_dot] % State & co-state dynamics
ans =
x2
- 8*lam2 - x1 - x2*(x2ˆ2/10 - 2)
lam2 - 2*x1
7 lam2*((3*x2ˆ2)/10 - 2) - lam1

To solve the boundary value ODE problem, we take the state and co-state dynamic equations de-
veloped above and insert into the two-point boundary value ODE routine bvp4c where we need
to specify the dynamics, the boundary conditions, and additionally estimate a suitable grid for
the collocation points. We also need to given an initial estimate trajectory for the state and co-state
profiles over the integration interval, but in this case I will simply supply constant trajectories.

zdot = @(t,z) ...

[z(2); ...
3 -8*z(4) - z(1) - z(2)*(z(2)ˆ2/10 - 2); ...
z(4) - 2*z(1) ;...
z(4)*(3/10*z(2)ˆ2-2) - z(3)]; % State & co-state dynamics

% Set up boundary conditions

8 x0 = [-5;-5]; lam_final = [0;0]; % Initial state conditions & terminal co-state conditions
BCres = @(a,b) [a(1:2)- x0; ...
b(3:4)- double(lam_final)]; % Boundary residuals
solinit = bvpinit(linspace(0,2.5,10),[-1 -1 -0.2 -0.5])

13 sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP

The optimum profile is given in Fig. 9.11 where we can quickly check that the boundary con-
ditions are satisfied. We can also visually check that the optimum input u⋆ (t) = −2λ2 (t) as
required. The circles plotted show the position of the collocation points and we should check
that they capture the general trend of the profile.
412 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

6
x1
4 x2

x(t)
0

−2

−4

−6
10

5
u(t)

−5
5

0
Figure 9.11: The optimum control policy for
λ(t)

the Rayleigh problem. The upper two trends −5 λ1

are the states, the middle is the optimal con- λ2
trol policy u⋆ (t), and the lower two trends are −10
0 0.5 1 1.5 2 2.5
the corresponding co-states. time

A N OPTIMAL BATCH REACTOR POLICY REVISITED

In the optimum profile determination for the batch reactor example in section 9.2.4, we approxi-
mated the continuous profile with a finite series of constant zeroth-order holds. In this example,
we will compute the true continuous curve.

Recall that the plant dynamics are given by the reaction kinetics and initial conditions are
dx1
= −2e−6/u x1 , starting from x1 (t = 0) = 0.9
dt (9.21)
dx2
= 2e−6/u x1 − 5e−11/u x2 , starting from x2 (t = 0) = 0.1
dt
and the objective function to be maximised is
Z t=2
J = x2 (t = 2) + 0 dt
0

which indicates that L(x, u, t) = 0. The Hamiltonian function, H = L + λT f , in this instance is

 
  −2e−6/u x1
H = 0 + λ1 λ2
2e−6/u x1 − 5e−11/u x2
 
= −2λ1 e−6/u x1 + λ2 2e−6/u x1 − 5e−11/u x2

The costates vary with time following λ̇ = −∂H/∂x and finish with
dλ1
= 2e−6/u (λ1 − λ2 ), ending at λ1 (t = 2) = 0
dt (9.22)
dλ2
= 5e−11/u λ2 , ending at λ2 (t = 2) = 1
dt
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 413

since φ = x2 . Now the optimum temperature trajectory is the one that maximises H or the
solution to dH/du = 0, which in this case can solved analytically to give
5
u=   (9.23)
55λ2 x2
ln 12x1 (λ2 −λ1 )

Once again constructing the costate dynamics and optimal input is easier with the help of a
symbolic manipulator as shown in Listing 9.4 below.

Listing 9.4: Analytically computing the co-state dynamics and optimum input trajectory as a
function of states and co-states
>> syms u lam1 lam2 x1 x2 real % define symbolic variables
def def
2 >> x = [x1 x2]'; lambda = [lam1 lam2]'; % x = [x1 , x2 ]T , λ = [λ1 , λ2 ]T

>> f = [-2*exp(-6/u)*x(1); ...

2*exp(-6/u)*x(1) - 5*exp(-11/u)*x(2)];
>> phi = x(2); L = 0;
7 >> H = L + lambda'*f; % H = L + λT f
>> lam_dot = -jacobian(H,x)'; % −∂H/∂x
lam_dot =
(2*lam1)/exp(6/u) - (2*lam2)/exp(6/u)
(5*lam2)/exp(11/u)
12 >> lam_final = jacobian(phi,x)'% λtfinal = ∂φ/∂x
lam_final =
0
1
>> uopt = solve(diff(H,u),u)
17 uopt =
5/log(-(55*lam2*x2)/(12*lam1*x1 - 12*lam2*x1))

Once again we can solve the two-point boundary value problem succinctly using bvp4c as
demonstrated in Listing 9.5. We need to supply the state and co-state dynamic equations, Eqn. 9.21
and Eqn. 9.22, the four required boundary conditions, the initial x1 (0), x2 (0) and the final λ1 (2), λ2 (2),
and the optimum input trajectory as a function of states and co-states.

Listing 9.5: Solving the reaction profile boundary value problem using the boundary value prob-
lem solver, bvp4c.m.
u = @(x) 5/log(55*x(4)*x(2)/12/x(1)/(x(4)-x(3))) % Optimum input trajectory, Eqn. 9.23
2

% State and co-state dynamics, Eqn. 9.21 & Eqn. 9.22.

zdot = @(t,x) ...
[-2*exp(-6/u(x))*x(1);
2*exp(-6/u(x))*x(1) - 5*exp(-11/u(x))*x(2);
7 2*exp(-6/u(x))*(x(3)-x(4));
5*exp(-11/u(x))*x(4) ];

% Boundary residuals x1 (0) = 0.9, x2 (0) = 0.1 and λ1 (2) = 0, λ2 (2) = 1

BCres = @(a,b) [a(1:2)-[0.9; 0.1]; ...
12 b(3:4)-[0;1]];

tf = 2; % Final time tfinal = 2

solinit = bvpinit(linspace(0,tf,10),[0.6,0.2,0.4 0.6])
sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP
414 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

17

xint = linspace(0,tf,1e3)'; Sxint = deval(sol,xint); % interpolate solution

plot(sol.x, sol.y,'ro',xint,[Sxint(1,:)',Sxint(2,:)'],'-') % See Fig. 9.12

The optimum input temperature policy is shown in Fig. 9.12 with the resultant state and costate
trajectories. The amount of desired product B that is formed using this policy is 0.438 and we can
be confident that no other temperature policy will improve on that result.

1
Ca
0.8 Cb
Concentration
0.6
Cb(final) = 0.438
0.4

0.2

0
8
Temperature

0
1.5
λ1
1
Co−states, λ

λ2

0.5

Figure 9.12: The concentrations, the opti- 0

mum temperature trajectory, and co-states
−0.5
for the batch reactor as evaluated using an 0 0.5 1 1.5 2
optimum control policy. time [hours]

In both the Rayleigh and the batch reactor optimal control examples given above we could ex-
tract analytical expressions for the optimal control policy as an explicit function of states and
co-states, u⋆ (t) = g(x, λ, t) which made the subsequent boundary-value problem easier to solve.
However in many practical cases, the solution of Eqn. 9.19 to give u⋆ (t) may not be possible to
find in closed form. In these cases, we may need to embed an algebraic equation solver such as
fsolve, or an optimiser such as fminunc.m inside the BVP solution routine. However the prob-
lem to be solved now is a differential algebraic equation or DAE which introduces considerable
computational problems.

9.3.4 P ROBLEMS WITH A SPECIFIED TARGET SET

In some optimal control problems we wish not only to establish the optimum profile for u(t)
starting from a given initial condition, x(0), but we might also require some (or all) of our states
to satisfy some arbitrary condition at the final time,
ψ (xf , tf ) = 0 (9.24)
9.3. THE GENERAL OPTIMAL CONTROL PROBLEM 415

These problems are known as OCP problems with a target or manifold set. Further details and
solution strategies for these sorts of problems are given in [45, 165]. The main difference here is
that we need to introduce a new vector of Lagrange multipliers, ν, the same dimension as the
number of target state constraints in Eqn. 9.24, and these get added to the modified objective
functional.

A simplified, but very common situation is where we wish some of our states to hit a specified
target, and perhaps leave the remainder of our states free. In this case the boundary conditions
simplify to:

1. if xi (tf ) is fixed, then we use that as a boundary constraint, i.e. xi (tf ) = xif ,
2. if xi (tf ) is free, then we use λi (tf ) = ∂φ/∂xi (tf ) as usual for the boundary constraint.

Note that together, these two conditions supply the additional n boundary conditions we need
to solve the two-point boundary value problem of dimension 2n.

Suppose we wish to find the optimum profile u(t) over the interval t ∈ [0, 1] to minimise
Z 1
J = u2 dt
0

subject to the dynamics

ẋ = x2 sin(x) + u, x(0) = 0
but where we also require the state x to hit a ‘target’ at the final time
x(1) = 0.5
In this case we have one state variable and one associated co-state variable, so we need two
boundary conditions. One is the initial state condition, and the other is the specified state target.
This means that for this simple problem we do not need to specify any boundary conditions for
the costate. The rest of the OCP problem is much the same as that given in Algorithm 9.1.

1 syms x u lam real % define symbolic variables, u(t), x(t), λ(t)

tf = 1; % final time
xtarget = 0.5; % target constraint

6 f = [xˆ2.*sin(x) + u]; % State dynamics ẋ = f (x, u)

phi = 0; L = uˆ2; % Terms in the objective function, φ, L

H = L + lam'*f % Hamiltonian, H = L + λf

11 lam_dot = -jacobian(H,x)'; % ensure column vector

uopt = solve(diff(H,u),u)

Running the above code gives the optimum input as u⋆ = −λ/2 and the the two-point boundary
problem to solve as
   
ẋ x2 sin(x) − λ2
=
(9.25)
λ̇ −λ x2 cos(x) + 2x sin(x)

in two variables. The two boundary conditions are

x(0) = 0, and x(1) = 0.5
416 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

% Define state & co-state dynamics, Eqn. 9.25.

ztmp = @(t,z1,z2) ...
3 [z1ˆ2*sin(z1) - z2/2;
-z2*(z1ˆ2*cos(z1) + 2*z1*sin(z1))];

zdot = @(t,z) ztmp(t,z(1),z(2));

x0 = [0];
8 BCres = @(a,b) [a(1)- x0; ...
b(1)- xtarget]; % Boundary residuals: Initial & terminal state conditions

xlam_init = @(t) [0.5*t; -1];

13 solinit = bvpinit(linspace(0,tf,10),xlam_init)
sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP

% Extract solution
t = sol.x; z = sol.y'; % could also check derivatives
18 x = z(:,1); lam = z(:,2);
u = -lam(:,1)/2;

Fig. 9.13 shows the result of this optimal control problem. We should note that the state, x, did
indeed manage to hit the specified target at t = 1. Finally we can establish the optimum profile
for u(t).

x
0.6
States

0.4
0.2
0

0.5

0.45
u

0.4

0.35

−0.6
λ
co−states

−0.8

−1
Figure 9.13: Optimal control with a target 0 0.2 0.4 0.6 0.8 1
where x(1) must equal 0.5. time

Problem 9.1 (These problems were taken in part from [51, p76].)

1. Show that to determine the optimal input trajectory u(t) given the one dimensional system,

ẋ = x + u, x(0) = 1
9.4. LINEAR QUADRATIC CONTROL 417

such that Z 1
1

J (u) = x2 + u2 dt
2 0

is minimised reduces to solving the two point boundary value problem,

     
ẋ 1 −1 x x(0) = 1
= ,
λ̇ −1 −1 λ λ(1) = 0

Can you find an analytical solution for this problem?

2. Construct the two point boundary problem as above, (but do not solve it) for the two state,
one input system      
0 1 0 1
ẋ = x+ u, x(0) =
0 0 1 0
R 1

with the cost functional J (u) = 12 0 x⊤ x + u2 dt

9.4 L INEAR QUADRATIC CONTROL

The general optimal controller from section 9.3 has some nice properties, not the least that the
trajectory is optimal. However there are some obvious disadvantages, some of which are sub-
stantial. Quite apart from the sheer complexity of the problem formulation and the subsequent
numerical solution of the two-point boundary value problem, there is the fact that the final re-
sult is a manipulated variable trajectory which is essentially an open loop controller. This is fine
provided we always start from the same conditions, and follow down the prescribed path, but
what happens if due to some unexpected disturbance we fall off this prescribed path? Do we
still follow the pre-computed trajectory for u(t), or must we re-compute a new optimal profile for
u(t)?

Ideally we would like our optimal controller to have some element of feedback, which after all is a
characteristic of all our other practical controllers. Furthermore if we restrict our attention to just
linear plant models, and use just a quadratic performance objective, then the subsequent prob-
lem setup, and implementation, is made considerably easier. These types of optimal controllers
designed to control linear plants with quadratic performance are known as Linear Quadratic
Regulators, or LQR.

9.4.1 C ONTINUOUS LINEAR QUADRATIC REGULATORS

The linear quadratic control problem is to choose a manipulated variable trajectory u(t), that
minimises a combination of termination error and the integral of the squared state and manipu-
lated variable error over a time horizon from t = 0 to some specified (although possibly infinite)
final time t = T ,
Z
1 1 T T

J = x(T )Sx(T ) + x Qx + uT Ru dt (9.26)
2 2 0
given the constraint of linear plant dynamics

ẋ = Ax + Bu, x(t = 0) = x0 (9.27)

As we assume linear plant dynamics, and the performance objective, Eqn. 9.26, is of a quadratic
form, this type of controller is known as a continuous Linear Quadratic Regulator or LQR.
418 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

The weighting matrices Q and S are positive semi-definite (and typically diagonal) matrices,
while R must be a strictly positive definite matrix. Note that any real positive diagonal matrix
is also positive definite. The expressions xT Qx and uT Ru are called quadratic forms and are
2 2
sometimes compactly written as ||x||Q or ||u||R .

Following the terminology introduced in the general optimisation criteria in Eqn. 9.8 we have

ẋ = Ax + Bu, x(t = 0) = x0 (9.28)

1 ⊤ 1 ⊤
L(x, u, t) = x Q x + u R u (9.29)
2 2
1 ⊤
φ = x (T ) S x(T ) (9.30)
2
for the state dynamics constraint, the ‘getting there’ cost or energy term, and the final cost term.

For many applications, there is no explicit termination cost, since all the cost of state deviations
is accounted for in the energy term, thus often simply the matrix S = 0. In addition, the upper
limit of the time horizon is often set to T = ∞, in which case we are obviously now no longer
concerned with the termination cost term.

Now applying the Euler-Lagrange equations (Eqns 9.13 and 9.14) gives1
 
1 ⊤ ⊤
dλ ∂H ∂ 2 x Qx + λ Ax
=− = (9.33)
dt ∂x ∂x
⊤
= −Qx − A λ (9.34)

Similarly using Eqn. 9.14 gives

∂H
= 0 = Ru + B⊤ λ (9.35)
∂u
which implies that the optimal control law is

u = −R−1 B⊤ λ (9.36)

Eqn. 9.36 defines the optimal u as a function of the co-states, λ. Note that now our problem has
2n unknown “states”, the original n x states and the n introduced co-states, λ. We can write this
in a compact form using Eqns 9.36, 9.28 and 9.34
" #
−BR−1 B⊤
  
ẋ A x
= (9.37)
λ̇ −Q −A⊤ λ

with mixed boundary conditions

x(0) = x0 , and λ(T ) = Sx

From Lyapunov’s theorem, we know that the costates are related to the states via

λ(t) = P(t)x(t) (9.38)

1 Note that Ogata, [150, Appendix A] has a very good summary of matrix operations including differentiation. Espe-

cially note the following:

∂Ax
= A⊤ (9.31)
∂x
∂x⊤ Ax
= 2Ax if A is symmetrical (9.32)
∂x
9.4. LINEAR QUADRATIC CONTROL 419

where P(t) is a time varying matrix. Differentiating Eqn. 9.38 using the product rule gives

λ̇ = Ṗx + Pẋ (9.39)

Now we can substitute the control law based on the costates (Eqn 9.36) into the original linear
dynamic model (Eqn 9.28) and further substitute the relation for the states from the costates
(Eqn. 9.38) giving

ẋ = Ax + Bu (9.40)
 
= Ax + B −R−1 B⊤ λ (9.41)
−1 ⊤
= Ax − BR B Px (9.42)

Now substituting Eqn. 9.42 into Eqn. 9.39 and equating with the Euler-Lagrange equation Eqn. 9.34
gives  
λ̇ = −Qx − A⊤ λ = Ṗx + P Ax − BR−1 B⊤ Px (9.43)

Rearranging, and cancelling x gives

Ṗ = −PA − A⊤ P + PBR−1 B⊤ P − Q (9.44)

which is called the differential matrix Riccati equation. It is a matrix differential equation where
the boundary requirement or termination value, from Eqn. 9.13 is

P(t = T ) = S (9.45)

and which can be solved backwards in time using the original dynamic model matrices A, B,
and the controller design matrices Q, R. While we need to know the model and design matrices
for this solution, we need not know the initial condition for the states. This is fortunate, since
it means that we can solve the matrix Riccati equation offline once, store the series of P(t), and
apply this solution to our problem irrespective of initial condition. We only need to repeat this
computation if the plant model or design criteria change.

In summary, we now have developed an optimal state feedback controller of the proportional
form
u = −K(t)x (9.46)
where the time varying matrix feedback gain is (from Eqn 9.36) defined as

K(t) = R−1 B⊤ P(t) (9.47)

where the time-varying P matrix is given by Eqn. 9.44 ending with Eqn. 9.45. Since we have
assumed that the setpoint is at the origin in Eqn. 9.46, we have a regulatory problem, rather than
a servo or tracking problem. Servo controllers are discussed in §9.4.6.

9.4.2 A NALYTICAL SOLUTION TO THE LQR PROBLEM

The problem with implementing the full time-varying optimal controller is that we must solve the
nonlinear differential matrix Riccati equation, Eqn. 9.44. This is a complicated computational task
and furthermore we must store offline the resultant gain matrices at various times throughout the
interval of interest. However there is an analytical solution to this problem which is described
in [62, §2.4] and [165, §5.2]. The downside is that it requires us to compute matrix exponentials,
and if we want to know P(t) as opposed to just the costates, we must do this computation every
sample time.
420 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

We start by solving for both states and costates in Eqn. 9.37, repeated here
    
ẋ A −BR−1 B⊤ x x(0) = known
= , (9.48)
λ̇ −Q −A⊤ λ λ(T ) = known
| {z }
H
where the matrix H is known as the Hamiltonian matrix. This approach has the attraction that
it is a linear, homogeneous differential equation with an analytical solution. Starting from any
known point at t = 0, the states and co-states at any future time are given by
   
x(t) x(0)
= eHt (9.49)
λ(t) λ(0)

However the outstanding problem is that we do not know λ(0) so we cannot get started!
def
If we define eHt = Φ(t), then we can re-write Eqn. 9.49 as
   
x(t) x(0)
= Φ(t) (9.50)
λ(t) λ(0)
and remember that with M ATLAB it is straight forward to compute Φ using matrix exponentials
since it is a function of known, constant matrices. We then partition the matrix Φ into four equal
sized n × n blocks as  
Φ11 (t) Φ12 (t)
Φ(t) =
Φ21 (t) Φ22 (t)
which means we can express Eqn. 9.50 as

x(t) = Φ11 (t)x(0) + Φ12 (t)λ(0) (9.51)

λ(t) = Φ21 (t)x(0) + Φ22 (t)λ(0) (9.52)

Employing the Riccati transformation, λ(t) = P(t)x(t), changes Eqn. 9.52 to

P(t)x(t) = Φ21 (t)x(0) + Φ22 (t)λ(0)

and multiplying Eqn. 9.51 by P(t) and equating to the above gives

P(t)Φ11 (t)x(0) + P(t)Φ12 (t)λ(0) = Φ21 (t)x(0) + Φ22 (t)λ(0)

Solving for the mystery λ(0) gives

−1
λ(0) = [Φ22 (t) − P(t)Φ12 (t)] [P(t)Φ11 (t) − Φ21 (t)] x(0)

Now at the final point, t = T , we know that P(T ) = S from the transversality conditions, so λ(0)
is given by
λ(0) = [Φ22 (T ) − SΦ12 (T )]−1 [SΦ11 (T ) − Φ21 (T )] x(0) (9.53)
which is something that we can compute since all the terms are known.

Now having established λ(0), we can use Eqn. 9.49 to compute analytically both states, x(t), and
costates, λ(t), at any time t. Consequently, we can compute P(t) at any intermediate time be-
tween 0 and T using

P(t) = [Φ22 (T − t) − SΦ12 (T − t)]−1 [SΦ11 (T − t) − Φ21 (T − t)] (9.54)

In many applications, it suffices to ignore the costates and just consider P(0) which is given
directly by
−1
P(0) = [Φ22 (T ) − SΦ12 (T )] [SΦ11 (T ) − Φ21 (T )] (9.55)
We will investigate further this important simplification in section 9.4.3.
9.4. LINEAR QUADRATIC CONTROL 421

A N LQR TIME - VARYING EXAMPLE

Suppose we wish to control the continuous plant

     
−2 −4 4 −2
ẋ = x+ u, x(0) =
3 −1 6 1

using an optimal controller of the form u = −K(t)x to drive the process from some specified
initial condition to zero. For this application, we decide that the manipulated variable deviations
are more costly than the state deviations,
   
1 0 0 0
Q= , R = 2, S=
0 1 0 0

Listing 9.6 first defines the plant, then solves for the initial costates, λ(0), using Eqn. 9.53, and
then finally simulates the augmented state and costate system given by Eqn. 9.48.

Listing 9.6: Computes the full time-evolving LQR solution

1 A = [-2 -4; 3 -1]; B=[4,6]'; x0 = [-2,1]'; % Plant: ẋ = Ax+Bu & initial state, x(t = 0)
n=length(A); % Problem dimension
Q = eye(2); R=2; S=zeros(size(A)); % Design criteria: Q, R and S
T = 0.5; % final time, T

6 H = [A, -B/R*B'; -Q, -A']; % Create Hamiltonian using Eqn. 9.48

% Φ(t) = eHt
def
Phi = expm(H*T);

Phi11 = Phi(1:n,1:n); Phi12 = Phi(1:n,n+1:end); % Construct Φ11 (t), Φ12 (t), . . .

Phi21 = Phi(n+1:end,1:n); Phi22 = Phi(n+1:end,n+1:end);
11 Lam_0 = (Phi22 - S*Phi12)\(S*Phi11 - Phi21)*x0; % costate initial condition, Eqn. 9.53

z0 = [x0;Lam_0]; % augment states & costates

G = ss(H,zeros(2*n,1),eye(2*n),0);
initial(G,z0,T) % simulate it

The results of this simulation are given in Fig. 9.14 which also compares the near identical results
from a slightly sub-optimal controller which uses simply the steady state solution to the Riccati
equation discussed next in section 9.4.3. Note that the performance drop when using the consid-
erably simpler steady-state controller is small, and is only noticeable when the total time horizon
is relatively short.

An alternative solution procedure to the analytical solution given in Listing 9.6 is to numerically
integrate the continuous time Riccati equation, Eqn. 9.44, backwards using say the workhorse
ode45 with ‘initial’ condition Eqn. 9.45. The trick that the function in Listing 9.7 does is to
collapse the matrix Eqn. 9.44 to a vector form that the ODE function can then integrate. The
negative sign on the penultimate line is because we are in fact integrating backwards.

Listing 9.7: The continuous time differential Riccati equation. This routine is called from List-
ing 9.8.
function dPdt = mRiccati(t, P, A, B, Q)
% Solve the continuous matrix differential Riccati equation
% Solves dP/dt = −AT P − PA + PBBT P − Q

5 P = reshape(P, size(A)); % Convert from (n2 × 1) vector to n × n matrix

422 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Time−evolving & steady−state: T =2 Time−evolving & steady−state: T =0.5

final final
2 2

0 0
states

states
−2 −2

−4 −4

0.5 0.5

0 0
u

u
u(t) u(t)
−0.5 u∞ −0.5 u∞

1 1

0.5 0.5
K

0 0

0.5 0.5

0 0
P

−0.5 −0.5

0.5 0.5

0 0
λ

−0.5 −0.5
0 0.5 1 1.5 2 0 0.1 0.2 0.3 0.4 0.5
time [s] time [s]

(a) A long time horizon, T = 2. (b) A shorter time horizon, T = 0.5.

Figure 9.14: Comparing the full time-varying LQR control with a steady-state sub-optimal con-
troller. The upper two plots trend the two states and one input for the steady-state case and for
the time varying case. The lower three plots trend the time-varying elements of the matrices
K(t), P(t), and the co-state vector λ(t) for the full time-varying optimal case.

dPdt = -A.'*P - P*A + P*B*B.'*P - Q; % Eqn. 9.44

dPdt = -dPdt(:); % Convert back from n × n to (n2 × 1)
return

We call the function containing the matrix differential equation in Listing 9.7 via the ode45 in-
tegrator as shown in Listing 9.8. Note that we can express BR−1 B efficiently as ZZT where
Z = B(chol(R)−1 ).

Listing 9.8: Solves the continuous time differential Riccati equation using a numerical ODE inte-
grator.
A = [-2 -4; 3 -1]; B=[4,6]'; % Plant dynamics of interest
2 Q = eye(2); R=2; S=zeros(size(A)); % Weighting matrices, Q, R
T = 1; % Final time for the finite time horizon

P0 = S; % “Initial” condition for P

9.4. LINEAR QUADRATIC CONTROL 423

[t,P] = ode45(@(t,P) mRiccati(t,P,A,B/(chol(R)),Q),[0 T],P0(:)); % Call listing 9.7

7 P = flipud(P); % reverse time order of P(t)

Now we have generated a trajectory for the four elements of the P(t) matrix as a function of time.
We could then pre-compute the time varying gains, K(t), and then store them ready to be used
in our closed loop simulation.

Compared to the analytical solution, this numerical integration scheme requires that we integrate
n2 equations as opposed to 2n when we combine the co-states. Furthermore we must store the
trajectory of pre-computed time-varying gains. In practice of course, this is storage requirement
rapidly becomes unwieldy and, as we shall see in the next section, rarely worth the effort.

9.4.3 T HE STEADY- STATE SOLUTION TO THE MATRIX R ICCATI EQUATION

In the optimal controller scheme presented above, the gain K(t), is a continuously time varying
matrix, as opposed to a more convenient constant matrix. A time varying controller gain requires
excessive computer resources even if we did discretise, since we need to store an (m × n) gain
matrix at every sample time in the control computer over the horizon of interest.

We can make an approximation to our already approximate optimal controller by using only the
steady-state gain derived from the steady-state solution of the matrix differential Riccati equation
when Ṗ = 0. With this approximation, the differential Eqn. 9.44 is simplified to the algebraic
equation,
0 = A⊤ P∞ + P∞ A − P∞ BR−1 B⊤ P∞ + Q (9.56)
where the steady-state solution, P∞ , is now a constant gain matrix which is considerably more
convenient to implement. The steady-state controller gain is still obtained from Eqn. 9.47,

K∞ = R−1 B⊤ P∞ (9.57)

and is also a constant gain matrix.

Whether the performance deterioration due to this approximation is significant depends on the
plant, and on the time horizon of interest. However as evident in Fig. 9.14, the differences in
performance are negligible for all but the shortest time horizons. Fig. 9.14(a) shows that even
for this relatively brief optimisation horizon, the gains are essentially constant for more than two
thirds of the horizon, and this feature is generally true for all plants, and optimisation scenarios
provided the horizon is reasonably long.

We can solve for the elements of the matrix P∞ in Eqn. 9.56 using one of several strategies. One
direct way is to by equating the individual elements in each of the matrices and thus reducing the
single matrix equation to a set of algebraic equations. However unlike the Lyapunov equation,
Eqn. 2.96, which we could collapse to a system of linear equations using vectorisation and Kro-
necker products as described in section 2.9.5, this time the steady-state Riccati equation includes
a quadratic term of the form P∞ BP∞ which means our resultant system of equations will not be
linear. However an iterative scheme based on this idea is given on page 425.

Another strategy is to compute the analytical expression for P(T ), Eqn. 9.55, with a large time
horizon T . Unfortunately Eqn. 9.55 is numerically very ill-conditioned when T is large, let alone
∞, and it is difficult to know a priori exactly what time horizon is sufficiently ‘large’. A final strat-
egy given later in Listing 9.12 is only applicable for the discrete version of the optimal controller.

The control toolbox in M ATLAB provides a reliable algebraic Riccati equation routine, care, that
424 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

solves Eqn. 9.56 for P∞ in a numerically reliable manner2 , and the ric routine verifies the com-
puted P from care (or equivalent) by calculating the residual of Eqn. 9.56. The linear quadratic
regulator lqr routine finds the controller gain directly which is slightly more convenient than
the two-step procedure solving the algebraic Riccati equation explicitly. All three routines are
demonstrated in Listing 9.9.

A STEADY- STATE CONTINUOUS LQR EXAMPLE

Simulating a steady-state optimal state-feedback controller is trivial in M ATLAB. We can try a

steady-state version of the optimal controller for the example given previously on page 421. A
script to solve the controller design problem using the toolbox functions are, care or lqr, and
to simulate the results in M ATLAB is given in Listings 9.9 and 9.10.

Listing 9.9: Calculate the continuous optimal steady-state controller gain.

>>A = [-2, -4; 3 -1]; B = [4; 6]; % Plant under consideration
>>Q = eye(2); R = 2; % Controller weighting matrices, Q, R.
3

>>P = are(A, B/R*B', Q) % Solve for X such that AT X + XA − XBX + Q = 0

P =
0.1627 -0.0469
-0.0469 0.2514
8 >>K = R\B'*P % Now compute gain from Eqn. 9.57, K∞ = R−1 B⊤ P∞ .
K =
0.1847 0.6603

>>[Kerr,Perr] = ric(A,B,Q,R,K,P) % Check results

13 Kerr =
0 0
Perr =
1.0e-014 *
0.0999 0.2720
18 0.1471 0.1887 % Residuals should ≈ 0 which they do

>> [P2,L,G]=care(A,B/chol(R),Q);% Solve for X such that AT X + XA − XBBT X + Q = 0

>> K2 = chol(R)\G; % Compute controller gain given that G = BT X

23

>>[K2,P2,E] = lqr(A,B,Q,R) % Gain from lqr should ≈ gain from care.

>> K2
K2 =
0.1847 0.6603

This completes the controller design problem, the rest is just a simple closed-loop simulation as
shown in Listing 9.10.

Listing 9.10: Closed loop simulation using an optimal steady-state controller gain.
C = eye(size(A)); D = zeros(2,1);
Gcl = ss(A-B*K,zeros(size(B)),C,D); % Hopefully stable optimal closed loop
3

T = 1; x0 = [-2,1]'; % final time & initial state, x0

[Y,t,X]=initial(Gcl,x0,T);
2 An older toolbox routine, are.m, essentially also does this calculation.
9.4. LINEAR QUADRATIC CONTROL 425

U = -X*K'; % back-calculate input, u(t), for plotting

8 subplot(2,1,1);plot(t,X); ylabel('states'); % Refer Fig. 9.15.

subplot(2,1,2);plot(t,U); ylabel('input'); xlabel('time (s)')

The state and input trajectories of this optimal controller given a non-zero initial condition are
given in the Fig. 9.15.

Continuous steady−state LQR

1

0
states

−1

−2

−3
0 0.2 0.4 0.6 0.8 1

0.4

0.2
input

−0.2

−0.4 Figure 9.15: A steady-state continuous LQR

0 0.2 0.4 0.6 0.8 1
time (s) controller.

The difference between the fully optimal time varying case and the simplification when using
the steady-state solution to the Riccati equation is given in Fig. 9.14 which uses the same example
from page 424. For short optimisation intervals there is a small difference between the steady-
state solution and the full evolving optimal solution as shown in Fig. 9.14(b). However as the
final time increases, there is no discernible difference in the trajectories of the states nor in the
input in Fig. 9.14(a). This goes along way to explain why control engineers hardly ever bother
with the full optimal solution. The lower two plots in Fig. 9.14 trend the time varying P(t) and
K(t) matrices.

S OLVING THE ALGEBRAIC R ICCATI EQUATION USING K RONECKER PRODUCTS

Notwithstanding that the algebraic Riccati equation of Eqn. 9.56 is nonlinear, we can devise an
iterative scheme to solve for P∞ in a manner similar to that for the Lyapunov equation using
vectorisation and Kronecker products described in section 2.9.5.

Collapsing Eqn. 9.56 to vectorised form gives

 
− I ⊗ AT + AT ⊗ I − I ⊗ P∞ BR−1 BT vec(P∞ ) = vec(Q) (9.58)

which we now must solve iteratively due to the unknown P∞ in the coefficient matrix in Eqn. 9.58.
Listing 9.11 uses this solution strategy which works (most of the time) for modest sized problems,
but is not numerically competitive with, nor as reliable as, the ARE scheme in M ATLAB.

Listing 9.11: Solving the algebraic Riccati equation for P∞ using Kronecker products and vec-
torisation given matrices A, B, Q and R.
1 n = size(A,1); I = eye(size(A)); % Identity, I
BRB = B/R*B'; % Constant matrix, BR−1 BT
D = kron(I,A') + kron(A',I); % vec(AT P + PA) = (I ⊗ AT + AT ⊗ I)vec(P)
426 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

P = I; % Start with P∞ = I
6 tol = 1e-8; dP = tol*1e2; % Suggested tolerance, ≈ 10−5
while norm(dP) > tol
Pnew = reshape(-(D-kron(I,P*BRB))\Q(:),n,n); % Solve Eqn. 9.58 iteratively for P∞
dP = Pnew - P;
P = Pnew;
11 end

In a similar manner to solving the Lyapunov equation using fsolve on page 81, we could also
construct the algebraic Riccati matrix equation to be solved and submit that to the general non-
linear algebraic equation solver fsolve. This works in so far as it delivers a solution, but the
solution may not be a stabilising solution, or in other words while the matrix P∞ does satisfy
the original matrix equation, Eqn. 9.56, it may not be symmetric, or lead to a stable closed loop
system. This is due to the fact that the matrix Riccati equation has multiple solutions and general
purpose numerical schemes such as fsolve will return the first solution it comes across as op-
pose to the stabilising one that we are interested in. For this reason it is better to use the dedicated
Riccati solving routines than the general purpose fsolve.

9.4.4 T HE DISCRETE LQR

In the discrete domain, the optimisation problem analogous to Eqn. 9.26 is to search for the N
future optimal control moves, u0 , u1 , · · · , uN −1 to minimise
N −1
1 1 X T

J = xN SxN + xk Qxk + uTk Ruk (9.59)
2 2
k=0

where the continuous plant dynamics, Eqn 2.37, are now replaced by the discrete equivalent
xk+1 = Φxk + ∆uk (9.60)
and the state feedback control law, Eqn. 9.46, is as before. Now the discrete matrix difference
Riccati equation analogous to Eqn. 9.44 is
 −1
Pk = Q + Φ⊤ Pk+1 Φ − Φ⊤ Pk+1 ∆ R + ∆⊤ Pk+1 ∆ ∆⊤ Pk+1 Φ (9.61)

with final boundary condition

PN = S (9.62)
and the feedback gain K at time t = kT is
 −1
Kk = R + ∆⊤ Pk+1 ∆ ∆⊤ Pk+1 Φ (9.63)

An alternative form to Eqn. 9.63 is

Kk = R−1 ∆T Φ−T (Pk − Q) (9.64)
which uses the current Pk rather than the future version, but has the disadvantage that Φ must
be invertible. Note that the optimal feedback gain K, is now an (m × n) matrix rather than a row
vector as in the SIMO pole placement case described in chapter 8.

It is possible to compute the value of the performance index J , of Eqn. 9.59 in terms of the initial
states and the initial P0 matrix,
1
J = xT0 P0 x0 (9.65)
2
although this precise value of J has probably little practical worth in most industrial control
applications.
9.4. LINEAR QUADRATIC CONTROL 427

U SING THE SLIGHTLY SUB - OPTIMAL DISCRETE STEADY- STATE GAIN

Clearly the discrete optimal gain given by Eqn. 9.63 is again time varying as it was in Eqn. 9.47 for
the continuous case, but this can be computed offline since it is only dependent on the constant
plant matrices Φ and ∆, the weighting matrices Q and R, and the time varying Pk .

However, once again as in the continuous case given in §9.4.3, we can use the considerably sim-
pler steady-state approximation to Eqn. 9.61 with little reduction in optimality. We solve for the
steady state covariance,
 −1
P∞ = Q + Φ⊤ P∞ Φ − Φ⊤ P∞ ∆ R + ∆⊤ P∞ ∆ ∆⊤ P∞ Φ (9.66)

by starting with P∞ = 0 and stepping, (rather than integrating), backwards in time until Pk
approximately equals Pk+1 . It follows that the resultant K given by Eqn. 9.63 will now also be
constant. This strategy is implemented in the listing 9.12 which is functionally equivalent, but
not as reliable to the lqr routine in the control toolbox.

Listing 9.12: Calculate the discrete optimal steady-state gain by ‘iterating until exhaustion’. Note
it is preferable for numerical reasons to use lqr for this computation.
function [K,P] = dlqr_ss(Phi,Delta,Q,R)
% Steady-state discrete LQR by unwinding the Riccati difference eqn, Eqn. 9.61.
% Routine not as reliable as dlqr.m
4 P = zeros(size(Phi)); % Start with P = 0 at t = ∞
tol = 1e-5; % default tolerance

P0 = P+tol*1e3;
while norm(P0-P,'fro') > tol
9 P0=P;
P = Q + Phi'*P*Phi - Phi'*P*Delta*((Delta'*P*Delta+R)\Delta')*P*Phi;
end % while

−1 T
K = (Delta'*P*Delta + R)\Delta'*P*Phi; % K∞ = R + ∆T P∞ ∆ ∆ P∞ Φ
end % end function dlqr_ss

However, the backward iteration calculation technique given in dlqr_ss in listing 9.12 is both
crude and computationally dangerous. For this reason it is advisable to use the functionally
equivalent, but more robust dlqr (or lqr) provided in the C ONTROL T OOLBOX.

Listing 9.13 compares a continuous LQR regulator for the system given on page 421 with a dis-
crete LQR sampled at a relatively coarse rate of Ts = 0.1s. Note that in both the discrete and
continuous cases we design the optimal gain using the same lqr function. M ATLAB can tell
which algorithm to use from the model characteristics.

Listing 9.13: Comparing the continuous and discrete LQR controllers.

A = [-2, -4; 3 -1]; B = [4; 6]; % Continuous plant, ẋ = Ax + Bu
2 C = eye(size(A)); D = zeros(2,1); G = ss(A,B,C,D)

Q = eye(2); R = 2; % Controller weighting matrices Q = I2 , R = 2

K = lqr(G,Q,R); % Optimal gain, K = [k1 , k2 ]
Gcl = ss(A-B*K,zeros(size(B)),C,D); % Continuous closed loop
7

T = 1; x0 = [-2,1]'; % final time & initial condition

[Y,t,X]=initial(Gcl,x0,T); U = -X*K'; % Simulate & back-calculate input
428 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

% Discrete version of the LQR controller

12 Ts = 0.1; % Rather coarse sampling time, Ts
Gd = c2d(ss(A,B,C,D),Ts); % Discrete plant, xk+1 = Φxk + Buk , Ts = 0.1

Kd = lqr(Gd,Q,R); % Design discrete LQR gain

Gcl_d = ss(Gd.A-Gd.b*Kd,zeros(size(B)),C,D,Ts);
17

[Yd,td,Xd]=initial(Gcl_d,x0,T); Ud = -Xd*Kd'; % Simulate & back-calculate input

subplot(2,1,1); plot(t,X,td,Xd,'o'); % Refer Fig. 9.16

subplot(2,1,2); plot(t,U,td,Ud);

0
states

−1

−2

−3
0 0.2 0.4 0.6 0.8 1

0.4

0.2
input

0 continuous input
Figure 9.16: Comparing discrete and
continuous LQR controllers. Note −0.2 discrete input, T =0.1
s
that even with a relatively coarse
−0.4
sample time, the controlled response 0 0.2 0.4 0.6 0.8 1
in both cases is practically identical. time (s)

Problem 9.2 1. Load in the Newell & Lee evaporator matrices (refer Appendix E.1) and design
an LQR with equal weighting on the Q and R matrices by typing Q = eye(A); R = eye(B);
and using the dlmr function with a sample time of 1.0. This feedback matrix should be the
same as the gain matrix given in [147, p55]3 . Comment on any differences in solution you
obtain.

2. Repeat the LQR design using the supplied dlqr function rather than the dlqr_ss func-
tion. Using dlqr however, the gain elements are now slightly different. Is this difference
important and how would you justify your answer?

3. Use either dlqr_ss or dlqr to generate feedback gains for the limiting cases when either
(or both) Q or R are zero. What difficulties do you encounter, and how do you solve them?

3 Newell & Lee define the control law in a slightly different way; u = Kx omitting the negative sign. This means that

the K on p55 should be the negative of that evaluated by M ATLAB.

9.4. LINEAR QUADRATIC CONTROL 429

C OMPUTING THE PERFORMANCE OF A STEADY- STATE DISCRETE LQR CONTROLLER IN M AT-

LAB

In the steady-state case, we are uninterested in the termination part of the performance index of
Eqn. 9.59 leaving just
∞
1X T

J = xk Qxk + uTk Ruk (9.67)
2
k=0
1 T
= x P∞ x0 (9.68)
2 0
reinforcing the fact that while the controller gain is not dependent on the initial condition, the
value of the performance index is.

Suppose we had a plant

     
0 0 1 2
xk+1 = xk + uk , starting from x0 =
−0.5 1 0 2

with controller design matrices Q = diag[1, 0.5], and R = 1, and we wanted to compute the
performance index J comparing the numerical summation given by Eqn. 9.67, and the analytical
strategy in Eqn. 9.68.

>> Phi = [0 0; -0.5, 1]; Del = [1 0]'; % Plant matrices Φ, ∆ and controller design Q, R
>> Q = diag([1 0.5]); r = 1;
>> [K,P,E] = dlqr(Phi,Del,Q,r) % Solve the Riccati equation for K∞ & P∞ .
4 K =
0.2207 -0.4414
P =
1.5664 -1.1328
-1.1328 2.7656

Now that we have the steady-state Riccati equation solution we can compute the performance
directly , J .

>> x0 = [2,2]'; % Initial condition x0

2 >> J = 0.5*x0'*P*x0 % Performance J = 12 xT0 P∞ x0
J =
4.1328

To obtain an independent assessment of this, we could ‘simulate until exhaustion’ and numeri-
cally compute Eqn. 9.67. Since we have simulated the closed loop, we need to ‘back-extract’ the
input by using the control law and the states.

1 Gcl = ss(Phi-Del*K,0*Del,eye(2),0, 1); % Form closed loop (Φ − ∆K)

[y,t,x] = initial(Gcl,x0); % Let M ATLAB establish a suitable on final time, k∞
u = -x*K'; % Back extract control input from closed loop simulation, u = −Kx

Now M ATLAB has returned a vector of input and states, and we need to compute P T the summa-
tion given in Eqn. 9.67. One elegant way to calculate expressions such as x Qx given the
states stacked in rows is to use the M ATLAB construction sum(sum(x.*x*Q’)). This has the
advantage that it can be expressed succinctly in one line of code and it avoids an explicit loop.
430 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

T
P

>> uRu = u'*u*r; % k uk Ruk , but only works for scalar r
1
xTk Qxk + uTk Ruk
P

2 >> J_numerical = 0.5*(sum(sum(x.*x*Q'))+uRu) % Performance 2 k
J_numerical =
4.1328

In this case the numerical value of J is very close to the analytical value.

VARYING THE Q AND R WEIGHTING MATRICES

The Q and R weighting matrices in the objective function Eqn. 9.26 reflect the relative importance
of the cost of state deviations with respect to manipulated variable deviations and are essentially
the tuning parameters for this control algorithm. If we increase the Q matrix, then the optimum
controller tries hard to minimise state deviations and is prepared to demand very energetic con-
trol moves to do it. Likewise, if we increase R, the weighting on the deviations from u, then
excessive manipulated variables moves are discouraged, so consequently the quality of the state
control deteriorates. If we are only really interested in output deviations, yT y, as opposed to
state deviations, then Q is often set equal to CT C. This follows simply from the output relation
y = Cx.

Fig. 9.17 and the listing below demonstrate the design and implementation of a discrete LQR
for a stable, interacting arbitrary plant. We will start from a non-zero initial condition and we
subject the system to a small external disturbance at sample time k = 20. In this listing, the state
deviations are considered more important than manipulated deviations.

1 % Create an simple, stable, interacting, 3*2 plant

Phi = [0.8 0.1 0.1; -0.1 0.7 0.2; 0 0.6 0.1];
Del = [1,2; -2,1; 0,-3]; C = [1,0,0]; D=[0,0];

t=[0:40]'; dist = 20; % sample vector & disturbance point

6 Q = eye(size(Phi)); [n,nb] = size(Del); R=eye(nb,nb); % tuning constants

K = dlqr(Phi,Del,100*Q,R); % Design LQR controller with Q = 100I, R = I

Gd = ss(Phi-Del*K,Del,C,D,1); % closed loop system
U = zeros(size([t,t]));U(dist,:) = [1,0.2]; % not much + disturbance
11 x0 = [-1,2,0]'; % given initial condition
[Y,T,X3] = lsim(Gd,U,t,x0); % do closed loop simulation
U3 = X3*K'; % extract control vector from x
stairs(t,[X3,U3]);
P T P T
16 % Compute performance indices: x Qx, & u Ru
perfx3 = sum(sum(X3.*(Q*X3')')), perfu3 = sum(sum(U3.*(R*U3')'))

Fig. 9.17 compares two responses. The left controlled response is when we consider state devia-
tions more important than manipulated variations, i.e. the weighting factor for the states is 100
times that for the manipulated variables. In the right plot, the opposite scenario is simulated.

Clearly in Fig. 9.17 Pthe left controlled response returns a relatively low ISE (integral squared
error)
P T on the states, xT Qx = 12.3, but at the cost of a high ISE on the manipulated variables,
u Ru = 1.57. Conversely the right controlled response shows the situation, where perhaps
owing to manipulated variable constraints, we achieve a much lower ISE on the manipulated
variables, but at a cost of doubling the state deviation penalty.
9.4. LINEAR QUADRATIC CONTROL 431

2 2

1 1
State Q=100

State Q=1
0 0

−1 Σx’Qx=12.3 −1 Σx’Qx=27.4

−2 −2
1 1

input R=100
0.5 0.5
input R=1

0
0
Σu’Ru=0.059
Σu’Ru=1.571 −0.5
−0.5
−1
−1
0 10 20 30 40 0 10 20 30 40
Sample number Sample number

(a) A discrete LQR where state deviations are con- (b) A discrete LQR where manipulated variable
sidered costly; q = 100, r = 1. deviations are considered costly; q = 1, r = 100.

Figure 9.17: Varying the state and manipulated weights of a discrete linear quadratic regulator
where there is a disturbance introduced at sample time k = 20. When state deviations are costly (q
is large compared to r), the states rapidly return to setpoint at the expense of a large manipulated
movement.

As Fig. 9.17 shows, the design of an optimal LQR requires the control designer to weigh the rel-
ative costs of state and manipulated deviations. By changing the ratio of these weights, different
controlled responses can be achieved.

As we did in §8.2, we can modify the simulation given above to account for non-zero setpoints,
x⋆ , by using the control law, u = K(x⋆ − x) rather than Eqn 9.46, so simulating the closed loop

xk+1 = (Φ − ∆K)xk + ∆Kx⋆k

thus enabling response to setpoint changes.

9.4.5 A NUMERICAL VALIDATION OF THE OPTIMALITY OF LQR

If the LQR is indeed an optimal controller, its performance must be superior than any arbitrarily
designed pole-placement controller such as those tested in section 8.2.1. Listing 9.14 designs an
optimal controller for the blackbox plant with equal state and input weights.

Listing 9.14: An LQR controller for the blackbox

>> G = ss(tf(0.98,conv([2 1],[1,1])),0.1) % Blackbox plant with sample time Ts = 0.1

3 >> [n,m] = size(G.b); % system dimensions


1 0
>> Q = eye(n); % state weight, Q =
0 1
>> R = eye(m); % Input weight, R = r = 1

>> K = dlqr(G.a,G.b,Q,R) % Compute optimal controller gain K.

8 K =
4.7373 -3.5942
432 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Now it would be interesting to compare this supposedly optimal controller with a pole-placement
controller with poles arbitrarily placed at say µT = [0.5 ± 0.5i].

Listing 9.15: Comparing an LQR controller from Listing 9.14 with a pole-placement controller
1 >> pc = [0.5-0.5i; 0.5+0.5i]; % Desired closed loop poles, µ = [0.5 ± 0.5i]
>> Kp = acker(G.a,G.b,pc); % Pole-placement controller gain
Kp =
13.6971 -5.7713

6 >> Gcl = G; Gcl.b = 0*G.b; Gcl.c = eye(n);

>> Gcl.a = G.a-G.b*K; % Closed loop system with LQR, (Φ − ∆K)
>> Gclpp = Gcl; Gclpp.a = G.a-G.b*Kp; % Closed loop system with pole-placement

>> x0 = [-1,2]'; % Simulate disturbance rejection

11 >> initial(Gcl,Gclpp,x0) % Refer Fig. 9.18.

The trajectory of the states for the two schemes are given in Fig. 9.18. However what is surprising
is that the pole-placement response looks superficially considerably better than the somewhat
sluggish, though supposedly optimal, LQR response. How could this be? Surely the optimally
designed controller should be better than any ad-hoc designed controller?

Comparing LQR & Pole−placement

2
pole−placement
0
1

−2
x

−4 LQR

−6
2

Figure 9.18: Comparing the state 0

trajectories for both pole-placement
2

−2
and LQR. Note that the optimal LQR
x

is considerably more sluggish to re- −4 dlqr

turn to the setpoint than the pole- Pole−placement λ =0.5±0.5i
placement controller. −6
0 0.5 1 1.5 2 2.5 3

What Fig. 9.18 has not shown us is the input trajectory. This is also important when compar-
ing controlled responses, although typically perhaps not quite as important as the state variable
trajectories.

We need to consider the input trajectory for at least two reasons. Excessive deviations in input
will almost certainly reach the input saturation constraints that are present in any real control
system. Secondly, excessive movement in the input can cause unreasonable wear in the actuators.
Finally in the case where the input magnitude is related to cost, say fuel cost in an engine, then
the larger the magnitude, the more expensive the operating cost of the corrective action will be.
Fig. 9.19 reproduces the same problem, but this time includes the input trajectory. In this instance,
the cost of the pole-placement controller is J = 1288, while for the LQR it is under half at 553.

Obviously different pole locations will give different performances, and it is up to us as control
designers to choose the most appropriate pole location for the particular control application at
hand. Using M ATLAB, it is easy to do an ‘exhaustive search’ over a reasonable domain to find the
best pole location.
9.4. LINEAR QUADRATIC CONTROL 433

Performance: LQR = 553, Pole−place = 1288

2
pole−placement
0
states

−2

−4 LQR

−6
10

0 Figure 9.19: Comparing pole-

placement and LQR, but this time
input

−10 including the input trajectories.

−20 dlqr Compared to Fig. 9.18, the input
Pole−placement λ =0.5±0.5i trajectory for the optimal LQR
−30 controller is considerably less active.
0 0.5 1 1.5 2 2.5 3

For real systems (that is systems with real coefficients), the desired poles must appear in conju-
gate pairs. That means that we only need to consider one half of the unit circle. Furthermore, we
should only consider the 1st quadrant, because in the 2nd quadrant, the response will be overly
oscillatory.

Naturally it goes without saying that in the discrete domain we must be inside the unit circle.

I have chosen a set of trial poles lying on a polar grid from 0 degrees round to a little past 90◦ ,
and from a magnitude of about 0.1 to 0.9. These locations are shown in Fig. 9.20.

1
0.5π/T
0.6π/T 0.4π/T
0.8 0.1
0.7π/T 0.3π/T
0.2
0.6 0.3
0.8π/T 0.4 0.2π/T
0.5
0.4 0.6
0.7
0.9π/T 0.8 0.1π/T
0.2 0.9
π/T
0
ℑ

π/T
−0.2
0.9π/T 0.1π/T
−0.4
0.8π/T 0.2π/T
−0.6
0.7π/T 0.3π/T
−0.8
0.6π/T 0.4π/T Figure 9.20: We will test a pole-placement
0.5π/T
−1
−1 −0.5 0 0.5 1 controller at each of the pole locations spec-
ℜ ified by a +. (See also Fig. 9.21 following.)

For each of these pole locations specified in Fig. 9.20, I will:

1. Design a pole placement controller, then

2. simulate the response for a reasonably long time which I assume will approximate infinity,
and then
P∞
3. compute the performance criteria, J = i=0 xT Qx + uT Ru.
434 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

As you can see from the resultant contour plot in Fig. 9.21, the performance of our previously
arbitrarily chosen pole location of µT = [0.5 ± 0.5i] is reasonable, but not great, and clearly
not optimal. From the plot, the best performance is located at a pole location of approximately
µ⋆ ≈ [0.75 ± 0.15i]T marked as a × in Fig. 9.21.

The question is: can we find this location without doing an exhaustive search, or indeed any type
of numerical optimisation search strategy?

11000
0.9 8000 43108102001010
5800 000 10500 10000
36
600
0.8000150 4100 9000
4100 3000

3000
0.7
021118000000

2100 81201 8000

100
0
0.6

15
00 7000

006
15

0
0.5

0
ℑ (λ)

1100 6000
0.6π/T 0.5π/T 0.4π/T 0.3π/T 0.2π/T 0.2 0.1
0.4
80.3 5000
0.400
0.3 0.5
603010500

2100

0.6
Figure 9.21: Trial pole-placement 0.7 0.1π/T 4000

800
0.2 0.8
performance. The performance
021118000000

3000

0.9 3000
when µT = [0.5 ± 0.5i] is approxi- 0.1
mately 1288, while the optimum per- 2000
80

0
0

formance is considerably lower at 1000

around 550 at pole location µ⋆ ≈ −0.1
0 0.2 0.4 0.6 0.8
[0.75 ± 0.15i]T . ℜ (λ)

When we look at the closed loop poles of the LQR controller,

Listing 9.16: Computing the closed loop poles from the optimal LQR controller from Listing 9.14.
>> [K,S,E] = dlqr(G.a,G.b,Q,R)
K =
4.7373 -3.5942 % Controller gain (as before).
4 S =
90.4087 -65.2391
-65.2391 50.4963
E =
0.7800 + 0.1664i % Optimum closed loop poles, µ⋆ .
9 0.7800 - 0.1664i

then we see that in fact the computed closed loop poles are intriguingly close to the optimum
found numerically in Fig. 9.21. It turns out that the LQR routine did indeed find the optimum
controller without going to the computational trouble of an exhaustive search. Just remember
of course, that this is not a proof of the optimality of the LQR, it is just a numerical validation.
Nonetheless, you should find it a convincing argument.

Note of course, if we change the dynamic system, or the controller design weights, the of course
the shape of the performance surface in Fig. 9.21, and the corresponding optimum will also
change.

Problem 9.3 1. Simulate the controlled response of the evaporator for about 500 time steps
(T = 0.5 time units) with the initial conditions
 ⊤
x0 = 1 −4.3 2.2
9.4. LINEAR QUADRATIC CONTROL 435

for the 4 cases:

(a) Q=R=I
(b) Q = 103 R = I
(c) 103 Q = R = I
(d) any other combination of your choice.
In each case, sketch the results of the input u, and the states x with time and calculate the
ISE for the states, manipulated variables and the weighted sum. What conclusions can you
deduce from the relative importance of Q and R?
2. Compare the performance of the full multivariable LQR controller with that of a controller
comprised of just the diagonal elements of the controller gain. What does this say about
the process interactions?

9.4.6 A N LQR WITH INTEGRAL STATES

The LQR developed in §9.4 is only a proportional controller so will suffer from the problem
of offset given setpoint changes as graphically demonstrated in the upper plot in Fig. 9.23. To
compensate however, the gains in the true multivariable LQR controller can be higher than in
multiple SISO controllers without causing instability, reducing, but not eliminating, the offset
problem. If the offset problem is still significant, one can introduce extra integral states, z defined
as the integral of the normal states Z
def
z= x dt (9.69)

and can be augmented to the original system as

      
ẋ A 0 x B
= + u (9.70)
ż I 0 z 0
and the feedback control law is now
 

x
u= Kp Ki (9.71)
z
where Kp are the gains for the proportional controller, and Ki are the gains for the integral
controller. Now the LQR is designed for the new system with n additional integral states in
the same manner as for the proportional state only case. In M ATLAB we can formulate the new
augmented system as

Ai = [A zeros(size(A)); eye(size(A)) zeros(size(A))]

Bi = [B; zeros(size(B))]

This modification improves the system type, (i.e. the number of poles at the origin), enabling our
controller to track inputs. [112, §10–14,p802] gives further details.

SISO SERVO INTEGRAL OUTPUT CONTROL

If we want to track setpoint changes (servo-control) for a SISO system, we need only to add one
extra integral state. The augmented system is
ẋ = Ax + Bu
ż = r − y
y = Cx
436 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

where x is the vector of original states, u, y and r are the scalar input, output and setpoint respec-
tively. We simply concatenate the new state to the old,
h i⊤
def
x̄ = ..
x1 x2 · · · xn . z

and the feedback gain row-vector in u = −K̄x̄ is

h i
K̄ = ..
k1 k2 · · · kn . kn+1

So the closed-loop system is

 
dx̄
0
= Ā − B̄K̄ x̄ + r
dt 1
where the augmented matrices are:
   
A 0 B
Ā = , B̄ =
−C 0 0
| {z } | {z }
(n+1)×(n+1) (n+1)×1

To implement, the augmented system must be controllable, which is a tougher requirement than
just ctrb(A, B). A block diagram of a pole-placement controller with a single integral state is
given in Fig. 9.22, after [112, p802]. The wide lines are vector paths.

integral state
R xn+1 ẋ x
✲ + ẋn+1✲ ✲−kn+1 ✲ + u✲ B ✲+ ✲ ✲ C ✲ y
R
r
✻− ✻− ✻

A ✛

K ✛
full state feedback

Figure 9.22: State feedback control system with an integral output state

We can compare the effect of adding an integral state to remove the offset given
   
4 3 2
ẋ = x+ u
−1 6 1

where we want to control x1 to follow reference r(t) using pole-placement. In other words,
y = [1, 0]x. I will place the closed-loop eigenvalues for the original system at
 
−6 + 2i
λ=
−6 − 2i

and with the extra integral state, I will add another pole at λ3 = −10. Fig. 9.23 compares the
response of both cases. Clearly the version with the integral state that tracks the setpoint with no
steady-state error is preferable.
9.4. LINEAR QUADRATIC CONTROL 437

Proportional control: λ = (−6 ± 2i)

2

−2

Integral states: λ = (−6 ± 2i, −10)

2

Figure 9.23: State feedback

0 with an integral state. Up-
per: response of original sys-
tem showing offset. Lower:
−2
0 5 10 15 improved response of the aug-
time [s] mented system with no offset.

S ERVO LQR CONTROL OF THE BLACK - BOX

A good example where poor trajectory-following con-

trol is advisable.

In experimental, rather than simulated, applications, since we do not measure the states directly,
we must use a plant model to produce state estimates based on the measured outputs. In the
experimental optimal regulator example shown in Fig. 9.24, we estimated online a simple 3 state
model using recursive least-squares after which we recomputed each sample time the optimal
steady-state gains. In this application the sample time was relatively long, T = 1.5 seconds, and
the RLS forgetting factor was λ = 0.995. No feedback control was used for the first 20 sample
times.

Problem 9.4 1. Design an integral state LQR for the Newell & Lee evaporator. Newell and
Lee suggest that the weightings on the integral states should be an order of magnitude less
than the weightings on the states themselves, hence the augmented Qi matrix is
 
Qi = diag 1 1 1 0.1 0.1 0.1
and the manipulated weighting matrix R, is as before.
2. Compare the feedback matrix from part 1 with that given in [147, p57]. Why are they
different? What are the eigenvalues of the closed loop response?
438 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

LQR of the blackbox ∆ t=1.50

0.6
output & setpoint

0.4

0.2

−0.2
0 50 100 150 200 250
dlqr servo: σ2= 1, σ2= 10
q r
0.5
Input

−0.5
0 50 100 150 200 250

RLS: λ=0.995
5

0
θ

−5
0 50 100 150 200 250

2
K1 and K2

−1
0 50 100 150 200 250
time (s)

Figure 9.24: An experimental application of LQR servo control of the black box with 10q = r.
Plots: (a) output and setpoint; (b) input; (c) model parameters θ; (d) controller gains.
9.5. ESTIMATION OF STATE VARIABLES 439

Disturbances & noise

❄
✲ actual noisy outputs
✲ Plant ✲ Meas

❄
✲ + ✲ corrected states
− ✲
inputs ✻ Estimator
✲
✲ Model ✲ g(x)
predicted
outputs
predicted states

inputs state feedback

✛ state ✛
control

Software

Figure 9.25: A state-based estimation and control scheme

9.5 E STIMATION OF STATE VARIABLES

Chapter 6 looked at various alternatives to identify the parameters of dynamic systems, given
possibly noisy input/output measured data. This section looks at ways to estimate not the pa-
rameters, (they are assumed known by now), but the states themselves, x, by only measuring the
inputs u, and the outputs y. This is a very powerful technique since if this estimation works, we
can use state feedback control schemes without actually going to the trouble of measuring the
states, but instead using only their estimates.

These estimators work by using a model to predict the output, ŷ, which can be compared to the
measured output, y. Based on this difference we can adjust, if necessary, the predicted states.
The estimation of states in this way is known, for historical reasons, as a Kalman filter. A block
scheme of how the estimator works is given in Fig. 9.25. Here we have a control scheme that
while demanding state-feedback, actually uses only state estimates rather than the actual hard-
to-obtain states.

There are many texts describing the development (eg: [50, 97]) and applications (eg: [37, 192]) of
the Kalman filter.

9.5.1 R ANDOM PROCESSES

The only reason that our estimation scheme will not work perfectly, at least in theory, is due to
the corrupting noise, and to a lesser degree, incorrect initial conditions. Now in practice, noise
440 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

is the garbage we get superimposed when we sample a real process, but when we run pure
simulations, we can approximate the noise using some sort of random number generator such as
described previously in §6.2. These are correctly termed pseudo random number generators, and
most of them are of the very simple linear congruent type, and are therefore deterministic and
not strictly random. Whole books have been written on this subject and many of them quote
John von Neumann saying in 1951 “Anyone who considers arithmetical methods of producing
random digits is, of course, in a state of sin.”

M ATLAB can generate two types of random numbers; namely uniform or rectangular, and normal
or Gaussian. Using the uniform scheme, we can generate any other arbitrary distribution we
may desire including Gaussian. However in most practical cases, it is the Gaussian distributed
random number we are more interested in. To visualise the difference, try:

1 x = rand(1000,1); hist(x) % [default] uniformly

y = randn(1000,1); hist(y) % normally distributed

Sometimes it is interesting to compare the actual probability density function or PDF obtained
using the hist command with the theoretical PDF. To directly compare these, we need to scale
the output of the histogram such that the integral of the PDF is 1. Fig. 9.26 compares the actual
(scaled) histogram from 104 samples of a random variable, N (µ = 12, σ = 2) with the theoretical
probability density function. The PDF of a normal variate is, [193, p60],
 2 !
1 1 x−µ
f (x) = √ exp − (9.72)
2πσ 2 2 σ

0.25

0.2

mean = 12.0116
frequency

0.15
σ2 = 4.03369
0.1

0.05

0
2 4 6 8 10 12 14 16 18 20
x

Figure 9.26: The probability density function of a normally distributed random variable x ∈
N (µ = 12, σ = 2); theoretical (solid) and experimental (histogram).

Listing 9.17: Comparing the actual normally distributed random numbers with the theoretical
probability density function.
xsig = 2; xmean = 12; % Variance, σ, & mean, x̄
x = xsig*randn(10000,1)+xmean; % sample (realisation) of r.v. x.
3 xv = linspace(min(x),max(x),100)'; % approx range of x
pdf= @(xv) 1/sqrt(2*pi)/xsig*exp(-0.5*(((xv-xmean)/xsig).ˆ2));% expected PDF, Eqn. 9.72.
area = trapz(xv,pdf(xv)); % Note integral of PDF should=1.0

[numx,xn] = hist(x,30); % Generate histogram [incidence #, x-value]

9.5. ESTIMATION OF STATE VARIABLES 441

8 dx = mean(diff(xn)); % assume evenly spaced

scale = 1/dx/sum(numx); % scale factor so total area = 1
freq = numx*scale; % normalise so total area =1
bar(xn,freq); hold on; plot(xv,pdf(xv));hold off % See Fig. 9.26

M ULTIVARIABLE RANDOM NUMBERS

For a single time series xk of n elements, the variance is calculated from the familiar formula
n
1X 2
Var(x) = σx2 = (xi − x) (9.73)
n i=1

where x is the mean of the vector of data. In the case of two time series of data (xk , yk ) the
variances of x and y are calculated in the normal fashion using Eqn. 9.73, but the cross-variance
or co-variance between x and y is calculated by
n
1X
Cov(x, y) = (xi − x) (yi − y) (9.74)
n i=1

Note that while the variance must be positive or zero, the co-variance can be both positive and
negative and that Cov(x, y) = Cov(y, x). For this two vector system, we could assemble all the
variances and cross-variances in a matrix P known as the variance-covariance matrix
 
σx2 Cov(x, y)
P=
Cov(y, x) σy2

which is of course symmetric with positive diagonals for time series of real numbers. In normal
usage, this name of this matrix is shortened to just the ‘covariance’ matrix.

For an n variable system, the co-variance matrix P will be a positive definite symmetric n × n
matrix. M ATLAB can calculate the standard deviation (σ) of a vector with the std command and
it can calculate the co-variance matrix of a multiple series using the cov command.

One characteristic about two independent random time series of data is that they should have no
cross-correlation, or in other words, the co-variance between x and y, should be approximately
zero. For this reason, most people assume that the co-variance matrices are simply positive diag-
onal matrices, and that the off-diagonal or cross-covariance terms are all zero. Suppose we create
three random time series vectors in M ATLAB (x, y, z).

x=randn(1000,1) +3; % generate 3 independent vectors

y=randn(1000,1)*6+1; % standard deviation, σ = 6
z=randn(1000,1)*2-2; % variance σ 2 = 22 = 4
4 a=[x y z];
cov(a) % find co-variance

The co-variance matrix of the three time series is in my case, (yours will be slightly different),
   
1.02 −0.11 0.10 1 0 0
Cov(x, y, z) =  −0.11 37.74 0.30  ≈  0 36 0 
0.10 0.30 4.22 0 0 4

which shows the dominant diagonal terms which are approximately the true variances (1, 36, 4)
and the small, almost zero, off-diagonal terms.
442 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

C OVARIANCE MATRICES AND ELLIPSES

In many cases, the random data does exhibit cross-correlation and we will see this clearly if the
x, y data pairs fall in a patten described by a rotated ellipse. We can compute the 95% percent
confidence limits of this ellipse using
T p(n − 1)
(x − µ) P−1 (x − µ) ≤ Fα,p,(n−p)
n−p
where p is the number of parameters (usually 2), and n is the number of data points (usually
> 100). Fα,p,(n−p) is the F -distribution at a confidence of α, in this case 95%. To compute values
from the F -distribution we could either use the functions given in Listing 9.18,

Listing 9.18: Probability and inverse probability distributions for the F -distribution.
pf = @(x,v1,v2) betainc(v1*x/(v1*x+v2),v1/2,v2/2);
qF = @(P,v1,v2) fsolve(@(x) pf(x,v1,v2)-P,max(v1-1,1))

or use equivalent routines from S TIX B OX toolbox from Holtsberg mentioned on page 3.

The easiest way to construct this ellipse is to first plot a circle parametrically, scale the x and y
axes, rotate these axes, then finally translate the ellipse so that the center is at the mean of the
data. The axes of the ellipse are given by the eigenvectors of P, and the lengths are given by the
square root of the eigenvalues of P.

Example of confidence limit plotting

Here we will generate 1000 random (x, y) samples with deliberate correlation and we will com-
pute the ellipse at a 95% confidence limit. Furthermore we can check this limit by using the
inpolygon routine to calculate the actual proportion of data points that lie inside the ellipse.
We expect around 95%.

1. First we generate some correlated random data,

x ∈ N (−0.3, 1) and y ∈ N (2, 1.1) + x
These data pairs are correlated because y is explicitly related to x.

Listing 9.19: Generate some correlated random data.

N = 1e3; % # of samples in Monte-Carlo
x = randn(N,1)-0.3;% N (−0.3, 1)
3 y = 1.1*randn(N,1)+x + 2; % Deliberately introduce some cross-covariance
plot(x,y,'.') % See Fig. 9.27(a).
9.5. ESTIMATION OF STATE VARIABLES 443

We can plot the (x, y) points and we can see a that the ‘buckshot’ pattern falls in a rotated
ellipse indicating correlation as expected as shown in Fig. 9.27(a).
2. The distribution of the data in Fig. 9.27(a) is more clearly shown in a density plot or a 3D
histogram. Unfortunately M ATLAB does not have a 3D histogram routine, but we can use
the normal 2D histogram in slices along the y-axis as demonstrated in the following code
snippet.

Listing 9.20: Plot a 3D histogram of the random data from Listing 9.19.
1 Nb = 30; % # of bins in a quick & dirty 3D histogram
xmin = min(x); ymin = min(y); xmax = max(x); ymax = max(y);
xv = linspace(xmin,xmax,Nb); yv = linspace(ymin,ymax,Nb);
[X,Y] = meshgrid(xv,yv); Z= 0*X;

6 for i=1:length(xv)-1
idx = find(x>xv(i)&x<xv(i+1));
Z(:,i) = hist(y(idx),yv)';
end % for
bar3(Z) % crude 3D histogram

This gives the density and 3D histogram plots given in Fig. 9.27(b) and (c).

x−y data plot Density plot

8 8 50

45
3D histogram
6 6
40
50
35
4 4
30

2 2 25
y

0
20
0 0
15

10
−2 −2
x y 5

−4 −4
−5 0 5 −5 0 5
x x

Figure 9.27: Correlated noisy x, y data. (a) Individual (x, y) data. (b) A density plot. (c) A
histogram in 2 independent variables.

3. To plot an ellipse, we plot first plot x = a cos(t), y = b sin(t) with t ∈ [0, 2π], rotate, and
finally then shift the origin to µ. The directions of the of the semi-major and semi-minor
axes of the ellipse are given by the eigenvectors of P and the lengths are given by the square
roots of the eigenvalues.
If V is the collection of eigenvectors vi of the (p × p) matrix P, then in the usual case of
p = 2,
def  
V = v1 v2
and the transformation to the un-rotated ellipse is

u = VT (x − µ)

This is shown in Listing 9.21 which calls the F -distribution using the anonymous functions
given previously in Listing 9.18.
444 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Listing 9.21: Compute the uncertainty regions from the random data from Listing 9.20.
X = cov(x,y); % sampled covariance matrix
[V,D] = eig(X); e = diag(D);

n = length(x); p = 2; % # of data points & # of parameters

5 alpha = 0.95; % confidence limit, α = 0.95
scale= p*(n-1)/(n-p) * qf(alpha,p,n-p); % Inverse F -distribution, See Listing 9.18.

t = linspace(0,2*pi,5e2)'; % parametric circle

uxc = sqrt(e(1)*scale)*cos(t); uyc = sqrt(e(2)*scale)*sin(t); % scaled circle
10

% Now rotate VT (x − x̄)

c = (inv(V')*[uxc, uyc]')'; % basically V−1 x
xc = c(:,1); yc = c(:,2);
xc = xc + mean(x); yc = yc + mean(y); % shift for mean

4. As a final check, we could use the inpolygon routine to see if the actual proportion of
points inside the ellipse matches the expected α%. The final data points and the 95% ellipse
is given in Fig. 9.28.

Listing 9.22: Validating the uncertainty regions computed theoretically from Listing 9.21.
1 % Now look at actual proportions
in = inpolygon(x,y,[xc;xc(1)],[yc;yc(1)]);
prop_inside = sum(in)/n;

plot(x(in),y(in),'r.','MarkerSize',1); hold on
6 plot(x(¬in),y(¬in),'b.','MarkerSize',1); plot(xc,yc,'k-');
hold off axis('equal'); grid

For my data, 94.6% of the 1000 data points lie inside the confidence limits which is in good
agreement to the expected 95%.

95% confidence limit

6

2
y

−1
Figure 9.28: Randomly generated data with
a 95% confidence limit superimposed. In −2
actual fact, 94.6% of the data points lie in- −3
side the confidence limits, which is a good −6 −4 −2 0 2 4
agreement. x
9.5. ESTIMATION OF STATE VARIABLES 445

9.5.2 C OMBINING DETERMINISTIC AND STOCHASTIC PROCESSES

If we have a perfect model of the process, no noise and we know the initial conditions x0 exactly,
then a perfect estimator will be simply the open loop predictions of x, given the system model
and present and past inputs u. However industrially one there is always noise present both in
the measurements and in the state dynamics, so this open loop prediction scheme will soon fail
even if the initial estimates and model are good. A more realistic description of an industrial
process is

xk+1 = Φxk + ∆uk + Γwk , wk ≡ N (0, Q) (9.75)

yk = Cxk + vk , vk ≡ N (0, R) (9.76)

where v, w are zero mean, white noise sequences with covariance matrices Q and R respectively.
Note how in Fig. 9.29, the two unobservable noise signals, v and w are wholly inside the dashed
box, as is the true states, xk coloured grey in this figure to highlight that it is essentially hidden.
We, as external observers, can only access those signals outside the dashed box, namely uk and
yk .

Plant noise, w

❄ Measurement noise, v
Γ

❄xk+1 xk ❄
uk ✲ ∆ ✲ + ✲ q −1 ✲ C ✲+ ✲ yk
✻

✛ Φ ✛

Figure 9.29: A block diagram of a stochastic state-space system

This model is still a discrete linear dynamic model, but the final terms are inserted to account for
all the non-deterministic components of the model.

M EAN AND VARIANCE PROPAGATION IN A DYNAMIC SYSTEM

Predicting the behaviour of the state mean and variance given a stochastic dynamic system
sounds quite complicated, but actually in the discrete domain it is simple. The mean of the
system, Eqn. 9.75 is the expected value,

x̄k+1 = E {Φxk + ∆uk + Γwk } (9.77)

= Φx̄k + ∆ūk + Γw̄k (9.78)

but since the input is deterministic, (ū ≡ u), and with no loss of generality the mean of the noise
is zero, (E(w) = 0), Eqn. 9.78 collapses to

x̄k+1 = Φx̄k + ∆uk (9.79)

446 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

which is simply the deterministic part of the full stochastic dynamic process, and also which
should be intuitive.

The propagation of the state variance, P is slightly more complex. The state variance at the next
sample time is, by definition,
def 
Pk+1 = E (xk+1 − x̄k+1 )(xk+1 − x̄k+1 )⊤ (9.80)

and if we substitute the new mean from Eqn. 9.79, the process, Eqn. 9.75 we get

Pk+1 = E (Φ(xk − x̄k ) + Γwk )(Φ(xk − x̄k ) + Γwk )⊤ (9.81)

which if we expand out, and note that the variance of w is Q, then we get the following matrix
equation

Pk+1 = ΦPk Φ⊤ + ΓPxk wk Φ⊤ + ΦPwk xk Γ⊤ +ΓQΓ⊤ (9.82)

| {z }
very small, ≈ 0
⊤ ⊤
= ΦPk Φ + ΓQΓ (9.83)

We can ignore the two inner cross-product terms in Eqn. 9.82) because of the uncorrelatedness of
states and the white noise w. In a similar manner to the above application of definition, expan-
sion and eliminating cross terms, one can find expressions for the propagation of the mean and
variance of the output variables, y. Further details are given in [122, pp62–64].

Eqn. 9.83 is called the discrete Algebraic Lyapunov difference Equation, or ALE. If the system is
stable, then the steady-state covariance, P∞ , will be given at sample k = ∞, or by solving the
following matrix equation,
P∞ = ΦP∞ Φ⊤ + ΓQΓ⊤ (9.84)
in exactly the same way we used in section 2.9.4 when establishing the stability of a linear
discrete-time system either using Kronecker products, or using the M ATLAB dlyap command.
Note that the final state covariance does not depend on the initial uncertainty, P0 .

The propagation of the mean and variance of a linear dynamic system, given stochastic distur-
bances, is fundamental to what will be developed in the following section which describes the
Kalman filter estimation scheme. The Kalman filter tries to reduce the variance over time which
in effect is to reduce the magnitude of the matrix P using a clever feedback scheme, thus obtain-
ing improved state estimates.

Problem 9.5 Verify by M ATLAB Monte-Carlo simulation that the steady-state covariance for a
stable linear system subjected to white noise input with known variance is given by the solution
to the ALE, Eqn. 9.84. Compare your simulated actual covariance (perhaps using cov) with the
predicted covariance (perhaps using dlyap).

9.5.3 T HE K ALMAN FILTER ESTIMATION SCHEME

The estimation scheme consists of two parts: a prediction and a correction. For the prediction
step, we will predict using the openloop model, the new state vector and the state covariance
matrix. The correction part will adjust the state prediction based on the current measurement,
and in so doing, hopefully reduce the estimated state covariance matrix.

We will denote the state vector as x and an estimate of this as x̂. The prediction of the state using
just the model is called the a priori estimate and denoted, x̂− . Once we have assimilated the
measurement information, we have the a posteriori estimate which is denoted simply as x̂.
9.5. ESTIMATION OF STATE VARIABLES 447

At sample time k, we have:

• The a priori estimate x̂−

k , which is the estimate of the true, but unknown, state vector xk
before we assimilate the current measurement, yk .
• We also assume we know the a priori covariance of x̂−
k,

− − T
P−
k = E{(xk − x̂k )(xk − x̂k ) }

Now we want to improve the estimate of xk by using the current measurement. We will do this
by the linear blending equation
−

x̂k = x̂−
k + Lk yk − Cx̂k (9.85)

where x̂k , is the corrected state estimate or a posteriori estimate, and Lk , the yet to be determined
blending factor or Kalman gain. We use the symbol L for the Kalman gain instead of the com-
monly used K because we have already used that symbol for the similar feedback gain for the
linear quadratic regulator.

The best value of L will be one that minimises the mean-square error of the a posteriori covariance,

Pk = E{(xk − x̂k )(xk − x̂k )T } (9.86)

and substituting in Eqn. 9.85, gives

 − 
Pk =E{ (xk − x̂− k ) − Lk (Cxk + vk − Cx̂k )
 − 
(xk − x̂−
k ) − Lk (Cxk + vk − Cx̂k ) }

Now noting that (xk − x̂−

k ) is uncorrelated with vk , then

T
Pk = (I − Lk C) P− T
k (I − Lk C) + Lk RLk (9.87)

is an expression for the updated covariance Pk in terms of the old covariance P−

k.

We want to find the particular Lk such that we minimise the terms along the diagonal of Pk .
Note that
d trace(AB)
= BT
dA
d trace(ACAT )
= 2AC
dA

So now expanding Pk from Eqn. 9.87 to explicitly show the terms in L gives
 
Pk = P− k − L k CP−
k − P− T T
C L k + Lk CP − T
C + R k LTk
| {z k } | k
{z }
linear in L
quadratic in L

We wish to minimise the trace of P since that is the sum of the diagonal elements which we want
to minimise. (Minimising the sum is the same as minimising each individually when positive.)

d trace(Pk )
=0
d Lk

T  T
= −2 CP−
k + 2Lk CP− T
k C + Rk
448 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Solving for the optimum gain gives

 −1
Lk = P−
k C T
CP− T
k C + Rk (9.88)

where the matrix Lk is called the Kalman gain.

Substituting Eqn. 9.88 into Eqn. 9.87 we have:

 −1
Pk = P−
k − Pk C
T
CP− T
k C + Rk CP−
k

or  −1
Pk = P−
k − L k CP− T
k C + R k Lk
or even
Pk = (I − Lk C) P−
k (9.89)
All expressions for computing the updated Pk from the prior P−
k are equivalent, but

1. All but Eqn. 9.87 are valid only for the optimum gain. (Eqn. 9.87 is valid for any gain.)

2. The last and simplest expression is the most used, but is suspectable to numerical ill-
conditioning.

Prediction updates

The updated estimate x̂−

k is obtained by using the discrete model

x̂−
k+1 = Φx̂k

Note that we can ignore wk since it has mean zero and is uncorrelated with any previous ws.

The a priori error is

−
e−
k+1 = xk+1 − x̂k+1
= (Φxk + wk ) − Φx̂k
= Φek + wk

and the error covariance associated with x̂−

k+1 is

P− − −T T
k+1 = E{ek ek } = E{(Φek + wk )(Φek + wk ) }

= ΦPk ΦT + Q

T HE PREDICTION

The best prediction we can make for the new state vector is by using the deterministic part of
Eqn. 9.75. Remember that Eqn 9.75 is our true process S(θ), and the deterministic part excludes
the noise term since we cannot predict it4 . We may also note that the expected value of the noise
term is zero, so if we were to add anything, it would be zero. Thus the prediction of the new state
estimate is at sample time kT
x̂−
k = Φx̂k−1|k−1 + ∆uk (9.90)
4 If we could predict it, then it would be part of the deterministic part!
9.5. ESTIMATION OF STATE VARIABLES 449

which is precisely Eqn. 9.79. In addition to predicting the states one sample time in the future,
we can also predict the statistics of those states. We can predict the co-variance matrix of the state
predictions Pk|k−1 , the a priori co-variance as

Pk|k−1 = ΦPk−1|k−1 Φ⊤ + ΓQΓ⊤ (9.91)

which again is Eqn. 9.83. Without any measurement information, the state estimates obtained
from Eqn 9.90 with variances from Eqn 9.91 are the best one can achieve. If our original dynamic
system is stable, (Φ is stable), then over time, the estimate uncertainty P will converge to a steady-
state value, and will not further deteriorate, despite the continual presence of process noise. This
is termed statistical steady-state. Conversely however, if our process is unstable, then in most
cases, the uncertainty of the states will increase without bound, and P will not converge.

Stable system or unstable, we can improve these estimates if we take measurements. This intro-
duces the correction step of the Kalman filter.

T HE CORRECTION

We first take a measurement from the real process to obtain yk . We can compare this actual
measurement with a predicted measurement to form an error vector ǫk known as the predicted
measurement residual error
ǫk = yk − ŷk = yk − Cx̂k|k−1 (9.92)
It would seem natural to correct our predicted state vector by a magnitude proportional to this
error

x̂k|k = x̂k|k−1 + Lk yk − Cx̂k|k−1 (9.93)
where Lk is called the Kalman gain matrix at time t = kT and is similar to the gain matrix in
the observers presented in chapter 8. The Kalman gain matrix Lk and co-variance of the state
variables after the correction Pk|k , the a posteriori co-variance, are
 −1
Lk = Pk|k−1 C⊤ CPk|k−1 C⊤ + Rk (9.94)
Pk|k = (I − Lk C) Pk|k−1 (9.95)

and note the similarity of these equations to Eqns 9.61 and 9.63. This predictor/corrector scheme
comprises the Kalman filter update.

I NITIALISING THE ALGORITHM

To start the estimation algorithm, we need estimates of the initial state, x̂0 , and covariance, P0|0 .
In practice, both these variables quickly converge to near optimal values, so the long-term per-
formance of the estimator is not overly sensitive to these parameters. As in the recursive least
squares estimator algorithms, it normally suffices to set P0|0 ≈ 106 I. If more outputs than states
are available (m > n), then it is possible to solve for the initial states (in a least squares sense) and
for the covariance given an initial measurement y0 ,
 −1
P0|0 = C⊤ R−1 C (9.96)
x̂0|0 = P0|0 C⊤ R−1 y0 (9.97)

The above two formulae can be derived (assuming no input) from Eqn 3.45 where the weight-
ing matrix W = R. However [76, p1539] argues that using the above equations for the initial
estimates is probably not worth the extra effort.
450 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Algorithm 9.2 Discrete Kalman filter algorithm

What we must know before we can implement an evolving Kalman filter:

• Model matrices Φ, ∆, C
• The covariance matrices Q, R of the noise sequences.
• Initial prior state estimate, x̂− −
0 and its error covariance, P0 .

At the kth iteration, do the following:

1. Compute Kalman gain, Lk , from a priori information,

 −1
Lk = P−
kC
T
CP− T
k C + Rk

2. Correction of the a priori state estimate and its error covariance using the current measure-
ment
−

x̂k = x̂−
k + Lk yk − Cx̂k
Pk = (I − Lk C) P−
k

3. Project ahead or prediction step using the model

x̂−
k+1 = Φx̂k

P− T
k+1 = ΦPk Φ + Q

4. Wait out the remainder of the sample time, increment counter, k ← k + 1, and then go back
to step 1.

Kalman filter example

Simulated results presented in Fig. 9.30 show the improvement of the estimated state when using
a Kalman filter with a randomly generated 2-state discrete model, and plant and measurement
noises with covariances
Q = 0.2I, R=I
Fig. 9.30 compares (a) an open loop model with (b) states computed by directly inverting the
measurement matrix, and (c) states obtained using the Kalman filter. The best practical scheme
is the one where the estimated states best approximates the true states.

To compare the performance quantitatively, the sum of the square errors, (sse), is computed for
each of the three alternatives. The Kalman filter, not surprisingly, delivers the best performance.

9.5.4 T HE STEADY- STATE FORM OF THE K ALMAN FILTER

Analogous to the optimal regulator gain given in Eqn. 9.47, the Kalman gain matrix Lk in Eqn. 9.93
is also time varying. However in practical instances, this matrix is essentially constant for most
of the duration of the control interval, so a constant matrix can be used with very little drop in
performance. This constant matrix is called the steady-state form of the Kalman gain matrix and
is equivalent to the steady-state LQR gain matrix used in §9.4.
9.5. ESTIMATION OF STATE VARIABLES 451

Open loop model: sse(ol) = 281 Measurements only: sse(y) = 201 State estimator: sse(KF) = 75
4 4 4
estimated states
2 2 2
1

0 0 0
x

−2 −2 −2
true states
−4 −4 −4

4 4 4

2 2 2
2

0 0 0
x

−2 −2 −2

−4 −4 −4
0 50 100 0 50 100 0 50 100

Figure 9.30: Comparing an openloop model prediction (left column), with direct measurement
inversion (middle column) and the Kalman filter (right column). The Kalman filter has the lowest
state error. See also the performance of the Kalman filter for a range of conditions in Fig. 9.34.

We can solve for the steady-state Kalman gain matrix by finding P such that
 −1
P = Q + ΦPΦ⊤ − ΦPC⊤ R + CPC⊤ CPΦ⊤ (9.98)

is satisfied. To solve Eqn. 9.98 for P, we could use an exhaustive iteration strategy where we
iterate until P converges, or we could notice that this equation is in the form of a discrete alge-
braic Riccati equation for which we can use the function dare. We should note that routine
dare solves a slightly different Riccati form than Eqn. 9.98, so follow the procedure given in
Listing 9.23.

Listing 9.23: Solving the discrete time Riccati equation using exhaustive iteration around
Eqn. 9.98 or alternatively using the dare routine.
P = eye(n); Pnew = 0; % Initial condition for P should be positive definite.
tol = 1e-6; i=0;
3 while norm(P-Pnew)>tol % Repeat until Pk+1 ≈ Pk .
P = Pnew; i=i+1;
Pnew = Q + A*P*A' - A*P*C'/(R+C*P*C')*C*P*A'; % Refer Eqn. 9.98.
end

8 % Now try using dare.m

−1
% Solve P = Q + ΦPΦ⊤ − ΦPC⊤ R + CPC⊤ CPΦ⊤
[Pdare,L,G] = dare(A',C',Q,R); % We are assuming E = I, S = 0

Once we have the steady-state matrix P, then the steady-state Kalman gain is given by Eqn. 9.94,
 −1
L = PC⊤ CPCT + R (9.99)

which is known as the prediction-form of the Kalman filter; see also Eqn. 9.102. Note also that I
have dropped the time subscripts on P and L since now they are both time invariant.

The M ATLAB routine dlqe (discrete linear quadratic estimator) uses an equivalent, but numer-
ically robust scheme to compute the steady-state discrete optimum Kalman gain matrix L and
steady-state covariance P.
452 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

9.5.5 C URRENT AND FUTURE PREDICTION FORMS

In fact there are two common ways to implement a discrete Kalman filter depending on what
computational order is convenient in the embedded system. Ogata refers to them as the Prediction-
type and the Current Estimation type in [150, p880-881], while the Control toolbox in M ATLAB
refers to them as ‘delayed’ and ‘current’ in the help file for kalman.

The main difference between the current estimate and the prediction type is that the current
estimate uses the most current measurements yk to correct the state x̂k , whereas the prediction
type uses the old measurements yk−1 to correct state x̂k .

For both cases, we assume our true plant is of the form we used in Eqns 9.75 and Eqn. 9.76,
repeated here with Γ = I

xk+1 = Φxk + ∆uk + wk

yk = Cxk + vk

and we want to estimate x in spite of the unknown noise terms.

Prediction-type
The prediction-type state estimate is given by

x̂k+1 = Φx̂k + ∆uk + L (yk − Cx̂k ) (9.100)

where the constant covariance and Kalman gain are

 −1
P = Q + ΦPΦ⊤ − ΦPC⊤ R + CPC⊤ CPΦ⊤ (9.101)
 −1
L = ΦPCT R + CPCT (9.102)

A block diagram of the prediction type Kalman filter implementation is given in Fig. 9.31.

Current estimation-type
The current-estimation state estimate is given by


x̂k+1 = zk+1 + L yk+1 − Czk+1 (9.103)
where zk+1 = Φx̂k + ∆uk (9.104)

and where the covariance is the same as above in Eqn. 9.101, but the Kalman gain is now
 −1
P = Q + ΦPΦ⊤ − ΦPC⊤ R + CPC⊤ CPΦ⊤ (9.105)
 −1
L = PCT R + CPCT (9.106)

A block diagram of the current estimator type Kalman filter implementation is given in Fig. 9.32.

Note that the prediction-type Kalman filter is obtained simply by multiplying the current esti-
mate gain with Φ as evident by comparing equations 9.102 and 9.106. Also note that the M ATLAB
routine dlqe solves for the current estimator type.

Listing 9.24 computes the Kalman gain for a discrete plant with matrices
 
1 2  
Φ= , C = 2 −3 , and with covariances Q = 2I, R=1
3 4
9.5. ESTIMATION OF STATE VARIABLES 453

process noise measurement noise

wk vk

❄ xk ❄
✲ ∆ ✲ + ✲ q −1 ✲ C ✲+ output, yk
✻

Φ ✛ True plant
input, uk ✲
Kalman gain
ǫ
L ✛

❄
✲ ∆ ✲ + ✲ q −1 x̂k ✲ −C ✲+✛
✻

Φ ✛
Plant model & estimator
❄
State estimates
available for feedback
xˆk

Figure 9.31: A block diagram of a steady-state prediction-type Kalman filter applied to a linear
discrete plant. Compare this with the alternative form in Fig. 9.32.

Listing 9.24: Alternative ways to compute the Kalman gain

>> A = [1,2; 3 4]; C = [2,-3]; % Discrete plant, here A used as Φ.
>> Q = 2*eye(2); R = 1; Gam = eye(2);

>> P = dare(A',C',Q,R) % Solve Eqn. 9.101 for P.

5 P =
27.0038 60.4148
60.4148 148.0538

>> Ke_ce = P*C'/(R+C*P*C') % Current estimator-type Kalman gain, Eqn. 9.106

10 Ke_ce =
-0.1776
-0.4513
>> Ke_pt = A*Ke_ce; % Prediction-type Kalman gain, Eqn. 9.102
Ke_pt =
15 -1.0801
-2.3377

>> Ke = dlqe(A,Gam,C,Q,R) % Note dlqe delivers the current estimator-type Kalman gain
Ke =
20 -0.1776
-0.4513

Both implementations of the steady-state Kalman filter are very similar to the estimator part of
454 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

process noise measurement noise

w v

❄ xk ❄
✲ ∆ ✲ + ✲ q −1 ✲ C ✲+ output, yk
✻
input, uk ✲
Φ ✛ True plant

✲ q −1 uk−1✲
∆

x̂k ❄ zk ✲
✛ ✲ q −1 ✲ Φ ✲+ −C ✲+✛
x̂k−1
❄
✛ +✛ L ✛

Plant model & estimator Kalman gain

Figure 9.32: A block diagram of a steady-state current estimator-type Kalman filter applied to a
linear discrete plant. Compare with the alternative prediction form in Fig. 9.31.

Fig. 8.5. The main difference between this optimal estimator and the estimator designed using
pole-placement in section 8.3 and 8.4 is that we now take explicit notice of the process and mea-
surement noise signals in the design of the estimator.

Of course knowing the states is not much use if you are not going to do anything with that
information, so of course we could add a feedback controller to this strategy in the same was as
we did for the pole-placement designed simultaneous state feedback controller and estimator in
Fig. 8.5.

IN CLOSED FORM SUITABLE FOR SIMULATION

If we combine the plant dynamics with the Kalman filter, we get the following system which is
convenient for simulation.

For the predictor form, by combining the plant dynamics in Eqns 9.75 and Eqn. 9.76 with the
Kalman filter update Eqn. 9.100, we get
 
       uk
xk+1 Φ 0 xk ∆ Γ 0
= +  wk  (9.107)
x̂k+1 LC Φ − LC x̂k ∆ 0 L
vk

where x is the true but unknown state vector, x̂ are the corrected best estimates of x, u are the
manipulated variable inputs, and w and v are the measurement and process noise. We colour
the true states grey, because they are unobservable and hence hard to see.
9.5. ESTIMATION OF STATE VARIABLES 455

For the current estimator form, by combining the plant dynamics with Eqns 9.103 and 9.104 we
get
 
       uk
xk+1 Φ 0 xk ∆ Γ 0 
= + wk  (9.108)
x̂k+1 LC Φ − LCΦ x̂k ∆ − LC∆ 0 L
vk

For simplicity, let us suppose that we want equal weightings on the state and measurements.
With this simplification we can write

Q = qI, R = rI

where q and r are positive scalars. As q/r → 0, L → 0 and the estimated dynamic equation
becomes xk+1|k = Φxk|k−1 + ∆uk which is the pure open-loop model prediction. Conversely as
q/r → ∞, we are essentially discarding all process information, and only relying on measurement
back-calculation.

Problem 9.6 Design a Kalman gain for the Newell & Lee evaporator. In this example, we are only
measuring level and pressure, so the measurement matrix is now
 
1 0 0
C=
0 0 1

The covariance matrices and optimum discrete gain are

Q = 0.1*eye(8); R = diag([0.1, 5.0]); L = dlqe(Phi,Delta,C,Q,R)

Compare this gain with that on [147, p63]. Is the difference significant?

9.5.6 AN APPLICATION OF THE K ALMAN FILTER

Let us suppose the composition meter on the evaporator failed, but we are required to estimate
it for a state feed-back controller. Since we have an industrial evaporator, we have noise super-
imposed on both the measurements and the process. We can simulate a scenario such as this.
We will use Eqn. 9.108 as a starting point, but we will also add another equation, that of the
uncorrected model predictions. Now we can compare 4 things:

1. The raw measurement unavoidably corrupted by noise, and possibly incomplete;

yk = Cxk + v (9.109)

2. The uncorrected model state predictions

x̄k+1 = Φx̄k + ∆uk (9.110)

3. The corrected state estimates using both model and the measurements processed by the
Kalman Filter;
x̂k+1 = f (x̂k , yk ) (9.111)

4. and of course the true state variable5

xk+1 = Φxk + ∆uk + Γw (9.112)

5 once again only available in “simulation land”
456 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

The simulation given in listing 9.25 uses a randomly generated discrete plant model. The actual
manipulated variable trajectories are unimportant for this example, and the measurement noise
variance is one fifth of the process noise variance. To further demonstrate the advantage of the
estimator, I will assume that we have an error in the initial condition of the states.

Listing 9.25: State estimation of a randomly generated discrete model using a Kalman filter.
n = 3;nc = 2;m = 2; % # of states, outputs, inputs
[Phi,Del,C,D]=drmodel(n,nc,m); % generate random discrete model
Gam = eye(n); D = zeros(size(D)); % process noise model & ignore any direct pass-through
4

t = [0:100]'; N = length(t); % discrete sample time

U = [square(t/10), square(t/11)];
x0_true = zeros(n,1); x0_est = [-10 5 -7]' % True & estimated initial conditions

2
9 v=0.5;w=0.1; Q=w*eye(n); R=v*eye(nc); % Noise covariances, Q = σw I, R = σv2 I
W=sqrt(w)*randn(N,n);V=sqrt(v)*randn(N,nc);% State & measurement noise sequences

[yp,xp] = dlsim(Phi,Del,C,D,U,x0_est); % True deterministic model

14 L = dlqe(Phi,Gam,C,Q,R); % Steady-state optimal Kalman gain, Eqn. 9.99.

Phia = [Phi,zeros(n);L*C, Phi-L*C*Phi]; % Augmented system for simulation, Eqn. 9.108.

Dela = [Del, Gam,zeros(size(L)); Del-L*C*Del, zeros(size(Gam)),L];
Ca=eye(2*n);Da=zeros(2*n,m+n+nc);
19 [y,x] = dlsim(Phia,Dela,Ca,Da,[U,W,V],[x0_true;x0_est]);

x_true = x(:,1:n); % true states, x

x_est = x(:,n+1:2*n); % Kalman estimated/corrected states, x̂
y_meas = (C*x_true')'+V; % Actual noisy measurement, y
24 y_est = (C*x_est')'; % Predicted measurement, ŷ = Cx̂

subplot(3,1,1); plot(t,x_true(:,1),t,x_est(:,1),'--',t,xp(:,1));
subplot(3,1,2); plot(t,x_true(:,2),t,x_est(:,2),'--',t,xp(:,2));
subplot(3,1,3); plot(t,x_true(:,3),t,x_est(:,3),'--',t,xp(:,3));

The most obvious scheme to reconstruct an estimate of the true states, is to use only the measure-
ments y. However using this scheme we have two difficulties, first we have noise superimposed
on the measurements, and second, we cannot reconstruct all the states since C is not inevitable,
not even square. Without the estimator, we can only use the incomplete raw output measure-
ments, rather than either the corrected states or filtered measurements. In addition, the poor
state estimate at t = 0 is compensated. We can improve on the measurements by using the model
information and the Kalman filter. One can see that the Kalman estimates (✷) are better than
using simply an open-loop model estimator (dashed line), particularly near the beginning of the
simulation given the incorrect starting conditions.

9.5.7 T HE ROLE OF THE Q AND R NOISE COVARIANCE MATRICES IN THE

STATE ESTIMATOR

The Kalman filter is an optimum model based filter, and is closely related to the optimum LQR
controller described in § 9.4. If properly designed, the Kalman filter should give better results
than a filter using only measurement or model information. We can rank different design alter-
natives by comparing the integral of the squared error between the true states x and our predic-
tions x̂. The only design alternative for the Kalman filter is the selection of the Q and R matrices.
9.5. ESTIMATION OF STATE VARIABLES 457

True states, corrected estimates & open−loop model

5

0
x1

−5

−10

5
x2

−5 Estimate
Open−loop model
10 True

5
Figure 9.33: Estimation using the Kalman
x2

0
filter. Each of the three trends shows the
−5
true state (solid), the corrected estimate (✷),
−10
0 5 10 15 20
and for comparison, the uncorrected open
sample time loop model trajectory, (dashed line).

The theory leads us to believe that if we select these matrices exactly to match the actual noise
conditions, then we will minimise the error between our prediction and the true states.

Normally the covariance matrices Q and R, are positive diagonal matrices, although strictly they
need only to be positive definite. The elements on the diagonal must be positive since they are
variances which are squared terms. If Q is large relative to R, then the model uncertainty is high
and the estimate values will closely follow the measured values. Conversely if R is large relative
to Q, then the measurement uncertainty is large, and the estimate will closely follow the model.

Suppose we have a very simple system discrete dynamic system

xk+1 = −0.5xk + wk (9.113)

yk = xk + vk (9.114)

if we were to design a Kalman filter with model and measurement covariances as Q = 1.5, R =
0.5, we could use dlqe as shown in Listing 9.26.

Listing 9.26: Computing the Kalman gain using dlqe.

>> Phi = -0.5; Gam = 1; C = 1; % System xk+1 = −0.5xk + wk with yk = xk + vk
2 >> Q = 1.5; R = 0.5; % Plant & measurement covariance matrices
>> L = dlqe(Phi,Gam,C,Q,R)
L =
0.7614

Now if we were lose all confidence in our measurements, effectively that means Q = 0 (or R
becomes very large), and we essentially discard any update. In this case the Kalman gain should
be L = 0.

>> L = dlqe(Phi,Gam,C,0,R) % Excellent measurements, bad plant noise, Q = 0

L =
458 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Alternatively if we lose all confidence in our model, then R becomes small, and the Kalman gain
in this case tends to 1 in the case of the current estimator. (Note that R 6= 0 because R must be
strictly positive definite.)

>> L = dlqe(Phi,Gam,C,Q,1e-6) % Bad measurements, R → 0

2 L =
1.0000

Now the update expression using the ‘current measurement estimator’ is

x̂k = x̂−
k + L(yk − xk )

so obviously if L = 1, then x̂k = x̂−

k + yk − xk = yk which means our state estimate is derived
completely from the measurement as anticipated given that we have a very poor model.

In the case of the prediction type Kalman filter, we get

>> L = Phi*dlqe(Phi,Gam,C,Q,1e-6) % Prediction-type KF, Eqn. 9.102.

2 L =
-0.5

which also makes sense intuitively. In this case the update is x̂k+1 = Φx̂k + L(yk − x̂k ) or in this
case x̂k+1 = −0.5x̂k − 0.5yk + 0.5x̂k = −0.5yk .

T HE OPTIMALITY OF THE K ALMAN FILTER

The optimality of the Kalman filter is best demonstrated using a Monte-Carlo simulation. The
script given in Listing 9.27 consists of two parts: first the inner loop simulates estimation of a
simple third order plant and calculates the ISE performance,
Z tf
⊤
J = (x̂ − x) (x̂ − x) dt (9.115)
0

using a steady-state Kalman filter, while the outer loop varies the Kalman design q/r ratios.
The inner loop is repeated many times (since slightly different results are possible owing to the
different noise signals) in order to obtain a statistically valid result.

In Listing 9.27, the true process has q/r = 20. We will then compare the performance of a Kalman
filter using different q/r values as the design parameters. If all goes well, we should detect a
minimum (optimum) near the known true value of q/r = 20.

Listing 9.27: Demonstrating the optimality of the Kalman filter.

qr_v = logspace(-4,3,20)'; % # of points to plot. Span % 10−4 to 103
2 nsum = 10; % # of inner iterations in Monte-Carlo simulation

s = tf([1 0],1); tau = 2; zeta = 0.5;

3s−1
G = tf((3*s-1)/(7*s+1)/(tauˆ2*sˆ2+2*zeta*s+1)); % G(s) = (7s+1)(τ 2 s2 +2ζτ s+1)
Ts = 0.2; % sample time
7 Gdss = ss(c2d(G,Ts,'zoh')); % discrete time model
9.5. ESTIMATION OF STATE VARIABLES 459

Phi = Gdss.a; Del = Gdss.b; C= Gdss.c; D = Gdss.d;

[n,m] = size(Del); p = length(C*C'); % system dimensions

t = Ts*[0:1000]'; N = length(t); % time vector

12 U = 0.2*square(t/80).*square(t/15) + square(t/50); % any input
x0 = [-1 1 0]'; % some starting point

Gam = eye(n);R = eye(p); % constant KF design

ves = 1e-6; wes = 20*ves; % measurement & process noise
17 x0_est = x0; % [0;0;0]; % (correct) estimate of x0

Jstats=[];
for i=1:length(qr_v) % outerloop: Design tests
L = dlqe(Phi,Gam,C, qr_v(i)*eye(n),R); % Design Kalman gain
22 LC=L*C; % Formulate the extended system

Phib = [Phi, zeros(n); LC, Phi-LC]; % Current predictor

Delb = [Del,Gam,zeros(size(L));Del+L*D,zeros(size(Gam)),L];
Ga = ss(Phib,Delb,eye(size(Phib)),0,Ts);
27

jx=[];
for j=1:nsum % don't repeat too often !
W = sqrt(wes)*randn(N,n); V = sqrt(ves)*randn(N,p); % Add noise
x = lsim(Ga,[U,W,V],t,[x0;x0_est]);
32 dx = x(:,[1:3])-x(:,[4:6]); % x − x̂
jx(j) = sum(sum(dx.*dx)); % Find state ISE performance
end % j
Jstats(i,:)=[mean(jx),std(jx),min(jx),max(jx)]; % collect statistics
end % i
37 Jm = Jstats(:,1);
h=loglog(qr_v,Jm,'-',qr_v,Jstats(:,[3 4]),':',...
qr_v,Jm*[1 1]+Jstats(:,2)*[1 -1],'--' );
set(line(qr_v,Jm),'MarkerSize',18,'LineStyle','.');
y = axis; y = y(3:4); h = line(wes/ves*[1,1],y);
42 set(h,'LineStyle','-.','color','k');
xlabel(sprintf('Q/R ratio (# of loops=%3g)',nsum)); ylabel('error')

The plot you obtain should be similar to Fig. 9.34 which was produced using 103 inner loops.
This curve shows that the actual minimum obtained by simulation agrees quite well with the
theoretically expected optimum expected at q/r = 20. Actually the curvature of this function near
the optimum is quite flat which means that one can design the q/r ratio an order of magnitude
in error, and it really does not degrade the system performance much. This is fortunate since it is
almost impossible to choose these design values accurately for any practical industrial process.

A DAPTING THE KF TUNING MATRICES

In any actual implementation of the Kalman filter, one does not usually know a priori appropri-
ate values for the Q and R covariance matrices, and therefore these can be effectively viewed
as tuning parameters. Tuning the KF is a cumbersome trial-and-error process that normally re-
quires expensive state verification. Some techniques make use of the fact that the innovations
(or residuals, see Eqn 9.92), of an optimal KF should be zero-mean white-noise. One can easily
store these innovations, and test this assumption once a sufficient number have been collected.
[25] describes one technique due to Mehra that updates the Kalman gain and the R matrix if the
whiteness condition of the innovations is not satisfied. There are many variations of adaptive
KF tuning based on this theme, but for a variety of reasons, they have found little acceptance in
460 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

5
x 10
2

1.8

1.6 maximum

1.4

1.2

error ±2σ
1

0.8 (q/r)*

0.6
Figure 9.34: The performance of a
Kalman filter for different q/r ratios. 0.4
The shaded areas give the approxi-
mate ±1 and 2σ uncertainty regions 0.2
of the performance. The optimum
0
performance should be at q/r = 20, 10
−4
10
−2 0
10 10
2

vertical dot-dashed line. Q/R ratio (# of loops=1000)

practice.

9.5.8 E XTENSIONS TO THE BASIC K ALMAN FILTER ALGORITHM

There are many extensions to the basic Kalman filter algorithm, and the next few sections briefly
describes some of the more important extensions. (A full treatment is given in [97] and in the later
chapters of [50] to name just a few.) The next few sections describe modifications to the update
equations that are more robust to numerical round-off, using the filter for nonlinear problems,
tricks to save computing time and estimating parameters along with states. These alternatives
are generally preferred to the standard algorithms.

N UMERICALLY STABLE K ALMAN FILTER DESIGN EQUATIONS

The Kalman filter design equations presented in §9.5, whilst correct, do have a tendency to fail
on actual industrial examples. This is due, in part to

◦ The covariance matrices becoming indefinite, (i.e., losing their positive definiteness), owing
to the limited precision of the machine, and

• the poor numerical properties of the covariance update equation.

This section describes an alternative scheme to design a Kalman filter, that tries to avoid these
two problems. Some solution strategies are very similar to the covariance concerns expressed in
the parameter identification sections. See [102] or [5, p147] for further details.

One way to ensure that the covariance matrix remains symmetric is to include the ‘hack’ some-
where in the update loop
P + P⊤
P←
2
9.5. ESTIMATION OF STATE VARIABLES 461

While this correction does force symmetry, it does not however force positive definiteness which
is a more expensive condition to check. To check positive definiteness, we could attempt a
Cholesky decomposition and keep a running check, using an optional output of the chol routine
with something like

[R,rc] = chol(P) % do decomposition & supply return code, rc

2 if (rc !=0), 'Matrix not +ve definite', end

Recall that the measurement update is

x̂k = x̄k + K (yk − Cx̄k ) (9.116)

Pk = (I − Kk C) Mk (9.117)

The troublesome equation is the measurement covariance update, Eqn 9.117. There are many
variations attempting to improve the numerics of the recursive update, and some of the more sim-
ple rearrangements are given in [122, p69-70]. Two of the most popular are the Upper-Diagonal
(U-D) factorisation due to Bierman, and Potter’s square root algorithm. The key idea for the U-D
algorithm is that the covariance matrix is factored into

P = UDU⊤

where U and D are upper triangular and diagonal matrices respectively, and these are the matri-
ces that are propagated rather than P. The algorithm for the U-D update is slightly complicated
and difficult to vectorise in a form elegant for M ATLAB, but many programs exist and one version
is given in [204, p167-168].

T HE C HOLESKY MATRIX SQUARE ROOT

Any positive semi-definite symmetric matrix can be factored into the product of a lower triangu-
lar matrix and its transpose. This is analogous to a square root in the scalar sense, and is called
the Cholesky matrix square root. If the Cholesky square root of matrix P is S, then by definition
def
P = SS⊤ (9.118)

For example      
1 2 3 1 0 0 1 2 3
 2 8 2 = 2 2 0  ×  0 2 −3  (9.119)
3 2 14 3 −2 1 0 0 1
| {z } | {z }
P S
M ATLAB can extract the Cholesky square root from a matrix using the chol command, although
this function actually returns the transpose ST .

U PDATING THE MATRIX SQUARE ROOT

The basic idea of the square root filter is to replace the covariance matrices with their square roots.
The covariance matrix can then never be indefinite, since it is calculated as the product of its two
square roots. Using the square roots also has the advantage that we can compute with double
accuracy. We can define the following square roots;
def def def def
M = SS⊤ , P = S+ S⊤
+, Q = UU⊤ , R = VV⊤ (9.120)
462 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Now the measurement covariance update equation can be written as

F = S⊤ C ⊤ (9.121)
 1/2
G = R + F⊤ F (9.122)
S+ = S − SFG−⊤ (G + V)−1 F⊤ (9.123)

and the correction is

x̂ = x̄ + SFG−⊤ G−1 (y − Cx̄) (9.124)
In Eqn. 9.122, one finds the square root of a matrix. M ATLAB can do this with the sqrtm (square
root matrix) command. The expression A−⊤ is a shorthand way of writing (A−1 )⊤ which is
also equivalent to (A⊤ )−1 . We can verify Eqn 9.123 by multiplying it with its transpose and
comparing the result with Eqn 9.117. This algorithm is often called Potter’s algorithm.

Listing 9.28 calculates the steady state Kalman gain using the square root method of Potter. The
functionality and arguments are the same as the built-in Matlab function dlqe.

Listing 9.28: Potter’s algorithm.

function [K,M,P] = potter(Phi,Gamma,C,Q,R);
% [K,M,P] = potter(Phi,Gamma,C,Q,R);
3 % Potter's square root filter algorithm to design a Kalman filter
% Ref: Algorithm from IEEE trans AC vol12 1971
% See dlqe for argument list & description

% Starting estimates & constants

8 GQG = Gamma*Q*Gamma'; K = zeros(C'); Kprev = -0.5*eye(C'); tol =
1.0e-6; P = zeros(Phi);

V = chol(R)'; % extract roots to start

13 while norm(K-Kprev,'fro') > tol

Kprev = K;
M = Phi*P*Phi' + GQG; % Time update

% Square root form for the measmnt update

18 S = chol(M)'; % Note transpose required from chol
F = S'*C'; G = sqrtm(R+F'*F);
Sp = S - (S*F/(G'))*((G+V)\F');
P = Sp*Sp'; % Will always be positive definite
K = S*F/(G')/G;
23 end % while
return % end function potter.m

Modify the potter algorithm to make it more efficient. (For example the calculation of K is not
strictly necessary within the loop.)

S EQUENTIAL PROCESSING

The Kalman filter was developed and first implemented in the 1960s when computational power
was considered expensive. For this reason, there has been much optimisation done on the algo-
rithm. One of these improvements was to sequentially process the measurements one at a time
rather than as a vector. [5, p142] gives two reasons for this advantage. First if the measurement
9.5. ESTIMATION OF STATE VARIABLES 463

noise covariance matrix R is diagonal, there is a reduction in processing time, and second, if for
some reason there is not enough time between the samples to process the entire data, (owing to
an interrupt for example), then there is only a partial loss of update information to the optimised
states, rather than a full loss of improvement.

In the sequential processing formulations of the Kalman filter equations, the Kalman gain matrix
L is in fact a single column vector, and the measurement matrix C is a row vector.

D IAGONALISING THE R MATRIX

If the measurement noises are correlated, then R will not be diagonal. In this rare case, we can
transform the system to where the measurements are uncorrelated, and work with these. The
transformation uses a Cholesky factorisation. R is factored into

R = SS ⊤ (9.125)

Now the transformed measurements are

ȳ = S −1 y (9.126)

9.5.9 T HE E XTENDED K ALMAN F ILTER

The Extended Kalman Filter or EKF is used when the process or measurement models are non-
linear. Once we have a nonlinear process, all the guarantees that the Kalman filter will converge
and that the filter will deliver the optimal state estimates no longer apply. Notwithstanding, it is
still worth a try, since in most cases even sub-optimal estimates are better than none. Almost all
the published accounts of using a Kalman filter in the chemical processing industries are actually
an extended Kalman filter applied to a nonlinear model.

The EKF is a logical extension from the normal linear Kalman filter. The estimation scheme is
still divided up into a prediction step and an estimation step. However for the model and mea-
surement prediction steps, the full nonlinear relations are used. Integrating the now nonlinear
process differential equation may require a simple numerical integrator such as the Runge-Kutta
algorithm. Thus the linear model update and measurement equations are now replaced by

x̂k|k−1 = f (x̂k−1|k−1 , uk , d, θ, t) (9.127)

ŷ = g (x̂) (9.128)

Given the nonlinear model and measurement relations, updating the covariance estimates is no
longer a simple, or even tractable task. Here we assume that over our small prediction step of
interest, the system is sufficiently linear that we can approximate the change in error statistics,
which are only an approximate anyway, with a linear model. Now all the other equations of
the Kalman filter algorithm are the same, except that the discrete state transition matrix Φ, is
obtained from a linearised version of the nonlinear model evaluated at the best current estimate,
x̂k|k−1 following the techniques described in §3.6. The creation of the linear approximation to the
nonlinear model can be done at every time step, or less frequently, depending on the application
and the availability of cheap computing. Linearising the measurement equation, Eqn 9.128 can
be done in a similar manner.
464 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

E STIMATING PARAMETERS ONLINE USING THE K ALMAN FILTER

Chapter 6 discussed estimating model parameters online which could then be subsequently used
in an adaptive control scheme. In addition, there are many text books explaining how to offline
regress parameters from experimental data. While there is some debate about the nomenclature,
parameters are generally considered to be constant, or at least change much more slowly than
the states. In a heat exchanger for example, the heat exchange surface area is constant, thus
a parameter, while the flow may vary depending on demand, thus a state. Some variables lie
somewhere in between, they are essentially constant, but may vary over a long time period. The
heat transfer coefficient may be one such variable. It will gradually get less as the exchanger fouls
(owing to rust for example), but this occurs over time spans much larger than the expected state
changes.

In many engineering applications, however, there is some, (although possibly not complete)
knowledge about the plant, and this partial knowledge should be exploited. This was not the case
for the identification given in chapter 6, where only an arbitrary ARMA model was identified.
The Kalman filter can be extended to estimate both states as given in §9.5 and also parameters.
Of course once parameters are to be estimated, the extended system generally becomes nonlin-
ear, (or if already nonlinear, now more nonlinear) so the Extended Kalman filter or EKF must be
used. In this framework, the parameters to be estimated are treated just like states. I typically
simply append the parameter vector (θ) to the bottom of the state vector x. Normally we would
like to assume that the parameter is roughly constant, so it is reasonable to model the parameter
variation as a random walk model,
dθ
=0+v (9.129)
dt
where v is a random vector of zero mean and known variance. A plot comparing normally
distributed white noise with a random walk is shown in Fig. 9.35 and can be reproduced using:

1 v = randn(500,1); % white noise: mean zero, variance 1.0, Normal distbn.

x = cumsum(v); % generate random walk
plot([v x])

Eqn 9.129 is the simplest dynamic equation for a parameter. In some specialised examples, it
may be worth while to have other simple dynamic models for the parameters, such as random
ramps to model instrument drift for example, but in general this extension is not justified. Of
course, with the extra parameters, the dimension of the estimation problem has increased, and
the observability criteria must still hold.

25

20
Random walk
15

10

Figure 9.35: An example of a ran- −5

dom walk process: θi+1 = θi + ν . −10
The solid line, ν, is white noise with White noise
a mean of zero, and a variance of 1. −150 100 200 300 400 500
The dashed line is θ. Sample #
9.5. ESTIMATION OF STATE VARIABLES 465

N ON -G AUSSIAN NOISE

The assumption that the measurement and process noise sequences are Gaussian, un-correlated,
and mean zero are usually not satisfied in practice. Coupled with the fact that the true system is
probably nonlinear, it is surprising that an optimal linear estimator, such as the Kalman filter, or
even the EKF works at all.

Most textbooks on regression analysis approximate the residual between actual experimental
data and a model as made up of two components; a systematic error and a random error. The
Kalman filter is based on the assumption that the only error is due to a random component. This
in general, is a very strong assumption, and not satisfied in practice. However coloured noise is
treated theoretically in [50, Chpt 5].

One easy test to make is how the Kalman filter performs when the noise is correlated. We can
generate correlated or coloured noise by passing uncorrelated or white noise through a low pass
filter.

Problem 9.7 Chapter 7 discussed auto-tuning by relay feedback. One of the key steps in con-
structing an automated tuning procedure is to identify the period and amplitude of the oscillation
in the measurement. Often, however, this is difficult owing to the noise on the measurement,
but we can use an Extended Kalman filter to estimate these quantities in the presence of noise.
Further details and alternative schemes are discussed in (author?) [11]. A signal that is the sum
of a sinusoid and a constant bias can be represented by the following equations;
x2
ẋ1 = 2π
x3
x1
ẋ2 = −2π
x3
ẋ3 = 0
ẋ4 = 0
y = x1 + x4

and the period and amplitude of the oscillation given by

q
P = x3 , ǫ = x21 + x22

The idea is to estimate the states of the process given the output y, and then to calculate the
period and amplitude online.

1. Linearise the dynamic equations to the standard state space form.

2. Design a steady-state Kalman filter.

3. Simulate the estimation process. Plot the estimates of the period and amplitude for a noisy
signal, and for a time varying signal.

9.5.10 C OMBINING STATE ESTIMATION AND STATE FEEDBACK

One of the key aims of the Kalman filter is to provide estimates of unmeasured states that can
then be used for feedback. When we combine state estimation with state feedback we have a
Linear Quadratic Gaussian or LQG controller.
466 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

Fig. 9.36 shows how we could implement such a combined estimator and controller in S IMULINK.
The plant model and estimator is in yellow, the regulator in blue, and the noise terms are in
orange. In this implementation we are using the 5 state aircraft model from [62] listed in Ap-
pendix E.2.

plant noise Measurement noise

5 3

5 5 1 5 3 3
K*u K*u
3 z 5 3
Del C output
5 Unit Delay

Phi
5
K*u
5

Ke
3
K*u
5

5 3
5 5 1 5 3
K*u K*u
3 z 5 3
Del1 Unit Delay1 C1
5

Phi1
5
K*u
5

L
5
K*u
3

Figure 9.36: An example of a discrete LQG optimal estimator/controller implemented in

S IMULINK. This is simply the estimator from Fig. 9.31 where the estimated states are fed back
through a static gain.

The design of the discrete regulator, perhaps using dlqr, and the design of the estimator, perhaps
using dlqe are completely independent.

9.5.11 O PTIMAL CONTROL USING ONLY MEASURED OUTPUTS

The optimal controllers of §9.4 had the major drawback that they required state feedback since
the control law was a function of the states, namely Eqn. 9.46 or u = −Kx. We addressed this
problem in §9.5 by first designing an observer to produce some estimated states which we could
use in place of the true, but unknown, states in the controller. However it would be valuable to
be able to design an optimal controller directly that uses output, rather than state feedback. Such
an optimal output feedback controller in fact exists, but it is difficult to compute, and may only
deliver a local optimum. The derivation of the optimum output controller can be found in [123,
chapter 8], and improvements to the design algorithm are given in [167].

Once again we assume a linear plant

ẋ = Ax + Bu, x(0) = x0
y = Cx
9.6. SUMMARY 467

but this time we desire an output controller of the form

u = −Ky

which minimises the familiar quadratic performance index

Z
1 ∞ T

J (K) = x Qx + uT Ru dt
2 0

where A, Q ∈ ℜn×n , B ∈ ℜn×p , C ∈ ℜr×n , R ∈ ℜp×p , K ∈ ℜp×r . Also we assume, Q > 0

(positive semi-definite) and R > 0.

The optimum gain Kis given by the simultaneous solution to the following design equations:

0 = ATc P + PAc + CT KT RKC + Q (9.130)

0 = Ac S + SATc
+X (9.131)
 −1
K = R−1 BT PSCT CSCT (9.132)

where the closed loop Ac and the covariance of the initial state are defined as
def 
Ac = A − BKC, X = E x(0)xT (0)

and the performance index is given by

J = trace (PX) (9.133)

The coupled nature of the multiple Lyapunov equations mean that we will need to use a nu-
merical technique to solve for the K that satisfies all three design equations. Our alternatives
are:

1. To use a general-purpose unconstrained optimiser to search for the values of K that min-
imise Eqn. 9.133 directly where P is given by the solution to Eqn. 9.130.

2. To use an unconstrained optimiser and exploit the gradient information since the gradient
of J with respect to K is

∂J  
= 2 RKCSCT − BT PSCT
∂K
where we have computed the auxiliary matrices P, S by solving the two Lyapunov equa-
tions Eqns 9.130–9.131 perhaps using the M ATLAB function lyap.

3. To follow a simple fixed-position algorithm where we guess a value for K and then iterate
around Eqns 9.130 to 9.132.

Numerical experiments show that the direct optimisation is prone to infinite looping, and the
fixed-point algorithm can converge to local optima.

9.6 S UMMARY

This chapter dealt with optimal control using classical methods. Optimal control is where we try
to control a process in some “best” manner. The term best in the engineering field usually means
468 CHAPTER 9. CLASSICAL OPTIMAL CONTROL

most economic (either most profit, or least cost). While it is naive to expect our controller to per-
form optimally in the field, in practice they have been found to out-perform more conventional
controllers designed via methods such as loop shaping or frequency response for example. I think
of optimal controllers as simply another set of design rules to construct a workable, satisfactory
controller, but would not expect them to be the “best” in any strict and absolute sense.

A convenient objective function is the quadratic objective function that tries to minimise the
weighted sum of state and manipulated deviations squared over time. The optimal controller
design problem given nonlinear process dynamics and constraints is, in general, too hard to solve
online, although offline open-loop optimal profiles are possible using numerical techniques. The
optimisation problem given linear dynamics is tractable and the solution is termed the linear
multivariable regulator (LMR or LQG). The LMR in its simple guise is a multivariable propor-
tional controller. The fact that estimation is closely related to regulation was shown in chapter 8
and the optimal estimator is the Kalman filter.
C HAPTER 10

P REDICTIVE CONTROL

To be computationally practical, the optimal controllers of chapter 9 required that the system
satisfy certain mathematical restrictions. To use variational calculus, the dynamic models must
be differentiable; to use LQG, we require linear models with no saturation constraints. In contrast
the controllers described in this chapter take into account constraints in the manipulated and
control variables, something which is crucial for practical control applications.

10.1 M ODEL PREDICTIVE CONTROL

Model predictive control, or MPC, first started appearing in a variety of flavours in the late 1970s
and has since then emerged as one of the most popular and potentially valuable control method-
ologies, particularly popular in the chemical processing industries. A good comprehensive sur-
vey of theoretical and practical applications is given in [52, 73, 164, 168].

The central theme of all these variants of predictive control is a controller that uses a process
model to predict responses in a moving or receding horizon. Unlike the optimal control of sec-
tion 9.3.1 or the linear cases following, this control strategy uses a finite optimisation horizon,
(refer Fig. 10.1). At every sample time, the optimiser built into the controller selects the best finite
number of future control moves over a horizon of 10 or so samples, of which the controller will
subsequently implement just the first. At the next sample time, the whole optimisation procedure
would be repeated over the new horizon, which has now been pushed back one step. This is also
known as receding horizon control. Since an optimisation problem is to be solved each sampling
time step, one hopes that the calculations are not too involved!

This long-term predictive strategy coupled with the seeking of multiple optimal future control
moves turns out to be rather a flexible control scheme where constraints in manipulated and
state variables can be easily incorporated, and some degree of tuning achieved by lengthening or
shortening the horizon. By predicting over the dead time and two or three time constants into the
future, and by allowing the controller to approach the setpoint in a relaxed manner, this scheme
avoids some of the pitfalls associated with the traditionally troublesome processes such as those
with inverse response or with excessive dead-time.

However relying predictions from models are not without their problems. I pity the Reserve Bank
of New Zealand which has the unenviable task of predicting the future time-varying interest rates
at quarterly intervals as shown in Fig. 10.2. The dark solid line shows the actual 90-day interest
rates from March 1999 until March 2003 compared to the predictions made at the originating

469
470 CHAPTER 10. PREDICTIVE CONTROL

✻ ✲ Future
Past ✛
Known future setpoint

Predicted (sampled) outputs

Past values
∆t
✛ ✲
✲ time
✲
Prediction horizon
✻
Current manipulated input
✠ .
..
.. ............... Planned future manipulated inputs
............... .. . ........................................
.. . ............
..............
Control horizon
✲
✲ time
Current time

Figure 10.1: Horizons used in model predictive control

point. This data was obtained from [195, p18] where is it cautioned: “While we consider our
approach generally useful, there are risks. For example, if the interest rate projections are taken
as a resolute commitment to a particular course of policy action, then those that interpret it as
such will be misled.”

Figure 10.2: Successive future 90-day interest

predictions by the Reserve Bank at quarterly
intervals.

The idea behind MPC is that we have a dynamic, possibly nonlinear, plant described as

ŷi+1 = f (yi , ui ) (10.1)

where we wish the plant output y(t) to follow as closely as practical a possibly varying setpoint
or reference r(t).
10.1. MODEL PREDICTIVE CONTROL 471

To establish the control move, we find a suitable u(t) solve that minimises the quantity
Np Nc
X 2
X
Jt = (rt+i − ŷt+i ) + w∆u2t+i−1 (10.2)
i=1 i=1
| {z } | {z }
prediction horizon control horizon

subject to some input constraints

ui ∈ U (10.3)
The set U in the constraint equation, Eqn. 10.3 is the set of allowable values for the input and
typically takes into account upper and lower saturation limits in the actuator. For example in the
case of a control valve, the allowable set would be from 0 (shut) to 100% (fully open).

The optimisation of Eqn. 10.2 involves finding not just the one next control move, but a series of
Nc future moves, one for each sample time over the future control horizon. In practice the con-
trol horizon is often about 5–15 samples long. The quantity optimised in Eqn. 10.2 is comprised
of two parts. The first expression quantifies the expected error between the future model pre-
dictions ŷ and the future setpoints, r. Typically the future setpoint is a constant, but if changes
in reference are known in advanced, they can be incorporated. The comparison between error
and expected output is considered over the prediction horizon, Hp , which must be longer than the
control horizon.

The second term in Eqn. 10.2 quantifies the change in control moves. By making the weighting
factor, w, large, one can discourage the optimal controller from making overly excited manipu-
lated variable demands.

At each sample time, a new constrained optimisation problem is solved producing an updated
vector of future control. At the next sample instance, the horizons are shifted forward one sample
time T , and the optimisation repeated. Since the horizons are continually moving, they are often
termed receding horizons.

The chief drawback with this scheme is the computational load of the optimisation every sample
time. If the model, Eqn. 10.1, is linear, and we have no constraints, then we are able to analyti-
cally extract the optimum of Eqn. 10.2 without resorting to a time consuming numerical search.
Conversely if we meet active constraints or have a nonlinear plant, then the computation of the
optimum solution becomes a crucial step. Further details for nonlinear predictive control are
given in the survey of [85].

Algorithm 10.1 Model Predictive Control (MPC) algorithm

The basic general predictive control algorithm consists of a design phase and a implementation
phase. The design phase is:

1. Identify a (discrete-time) model of the plant to be controlled.

2. Select a suitable sampling time and choose the lengths of the control and prediction hori-
zons, Hc , Hp , where Hc ≤ Hp and the control move weightings, w. Typically Hp is chosen
sufficiently long in order to capture the dominant dynamics of the plant step response.

3. For further flexibility, one can exponentially weight future control or output errors, or not
include the output errors in the performance index for the first few samples. Making the
prediction horizon longer than the control horizon effectively adds robustness to the algo-
rithm, without excessively increasing the optimisation search space.
472 CHAPTER 10. PREDICTIVE CONTROL

4. Incorporate constraints in the manipulated variable if appropriate. In the absence of con-

straints, and as the prediction horizons tend to ∞, the MPC will lead to the standard linear
quadratic regulator as described in §9.4.

The implementation phase is carried out each sampling time.

1. Use an optimiser to search for the next future Nc optimum control moves ut starting from
the current position.
2. Implement only the first optimum control move found from step 1.
3. Wait one sample period, then go back to step 1. Use the previous optimum ut vector,
(shifted forward one place) for the initial estimate in the optimiser.

The following sections demonstrate various predictive controllers. Section 10.1.1 applies a pre-
dictive controller to a linear plant, but with input constraints. Because this is a nonlinear optimi-
sation problem, a generic nonlinear optimiser is used to solve for the control moves. As you will
see if you run the example in Listing 10.1, this incurs a significant computational burden. If we
ignore the constraints, then we can use a fast analytical solution as shown in section 10.1.2, but
this loses one of the big advantages of MPC in the first place, namely the ability to handle con-
straints. The M ODEL P REDICTIVE C ONTROL T OOLBOX for M ATLAB described in [26] is another
alternative to use for experimenting with MPC.

10.1.1 C ONSTRAINED PREDICTIVE CONTROL

This section demonstrates a constrained predictive control example using the following linear
non-minimum phase plant,
1.5(1 − 2.5s)
Gp = 2 (10.4)
4s + 1.8s + 1
but where the manipulated variable is constrained to lie between the values −3.5 < u < 3.

Sampling Eqn. 10.4 at T = 0.75 with a zeroth-order hold gives the discrete system
−0.005z −1 − 0.4419z −2
Gp (z) =
1 − 1.596z −1 + 0.7136z −2
whose open loop response to a step change is illustrated in Fig. 10.3.

To make the control problem more interesting, we will assume that

Since we have a constraint in the manipulated variable, I will use the general nonlinear con-
strained optimiser fmincon from the O PTIMISATION T OOLBOX. The function to be minimised is
fmodstpc in Listing 10.2 which returns the performance index.

We can see from the discrete step response that a sample time of T = 0.75 is a reasonable choice.
However the response drops before it climbs to the final steady-state, exhibiting an inverse re-
sponse or non-minimum phase behaviour owing to the right hand plane zero at s = +2.5. With
this sort of inverse behaviour, we must take care that the prediction horizon is sufficiently long
to span the initial inverse response which again we can estimate approximately from the discrete
step response. In this case I will set Hp = 20 and optimise for 10 inputs, Hc = 10. The control
weightings are set to w = 1.

The script file to run the simulation and to obtain results similar to Fig. 10.4 is given in List-
ing 10.1.
10.1. MODEL PREDICTIVE CONTROL 473

Step Response

1.5

1
Amplitude

0.5

1.5(1−2.5s)
−0.5
Gp = 4s2 +1.8s+1 , Ts = 0.75

−1
0 5 10 15 20 25 Figure 10.3: An inverse plant model sam-
Time (sec) pled at Ts = 0.75s.

Listing 10.1: Predictive control with input saturation constraints using a generic nonlinear opti-
miser
ntot=150; dt=0.75; t=dt*[0:ntot-1]'; % time vector
2

r = [2,3,-2,-7,2]; % Interesting setpoint

Yspt = ones(size(t(1:fix(length(t)/length(r))+1)))*r(:)';
Yspt = Yspt(1:length(t))'; % Cut off excess data points

7 tau = 2.0; zeta = 0.45; Kp=1.5;taup=2.5; % Create non-minimum phase plant

Gss = ss(c2d(tf(Kp*[-taup 1],[tauˆ2 2*tau*zeta 1]),dt)); % discrete plant

Hc = 10; % control horizon, Hc

Hp = Hc+10; % prediction horizon, Hp > Hc
12 Umin = randn(Hc+1,1); % start guess
Ulb = -3.5*ones(size(Umin)); Uub = 3*ones(size(Ulb)); % bounds
Uopt=zeros(size(t)); Y=zeros(size(t)); % initialise u(t)
x = [0,0]'; % initialise starting point
optns = optimset('Display','off','TolFun',1e-3,'TolX',1e-3); % optimiser options
17

for i=1:ntot-Hp-1; % for each sample time ...

Umin = [Umin(2:end);Umin(end)]; % roll & repeat
Umin = fmincon(@(u)fmodstpc(u,Gss,x,Yspt(i:i+Hp-1),Hp),Umin, ...
[],[],[],[],Ulb,Uub,[],optns);
22

x = Gss.a*x + Gss.b*Umin(1,:)'; % do 1-step prediction

Y(i,:) = Gss.c*x; Uopt(i,:) = Umin(1)'; % collect
end % for i
Yspt(1) = []; Yspt=[Yspt;1]; % shift due to fence-post error
27

subplot(2,1,1); % Now plot results

plot(t,Y,t,Yspt,'r--') % plot results
ylabel('output & setpoint')
subplot(2,1,2);
32 [ts,Us] = stairs(t,Uopt);
plot(ts,Us,t,[ones(size(t))*mean([Ulb,Uub])],'k--')
xlabel('time [s]'); ylabel('Input')
474 CHAPTER 10. PREDICTIVE CONTROL

The objective function to be minimised called by the optimiser is given in Listing 10.2. Note that
auxiliary variables, such as the model, and horizon lengths are passed as anonymous variables
through to the function to be optimised.

Listing 10.2: Objective function to be minimised for the predictive control algorithm with input
saturation constraints
1 function Perf = fmodstpc(U,G,x0,Yspt,Hp)
% Short-term-prediction error called via fmincon via TGPC.M

if Hp > length(U), % normally the case

U = [U;U(end)*ones(Hp-length(U),1)]; % pad extra
6 end % if
% Control weightings N 2
P c
ssdu = sum(diff(U).ˆ2); i=1 ∆ut+i−1
[Y,t,X] = lsim(G,U,[],x0); % do linear model prediction

Perf = sum((Y-Yspt).ˆ2) + ssdu; % Performance index J

11 return %

The controlled performance as shown in Fig. 10.4 is very good, as indeed it should be.

MPC: T =0.75, H =20, H =10

s p c
5
output & setpoint

−5

−10
4
umax
2
Input

−2
umin
−4
0 20 40 60 80 100 120
time [s]

Figure 10.4: Predictive control of a non-minimum phase system with constraints on u with a
control horizon of Hc = 10 and a prediction horizon of Hp = 20. Refer also to Fig. 10.5 for a
zoomed portion of this plot.

Especially note that since we are making use of a vector of future setpoints, the control algorithm
starts preparing for the setpoint change before the actual setpoint changes. This acausal behaviour,
illustrated more clearly in Fig. 10.5, is allowed because we assume we know future setpoints,
improves the response compared with a traditional feedback only PID controller. Naturally the
control becomes looser as the manipulated variable hits the upper and lower bounds.

You will soon realise, if you run this example, that the computational load for the optimal in-
put trajectory is considerable. With such a performance load, one needs to investigate ways to
increase the speed of the simulation. Reasonable starting estimates for the new set of future con-
trol moves are simply obtained from the previous optimisation problem. Good starting guesses
for the nonlinear optimiser are essential to ensure that the controller manages to find a reason-
able output in the time available. Even though our model, ignoring the manipulated variable
10.1. MODEL PREDICTIVE CONTROL 475

MPC: Ts=0.75, Hp=20, Hc=10

5
output & setpoint

−5

H H H
p c p
−10
4

2
Input

0
Figure 10.5: A zoomed portion of Fig. 10.4
−2 centered about the setpoint change around
t = 46 showing clearly the acausal response
−4
30 35 40 45 50
possible if we take advantage of the vector of
time [s] future setpoints.

constraints, is linear, we have used a general nonlinear optimiser to solve the control problem.
This means that the same algorithm will work in principle for any plant model, whether linear
or nonlinear, and any constraint formulation. Naturally there are many extensions to the basic
algorithm that address this issue such as only optimising every n-th sample time, or lowering Nc .

However substantial simplifications can be made if we restrict our attention to just linear plants
and bounded output, input and input rate constraints. Under these restrictions, the optimal
MPC formulation can be re-written as a standard quadratic program, or QP for which there are
efficient solution algorithms. Further details of this are given in [24, 166], and a commercial
M ATLAB implementation is described in [27].

VARYING THE CONTROL AND PREDICTION HORIZONS

Investigating different controller design decisions is computationally expensive for this applica-
tion, but we can relax the termination requirements for the optimiser with little noticeable change
in results. From a computational complexity viewpoint, we want to reduce the search space, and
hence reduce the horizons. However, reducing the horizons excessively, will result in a poor
controlled response.

Fig. 10.6 demonstrates what is likely to happen as the control and prediction horizons grow
shorter. For a short control horizon, Nc = 2, and a reasonably short prediction horizon, Np =
8, but one that still covers inverse response, the controlled response is still quite reasonable as
shown in Fig. 10.6(a). However decreasing the prediction horizon any further, even just slightly
such as to Np = 7, gives problems as exhibited in Fig. 10.6(b). The inverse response horizon is
≈ 5.

MPC ON THE LABORATORY BLACKBOX

Fig. 10.7 shows the result of using the constrained MPC on the black box assuming a constant
model of the blackbox as
0.98
M: G(s) =
(3s + 1)(5s + 1)
476 CHAPTER 10. PREDICTIVE CONTROL

GPC: T =0.75, H = 8, H = 2 GPC: T =0.75, H = 7, H = 2

s p c s p c
5 5
output & setpoint

output & setpoint

0 0

−5 −5

−10
4 −10
4

2 2
Input

Input
0 0

−2 −2

−4 −4
0 20 40 60 80 100 120 0 20 40 60 80 100 120
time [s] time [s]

(a) Short control & reasonable prediction horizons, (b) Short control & prediction horizons,Hc = 2, Hp = 7.
Hc = 2, Hp = 8.

Figure 10.6: Varying the horizons of predictive control of a non-minimum phase system with
constraints on u.

sampling at T = 0.75 seconds with a prediction horizon of Hp = 20, and control horizon of
Hc = 10 samples. Clearly we should also estimate the controller model online perhaps using a
recursive least-squares type algorithm.

Black−Box: MPC ∆t=0.75, H =20, H =10

p c
0.5
output & setpoint

−0.5

−1
1

0.5
input

−0.5

−1
0 20 40 60 80 100 120 140 160 180 200
time (s)

Figure 10.7: Constrained MPC applied to the laboratory blackbox with sampling at T = 0.75 sec
and prediction & control horizons Hp = 20, Hc = 10.

10.1.2 D YNAMIC MATRIX CONTROL

If you actually ran the simulation of the general optimal predictive controller from §10.1.1, and
waited for it to finish, you would have stumbled across the one big drawback — computational
load. As previously anticipated, one way to quicken the search is to assume a linear system,
ignore any constraints, and make use of the analytical optimisation solution. To do this, we must
10.1. MODEL PREDICTIVE CONTROL 477

express our model as functions of input only, that is, a finite impulse response model, rather
than the more parsimonious recursive infinite impulse response model. There are again many
different flavours of this scheme, but the version here is based on the step response model,

∞
X
y(t) = gi ∆u(t − i) (10.5)
i=1

where the gi s are the step response coefficients, and ∆u(t) is the change in input, or ut − ut−1 , refer
Fig. 10.8. Suppose we start from rest, and make a unit step change in ∆u at time t = 0. Using
Eqn. 10.5, the first few output predictions are

y1 = g1 ∆u0 + g2 ∆u−1 + · · · (10.6)

|{z} | {z }
=1 =0
y2 = g1 ∆u1 + g2 ∆u0 + g3 ∆u−1 + · · · (10.7)
|{z} |{z} | {z }
=0 =1 =0
y3 = g1 ∆u2 + g2 ∆u1 + g3 ∆u0 + · · · (10.8)

Since all the control moves are zero, except ∆u0 = 1, then it follows that y1 = g1 , y2 = g2 , y3 = g3
and so on. So the values of the output at each sample time yk in response to a unit step at k = 0
are the step response coefficients, gk . These coefficients form an infinite series in general, but for
stable systems, converge to a limit as shown in Fig. 10.8.

output
✻ gNp
.. .. ..
... ... ... ... steady-state
.. .. .. ..
impulse coeff, h5 ✻ ... .. .. .. ..
... .. ... ... ...
❄.. .. ..
.. ... ... ...
g3 .. .. .. .. .. ..
.. ... ... .. .. .. ..
.. .. . .. .. .. .. .. .. ✲ time
.. ..
0 .. .. ✛ ✲
.. .
.
g2 Step response coefficients ∆t
g1

✻ Manipulated input
1
unit step
✲ time
0
Past ✛ ✲ Future

Figure 10.8: Step response coefficients, gi , for a stable system.

478 CHAPTER 10. PREDICTIVE CONTROL

We can write the response of the model in a compact matrix form as

 
  g1 0 0 ··· 0  
y1 ∆u0
 g2 g1 0 ··· 0 
 y2    ∆u1 

 ..
 
= g3 g2 g1 ··· 0 
 ..


 .   .. .. .. .. ..  . 
 . . . . . 
yN p ∆uNc −1
gNp gNp −1 gNp −2 · · · gNp −Nc +1
| {z }
dynamic matrix

yforced = G ∆u (10.9)

The Np × Nc matrix G in Eqn. 10.9 is referred to as the dynamic matrix, hence the term dynamic
matrix control or DMC. It is also closely related to a Hankel matrix which provides us with a
convenient, but not particularly efficient, way to generate this matrix in M ATLAB. Eqn. 10.9
tells us how our system will respond to changes in future moves in the manipulated variable.
Specifically Eqn. 10.9 will predict over the prediction horizon, Np , the effect of Nc control moves.
As in the general optimisation case, the input is assumed constant, ∆u = 0, past the end of the
control horizon.

All we need to do for our optimal controller, is to find out what to set these future control moves
to, in order that the future outputs are close to the desired setpoints. Actually Eqn. 10.9 only
predicts how our system will behave given future control changes, (termed the forced response),
but it ignores the ongoing effect of any previous moves, (termed the free response). We must
add the effect of the free response, if any, to Eqn. 10.9 to obtain a valid model prediction. This
simple addition of two separate responses is a consequence of the linear superposition principle
applicable for linear systems. We can calculate the free response over the prediction horizon,
yfree simply by integrating the model from the current state with the current constant input for
Np steps. Therefore to ensure our future response is equal to the setpoint, r, at all sample times
over the prediction horizon, we require

r = yfree + yforced (10.10)

Again, since we will make the requirement that the control horizon is shorter than the prediction
horizon, our dynamic matrix, G, is non-square, so the solution of Eqn. 10.9 for the unknown con-
trol moves involves a pseudo inverse. Typically we further weight the control moves to prevent
an over-excited input demands by a positive weight λ, so the optimal solution to the predictive
control problem is
 −1
∆u = G⊤ G + λI G⊤ (r − yfree ) (10.11)

This analytical solution to the optimal control problem is far preferred above the numerical search
procedure required for the general nonlinear case. Note that the matrix inversion in Eqn. 10.11 is
only done once, offline and thus does not constitute a severe online numerical problem. The effect
of the control move weighting improves the numerical conditioning of the matrix inversion.

Fig. 10.9 shows in detail how this control philosophy works. If we are controlling a process with
a controller that uses a prediction horizon of 12 samples and a controller horizon of 5, and we
take a snap-shot at t = 135. The thick solid line (upper plot of Fig. 10.9) shows how the controller
predicts the process will respond given the optimally calculated dashed input (lower plot) at time
t = 135. Since the model inside the controller is slightly wrong owing to model/plant mismatch,
the process would, in fact, follow the thin solid line if given the dashed pre-supposed optimal
input profile. In actual fact, since only the first input is implemented, and the whole optimisation
repeated at the next time step, the process follows the · · · ◦ · · · line given the solid input trajectory.

A dynamic matrix control (DMC) example

10.1. MODEL PREDICTIVE CONTROL 479

DMC dt= 1.0 Hp= 12 Hc= 5

6

2
Output

-2

-4
134 136 138 140 142 144 146 148 150

1
Input

-1

-2
134 136 138 140 142 144 146 148 150
time w= 1 noise var=0.04

Figure 10.9: DMC predictive control showing at t = 135 how the controller predicts the process
will respond (heavy-solid line) given the current calculated optimal (dashed) input trajectory,
how the process would really respond (light-solid line), and how in fact eventually the process
was controlled (· · · ◦ · · · ) given the solid input trajectory.

Fig. 10.10 shows the performance of a DMC controller for the non-minimum phase plant of
Eqn. 10.4 from §10.1.1. I generated the step response coefficients elegantly from the transfer
function model simply using the dstep command. For illustration purposes in this simulation I
have varied the control move weighting, λ in Eqn. 10.11 as λ = 100 for t < 50, then λ = 1 for the
interval 50 < t < 100, and then finally λ = 0.

The controller uses a prediction horizon of Hp = 25, a control horizon of Hc = 8 at a sample time
of T = 0.5. The manipulated variable constraints present in §10.1.1 are relaxed for this example.
As the control move weighting is decreased, the output performance improves at the expense of
vigorous manipulated variable movements. Note that a weighting λ = 0 is allowable, but will
result in an over-excited controller.

The m-file for this DMC example, but using a constant weight of λ = 1 is given in Listing 10.3.

Listing 10.3: Dynamic Matric Control (DMC) control

ntot=250; dt=0.5; t=dt*[0:ntot-1]';% time vector
r = [1,3,-1,-2.5,3.5,5.4,-4,-6,5,2.2]; % random square wave setpoint
Yspt = ones(size(t(1:fix(length(t)/length(r))+1)))*r(:)';
4 Yspt = Yspt(1:length(t))'; % Cut off excess data points

Kp=1.5;taup=2.5; tau = 2.0; zeta = 0.45; % process model

Gpc = tf(Kp*[-taup 1],[tauˆ2 2*tau*zeta 1]); % Kp (−τp s + 1)/(τ 2 s2 + 2ζτ s + 1)
Gp = c2d(Gpc,dt); % discrete plant
 480 CHAPTER 10. PREDICTIVE CONTROL

T =0.5, H =25, H = 8
s p c
1

0.5
Y & spt

−0.5

−1
1

0.5
U

−0.5
λ = 100 λ = 20 λ = 0.1
−1
0 50 100 150
sample time

Figure 10.10: DMC predictive control of a non-minimum phase system with Hc = 8, Hp = 25

showing the effect of varying the control move weighting λ.

9 [Phi,Del,C,D] = ssdata(Gp); % state space for x0

Hc = 8; % Control horizon, Hc
Hp = 25; % Prediction horizon, Hp > Hc

14 [g,ts] = step(Gp,dt*Hp); % step response coefficients

g(1) = []; % only consider Hp points (ignore first zero)

% Formulate DMC G matrix & inverse once; (offline) Hankel version

G = flipud(hankel(flipud(g))); G(:,Hc+1:end) = [];
19 weight=1; % control move weighting w ≥ 0
GG = G'*G; % G⊤ G

−1 ⊤
invG = (GG + weight*eye(size(GG)))\G'; % G⊤ G + λI G

x = [0,0]'; y=C*x; u=0; p=0; % cheat here but Ok

24 Uopt=zeros(size(t)); Y=zeros(size(t)); % initialise u(t)
for i=1:ntot-Hp-1;
du = invG*(Yspt(i:i+Hp-1)-p); % analytical optimisation
u = du(1)+u ; % only take first change
x = Phi*x + Del*u; y = C*x; % apply 1 control move
29

% Calculate future free response (assume all future du=0)

[p,px] = dlsim(Phi,Del,C,D,u*ones(Hp+1,1),x);
p(1)=[]; % kill repeat of x0

34 Y(i,:) = y; Uopt(i,:) = u;
end % for i

Yspt(1) = []; Yspt=[Yspt;1];

subplot(2,1,1); plot(t,Y,t,Yspt,':') % plot results
39 subplot(2,1,2); stairs(t,Uopt);

The high fidelity of the controlled response of both predictive control algorithms, (Figures 10.4
and 10.10) is helped by the lack of model/plant mismatch in the previous simulations. In practice,
faced with the inevitable mismatch, one can extend the basic DMC algorithm such as varying the
10.1. MODEL PREDICTIVE CONTROL 481

weights and/or horizon lengths or by including an adaptive model and therefore performing the
matrix inversion online. In some cases, particularly for unstable processes, or for cases where the
sampling rate is either very fast or very slow compare with the step response of the system, the
DMC algorithm will fail primarily owing to the poor conditioning of G. In these cases singular
value decomposition may help.

DMC OF THE BLACKBOX

A DMC controller is tried on the laboratory blackbox in Fig. 10.11. Unlike the GPC example on
476 however, here we will identify the model online using a recursive least-squares estimation
algorithm. The model-based controller assumes the structure

b0 + b1 q −1 + b2 q −2
M: G(q −1 ) =
1 + a1 q −1 + a2 q −2 + a3 q −3

for the blackbox with a sample time of T = 1.5 seconds and a forgetting factor of λ = 0.995. The
controller horizon is set to 8 samples, and the prediction horizon is set to 20 samples. The control
move weighting is set to 0.2.

DMC of the blackbox ∆ t=1.50, H =25, H = 8

p c
1

0.5
output & setpoint

−0.5

−1
0 50 100 150 200 250 300 350 400

RLS Model ID λ=0.20, n = 3, n = 2

a b
1

0.5
Input

−0.5

−1
0 50 100 150 200 250 300 350 400
time (s)

Figure 10.11: Adaptive dynamic matrix control (DMC) of the blackbox with control horizon Hc =
8 and prediction horizon Hp = 20.
482 CHAPTER 10. PREDICTIVE CONTROL

10.2 A M ODEL P REDICTIVE C ONTROL T OOLBOX

A Model Predictive Control toolbox developed by Currie, [56, 57] allows one to rapidly build and
test various MPC configurations for a wide variety of linear, or linearisable nonlinear plants with
both soft and hard constraints on the output, the input and input rate of change. The following
sections describe this toolbox in more detail.

10.2.1 A MODEL PREDICTIVE CONTROL GUI

The easiest way to get started is to use the Model Predictive Control toolbox GUI as shown in
Fig. 10.12. In this M ATLAB GUI, one can load and experiment with different plants (and models of
those plants), adjust prediction horizons, operating constraints, and setpoints. What is interesting
to observe in the GUI application are animations of the future output and input predictions given
to the right of the solid line in the scrolling plot.

Figure 10.12: An MPC graphical user interface

10.2.2 MPC TOOLBOX IN M ATLAB

The MPC toolbox is easy to use in script mode. This section shows how to build a multivariable
MPC controller using the Wood-Berry column plant in Eqn. 3.24 from section 3.3.3. This plant
has 2 inputs (reflux and steam) and two outputs (distillate and bottoms concentration) and we
will use a sampling time of Ts = 2 seconds. For this application we will assume no model/plant
mismatch.
10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 483

Listing 10.4: Setting up an MPC controller

1 G = tf({12.8, -18.9; 6.6 -19.4}, ...
{ [16.7 1],[21 1]; [10.9 1],[14.4 1]}, ...
'ioDelayMatrix',[1,3; 7,3]) % The Wood-Berry column plant from Eqn. 3.24

Ts = 2; Gd = c2d(G,Ts); % Discretise the plant at Ts = 2 seconds

6 Plant = jSS(Gd); % Convert the LTI model to a special form required for the MPC controller
Model = Plant; % Assume (for the moment) no model-plant mismatch

Now we are ready to specify the controller tuning parameters which include the length of the
control and prediction horizons, the values of the input and output saturation constraints, and
the relative input and output weights in the objective function.

Np = 10; % Prediction Horizon

Nc = 5; % Control Horizon
3

con.u = [-1 1 0.1; ... % Input constraints: umin , umax , ∆umax

-1 1 0.1]; % One row for each input
con.y = [-2 2; % Output constraints: ymin , ymax
-2 2] ;
8

uwt = [1 1]'; % Individual weighting on the inputs

ywt = [1 0.1]'; % We are less interested in controlling bottoms, y2

Kest = dlqe(Model); %Estimator Gain

Now we are ready to initialise the model (and plant). To that, we need to know how many inter-
nals states we have in our discrete model. That number depends on the discretisation method,
and the sampling rate, so the easiest way is to inspect the properties of the plant and model.

>> Plant
Plant =
3 -- jMPC State Space Object --
States: 10
Inputs: 2
Outputs: 2
Sampling Time: 2 sec
8

>> Plant.x0 = zeros(10,1); % Initial condition of the 10 states

>> Model.x0 = zeros(10,1);

Finally we are ready to design a setpoint trajectory, assemble the MPC controller, and simulate
the results. In this case we are using the high-speed MEX version of the optimisation routines.

T = 200; % Desired simulation length

setp = 0.5*square(0.05*Ts*[0:T]',50)+0.7; % Design interesting setpoint
setp(:,2) = 0.5*square(0.1*Ts*[0:T]',50)-0.7;
setp(1:10,:) = 0;
5

MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest); % Construct an MPC controller

simopts = jSIM(MPC1,Plant,T,setp); % Set some simulation options

simresult = sim(MPC1,simopts,'Mex'); % Simulate & plot results

484 CHAPTER 10. PREDICTIVE CONTROL

10 plot(MPC1,simresult,'summary');
suptitle(sprintf('Horizons: N_p=%2i, N_c=%2i',Np,Nc))

The results for this multivariable MPC example are shown in Fig. 10.13. Note how the control of
y2 is considerably looser than that for y1 . This is a consequence of the output weights which are
ten times higher for y1 . Also note that the change in input ∆u never exceeds the required ±0.1.

Horizons: Np=10, Nc= 5

Outputs: y (k) States: x (k)
p p
6

1
4
Amplitude

Amplitude
0 2

0
−1

−2
0 50 100 150 200 0 50 100 150 200

Input: u(k) Change in Input: ∆ u(k)

0.6 0.1

0.4 0.05
Amplitude

Amplitude

0.2 0

0 −0.05

−0.2 −0.1
0 50 100 150 200 0 50 100 150 200
Sample Sample

Figure 10.13: Multivariable MPC of the Wood-Berry column.

10.2.3 U SING THE MPC TOOLBOX IN S IMULINK

It is also easy to implement MPC controllers in S IMULINK. Fig. 10.14 shows an MPC controller
from the jMPC toolbox connected to an unstable, non-minimum phase plant
s−1
G(s) = (10.12)
(s − 2)(s + 1)
which is difficult to control and has been used in various benchmark control studies, [104].

double (s−1) double

u
mdist (s−2)(s+1)
Plant Scope
delu
double
setp
ym
Signal
Generator yp xm

Discrete MPC Controller

Figure 10.14: The MPC block in the S IMULINK environment.

To use the MPC controller block in S IMULINK, we first must design an MPC controller in the
same way as we did in section 10.2.2. Once we have designed the controller, we can simply
10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 485

insert it into the discrete MPC controller block and connect it to our plant. However in this case
instead of using the true plant inside the model-based MPC controller, we will inadvertently use
the slightly different model
0.95(s − 1)
Ĝ = (10.13)
(s − 2.1)(s + 1)
thereby introducing some model/plant mismatch.

Model = jSS(zpk([1],[2.1 -1],0.95));% Mismatch: Compare Eqn. 10.12 with Eqn. 10.13.
Ts = 0.1; Model = c2d(Model,Ts); % Discretize Model

4 Np = 40; Nc = 5; % Prediction & control Horizons

con.u = [-5 5 0.3]; % Input & output constraints
con.y = [-2 2];
uwt = 1; ywt = 0.5; % input & output weightings
Kest = dlqe(Model);
9

T = 800; setp = square(0.2*Ts*[0:T]',50); % Setpoint

setp(1:10,:) = 0;
Model.x0 = zeros(2,1); % Initial values

14 MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest); % Build the MPC controller

The last line created an MPC controller object called MPC1 which we can place in the MPC
controller block in the S IMULINK diagram in Fig. 10.14. Note that the true plant, Eqn. 10.12 is
never actually used in the MPC controller. The results of the subsequent S IMULINK simulation in
Fig. 10.15 show that we hit, but do not exceed, the input rate constraint during the transients.

0
y

−1

−2
4

0
u

−2

−4
0.5
∆u

0
Figure 10.15: MPC of a SISO non-minimum
−0.5 phase unstable plant with model/plant
0 10 20 30 40 50 60 70
time mismatch from Fig. 10.14.

10.2.4 MPC CONTROL OF AN UNSTABLE 3 DEGREE OF FREEDOM HELICOPTER

The 3-DOF helicopter manufactured by Quanser1 shown in Fig. 10.16 is a popular test rig for
testing multivariable controllers, [55]. The helicopter can be modelled using 6 states,
def  
xT = ǫ p λ ǫ̇ ṗ λ̇
1 www.quanser.com
486 CHAPTER 10. PREDICTIVE CONTROL

which are the angles (measured in radians) of the elevation, ǫ, the pitch, p, and the travel, λ, from
a nominal zero position, and their corresponding rates of change. The physical limitations of
the elevation angle are about ±50◦, while the pitch is limited to about ±100◦ . The travel angle
is unlimited due to the slip rings on the central shaft. These state constraints provide a good
incentive to use MPC control.

The angles are directly measured via a rotary encoder, but the time derivatives, (ǫ̇, ṗ, λ̇) must be
estimated either using a state estimator, or some sort of carefully designed differentiator. The two
control inputs are the two independent voltages applied to the two motors driving the front and
def
back rotors, uT = [Vf , Vb ].

Figure 10.16: A 3 degree of free-

dom helicopter manufactured by
Quanser.

The nonlinear equations of motion for the helicopter are as follows:

 
K f la Tg
ǫ̈ = (Vf + Vb ) cos(p) − (10.14)
Je Je
 
K f lh
p̈ = (Vf − Vb ) (10.15)
Jp
 
K f la
λ̈ = − (Vf + Vb ) sin(p) (10.16)
Jt
with the model parameters either directly measurable from the equipment, or alternatively avail-
able from the Quanser documentation. Note that the sin(·) and cos(·) terms introduce both non-
linearities and cross-coupling to the model and that the double integrators for all three states
mean that this system is ‘triply’ unstable.

Fig. 10.17 shows how we would control the 3DOF helicopter using an MPC controller. The plant
block in Fig. 10.17(a) expanded in Fig. 10.17(b) contains the the dynamic equations given above,
or we could directly control the actual helicopter. The former allows us to test the controller in
a simulation mode, the latter allows us to actually control some external hardware provided we
have suitable external connections.

In order to build a MPC controller of helicopter, the model equations, Eqns (10.14)–(10.16) are
entered into M ATLAB as a normal ODE function file as shown in Listing 10.5. We will use this
nonlinear model as the base for the MPC controller design.

Listing 10.5: Nonlinear dynamics of a helicopter

10.2. A MODEL PREDICTIVE CONTROL TOOLBOX 487

(a) MPC control of a Helicopter in S IMULINK

(b) The internals of the nonlinear Helicopter plant above following Eqns 10.14–10.16.

Figure 10.17: MPC control of a helicopter.

1 function xdot = heli_nlQ(t,x,u,param)

% Nonlinear helicopter dynamics following Eqns 10.14–10.16

[Je,La,Kf,Fg,Tg,Jp,Lh,Jt,m_f,m_b,m_w,Lw,g] = param{:}; %Assign Parameters

6 xdot(1,1) = x(4); % elevation velocity

xdot(2,1) = x(5); % pitch velocity
xdot(3,1) = x(6); % rotation (travel) velocity

xdot(4,1) = (Kf*La/Je)*(u(1)+u(2))*cos(x(2))-Tg/Je; % ǫ̈, Eqn. 10.14

11 xdot(5,1) = (Kf*Lh/Jp)*(u(1)-u(2)); % p̈, Eqn. 10.15
xdot(6,1) = -(Kf*La/Jt)*(u(1)+u(2))*sin(x(2)); % λ̈, Eqn. 10.16
return

We can either use this nonlinear model as the basis of the MPC design (as shown in the listing be-
low), or we can deliberately use a different model, perhaps to test some model/plant mismatch.
488 CHAPTER 10. PREDICTIVE CONTROL

[Kf,m_h,m_w,m_f,m_b,Lh,La,Lw, ...
2 g,K_EC_T,K_EC_P,K_EC_E,Je,Jt,Jp,Fg,Tg] = setup_constants();

C = eye(3,6); % Output Matrix, C. We only measure the angles, not the rates of change
Vop = g*(Lw*m_w - La*m_f - La*m_b)/(2*La*Kf); % Operating Point

7 param = {Je,La,Kf,Fg,Tg,Jp,Lh,Jt,m_f,m_b,m_w,Lw,g}; %parameter cell array

Plant = jNL(@heli_nlQ,C,param); % See Listing 10.5 for the nonlinear ODEs
u0 = [Vop;Vop];

Now we need to linearise the plant about some operating point in order to build the linear inter-
nal model that will be used in the MPC controller.

1 Model = linearize(Plant,u0,zeros(6,1)); % Automatically formulate a linearized model

UD = Model.UserData;

Ts = 0.04; % Discretize model at Ts & add 2 samples of deadtime

set(Model,'iodelay',2*Ts);
6 Model = c2d(Model,Ts);
Model.UserData = UD;

Model = jSS(Model); %Build jSS object

Now that we have a model, we need to specify the MPC tuning options such as control and
prediction horizons, the input and output constraints, the various control weightings, and the
design of the estimator.

1 Np = 45; Nc = 5; % Prediction, Np , & control, Nc , horizons

con=[]; pitch = 30*pi/180; %Constraints

con.u = [-20 20 10;
-20 20 10];
6 con.y = [-Inf Inf; % Or could limit elevation to ±27.5◦
-pitch pitch; % Limit pitch to ±30◦
-Inf Inf]; % Or could limit rotation to ±180◦ degrees

con.slack = [Inf 5e3 Inf]; % Heavy penalty for soft constraints

11

uwt = [1 1]'; %Input & output Weighting

ywt = [10 5 15]';
%Estimator Gain
Q = 2*eye(10); R = 0.1*eye(3); Q(4,4) = 1; Q(5,5) = 1; Q(6,6) = 0.1;
16 Kest = dlqe(Model,Q,R);

Finally we are ready to construct the MPC controller. It might be useful to include descriptive
variable names and units.

opts=jMPCset('InitialU',u0,'QPMaxIter',20,...
'InputNames',{'Motor 1 Input','Motor 2 Input'},...
'InputUnits',{'V','V'},...
4 'OutputNames',{'Elevation Angle','Pitch Angle','Rotation Angle'},...
'OutputUnits',{'rad','rad','rad'});
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 489

% Build MPC controller object & place in the S IMULINK block in Fig. 10.17(a).
MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest,opts);

The control object, MPC1 in the above listing, can now be inserted into the S IMULINK MPC control
block in Fig. 10.17(a). We are now ready to test out MPC controller.

The resultant MPC control given in Fig. 10.18 shows the results of an extremely large travel set-
point change of 400◦ , over a full circle. Notice how the MPC controller does violate the (soft)
state constraints at times, but does eventually manage to bring the helicopter under control. For
comparison, a linear controller such as LQR/LQG would not be able to maintain stability for
such a large setpoint change.

10
elevation [deg]

−10

−20

−30
40

20
pitch [deg]

−20

0
travel [deg]

−200

−400

−600
20
Motor voltages

10

0
Figure 10.18: MPC control of a
−10
helicopter for an extremely large
−20 travel setpoint change of over 1 full
0 5 10 15 20 25 30 35 40
time [s] rotation.

10.2.5 F URTHER READINGS ON MPC

Model predictive control is now sufficiently mature that it has appeared in numerous texts,
[134, 175, 200], all of which give an overview of the underlying theory, and insight into the in-
dustrial practice. Industrial applications using commercial implementations such as Pavilion
Technologies’ Pavilion8, Honeywell’s RMPCT or Perceptive Engineering have been surveyed in
[164]. A commercial MPC toolbox for M ATLAB is available in [27] while freeware versions also
exist from [56] and [4].

10.3 O PTIMAL CONTROL USING LINEAR PROGRAMMING

Optimal control problems quickly demand excessive computation, which all but eliminates them
from practical real-time applications. If however our model is a linear discrete-time state-space
model, our performance objective is to minimise the sum of the absolute value, rather than the
490 CHAPTER 10. PREDICTIVE CONTROL

more normal least-squares, and we have linear constraints on the manipulated variables, then
the general constrained optimisation problem becomes a linear program. Linear programs or LPs
are much more attractive to solve than a general constrained nonlinear optimisation problem of
comparable complexity because, in theory, they can be solved in a finite number of operations,
and in practice they are far more successful at solving problems with more than 50 variables and
100 constraints on modest hardware. The theory of linear programming is well known, and if the
problem is poorly conditioned, or not ‘well-posed’, then the software will quickly recognise this,
ensuring that we do not waste time on impossible, or poorly thought out problems.

There are many computer codes available to solve linear programs, including linprog from
the O PTIMIZATION T OOLBOX. An introduction to linear programming with emphasis on the
numerical issues with M ATLAB is given in [205], and an experimental application is given in
[148] from which this section was adapted.

As in the predictive control development, we want a controlled response that minimises the error
between setpoint, r, and the actual state, say something like Eqn. 9.8
Z ∞
J = f (x, r)
0

In this application we will take the absolute value of the error, partly because this is a more robust
measure than the classical least-squares, but mainly because a squared error will introduce a non-
linearity into the performance objective, which will render the linear programming inapplicable.
We will also consider only a finite time horizon, and since the model is discrete, we will use a
simple sum in place of the integral. The optimisation problem is then to choose the set of future
manipulated variables uk over the time horizon such that the performance index
N X
X n
J = |ri − xi | (10.17)
k=0 i=1

is minimised subject to the process model. In Eqn. 10.17, n is the number of states, and N is the
number of samples in the time horizon, (typically around 10–20). If desired, Eqn. 10.17 could be
further modified by adding state weighting if some states are more important than others.

10.3.1 D EVELOPMENT OF THE LP PROBLEM

The hardest part of this optimal controller is formulating the LP problem. Once that is done, it
can be exported to any number of standard numerical LP software packages to be solved.

We will use a discrete state model for the predictive part of the controller.

xk+1 = Φxk + ∆uk (10.18)

Unwinding the recursion in Eqn. 10.18, we can calculate any subsequent state, xk given the ma-
nipulated variable history, and the stating condition, x0 , (refer Eqn. 2.81,

xk = Φk x0 + Φk−1 ∆u0 + Φk−2 ∆2 u1 + · · · + ∆uk−1 (10.19)

If rk is the setpoint at sample time k, then the error is

rk − xk = rk − Φk x0 − Φk−1 ∆u0 − · · · − ∆uk−1 (10.20)

We desire to minimise the sum of the absolute values of these errors, as given in Eqn. 10.17, but
the absolute value sign is difficult to incorporate in a linear programming format. We can instead
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 491

introduce two new non-negative variables,


ǫ1i := ri − xi
if ri ≥ xi (10.21)
ǫ2i := 0

otherwise

ǫ1i := 0
if xi ≥ ri (10.22)
ǫ2i := xi − ri

Note that we could write the above four relations without the IF statement as something like:

e1 = abs(r-x)*round(sign(r-x)/2 + 0.5)
e2 = abs(x-r)*round(sign(x-r)/2 + 0.5)

for scalar x and r.

Using definitions Eqns 10.21–10.22, the absolute value is

|ri − xi | = ǫ1i + ǫ2i (10.23)

and the error

ri − xi = ǫ1i − ǫ2i (10.24)

Both relations are valid at all times because only one of either ǫ1 or ǫ2 is non-zero for any particular
state at any particular time.

Re-writing the objective function, Eqn. 10.17 in terms of the new variables using Eqn. 10.23 gives
the linear relation without the troublesome absolute values.

N X
X n
J = (ǫ1i + ǫ2i )k (10.25)
k=0 i=1

The penalty paid is that we have doubled the number of decision variables, thereby doubling the
optimisation space.

Eqn. 10.25 is a linear objective function suitable for a linear program. The constraints arise from
the model dynamics which must be obeyed. Substituting Eqn. 10.24 into Eqn. 10.20 gives N
blocks of equality constraints, each with n equations. To make things clearer, the first few blocks
are written explicitly out below.

ǫ11 − ǫ21 +∆u0 = r1 − Φx0

| {z }
r1 −x1
ǫ12 − ǫ22 +Φ2 ∆u0 + ∆u1 = r2 − Φ2 x0
| {z }
r2 −x2
..
.
ǫ − ǫ +Φk−1 ∆u0 + Φk−2 ∆u1 + · · · + ∆uk−1 = rk − Φk x0
| 1k {z 2k}
rk −xk
492 CHAPTER 10. PREDICTIVE CONTROL

Naturally this is best written in a matrix form.

 
u0
 
 u 
 1 
 
 . 
 . 
 . 
 
u
 N −1 
 
∆ 0 · · · 0 I −I    
  ǫ11 
 r1 − Φx0

 Φ∆ ∆ ··· 0 I −I

   r2 − Φ2 x0 
   ǫ21  
= ..

 (10.26)

 .
.. .
.. . .. 0 . ..

   . 
  ǫ12 
ΦN x 0
  rN −
ΦN −1 ∆ ΦN −2 ∆ · · · ∆
 
I −I  
 ǫ22 
 
 
 .. 
 . 
 
 ǫ 
 1N 
 
ǫ2N

The dimensions of the partitions in this large constraint matrix are given in Fig. 10.19 where
N is the number of future sample horizons, n is the number of states, and m is the number of
manipulated variables.

m n
z }| { z }| {
 
n  ∆ 0 ··· 0 I −I n
 ···
 Φ∆ ∆ 0 I −I
Nn  .. .. .. .. ..
. . . . .

 N −1 N −2 ··· I −I
Φ ∆ Φ ∆ ∆
| {z }| {z }
Nm 2N n

Figure 10.19: LP constraint matrix dimensions

The constraint matrix in Fig. 10.19 is a N n × (N m + 2N n) matrix, the decision variable vector has
N m manipulated variables and 2N n auxiliary variables comprising of (N m + 2N n) × 1 vector,
and the right-hand side is a N n × 1 vector.

In general applications it is necessary for the control variable, u, to be unrestricted in sign, which
is counter to the formulation required in most linear programming packages. To achieve this we,
must introduce two further new variables, both non-negative, for each manipulated variable.

u := s − t, s, t ≥ 0 (10.27)

If the unrestricted u > 0, then t = 0, otherwise u is negative and s = 0. This substitution modifies
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 493

the A matrix in the constraint equation, Eqn. 10.26, to

 
∆ 0 ··· 0 −∆ 0 ··· 0 I −I
 
 
 Φ∆ ∆ ··· 0 −Φ∆ −∆ ··· 0 I −I 
 

 .. .. .. .. .. .. .. 

 . . . 0 . . . 0 . 
N −1 N −2 N −1 N −2
Φ ∆Φ ∆ · · · ∆ −Φ ∆ −Φ ∆ · · · −∆ I −I

and the decision variables are now

h i⊤
.. ..
s0 s1 · · · sN −1 . t0 t1 · · · tN −1 . ǫ11 · · · ǫ2N

Using this ordering of decision variables, the objective function, Eqn. 10.25, is the sum of the ǫs.
In linear programming, the objective function is written as J = c⊤ x, where c is termed the cost
vector, and x is the decision variable vector to be optimised. In our case the cost vector is given
as " #⊤
0 0 ··· 0 1 1 1 1 ··· 1
c= | {z } | {z } (10.28)
2N m 2N n

Note that the manipulated variables do not explicitly enter the objective function. Naturally the
ones in Eqn. 10.28 could be replaced by different positive individual time and state weightings,
if so desired.

This completes the formal development of the LP as far as the unconstrained optimum controller
is concerned. However since we are already solving a constrained optimisation problem, there
is little overhead in adding extra constraints such as controller upper and lower bounds to the
problem. In practice the manipulated space is limited, (a control valve can only open so far, an
electric drive is limited in top speed), so we are interested in constraints of the form

ulower ≤ u ≤ uupper (10.29)

These constraints can be appended to the other constraint equations.

S IMPLIFICATIONS WHEN USING THE OPTIMISATION TOOLBOX

When using linprog.m from the O PTIMIZATION T OOLBOX, certain simplifications are possible
to the standard LP problem form. These simplifications reduce the problem size, and therefore
also reduce the solution time. Since the algorithm does not make the non-negative constraint
assumption, we do not need to introduce the variable transformation given by Eqn. 10.27, and
suffer the associated growth of the constraint matrix. This is not the case in general, if we were to
use other LP routines such as described in [205] or [79], then we would need to use non-negative
variables.

Most LP programs assume the constraints are given as Ax ≤ b, i.e. less-than inequalities rather
than equalities as in Eqn. 10.26. While the conversion is trivial, it again increases the number of
constraint equations. However the routine linprog conveniently allows the user to specify both
equality and inequality constraints.

The third simplification is to exploit how linprog handles constant upper and lower bounds
such as needed in Eqn. 10.29. These are not added to the general constraint matrix, Ax ≤ b, but
are supplied separately to the routine. All these modifications reduce the computation required
to solve the problem.
494 CHAPTER 10. PREDICTIVE CONTROL

O PTIMAL CONTROL USING LINEAR PROGRAMMING

Listing 10.6 demonstrates a finite time-horizon optimal controller with a randomly generated
stable multiple-input/multiple output (MIMO) model. The optimisation problem is solved using
the linear program linprog from the O PTIMISATION T OOLBOX.

Listing 10.6: Optimal control using linear programming

n = 2;% # of states
2 m = 2;% # of manipulated variables
[Ac,Bc,C,D]=rmodel(n,1,m); % Some arbitrary random model for testing

Gc = ss(Ac,Bc,C,D);
dt = min(1. ./abs(eig(Ac)))/0.5; % sparse sample time reasonable guess
7 G = c2d(Gc,dt,'zoh');
G.d = zeros(size(G.d)); % clear this
if rank(ctrb(G)) 6= n
error('System not controllable')
end % if
12

N = 20; % future optimisation horizon

x0 = randn(n,1); % random start point
r = [ones(floor(N/2),1)*randn(1,n); ...
ones(ceil(N/2),1)*randn(1,n)]; % setpoint change
17

%% start construction of the LP matrix

Phin = G.b; Aeq = G.b; % better for when m>1
for i=2:N
Phin = G.a*Phin;
22 Aeq = [Aeq; Phin]; % rough method
end

Z = zeros(size(G.b)); % padding
for i=2:N
27 Aeq = [Aeq, [Z;Aeq(1:(N-1)*n,(i-2)*m+1:(i-1)*m) ]];
end % for i

% weighting Could use blkdiag here (but messy)

W = eye(n); % change if want different state weighting
32 A2 = zeros(N*n,2*N*n); % initialise
for i=1:N
A2((i-1)*n+1:i*n,2*(i-1)*n+1:2*i*n) = [W, -W];
end % for
Aeq = [A2,Aeq]; % put +ve variables before general +/- variables
37

% Now right-hand side .....

U = zeros(N+1,m); % unforced, therefore zero input (drop start term)
[yuf,t,xuf]=lsim(G,U,[],x0); % simulate unforced response
xuf(1,:) = []; % drop start term since dlsim repeats it.
42 b = r - xuf; % error
b = b'; beq = b(:); % columnerise

% Now objective function

[nr,nc] = size(Aeq); % current size
47 c = zeros(1,nc); % row vector for the moment
c(1:2*N*n) = 1; % unity weights for the moment !
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 495

% Set up A*x < b (or -Ax > 0)

b = zeros(2*N*n,1); % All #s are positive
52 A = zeros(2*N*n,nc)
A(:,1:2*N*n) = -eye(2*N*n);

tic; % now ready to solve LP ..................

ulow = -13; uhigh = 14; % upper/lower constraints
57 xlower = [zeros(2*N*n,1); ulow*ones(N*m,1)];
xupper = [1e6*ones(2*N*n,1); uhigh*ones(N*m,1)];
xsol=linprog(c',A,b,Aeq,beq,xlower,xupper); % first 2Nn are +ve
toc

62 % extract solution, if found

U = xsol(2*N*n+1:end); % must reshape if m6=1
U = reshape(U,m,length(U)/m)'; % MI system

[y,t,xopt] = lsim(G,U,[],x0); % Now test system response

67 %k = [0:length(xopt)-1]';
[ts,rs] = stairs(t,r); [ts,Us] = stairs(t,U);

subplot(2,1,2);
plot(ts,Us,t,ones(size(t))*[ulow, uhigh],'--');
72 h = line(ts,Us); set(h,'LineWidth',2);
%set(gca,'YLim',[-4,4.5])
xlabel('sample #'); ylabel('Manipulated')

% Sub-sample output response

77 nsub = 500; % # of sub-samples total (approx 500)
ts2 = ts(:,1) + [1:length(ts)]'/1e10; ts2(1) = 0; % destroy monotonicity
t2 = linspace(0,t(end),nsub)';
U2 = interp1(ts2,Us,t2);

82 [y,t2,xoptsc] = lsim(Gc,U2,t2,x0);
subplot(2,1,1);
plot(ts(:,1),rs,':',t2,xoptsc);
h = line(t,xopt); set(h,'Marker','.','MarkerSize',24,'LineStyle','none');
ylabel('state'); title('LP optimal control')

You should run the simulation a number of times with different models to see the behaviour of
the optimal controller. One ideal 2-input/2 state case is given in Fig. 10.20. Here the states reach
the setpoint in two sample times, and remain there. The manipulated variables remain inside the
allowable upper (4) and lower (−3) bounds. It is clear that the trajectory is optimal, since the sum
of the absolute error at the sample points is zero. One cannot do better than that.

In another case, the manipulated bounds may be active and consequently the setpoint may not
be reached in the minimum time, and our objective function will not reach zero. In Fig. 10.21,
both steady-states are reachable, but owing to the upper and lower bounds on u, they cannot be
met in just two sample times. In other words we have hit the manipulated variable constraints
during the transients.

Our system does not need to be square, Fig. 10.22 shows the results for a 3-state, 2 input problem.
In this case, one state is left ‘floating’. Some small inter-sample ripple is also evident in this case.

Note that since this is a predictive controller, and it knows about future setpoint changes, it
anticipates the changes and starts moving the states before the setpoint change. Fig. 10.23 shows
the case for a 2 × 3 system where owing to the active upper and lower bounds on u, the controller
knows that it cannot reach the setpoint in one sample time, hence it chooses a trajectory that will
496 CHAPTER 10. PREDICTIVE CONTROL

LP optimal control
0

−0.5
state 1

−1

−1.5
2

1
state 2

−1

−2
20
Manipulated

−20
0 5 10 15 20
sample #

Figure 10.20: LP optimal control with no active manipulated variable bounds. Note the “perfect”
controlled performance. We hit the setpoint in one sample time, and stay there. One cannot
do much better than that within the limitations of discrete control. The plot was generated by
Listing 10.6.

LP optimal control
2

1
state

−1

−2
0 1 2 3 4 5 6 7 8 9

4
Figure 10.21: LP optimal control
2
with active manipulated variable
Manipulated

constraints. During transients, the 0

manipulated variables hit the up-
−2
per constraints, so compromising the
controlled performance for a short −4
0 1 2 3 4 5 6 7 8 9
period. sample time
10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING 497

LP optimal control
2

0
state

−1

−2

−3
0 1 2 3 4 5 6 7 8 9

2
Manipulated

0
Figure 10.22: Non-square LP opti-
−2
mal control with 3 states and 2 in-
−4 puts. Note the small inter-sample
0 1 2 3 4 5 6 7 8 9
sample time ripple.

minimise the absolute error and in doing so, starts the states moving before the setpoint.

LP optimal control
1

0.5

0
state

−0.5

−1

−1.5

−2
0 1 2 3 4 5 6 7 8 9

2
Manipulated

−2
Figure 10.23: LP optimal
−4 control showing acausal
0 1 2 3 4 5 6 7 8 9
sample time behaviour
498 CHAPTER 10. PREDICTIVE CONTROL
C HAPTER 11

E XPERT SYSTEMS AND NEURAL

NETWORKS

Doctor Whirtzle’s controversial “talking brain in a jar”

is revealed for the shameless fraud that it was.
©1995 David Farley

11.1 E XPERT SYSTEMS

An expert system is a computer programme that uses knowledge and techniques in much the
same way as a human expert would to solve a wide variety of problems. Typically expert systems
are used for types of problems that cannot be successfully solved using traditional procedural
type programmes. Expert systems do not use numeric data like conventional programmes, but
they manipulate symbolic data instead.

499
500 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

Human experts have the following characteristics that
expert systems try to duplicate:
• A narrow field of expertise (domain)
• Usually are formal about decision making and
can justify their reasoning.
◦ Use all possible senses (touch, hearing, sight,
smell etc.)
• They do not do large quantitative number
crunching in their head,– at least initially.
• Can deal with vague and uncertain or even ir-
relevant input data.
Expert systems differ from human experts in that:
• They do not die, retire or go to sleep, and will
diligently work 24 hours a day.
• They are easily duplicated – One just copies the
programme and supplies additional hardware
if required.
◦ Generally cheap to maintain once the original
development is done.
• Expert systems find it difficult to use “common
sense” or to realise if a situation is sufficiently
novel that it should fail gracefully.

• Learn from past experience.

The concept of Expert systems, or machines that can reason in a manner similar to humans, seems
to worry some people. One strong argument against the algorithmic view of the human brain
was put forward by Roger Penrose in the popular, but detailed work The Emperor’s New Mind.

11.1.1 W HERE ARE THEY USED ?

Expert systems originally grew out of the field of artificial intelligence (AI) and is now used
in game playing, theorem proving, pattern recognition and diagnosis, [82]. Languages used in
AI are typically different from conventional languages such as FORTRAN, Cobol, Pascal etc.
AI languages deal with symbolic notation rather than numeric data and include LISP, Prolog,
POP and OOPs. However, some applications in AI use conventional languages with extensions
(Object Oriented Pascal, C++).

Artificial intelligence is sometimes difficult to distinguish from monkey business.

Conventional programmes (rather than expert systems) are used for applications such as payroll
calculations, finite element analysis, 3D flight simulators etc. Here the problem can be rigorously
defined and an accurate numerical answer is required.

Some well known expert systems are:

M YCIN Medical diagnosis for bacterial infections. This has been shown, in its admittedly limited
domain, to be better than expert human doctors .

E MYCIN : (Empty M YCIN) Similar programme as the above M YCIN, but without the data for
bacterial infections. This allows other applications to use the same programme shell but in
different domains.

P ROSPECTOR : Gives advice about mining and drilling operations. This programme pointed to
a $2 billion molybdenum deposit that human experts had previously overlooked.
11.1. EXPERT SYSTEMS 501

All the above expert systems have enjoyed good success, but then they also have had hundreds
of human years effort. However in the case of P ROSPECTOR, which is invariably cited by AI
enthusiasts as a particularly good example, [103] notes that it is credited for finding only one
mineral deposit around 1982, and this was achieved using a never-released experimental version
rather than the publicly available version. Since that time, there have been no other reported
mineral deposit finds.

11.1.2 F EATURES OF AN EXPERT SYSTEM

Most expert systems have in addition to a knowledge base, a facility to explain how it arrived at
a decision, some sort of question and answer format, and an accompanying probability analysis
to address the degree of uncertainty.

In structure they have an inference engine, a knowledge base, and after time, some case specific
data.

A N INFERENCE ENGINE

This is the part of the programme that decides what knowledge is relevant and what further
questions need to be asked to arrive at a conclusion. It is the central core or kernel. There are two
different styles of inference engine:

1. Goal driven or top down

2. Data driven or bottom up

The style which is used depends on the application. A goal driven system asks questions that
work backwards from a possible goal to find pre-requisites (sub goals) until the initial conditions
required by the proposed goal are the same as the actual initial conditions.

A data driven method looks at the data and examines which rules are satisfied. The rules that
are satisfied are said to “fire” which in turn may satisfy conditions for other rules to fire. This
continues until a goal is satisfied. An example of the two types of inference engine methods are
from HAZOP (hazard and operability) studies.

• Consider causes: top down

• Consider consequences: bottom up

G OAL DRIVEN

Suppose you are an expert consultant employed by a multinational oil company which is won-
dering why their plant has just exploded. You propose (guess) that a pipe carrying octane cor-
roded at an elbow spraying flammable material around. A dialogue between an expert (you) and
an eye witness might go something like

Was there a large plume after the explosion?

... Yes
Was the centre of the explosion 50m down wind of the suspected leak?
... Well it was about 100m I suppose.
502 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

You are trying to establish sufficient evidence to support your initial hypothesis.

D ATA DRIVEN

You are designing a new plant and carry out a consequence analysis as part of a preliminary
H AZOP (Hazard and operability study). You might ask:

What would happen if pump #23 failed? Would the pressure be high enough to
rupture the vessel? Would the contents find a source of ignition within the
explosive limit?

Here you are taking the initial data and evaluating what would happen given that data. You then
match that with what really happened.

K NOWLEDGE REPRESENTATION IN EXPERT SYSTEMS

Information in the knowledge base has two forms; facts and rules. The fact base contains a
collection of fact statements. These facts may be stored in the form older(John,Paul) which
is read as “John is older than Paul”. This way of expressing facts is derived from the P ROLOG
programming language. Rules in a more engineering context could be:

saturated(steam,100C, 1 Bar) saturated(freon,-36C,1 Bar) rupture(boiler, 15

Bar)

These facts can be dynamically updated or deleted from the knowledge base as the expert system
grows and gains experience. The second type of knowledge representation is as rules. The rules
are usually of the simple I F . . . T HEN . . . E LSE type.
I F the flame colour is blue AND the flow rate is high THEN a secondary reaction is occurring.
Other types of knowledge representations are possible such as semantic nets or neural networks.
Neural networks are introduced in §11.2.

11.1.3 T HE USER INTERFACE

The user interface is the part of the programme that asks the questions and receives the answers.
Generally the users of the expert system are not computer experts and would like to communicate
in as near to natural language as possible. An example that requires a good user interface is one
where a group of doctors installed a common disease diagnosis expert system in the waiting
room of their surgery. The waiting patients could ask the Expert system, in the same way they
could ask the doctor what was wrong with them, whilst waiting for the doctor. Supposedly the
programme was not too successful, or the doctors might soon be out of a job. The user interface
can operate on at least 3 levels:

1. Solves problems for the client. (normal mode)

2. The user is an expert who is extending the knowledge base of the system.

3. The user is extracting knowledge from the programme. (Industrial espionage?)

11.1. EXPERT SYSTEMS 503

11.1.4 E XPERT SYSTEMS USED IN PROCESS CONTROL

Why is it that an experienced human operator can often satisfactorily control large complex
plants when they exhibit nonlinear time varying behaviour and where there is little or no precise
knowledge available? Examples are control of processes such as sugar crystallisation, cement
kilns, floatation separations etc. All these plants involve imprecise measurements such as sharp-
ness of crystals, colour of the flame, shape and size of bubbles, and stability of boiling. Given
that conventional controllers have difficulty and are substandard when compared to the human
operators, it seems logical to try and develop non-conventional computer based controllers that
duplicate what the human operator is doing. Here the human operator is considered an expert
and one tries to capture their way of doing things in a computer programme such as an expert
system. Fuzzy control is a further attempt to duplicate how a human makes decisions given
vague and qualitative information. Fuzzy control is quite closely related to expert systems, and
is described formally in [74] and an application to the nonlinear evaporator is given in chapters
13 and 14 of [147].

Outside regulatory control, expert systems have also been used in:

◦ Tuning industrial PID controllers, (pattern recognition)

• Monitoring equipment for faults (diagnosis)
• Analysing alarms on complex plants
◦ Designing distributed computer control (DCC) configurations
◦ Qualitative simulation
• Prediction, planning and design

A UTO - TUNERS

PID controllers are very common on chemical processing plants. However they will only work
effectively if they are well tuned. Tuning PID controllers is where one selects the optimum Gain
(Kc ), integral time τi and derivative time τd . These 3 variables will be dependent on the process
being controlled. Traditionally instrument engineers tuned the PID controllers during start-up,
but often never re-tuned them afterwards. Auto-tuners such as the EXACT controller manufac-
tured by Foxboro will re-tune the controller if the operator demands [17, pp445-446]. This means
that the controller can be frequently retained by inexperienced personnel. The method of tuning
is based on analysis of the transient response of the closed loop system to changes in setpoint
and load. It uses the methods of Ziegler-Nichols. Heuristic logic is used to detect if a proper dis-
turbance has occurred and when the peaks subsequently occur. Hence the Expert system. This
system has been widely accepted in industry.

FAULT DIAGNOSIS

Expert systems are ideal for fault diagnosis. Here one constructs a fault tree which is some type
of conditions connected by branches. Given a fault such as a high exit temperature, you can trace
back through the tree and find the cause such as a pump failure.

Unfortunately these trees can get rather large very quickly, so some heuristic rules must be em-
ployed to reduce the scope of the tree. Also most people will ignore events that have a low
probability associated with them (asteroids hitting the distillation column for example).
504 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

A LARM SYSTEM ANALYSIS

Related to fault diagnosis is alarm analysis. In a large chemical plant, there may be thousands of
variables and hundreds of controllers. If each controller has two alarms (high & low) associated
with it, then there will also be hundreds of possible alarms. Often one alarm will trigger other
alarms which in turn also trigger alarms. Consequently a single fault may generate many differ-
ent alarms making the control room display light up like a “Christmas tree”. Faced with so many
alarms, the operator may find it difficult to evaluate exactly what has happened and what should
be done to correct the situation. Experience has shown that typically operators, who otherwise
know the process better than anybody else, make poor decisions when faced with the emergency
situation of so many alarms.

Expert systems have been developed to analyse these alarms (what order, which type etc) and
to try to deduce what was the crucial initial fault that triggered all the subsequent alarms. Once
the programme has established a probable cause, it advises the operator the cause and a suitable
remedy. Programmes of this type typically have heuristic rules and plant models built into them.
To be useful however, they must generate good answers quickly (within say 5 minutes), and
reject the inevitable false alarms.

Q UALITATIVE SIMULATION (QSIM)

This combines the fields of exact mathematical modelling (simulation) and the heuristic approach
of expert systems and is described in [59].

In an emergency situation such as a major plant failure, the human operators want to be able
to predict quickly what will happen if they take some particular course of action such as open
a valve or shut down a reactor for example. They do not have time or even are interested in
exact results, just general trends. QSIM is designed to provide reasonably confident results in
the face of uncertainty without having to resort to a time consuming major plant simulation.
Essentially QSIM is a method that tries to provided knowledge of the approximate dynamics
of the system from incomplete knowledge of the process. Normally everything must be exactly
known (or estimated exactly) before any traditional equation solver or dynamic simulator such
as those discussed in chapter 3 can begin. This condition is not required for the QSIM algorithm.

The QSIM algorithm is a little like a programming language that includes operators, constraints,
relations and values. Operators include the familiar mathematical operators such as:

MULT(mass,temp,enthalpy) DERIV(enthalpy,heat_flow)

where the first entry can be read as mass∗temperature = enthalpy. Other operatives include
qualitative relations such as:

M+(temp,press) M-(press,vol)

The first qualitative relation says that pressure increases with temperature for a particular pro-
cess. Nothing is said about how much increase or does it give particular values. The second
relation says that volume decreases with pressure in some unspecified manner. These style of
relations incorporate uncertainty and cannot be duplicated in normal simulation computer pro-
grammes.

Values can equal landmark values such as −∞, 0, +∞ or somewhere in between. Values can also
be increasing or decreasing. If a temperature (in Kelvin) is decreasing, then the qualitative state
11.2. NEURAL NETWORKS 505

will be T = ((0,+∞),dec). Dynamic modelling is possible with QSIM just as with conven-
tional quantitative simulation but for QSIM the output results are now not numerical values, but
qualitative states.

D IFFICULTIES OF QSIM

• Most implementations today take more computer time than an equivalent numerical solu-
tion.

• Only very simple cases are described.

• Problems arise when two numbers close in magnitude are subtracted since now the sign of
the result is unknown! This causes big ambiguities in the result.

Problem 11.1 Tuning PID controllers in industrial conditions is somewhat of an art. Construct a
rule-based expert system to aid a control engineer to tune a PID controller.

1. Write down a list of rules that you would use in your expert system. You should incorporate
heuristics and mathematical calculations in this rule base. You may wish to add a classifier
to judge what type of response you get, (non-minimum phase, oscillatory, excessive dead
time etc).

2. Structure the rules so the most important ones are searched first. Draw a diagram of the
logic tree and show an example for a typical case. What happens if you have an open loop
unstable process, or a process that is difficult to control?

3. What sort of diagnostics and explanation facilities would you incorporate in your expert
system?

11.2 N EURAL NETWORKS

“We’ve replaced the bad cybernetic brain that went

berserk and tore people limb-from-limb with a good
one that does not.”
©1994 David Farley

Accompanying expert systems, the other popular development in the research in artificial in-
telligence with relevance for process control is Artificial Neural Networks or ANN. Artificial
neural networks are computer programs that attempt to duplicate how the brain works, albeit
506 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

at a very limited level, by simulating the highly interconnected biological neurons. The relation
between the biological brain and even the most complex computer programs is really quite weak,
and nowadays most people do not expect that current ANNs will ever convincingly duplicate a
human’s cognitive ability. However ANNs do represent a novel and refreshing way of solving
certain types of problems. Neural networks are also referred to as connectionist models, parallel
distributed models, or neuromorphic systems.

Of course, any program that claims to duplicate the brain’s working, must do at a very crude
level and this is certainly the case for ANN. Both the artificial and biological neural networks are
composed of simple elements called neurons that are joined together in a particular topology or
network. The connections between neurons are referred to as dendrites, and the inputs and out-
puts are called axons. However in the case of the artificial neural network, the size and topology
of the network is much much simpler than those even in an ant’s biological brain. We will not be
concerned further with biological neural networks, so henceforth I will simply refer to ANN as
neural networks with the artificial term implied. The neural network attempts to model complex
behaviour with a dense interconnection of simple computational elements, hence the descriptive
term connectionist models referred to above, and the obvious parallelising of the computation.

Table 11.1 compares the characteristics between expert systems and neural networks. The M AT-
LAB neural network toolbox, [60] supplies tools and gives a methodology for elegantly simulat-
ing neural networks within the matrix framework. There are many introductory texts explaining
neural networks, [156] is just one.

Table 11.1: A comparison of characteristics between expert systems and artificial neural networks

Neural Networks Expert systems

example based rule based
domain free domain specific
finds rules needs rules supplied
easy to program & maintain hard to program & maintain
needs database needs human expert
adaptive needs re-programming
time consuming generally quicker

It is clear by now, that successful advanced control for chemical processes hinges on the devel-
opment of accurate suitable models, and that these models are expensive and time consuming to
develop and verify. Neural networks are interesting because they can be made to model complex
nonlinear systems with very little attention paid to the underlying structure of the model. In this
sense, ‘one size fits all’, which has its obvious advantages when one is trying to model complex
behaviour for which no feasible mechanism is known. If ANN are to be useful for control prob-
lems we must compared their performance to more classical and traditional techniques such as
model based control, adaptive control and so forth. We should carefully look at the quality of the
model/plant matching, the efficiency of parameter use, online and offline computer usage and
ease of tuning and configuration. [143, p151] cite some additional issues such as the high degree
of parallelisation exhibited by neural networks ensures a certain fault tolerancy, and speed of op-
eration if the hardware is optimised for the task. One of the most successful areas of application
of ANN is the fusion between symbolic information such as that used by an Expert System and a
deterministic model. By combining both sources of information, one would naturally expect an
improved result.

As a final caution consider carefully the following quote by the computer scientist A. K. Dewdney
in his expose of bad science, Yes, We Have No Neurons.
11.2. NEURAL NETWORKS 507

(a) Hard limit (b) Linear with possible saturation (c) Nonlinear
output output output
✻ ✻ ✻

log-sig
✲ ✲ ✲
input input input

tanh(x)

Figure 11.1: Possible neuron activation functions.

“As computer recreations columnist for a number Of years with Scientific American
magazine, I had ample opportunity to review the literature on neural nets and may
have unwittingly helped to spark the revolution by writing more than one article on
the subject. Only now may I say what I never could in the column: The appearance
of an idea in Scientific American does not automatically make it the coming thing. The
number of people who seemed unable to spot the word “recreations” in my column
was truly frightening. Although neural nets do solve a few toy problems, their powers
of computation are so limited that I am surprised anyone takes them seriously as a
general problem solving tool. Most computer scientists who work in the field known
as computational complexity understand this very well.
Neural nets have already entered the long, slow decline predicted by Irving Langmuir
(see the end of chapter 1) for all bad science-or for that matter, technology.”

11.2.1 T HE ARCHITECTURE OF THE NEURAL NETWORK

Neural networks have three basic characteristics; their architecture or how the neurons are con-
nected to each other, the type of processing element in the individual neuron and the training
rule used to initialise the network.

T HE NEURON MODEL

The neuron is a very simple element that takes as input a value p, multiplies this by a weight w,
and then passes this weighted input to a transfer or activation function f (). There are a variety of
possible transfer functions and some are given in Fig. 11.1. The simplest transfer (or activation)
function is a binary switch;- either it fires or it does not (see Fig. 11.1 (a)). This is at the heart of the
perceptron, but it is limited as to what behaviour it can model. A more flexible transfer function
is a linear transfer function, although in practice the output may be constrained by limits (see
Fig. 11.1b) but it is the nonlinear activation functions, (see Fig. 11.1c) that give the neural network
the most power to model interesting and nonlinear relations.

The two most common nonlinear activation functions are the hyperbolic tangent,

y = tanh(x), (11.1)
508 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

and the log sigmoidal,

1
y= . (11.2)
1 + exp(−x)
Both of these are sigmoidal or “S” shaped curves that are bounded both above and below and
are continuously differentiable.

M ULTIPLE NEURONS IN LAYERS

There are two obvious extensions to the single neuron example given above. First we can deal
with multiple inputs to the single neuron. In this case, the inputs are individually scaled by a
weight, w, and then summed to form a single input to the activation function in the same manner
as before as shown in Fig. 11.2.

Suppose we have N inputs and corresponding weights, then the output of the neuron y is
N
!
X
y=f wi pi + w0 (11.3)
i=1

where w0 is a bias weight. Bias weights have an implied input of 1 and are used to enable the
neuron model to give a nonzero output for a zero input. This adds flexibility to the neuron model.

synaptic connections

 ✲w1





 nonlinear function
✲w2



s P
 si ✲ ✲ output, yi

③

inputs ✿


 ✲ .. ✸✒



 . activation function f (·)





✲wL
single neuron

1 ✲w0

bias
weighted
summer

Figure 11.2: A single neural processing unit with multiple inputs

If we duplicate the single neuron given in Fig. 11.2, we can build up a one layer feed-forward
neural network. Fig. 11.3 shows the topology for a single layer neural network with L inputs and
N outputs or neurons. The weight wij is the weight from the jth input to the ith neuron.

Given this multivariable structure, a convenient computational molecule employed by the Neu-
ral Network toolbox is to use a vector/matrix version of Eqn. 11.3,

y = f (Wx) (11.4)
11.2. NEURAL NETWORKS 509

w11 ✲1 ✲ y1
 x1 

 w21 ✯
✼ 


 


 


 

 w12 
❥
 
✲2 ✲ y2

 


 x2 

L inputs ❃ N outputs


 wN 2 



 .. .. 

 


 . w1L . 



 


 w2L 

 ⑦
✲
✇N ✲ yN

xL
wN L

Figure 11.3: Single layer feedforward neural network

where y is a N × 1 vector of outputs, x is a L vector of inputs, and W is a N × L matrix of weights,

 
w11 w21 · · · w1L
 w21 w22 · · · w2L 
 
W= . .. .. .. 
 .. . . . 
wN 1 wN 1 · · · wN L

The neural network is comprised of two mappings, the Wx linear transformation, and the non-
linear f (·).

The second extension is to feed the output of one neuron into the input of another. This layering
of neurons forms network part of the neural network as shown in Fig. 11.4. Organising the
neurons in layers allows the technique far more flexibility to model things than before. Most
neural networks in practice have sigmoidal processing elements, are arranged in layers with at
least one center or hidden layer and have at least one bias input.

input output
✲ ✲
✯
s ✕
✸ ❥
✲ ✕ ✲
✸
s❯
✯ ❯
s
✲ ✲

hidden layer

✲ information flow ✲

Figure 11.4: A 3 layer fully interconnected feedforward neural network

Layering the neurons gives a great deal of flexibility in what can be approximated by the net-
work. In fact one can prove using the Stone-Weierstrass theorem that just a two layer network
510 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

is sufficient to model any continuous function. Unfortunately this theorem does not tell us how
many neurons we need, and it should be mentioned that simple polynomials or other orthogonal
expansions can also approximate to arbitrary accuracy any continuous function, although again
the theorem is silent regarding how many coefficients are required. (See for example [105, p289].)
So two main design problems remain; how many layers we should use, and how many nodes in
each layer. Naturally we wish to minimise the number of parameters to search for, but still obtain
a reasonable result.

R ECURRENT NETWORKS

In the network structure given in Fig. 11.4, the information travels from left to right, analogous
to an open loop process producing an output given an input. These are called statical networks.
However for the analysis of dynamic models, it is natural to use a neural network structure with a
feedback through a unit time delay duplicating the nonlinear discrete time model, xk+1 = f (xk ).
Fig. 11.5 shows a neural network suitable for an autonomous (no input) dynamic system. The
outputs are delayed one sample time and then fed back to the inputs.

unit delays

z −1 ✛

z −1 ✛

z −1 ✛

x1 (0) ✲ ✲y1
✯ 1
✒
❥
✲ ✲y2
x2 (0)
✯ 2
❥❘
xN (0) ✲ ✲yN
N
| {z } | {z }
initial feedforward part
conditions

Figure 11.5: Single layer network with feedback

Other possibilities include using a two layered network with a delays between the layers, or
feeding only some of the states back. Adding provision for a control signal is simply a matter of
adding another node in the first layer joined to the input at time k.

T RAINING OR REGRESSING THE WEIGHTS

Once we have decided on a network structure, (number of hidden layers, number of neurons
in each layer etc), and have decided on the type of neuron, (sigmoid, linear etc), then we must
calculate appropriate values for the weights between the neurons. In essence this is a nonlinear
11.2. NEURAL NETWORKS 511

regression problem, where inputs and known outputs are presented to the network in a super-
vised learning mode. The weights are adjusted by the optimiser so that the network duplicates
the correct answers. Once the network can do this, and the weights have converged, it may be
able to extrapolate and produce reasonable answers to inputs that it had not seen previously.

11.2.2 C URVE FITTING USING NEURAL NETWORKS

Fitting functions to data is a classic engineering pastime. When faced with some data say such as
given in Fig. 11.6, our first, and often hardest choice, is to decide what is an appropriate under-
lying structure for the function that stands some chance of going through the data. This is where
strategies that make minimal demands on any a priori information such as neural networks have
advantages.

−1

−2 Figure 11.6: An input/output data series ◦

and initial trial fit (solid) to be approximated
−3 data using a neural network. The initial fit was
Initial NN fit obtained by randomly setting the neural-
−4 network weights.
−2 −1 0 1 2

We are going to make use of the neural network characteristic of ‘one size fits all’ mentioned
above, in that we will not actually pay much attention to the structure of the model to which
we will regress. To speed up the prototyping, I am going to use the Neural Network toolbox for
M ATLAB, but constructing this example from scratch is also very simple. I will also closely follow
the example for curve fitting from the NN toolbox manual.

Traditionally to select a model structure for curve fitting, I by looking at the data, would choose
some sort of growing exponential sinusoidal family with say 3–6 parameters and then use a non-
linear regression package. However in this case we avoid the problem specific model selection
step and just to be different, will use neural networks. It this advantage, that of eliminating the
requirement to select a model structure that is why neural networks are currently so popular.

We will search for the values of the weights inside the neural network (also termed parameters)
such that our neural network approximates the unknown function. We will use a crude, but
effective optimisation technique to search for these weights called back propagation. Searching for
the optimal neural network weights is termed training the network, but in reality it is simply a
nonlinear regression problem.

H OW MANY NEURONS AND IN WHAT STRUCTURE ?

It turns out that we can, at least in principle, fit any function with a finite neural network with
only two layers, the first a nonlinear layer, and the output, a linear layer. Unfortunately, this
512 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

result does not tell us how many neurons are required for a given accuracy specification, so it
is not much use in practice. Hence we will use a three layer network (two hidden which are
nonlinear, and one linear output). Current thinking is that if the network is too large, it will not
be able to generalise very well, despite the good performance on the training runs.

I will use the hyperbolic tangent activation function, (see Fig. 11.1c), for the nonlinear neurons.
We are already constrained by the number of input and output neurons, that is given by the
problem. In our case we have 1 input (the independent variable) and 1 output (the dependent
variable). However we do have to choose how many neurons we want in our two hidden layers.
Too many neurons take too much memory, and can over-fit the data. Too few may never fit the
data to the desired level of accuracy. I will choose 10 neurons for each of the hidden layers.

The M ATLAB code in Listing 11.1 first generates some arbitrary data to be fitted and initialises
the neural network with random weights.

Listing 11.1: Generate some arbitrary data to be used for subsequent fitting
x = linspace(-2,2,20); % Note: row vector for input
F = @(x) 1./(1+2*x.ˆ2).*cos(3*(x-2)); % create "unknown" Target function to fit!
y=F(x);
4

xi = linspace(x(1), x(end),50); % Interpolating input vector validate

% Now create Network Skeleton

[R,Q] = size(x); % dimension of input
9 S1 = 10; S2 = 10;% # of hidden nodes (2 layers)
[S3,Q] = size(y);% dim of output

% Initialize weights and biases.

[W1,B1] = rands(S1,R); % random weights to start
14 [W2,B2] = rands(S2,S1); [W3,B3] = rands(S3,S2);

% try out our initial guess

plotfa(x,y,xi,purelin(W3*tansig(W2*tansig(W1*xi,B1),B2),B3));

Clearly when using random weights, any fit will be purely fortuitous. More likely as a first
estimate, we will see a fit similar to the attempt in Fig. 11.6. However hopefully once we have
optimised the network weights the fit will improve.

Now we are ready to train the network using the back-propagation algorithm. We specify a max-
imum of 2000 iterations, and a final error allowance of 10−3 . There is scope for many more tuning
parameters just for the optimiser, and these are specified in the following code. The algorithm
should work without them, but it will be much less efficient. We will use the supplied trainbpx
m-file to perform this optimisation.

disp_freq = 500; % plotting frequency

max_epoch = 20000; % maximum allowable iterations
3 err_goal = 1.0e-3;% termination error
lr = 0.01; % learning rate
TP = [disp_freq max_epoch err_goal lr];

% adaptive learning rate add-ons

8 lr_incr=1.01; % learn rate increase
lr_decr=0.99; % decrease
momentum = 0.9;% low pass filter (sort of)
err_r = 1.04;
11.2. NEURAL NETWORKS 513

TPx = [TP,lr_incr, lr_decr, momentum, err_r];

13

% Now do the training by accelerated back-prop.

[W1x,B1x,W2x,B2x,W3x,B3x,epoch,errors] = ...
trainbpx(W1,B1,'tansig',W2,B2,'tansig',W3,B3,'purelin',x,y,TPx);

18 % finally when finished plot the best approximated curve

plotfa(x,y,xi,purelin(W3x*tansig(W2x*tansig(W1x*xi,B1x),B2x),B3x));

If you have tried the above, you will have learned the first important feature of neural networks
— they are not very computationally efficient! The result of my best fit is given in Fig. 11.7. The fit
is good considering we are using 140 parameters! Just for comparison, the dashed line in Fig. 11.7
shows the fit only using 3 neurons in the hidden layers or 9 fitted parameters.

1.2
data
1 NN fit
10,10
0.8 NN
2,1

0.6

0.4 Figure 11.7: An unknown input/output func-

tion approximated using a neural network
0.2
with two hidden layers of 10 neurons each
0 with 140 fitted parameters, (solid) and (2,1)
−0.2
neurons in the two hidden layers with 9 pa-
rameters, (dashed). Both optimisers were
−0.4 constrained to 2000 iterations.
−2 −1 0 1 2

Actually one must be weary of comparisons such as these since in both cases the optimiser “timed
out” by exceeding the maximum allowed number of iterations rather than actually finding the
appropriate minimum. On the other hand however, I do not have time to waste! For the case
with 5 neurons in the hidden layers, a good fit is obtained with 16,000 iterations.

P REDICTING THE TIDES

The high and low tides in Auckland habour (36◦ 50′ 39′′ S, 174◦ 46′ 6′′ E) is shown in Fig. 11.8(a)
over 4 months.1 Note that this time series only shows the extreme values of the data, and it is
evident from the plot that Auckland experiences what is known as semi-diurnal tides but with a
noticeable mixed component. That is, Auckland has two low tides and two high tides each day,
but one of the tides is larger than the other.

Recreational boaters need to know the tides, and it is interesting to try and fit a function to this
data. In fact this is quite a challenge because there are really 4 curves in Fig. 11.8(a). Suppose to
make the problem easier, we just take one of the high tides, and use this as a test case for fitting a
neural network function.

Fig. 11.8(b) shows the result of fitting a large neural-network with 2 layers of 30 neurons in each.
Note that the last month of data was withheld from the neural network and gives an indication
of the quality if the extrapolation that we can expect. Not too good I’m afraid.

Problem 11.2 1. Repeat the curve fitting using a neural network with only one hidden layer.
Investigate the effect of varying the number of hidden neurons.
1 This data was obtained from http://www.niwascience.co.nz/services/tides.
514 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS

1.5
Height of High & Low tides

0.5

0 Mean Sea Level

−0.5

−1

−1.5

−2
May Jun Jul Aug Sep Oct
Date

(a) The high and low tides in Auckland harbour over 4 months.

2
data
Final NN fit
1.8
Height of High tides

1.6

1.4

1.2

1
Validation
0.8
May Jun Jul Aug Sep Oct
Date

(b) High tide predictions. Note that the September data was not used for the curve fitting, and is
used to validate the fit.

Figure 11.8: Fitting a neural network function to tide data

2. Fit the data given in Fig. 11.6 using the conventional nonlinear least squares optimisation.
(The optimisation toolbox procedure fminu may be useful.) You will need to select an
appropriate model and suitable initial parameters. How does this compare to the neural
network approach in terms of computational effort required, user understanding, robust-
ness etc?

S OME PRELIMINARY CAUTIONS

A very short readable criticism of the whole neural network idea is given by Ponton, [162]. Pon-
ton makes the point that there is nothing particularly special about neural networks, and that
often fitting a known nonlinear model will probably work just as well, if not better. Ponton also
asserts that a neural network function does not generally correspond to physical reality, and in
general the network output cannot be extrapolated beyond the domain used to train (or in other
words, fit) the network parameters. Ponton advises that one can use a neural network when
nothing is known about a particular relationship except that it is nonlinear. In many engineering
applications however, something is known (or at least partially known), and that this informa-
tion is useful in the model. This approach is known as using the a priori knowledge available.
Ponton ends his short critique of neural networks with the following two statements found from
experience:

• A simpler arbitrary function can probably be found to perform just as well as a ANN, and
• a model incorporating almost any engineering or scientific knowledge will do much better.
11.3. SUMMARY 515

In a similar manner, the September 1990 issue of B YTE magazine published a dialogue from a
discussion of experts in the field of artificial intelligence. B YTE asked these experts “Why have
we failed to come up with real AI?” The replies ranged from It’s an extremely hard problem (Danny
Hillis) to AI started to work on robotics and expert systems, which were very easy in comparison to
really understanding intelligent behaviour and learning at the early stages (Nicholas Negroponte) to the
dramatic The computation power required . . . is enormous (Federico Faggin), to At a very fundamental
level, we do not know how the brain works (Dick Shaffer). Remember some of these concerns when
next investigating the claims of new expert systems and artificial intelligence.

11.3 S UMMARY

Expert systems have been around for about 25 years and have been popular for about the last 10
years. They represent a major departure from the traditional way of using computers as that of
a precise exact machine solving engineering problems exactly to solving more vague problems
that require concepts that humans have traditionally been better then computers at. With this
new technology has arrived new programming languages and new computer architectures.

Expert systems in the process control field is still in its infancy, and has had mixed success. Expert
systems are often applied to control problems that are traditionally control ‘hard’ such as the
control of cement kilns and pulp digestors. These applications are characterised by complicated
physiochemical phenomena with scarce reliable online measurements. Notwithstanding, human
operators do manage to operate these plants, so one predicts there is scope for an expert system.
Usually the expert system is configured to operate in an advisory mode which the operator can
override. Installation costs are high hence they tend to be used only for projects that show a large
potential pay back.

Recently there has been some published disillusioned analysis of expert systems, and many re-
searches abandoned the field moving to the now more popular neural network field. Whether
this currently popular ship will also capsize in the near future makes interesting speculation.

Like expert systems, neural networks have also been around for 25 years or more, and is only now
gaining interest in the general engineering community. The neural network is the quintessential
black-box model, that hopefully after it has been trained on some data, can make useful predic-
tions with similar, but slightly different data. The training of the network, or alternatively the
regression of the network weights is a time consuming, expensive nonlinear optimisation prob-
lem, but needs to be done only once offline. The actual operation of the neural network once
trained, is trivial.
516 CHAPTER 11. EXPERT SYSTEMS AND NEURAL NETWORKS
A PPENDIX A

L IST OF SYMBOLS

S YMBOL D ESCRIPTION U NITS Pu ultimate period s

t time s ωm phase margin s
T, ∆t sample time s γ, α actual PID parameters -
d, θ dead time samples, s
τ time constant s
P period s
K Gain
ζ shape factor - Φ signal power
δ(·) Dirac delta function - W weight kg
f (t) function of time - M model -
f ∗ (T ) sampled function f (T ) - S system -
L(·) Laplace transform - θ vector of parameters -
Z(·) z transform - ǫ error vector -
s Laplace variable - t1−α/2 t-statistic -
z z transform variable - cov(·) co-variance
q −1 backward shift operator - C past input/output data -
ω radial velocity radians/s K gain matrix -
f frequency Hz P co-variance matrix -
V(x) scalar Lyapunov function - I identity matrix -
P, Q positive definite matrices - λ forgetting factor -
J Jacobian matrix - f factor ≈4
num numerator - E{·} expected value
den denominator - ∆t sample time s
M mass kg x state vector n×1
F material flow kg/s u input vector m×1
ρ density kg/m3 y output vector r×1
h height m d disturbance vector p×1
A cross sectional area m2 A system matrix n×n
∆P pressure differential kPa B control matrix n×m
T temperature K,◦ C D disturbance matrix n×p
H Enthalpy kJ C measurement matrix r×n
I current mA Φ transition matrix n×n
cp heat capacity kJ/kg.K ∆ discrete control matrix n×m
∆T temperature differential K Γ discrete disturbance matrix n × p
Q, q heat loss (gain) kJ/s θ time delay s
J moment of inertia - β # whole samples delay -
θ angle radians z augmented state vector -
Λ relative gain matrix - Co controllability matrix -
u manipulated variable - Ob observability matrix -
ǫ error - K controller gain -
L observer gain -
τi integral time s
τd derivative time s J,j performance index $
Kc proportional gain - Q, R weighting matrices -
Ku ultimate gain - x̄ mean of x -
ωu ultimate frequency rad/s σx standard deviation of x -

517
518 APPENDIX A. LIST OF SYMBOLS

λ, ϕ Lagrange multipliers
H Hamiltonian
φ termination criteria

S UBSCRIPTS D ESCRIPTION
N Nyquist
u,l upper & lower
ss steady-state

S UPERSCRIPTS D ESCRIPTION
x̂ estimate (of x)
x̄ mean (of x)
x⋆ desired value (of x)
A PPENDIX B

U SEFUL UTILITY FUNCTIONS IN

M ATLAB

The routine in Listing B.1 adds two or more polynomials of possibly differing lengths. The resultant
polynomial order is the same as the maximum order of the input polynomials. Note that this routine
pads with zeros to the left so that all the polynomials are of the same size. It assumed (but not checked)
that the polynomials are row vectors in descending order.

Listing B.1: Polynomial addition.

1 function R = polyadd(A,varargin)
% Adds two or more row vectors of possibly differing lengths.
% R(x) = A(x) + B(x) + C(x) + · · · .
np = length(varargin); nmax = length(A);
for i=1:np % find maximum order in the input argument list
6 nmax = max(nmax,length(varargin{i}));
end % if
R = [zeros(1,nmax-length(A)),A];
for i=1:np
varargin{i} = [zeros(1,nmax - length(varargin{i}),1), varargin{i}];
11 R = R+varargin{i};
end % if
return

This routine is useful for checking the results of a Diophantine solution, A(x)R(x) + B(x)S(x) = T (x).

Tc = polyadd(conv(A,R),conv(B,S))

The routine in Listing B.2 convolves or multiplies two or more polynomials. Unlike polynomial addi-
tion, we do not need to ensure that all the polynomials are of the same length.

Listing B.2: Multiple convolution.

function y = mconv(a,varargin)
2 % Multiple convolution of polynomial vectors
% R(x) = A(x)B(x)C(x) · · · .

% Would be faster to do “divide & conquer”, and lots of smaller first.

if nargin < 2

519
520 APPENDIX B. USEFUL UTILITY FUNCTIONS IN MATLAB

7 error('Need two or more arguments')

end % if

y = conv(a,varargin{1}); % do the first pair

for i=2:nargin-1 % now do any remaining
12 y = conv(y,varargin{i});
end return

Listing B.3 strips the leading zeros from a row polynomial, and optionally returns the number of zeros
stripped which may, depending on the application, be the deadtime.

Listing B.3: Strip leading zeros from a polynomial.

function [Br,i] = stripleadzeros(B)
2 % Strip the leading zeros from a (vector) polynomial & return deadtime (# of leading zeros)
% [Br,i] = stripleadzeros(B)

for i=0:length(B)-1
if B(i+1) % if Bi 6= 0
7 Br = B(i+1:end);
return % jump out of routine
end
end % for
return
A PPENDIX C

T RANSFORM PAIRS

time function Laplace transform z-transform

f (t) F (s) Z
1 z
unit step 1
s z−1
impulse δ(t) 1
1 Tz
unit ramp t
s2 (z − 1)2
2 T 2 z(z + 1)
t2
s3 2(z − 1)3
n!
tn n+1
s
1 z
e−at
s+a z − eaT
1 T zeaT
te−at
(s + a)2 (z − eaT )2
s
(1 − at)e−at
(s + a)2
a z sin(aT )
sin(at)
s2 + a2 z 2 − 2z cos(aT ) + 1
s
cos(at)
s2 + a2
−bt
e − e−at 1
a−b (s + a)(s + b)
−bt a
e sin(−at)
(s + b)2 + a2
s+b
e−bt cos(−at)
(s + b)2 + a2
1 ae−bt − be−at 1
+
ab ab(b − a) s(s + a)(s + b)

521
522 APPENDIX C. TRANSFORM PAIRS
A PPENDIX D

A COMPARISON OF M APLE AND

M U PAD

Much of the functionality of the symbolic manipulator M APLE, www.maplesoft.com is duplicated

in the freeware alternative M U PAD, www.sciface.com. Below both packages are demonstrated in
parallel.

D.1 PARTIAL FRACTIONS

Suppose we want to decompose

s3 + s2 + 1
G(s) =
s2 (s2 − 1)
into partial fractions.

MuPad uses the partfrac command.

Maple uses the parfrac option in the convert
command. partfrac((sˆ3+sˆ2+1)/(sˆ2*(sˆ2-1)));

> G := (sˆ3+sˆ2+1)/(sˆ2*(sˆ2-1)): 3 1 1
> convert(G,parfrac,s); --------- - -- - ---------
2 (s - 1) 2 2 (s + 1)
1 3 1 1 1 s
− 2
+ −
s 2s−1 2s+1 Note the different spelling!

D.2 I NTEGRAL TRANSFORMS

First we must load the relevant package to do integral transforms.

MuPad
Maple
In this case Laplace, Fourier, z and Mellin trans-
In this case for integral transforms.
forms.
>with(inttrans):
export(transform);

523
524 APPENDIX D. A COMPARISON OF MAPLE AND MUPAD

Now we find a Laplace transform of a common function, L{sin(at)}

laplace(sin(a*t),t,s);
laplace(sin(a*t),t,s);
a
a -------
a2 + s2 2 2
a + s
The inverse Laplace transform of L−1 {K/(s2 + s + 1)}
ilaplace(K/(sˆ2+s+1),s,t);

invlaplace(K/(sˆ2+s+1),s,t); / 1/2 \
1/2 / t \ | t 3 |
−1 √  ((− 12 + 21 √−3)t) √  2 K 3 exp| - - | sin| ------ |
− e((− 2 − 2 −3)t)
1 1
K −3 e \ 2 / \ 2 /
3
---------------------------------
3
We can try a Laplace transform the ‘hard way’ following the mathematical definition (without using the
integral transform packages), Z ∞
def
L{sin(at)} = sin(at) e−st dt
0

f := cos(a*t):
int(f*exp(-s*t),t=0..infinity);

However M APLE returns a solution accompa-

nied with a warning:
Definite integration: Can’t
MuPad, however, has no resevations.
determine if the integral is
convergent. Need to know the f := cos(a*t):
sign of --> x Will now try int(f*exp(-s*t),t=0..infinity);
indefinite integration and
then take limits.
s
-------
−se−st cos(at) + ae−st sin(at) + s 2 2
lim
t→∞ s2 + a2 a + s
> assume(s,positive);
> simplify(%);

s
s2 + a2

D.3 D IFFERENTIAL EQUATIONS

Find the solution to the linear ordinary differential equation with initial conditions
dx dx dx
3 + + x = 4e−t , x(0) = 1, (0) = 2
dt dt dt

In M APLE we use the dsolve command to solve differential equations.

D.4. VECTORS AND MATRICES 525

>ode2 := 3*diff(x(t),t,t) + diff(x(t),t)+x(t)=4*exp(-t);

   
∂2 ∂
ode2 := 3 x(t) + x(t) + x(t) = 4e−t
∂t2 ∂t

>dsolve({ode2,x(0)=1,D(x)(0)=2},x(t));

   
4 1 1 1√ 59 − 1 t 1√ √
x(t) = e−t − e− 6 t cos 11t + e 6 sin 11t 11
3 3 6 33 6

In M U PAD we construct an ‘ode’ object group and use the normal solve command.

>eqn := ode({3*x’’(t) + x’(t) + x(t)=4*exp(-t),x(0)=1, D(x)(0)=2},x(t));

ode({D(x)(0) = 2, x(0) = 1, x(t) + diff(x(t), t) + 3 diff(x(t), t, t) =

4 exp(-t)}, x(t))

>solve(eqn);

{ / 1/2 \ / 1/2 \ }
{ / t \ | t 11 | 1/2 / t \ | t 11 | }
{ exp| - - | cos| ------- | 59 11 exp| - - | sin| ------- | }
{ 4 \ 6 / \ 6 / \ 6 / \ 6 / }
{ -------- - ------------------------- + ---------------------------------- }
{ 3 exp(t) 3 33 }

D.4 V ECTORS AND MATRICES

To manipulate vectors and matrices in M U PAD, we must load the linear algebra package.

info(linalg):

which returns a list of functions available in the linear algebra package. We start by defining the domain
of the matrices we are dealing with. In our case we will stick with the default and call that domain type
M:

M := Dom::Matrix();
Dom::Matrix(Dom::ExpressionField(id, iszero))

Now we are ready to input a test matrix, A, and perform some arithmetic operations such as addition,
inverse, matrix exponentiation.

A:=M([[1,3],[4, 5]]);

+- -+
526 APPENDIX D. A COMPARISON OF MAPLE AND MUPAD

| 1, 3 |
| |
| 4, 5 |
+- -+

A+A;
+- -+
| 2, 6 |
| |
| 8, 10 |
+- -+

exp(A,T);

+- -+
| 3 exp(-T) exp(7 T) 3 exp(-T) 3 exp(7 T) |
| --------- + --------, - --------- + ---------- |
| 4 4 8 8 |
| |
| exp(-T) exp(7 T) exp(-T) 3 exp(7 T) |
| - ------- + -------- , ------- + ---------- |
| 2 2 4 4 |
+- -+

Of course we could involve variables in the matrix, and perform further manipulations

A:=M([[1,3],[4, x]]);
+- -+
| 1, 3 |
| |
| 4, x |
+- -+
A+Aˆ(-1);

+- -+
| 12 3 |
| ------ + 2, - ------ + 3 |
| x - 12 x - 12 |
| |
| 4 1 |
| - ------ + 4 , x + ------ |
| x - 12 x - 12 |
+- -+

linalg::eigenValues(A);

{ 2 1/2 2 1/2 }
{ x (x - 2 x + 49) x (x - 2 x + 49) }
{ - + ------------------ + 1/2, - - ------------------ + 1/2 }
{ 2 2 2 2 }
A PPENDIX E

U SEFUL TEST MODELS

The following are a collection of multivariable models in state-space format which can be used to test
control and estimation algorithms.

E.1 A FORCED CIRCULATION EVAPORATOR

The linearised forced circulation evaporator developed in [147, chapter 2] and mentioned in §3.3.2 in
continuous state-space form, ẋ = Ax + Bu, is

 
0 0.10455 0.37935
A= 0 −0.1 0  (E.1)
0 −1.034 · 10−2 −5.4738 · 10−2
−3.636 · 10−2
 
−0.1 −0.37266 0 0.36676 0.38605 0 0
B =  −0.1 0 0 0 0.1 0.1 0 0 
0 3.6914 · 10−2 −7.5272 · 10−3 3.6302 · 10−2 3.2268 · 10−3 0 3.5972 · 10−3 1.7785 · 10−2
(E.2)

with state and input variables defined as in Table 3.3,

 T
x= L2 P2 x2 (E.3)
h iT
u= .. (E.4)
F2 P100 F200 . F3 F1 x1 T1 T200

In this case we assume that the concentration cannot be measured online, so the output equation is

 
1 0 0
y= x (E.5)
0 1 0

527
528 APPENDIX E. USEFUL TEST MODELS

E.2 A IRCRAFT MODEL

The following model is of an aircraft reported in [62, p31]. The state model is
   
0 0 1.132 0 −1 0 0 0
 0 −0.0538 −0.1712 0 0.0705   −0.12 1 0 
   
A=  0 0 0 1 0 
, B =  0 0 0 (E.6)
 

 0 0.0485 0 −0.8556 −1.013   4.419 0 −1.665 
0 −0.2909 0 1.0532 −0.6859 1.575 0 −0.0732
 
1 0 0 0 0
C= 0 1 0 0 0  (E.7)
0 0 1 0 0

where the aircraft states & control inputs are defined as:

state description units measured

x1 altitude m yes
x2 forward velocity m/s yes
x3 pitch angle degrees yes
x4 pitch rate, x˙3 deg/s no
x5 vertical speed m/s no
u1 spoiler angle deg/10
u2 forward acceleration m/s2
u3 elevator angle deg

Note that we only measure altitude, velocity and pitch angle, we do not measure the rate states.

A reasonable initial condition is

 ⊤
x0 = 10 100 −15 1 25 .
B IBLIOGRAPHY
[1] J. Abate and P.P. Valkó. Multi-precision Laplace
transform inversion. International Journal for Nu-
merical Methods in Engineering, 60:979–993, 2004.
32, 33
[2] James L. Adams. Flying Buttresses, Entropy, and O-
rings: The World of an Engineer. Harvard University
Press, 1991. 4
[3] Advantech Co. Ltd. PC-Multilab Card User’s Man-
ual, 1992. 14
[4] Johan Åkesson. MPCtools: A toolbox for simu-
lation of MPC controllers in Matlab. Technical re-
port, Lund Institute of Technology, Lund, Sweden,
January 2006. 489
[5] Brian D.O. Anderson and John B. Moore. Opti-
mal Filtering. Information and System Sciences.
Prentice-Hall, 1979. 460, 462
[6] Jim Anderson, Ton Backx, Joost Van Loon, and
Myke King. Getting the most from Advanced Pro-
cess Control. Chemical Engineering, pages 78–89,
March 1994. 5
[7] Anon. Experience with the X-15 Adaptive Flight
Control System. Technical Report NASA TN D-
6208, NASA Flight Research Centre, Edwards,
California, Washington, D.C., March 1971. 321
[8] K-E Årzén. Realisation of expert system based feed-
back control. PhD thesis, Department of Automatic
control, Lund Institute of Technology, Lund, Swe-
den, 1987. 148
[9] K. J. Åström. Computer Control of a Paper Ma-
chine – an Application of Linear Stochastic Con-
trol Theory. IBM Journal of Research and Develop-
ment, 11(4):389–405, 1967. 326
[10] K. J. Astrom. Maximum likelihood and prediction
error methods. Automatica, 16:551–574, 1980. 283
[11] K. J. Åström. Ziegler–Nichols Auto-tuners. Tech-
nical Report LUTFD2/(TFRT–3167) /01–025/,
Lund University, Lund, Sweden, 1982. 169, 173,
465
[12] K. J. Åström and B. Wittenmark. On Self Tuning
Regulators. Automatica, 9:185–199, 1973. 326
529
[13] Karl J. Åström and Tore Hägglund. Advanced PID
Control. ISA, Research Triangle Park, NC, USA,
2006. 132, 181
[14] Karl-Johan Åström. Introduction to stochastic con-
trol theory. Academic press, 1970. 326
[15] Karl-Johan Åström and Tore Hägglund. Automatic
tuning of PID controllers. Instrument Society of
America, Research triangle park, NC, 1988. 132,
141, 155, 166, 167, 325
[16] Karl-Johan Åström and Tore Hägglund. Revisiting
the Ziegler-Nichols step response method for PID
control. Journal of Process Control, 14:635–650, 2004.
152
[17] Karl-Johan Åström and Björn Wittenmark. Adap-
tive control. Addision–Wesley, 1989. 5, 310, 312,
503
[18] Karl-Johan Åström and Björn Wittenmark. Adap-
tive Control. Addision–Wesley, 2 edition, 1995. 243,
332, 354
[19] Karl-Johan Åström and Björn Wittenmark.
Computer-Controlled Systems: Theory and Design.
Prentice–Hall, 3 edition, 1997. 61, 141, 363, 374
[20] K.J. Åström and R.D. Bell. Drum-boiler dynamics.
Automatica, 36(3):363–378, 2000. 181
[21] K.J. Aström and B. Wittenmark. Adaptive Control.
Addison–Wesley, 1989. 330
[22] K.J. Åström and B. Wittenmark. Computer Con-
trolled Systems: Theory and Design. Prentice–Hall,
2 edition, 1990. 314, 348, 350, 358
[23] Yonathan Bard. Nonlinear parameter estimation.
Academic Press, 1974. 277
[24] R.A. Bartlett, A. Wächter, and L. T. Biegler. Ac-
tive set vs. interior point strategies for model pre-
dictive control. In Proceedings of the American Con-
trol Conference, pages 4229–4233, Chicago, Illinois,
USA, 2000. 475
[25] B. Bellingham and F.P. Lees. The detection of
malfunction using a process control computer:
A Kalman filtering technique for general control
loops. Trans. IChemE, 55:253–265, 1977. 459
530
[26] Alberto Bemporad, Manfred Morari, and
N. Lawrence Ricker. Model Predictive Control
Toolbox. The MathWorks Inc., 2006. 472
[27] Alberto Bemporad, Manfred Morari, and
N. Lawrence Ricker. Model predictive con-
trol toolbox 3. Technical report, The Mathworks,
2009. 475, 489
[28] B. Wayne Bequette. Nonlinear control of chemical
processes: A review. Ind. Eng. Chem. Res., 30:1391–
1413, 1991. 386
[29] A. Besharati Rad, Wai Lun Lo, and K.M. Tsang.
Self-tuning pid controller using newton-raphson
search method. Industrial Electronics, IEEE Trans-
actions on, 44(5):717–725, Oct 1997. 155, 256
[30] Torsten Bohlin. A grey-box process identification
tool: Theory and practice. Technical Report IR-
S3-REG-0103, Department of Signals Systems &
Sensors, Royal Institute of technology, SE-100 44
Stockholm, Sweden, August 2001. 86
[31] C. Bohn and D.P. Atherton. An Analysis Pack-
age Comparing PID Anti-Windup Strategies. IEEE
Control Systems, 15:34–40, 1995. 141
[32] G.E.P. Box and G.M. Jenkins. Time Series Analysis:
Forecasting and Control. Holden–Day, 1970. 236,
243, 311, 312
[33] M. Braun. Differential Equations and their Applica-
tions. Springer–Verlag, 1975. 86, 119
[34] John W. Brewer. Kronecker products and matrix
calculus in system theory. IEEE Transactions on Cir-
cuits and Systems, 25(9):772–781, September 1978.
81
[35] E.H. Bristol. On a new measure of interaction for
multivariable process control. IEEE transactions on
Automatic Control, AC–11(133), 1966. 103
[36] Jens Trample Broch. Principles of Experimental Fre-
quency Analysis. Elsevier applied science, London
& New York, 1990. 220
[37] Robert Grover Brown and Patrick Y.C. Hwang. In-
troduction to Random Signals and Applied Kalman Fil-
tering. John Wiley & Sons, 2 edition, 1992. 439
[38] Arthur E. Bryson and Yu-Chi Ho. Applied Optimal
Control. Ginn & Co., Waltham, Mass., 1969. 407
[39] James L. Buchanan and Peter R. Turner. Numerical
Methods and Analysis. McGraw–Hill, 1992. 120
[40] F. Buchholt and M. Kümmel. A Multivariable Self-
tuning Regulator to Control a Double Effect Evap-
orator. Automatica, 17(5):737–743, 1981. 94
APPENDIX E. USEFUL TEST MODELS
[41] Richard L. Burden and J. Douglas Faires. Numeri-
cal Analysis. PWS Publishing, 5 edition, 1993. 120
[42] C. Sidney Burrus, James H. McClellan, Alan V.
Oppenheim, Thomas W. Parks, Ronald W. Schafer,
and Hans W. Schuessler. Computer-Based Exercises
for Signal Processing using Matlab. Prentice–Hall,
1994. 17, 204
[43] B. Carnahan and J.O. Wilkes. Numerical solution
of differential equations - an overview. In R. Mah
and W.O. Seider, editors, FOCAPD ’80, Henniker,
New Hampshire, 1980. 86
[44] C.C. Hang and K. J. Åström and Q.G. Wang. Re-
lay feedback auto-tuning of process controllers –
A tutorial overview. J. Process Control, 12:143–162,
2002. 177
[45] B. Chachuat. Nonlinear and dynamic op-
timization: From theory to practice. Tech-
nical report, Automatic Control Laboratory,
EPFL, Switzerland, 2007. Available from
infoscience.epfl.ch/record/111939/files/.
407, 415
[46] Edward R. Champion. Numerical Methods for En-
gineering Applications. Marcel Dekker, Inc., 1993.
120
[47] Cheng-Liang Chen. A simple method of on-line
identification and controller tuning. 35(12):2037–
2039, 1989. 159
[48] Cheng-Liang Chen. A closed loop reaction–curve
method for controller tuning. Chemical Engineering
Communications, 104:87–100, 1991. 159, 166
[49] Robert Chote. Why the Chancellor is always
wrong. New Scientist, page 26, 31 October 1992.
91
[50] C.K. Chui and G. Chen. Kalman Filtering with Real-
Time Applications. Springer–Verlag, 1987. 439, 460,
465
[51] C.K. Chui and G. Chen. Linear Systems and Optimal
Control. Springer–Verlag, 1989. 407, 408, 416
[52] David Clarke. Advances in Model-Based Predictive
Control. Oxford University Press, 1994. 185, 322,
469
[53] David W. Clarke. PID Algorithms and their Com-
puter Implementation. Trans. Inst. Measurement
and Control, 6(6):305–316, Oct-Dec 1984. 132
[54] J.D. Cryer. Time series analysis. PWS, 1986. 243
[55] Jonanthan Currie, Arrian Prince-Pike, and
David I. Wilson. A cost effective high-speed
auto-coded embedded model predictive con-
troller. International Journal of Intelligent Systems
E.2. AIRCRAFT MODEL
Technologies and Applications, 13(1/2):37–55, 2014.
485
[56] Jonathan Currie and David I. Wilson. A Model
Predictive Control toolbox intended for rapid pro-
totyping. In Tim Molteno, editor, 16th Electron-
ics New Zealand Conference (ENZCon 2009), pages
7–12, Dunedin, New Zealand, 18–20 November
2009. 482, 489
[57] Jonathan Currie and David I. Wilson. Lightweight
Model Predictive Control intended for embedded
applications. In 9th International Symposium on Dy-
namics and Control of Process Systems (DYCOPS),
pages 264–269, Leuven, Belgium, 5–7 July 2010.
482
[58] James B. Dabney and Thomas L. Harman. Master-
ing Simulink 4. Prentice–Hall, 2001. 403
[59] D.T. Dalle-Molle, T. Edgar, and B.J. Kuipers. Qual-
itative modelling of physical systems. In 3rd Int.
Symposium on Process Systems Engineering, pages
169–174, August 1988. 504
[60] Howard Demuth and Mark Beale. Neural Network
Toolbox User’s Guide. The MathWorks Inc., June
1992. 506
[61] J.J. DiStefano, A.R. Stubberud, and I.J. Williams.
Feedback and Control Systems. Schaum’s Outline Se-
ries, McGraw-Hill, 1990. 71
[62] Peter Dorato, Chaouki Abdallah, and Vito Cerone.
Linear-quadratic Control: An Introduction. Prentice–
Hall, 1995. 419, 466, 528
[63] Richard Dorf. Modern control systems. Addison–
Wesley, 5 edition, 1989. 65
[64] M. Drouin, H. Abou-Kandil, and M. Mariton.
Control of complex systems. Methods and technology.
Plenum Press, 223 Spring St, New York NY 10013,
1 edition, 1991. 5, 407
[65] D.B. Ender. Control analysis and optimisation.
Technical report, Instrument Soc. America, Tech-
mation, Tempe, Arizona 85282, 1990. 149
[66] D. Grant Fisher and Dale E. Seborg. Multivariable
Computer Control. North-Holland, 1976. 94
[67] Samuel C. Florman. The Existential Pleasures of En-
gineering. St. Martins Press, New York, 1976. 4
[68] T. R. Fortescue, L. S. Kershenbaum, and B. E.
Ydstie. Implementation of self-tuning regula-
tors with variable forgetting factors. Automatica,
17(6):831–835, 1981. 299
531
[69] Lloyd D. Fosdick, Elizabeth R. Jessup, and Car-
olyn J.C. Schauble. Elements of Matlab. High
Performance Scientific Computing, University
of Colorado, January 1995. Available from
cs.colorado.edu/pub/HPSC. 3, 4
[70] A. S. Foss. Critique of chemical process con-
trol theory. American Inst. Chemical Engineers J,
19(2):209–214, 1973. 45
[71] G.F. Franklin and J.D. Powell. Digital control of dy-
namic systems, pages 131–183. Addison–Wesley,
1980. 42, 68, 373
[72] G.F. Franklin, J.D. Powell, and M.L. Workman.
Digital Control of Dynamic Systems. Addison–
Wesley, 3 edition, 1998. 277
[73] C.E. Garcia, D.M. Prett, and M. Morari. Model pre-
dictive control: Theory and practice — A survey.
Automatica, 25(3):335–348, 1989. 469
[74] H.P. Geering. Introduction to Fuzzy Control. Tech-
nical Report IMRT-Bericht Nr. 24, Eidgenössische
Technische Hochschule, February 1992. 503
[75] Paul Geladi and Bruce R. Kowalski. Partial Least-
Squares Regression: A Tutorial. Analytica Chimica
Acta, 185:1–17, 1986. 111
[76] S.F. Goldmann and R.W.H. Sargent. Applications
of linear estimation theory to chemical processes:
A feasibility study. Chemical Engineering Science,
26:1535–1553, 1971. 449
[77] Graham C. Goodwin, Stefan F. Graebe, and
Mario E. Salgado. Control System Design. Prentice–
Hall, 2001. 125
[78] Felix Gross, Dag Ravemark, Peter Terwiesch, and
David Wilson. The Dynamics of Chocolate in Beer:
The Kinetic Behaviour of theobroma cacao Paste in
a CH3 CH2 OH–H2 O–CO2 Solution. Journal of Irre-
producible Results, 37(4):2–4, 1992. 238
[79] Tore K. Gustafsson and Pertti M. Mäkilä.
L1 Identification Toolbox for Matlab. Åbo
Akademi, Finland, August 1994. From
ftp.abo.fi/pub/rt/l1idtools. 493
[80] I. Gustavsson. Survey of Applications of Identifi-
cation in Chemical and Physical Processes. Auto-
matica, 11:3–24, 1975. 236
[81] Juergen Hahn, Thomas Edison, and Thomas F.
Edgar. A note on stability analysis using bode
plots. Chemical Engineering Education, 35(3):208–
211, 2001. 73
[82] F. Hayes-Roth. Building Expert Systems. Addison–
Wesley, Reading Massachusetts, 1983. 500
532
[83] A. Helbig, W. Marquardt, and F. Allgower. Non-
linearity measures: definition, computaiton and
applications. Journal of Process Control, pages 113–
123, 2000. 129
[84] M.A. Henson and D.E. Seborg. Input-output Lin-
earization of General Nonlinear Processes. Amer-
ican Inst. Chemical Engineers J, 36(11):1753–1757,
1990. 121, 386
[85] Michael A. Henson. Nonlinear model predictive
control: current status and future directions. Com-
puters in Chemical Engineering, 23(2):187–202, 1998.
471
[86] Michael A. Henson and Dale E. Seborg. Nonlinear
Process Control. Prentice Hall, Saddle River, New
Jersey, 1997. 92
[87] David R. Hill. Experiments in Computational Matrix
Algebra. The Random House, 1988. 54
[88] David M. Himmelblau. Process Analysis by Statis-
tical Methods. John Wiley & Sons, 1970. 116, 259
[89] Roger A. Horn and Charles R. Johnson. Topics
in Matrix Analysis. Cambridge University Press,
1991. 81
[90] Morten Hovd and Sigurd Skogestad. Pairing Cri-
teria for Unstable Plants. In AIChE Annual Meet-
ing, page Paper 149i, St. Louis, Nov 1993. 357
[91] P. J. Huber. Robust regression: Asymptotics, con-
jectures, and Monte Carlo. Ann. Math. Statist.,
1(5):799–821, 1973. 283
[92] Enso Ikonen and Kaddour Najim. Advanced Pro-
cess Identification and Control. Marcel Dekker, 2002.
265
[93] Vinay K. Ingle and John G. Proakis. Digital Signal
Processing using Matlab V.4. PWS Publishing Com-
pany, 1997. 204
[94] Rolf Isermann. Paramater adaptive control
algorithms—A tutorial. Automatica, 18(5):513–528,
1982. 312
[95] M.L. James, G. M. Smith, and J. C. Wolford. Ap-
plied Numerical Methods for Digital Computation
with Fortran and CSMP. Harper & Row, 2 edition,
1977. 120
[96] R.K. Jaspan and J. Coull. Trajectory Optimization
Techniques in Chemical Reaction Engineering II:
Comparison of Methods. American Inst. Chemical
Engineers J, 18(4):867–869, July 1972. 410
[97] Andrew H. Jazwinski. Stochastic Processes and Fil-
tering Theory. Academic Press, 111 Fifth Avenue,
New York, 1970. 439, 460
APPENDIX E. USEFUL TEST MODELS
[98] A. Johnson. LQG Applications in the Process In-
dustries. Chemical Engineering Science, 48(16):2829–
2838, 1993. 407
[99] Michael A. Johnson and Mohammad H. Moradi.
PID COntrol: New Identification and Design Meth-
ods. Springer–Verlag, London, UK, 2005. 132, 134
[100] Arthur Jutan and E. S. Rodriguez II. Extension of
a New method for On-Line Controller Tuning. The
Canadian Journal of Chemical Engineering, 62:802–
807, December 1984. 159
[101] Thomas Kailath. Linear Systems. Prentice–Hall,
1980. 407
[102] Paul G. Kaminski, Arthur E. Bryson, and Stan-
ley F. Schmidt. Discrete Square Root Filtering: A
survey of current techniques. IEEE Trans. Auto-
matic Control, AC-16:727–735, 1971. 460
[103] Solomon S. Katz. Emulating the PROSPECTOR Ex-
pert System with a Raster GIS. Computers & Geo-
sciences, 17(7):1033–1050, 1991. 501
[104] L.H. Keel and S.P. Bhattacharyya. Robust, fragile,
or optimal? Automatic Control, IEEE Transactions
on, 42(8):1098 –1105, Aug 1997. 484
[105] David Kincaid and Ward Cheney. Numerical Anal-
ysis. Mathematics of Scientific Computing. Brooks/-
Cole, 1991. 120, 510
[106] Costas Kravaris and Jeffery C. Kantor. Geometric
Methods for Nonlinear Process Control. 1 Back-
ground. Ind. Eng. Chem. Res., 29:2295–2310, 1990.
386
[107] Costas Kravaris and Jeffery C. Kantor. Geomet-
ric Methods for Nonlinear Process Control. 2 Con-
troller Synthesis. Ind. Eng. Chem. Res., 29:2310–
2323, 1990. 386
[108] Erwin Kreyszig. Advanced Engineering Mathemat-
ics. John Wiley & Sons, 7 edition, 1993. 220
[109] B. Kristiansson and B. Lennartson. Robust and op-
timal tuning of PI and PID controllers. IEE Proc.-
Control Theory Applications, 149(1):17–25, January
2002. 190
[110] R. Kulhavý and M.B. Zarrop. On a general concept
of forgetting. Int. J. Control, 3(58):905–924, 1993.
315
[111] Benjamin C. Kuo. Automatic Control Systems.
Prentice–Hall, 6 edition, 1991. 40
[112] Benjamin C. Kuo. Automatic Control Systems.
Prentice–Hall, 7 edition, 1995. 435, 436
E.2. AIRCRAFT MODEL
[113] I. D. Landau. Identification in closed loop: a pow-
erful design tool (better design models, simpler
controllers). Control Engineering Practice, 9:51–65,
2001. 312
[114] Yoan D. Landau. Adaptive Control: The model refer-
ence approach. Marcel Dekker, 1979. 301
[115] Leon Lapidus and John H. Seinfeld. Numerical So-
lution of Ordinary Differential Equations. Academic
Press, 1971. 86
[116] Alan J. Laub. Matrix Analysis for Scientists and En-
gineers. Society for Industrial and Applied Mathe-
matics, 2004. 81
[117] P.L. Lee and G.R. Sullivan. Generic Model Con-
trol (GMC). Computers in Chemical Engineering,
12(6):573–580, 1988. 378
[118] T.H. Lee, Q.G. Wang, and K.K. Tan. Knowledge-
based process identification from relay feedback.
Journal of Process Control, 5(6):387–397, 1995. 177,
179
[119] S. Lees and J.O. Hougen. Determination of pneu-
matic controller characteristics by frequency re-
sponse. Ind. Eng. Chem., 48:1064, 1956. 247
[120] J. R. Leigh. Control Theory: A guided tour. IEE Con-
trol Series 45. Peter Peregrinus Ltd., 1992. 73
[121] A. Leva, C. Cox, and A. Ruano. Hands-on
pid autotuning: a guide to better utilisation. [138] Manfred Morari. Three critiques of process control
In Marek Zaremba, editor, IFAC Professional
Brief, chapter 3. IFAC, 2002. Available from
www.ifac-control.org/publications/pbriefs/PB_Final_LevaCoxRuano.pdf.
240
[122] Frank L. Lewis. Optimal Estimation. John Wiley &
Sons, 1986. 285, 446, 461
[123] Frank L. Lewis and Vassilis L. Syrmos. Optimal
Control. John Wiley & Sons, 2 edition, 1995. 466
[124] R.J. Litchfield, K.S. Campbell, and A. Locke. The
application of several Kalman filters to the con-
trol of a real chemical reactor. Trans. IChemE,
57(2):113–120, April 1979. 382
[125] Lennart Ljung. System Identification: Theory for the
User. Prentice–Hall, 1987. 236, 244, 271, 283, 314
[126] Lennart Ljung. System Identification Toolbox, for use
with Matlab. The MathWorks Inc., May 1991. 236,
310, 313
[127] Lennart Ljung. System Identification: Theory for the
User. Prentice–Hall, 2 edition, 1999. 280
[128] Lennart Ljung. Perspectives on system identifica-
tion. Annual Reviews in Control, 34(1):1–12, 2010.
237
533
[129] Lennart Ljung and Torkel Glad. Modeling of Dy-
namic Systems. Prentice–Hall, 1994. 236
[130] Charles F. Van Loan. The ubiquitous kronecker
product. Journal of Computational and Applied Math-
ematics, 123:85–100, 2000. 81
[131] W. L. Luyben. Process Modeling, Simulation, and
Control for Chemical Engineers. McGraw-Hill, 2 edi-
tion, 1990. 86
[132] W.L. Luyben. Process modelling simulation and con-
trol for chemical engineers. McGraw-Hill, 1973. 98
[133] Paul A. Lynn and Wolfgang Fuerst. Introductory
Digital Singnal Processing with Computer Applica-
tions. John Wiley & Sons, 2 edition, 1994. 208, 220
[134] J. M. Maciejowski. Predictive Control with Con-
straints. Prentice–Hall, 2002. 489
[135] Sven Erik Mattsson. On Modelling and Differen-
tial/Algebraic Systems. Simulation, pages 24–32,
January 1989. 129
[136] Cleve Moler and John Little. Matlab User’s Guide.
The MathWorks Inc., December 1993. 1
[137] B. De Moor, P. De Gersem, B. De Schutter, and
W. Favoreel. DAISY: A database for identification
of systems. Journal A, 38(3):4–5, 1997. 316
revisited a decade later. In D.M. Prett, C.E. Gracı́a,
and B.L. Ramaker, editors, The Shell Process control
workshop, pages 309–321. Butterworths, 1987. 45,
366
[139] Manfred Morari. Process control theory: Reflec-
tions on the past and goals for the next decade.
In D.M. Prett, C.E. Gracı́a, and B.L. Ramaker, edi-
tors, The second Shell Process control workshop, pages
469–488. Butterworths, December 1988. 95
[140] Manfred Morari. Some Control Problems in the
Process Industries. In H.L. Trentelman and J.C.
Willems, editors, Essays on Control: Perspectives in
the Theory and its applications. Birkhäuser, 1993. 141
[141] Manfred Morari. Model Predictive Control: Multi-
variable Control Technique of Choice in the 1990s.
In David Clarke, editor, Advances in Model-Based
Predictive Control, pages 22–37. Oxford University
Press, 1994. 45, 221
[142] Manfred Morari and Evanghelos Zafiriou. Robust
Process Control. Prentice–Hall, 1989. 101, 122
[143] Jerzy Moscinski and Zbigniew Ogonowski. Ad-
vanced Control with Matlab and Simulink. Ellis Hor-
wood, 1995. 248, 506
534
[144] Frank Moss and Kurt Wiesenfeld. The benefits of
Background Noise. Scientific American, pages 50–
53, August 1995. See also Amateur Scientist in the
same issue. 195
[145] Ivan Nagy. Introduction to Chemical Process Instru-
mentation. Elsevier, 1992. 6
[146] Ioan Nascu and Robin De Keyser. A novel appli-
cation of relay feedback for PID auto-tuning. In
IFAC CSD03 Conference on Control systems Design,
Bratislava, Slocak Republic, 2003. 177
[147] R.B. Newell and P.L. Lee. Applied Process Control –
A Case Study. Prentice–Hall, 1989. 94, 95, 428, 437,
455, 503, 527
[148] R.E. Nieman and D.G. Fisher. Experimental Eval-
uation of Optimal Multivariable Servo-control in
Conjunction with Conventional Regulatory Con-
trol. Chemical Engineering Communications, 1, 1973.
490
[149] R.E. Nieman, D.G. Fisher, and D.E. Seborg. A re-
view of process identification and parameter esti-
mation techniques. Int. J of control, 13(2):209–264,
1971. 236
[150] Katsuhiko Ogata. Discrete Time Control Systems.
Prentice–Hall, 1987. 15, 23, 25, 28, 31, 35, 38, 49,
54, 62, 74, 75, 77, 82, 108, 231, 314, 345, 358, 359,
361, 372, 373, 375, 418, 452
[151] Katsuhiko Ogata. Discrete-Time Control Systems.
Prentice–Hall, 1987. 26
[152] Katsuhiko Ogata. Modern Control Engineering.
Prentice–Hall, 2 edition, 1990. 71, 76, 77, 80, 109,
151, 168, 358, 359, 361, 397
[153] B. A. Ogunnaike, J. P. Lemaire, M. Morari, and
W. H. Ray. Advanced multivariable control of a
pilot-plant distillation column. 29(4):632–640, July
1983. 97
[154] Alan V. Oppenheim and Ronald W. Schafer. Digi-
tal Signal Processing. Prentice–Hall, 1975. 207
[155] Overschee and De Moor. Subspace Identification.
Kluwer Academic Publshers, 1996. 277
[156] G.F. Page, J.B. Gomm, and D. Williams. Applica-
tions of Neural Networks to Modelling and Control.
Chapman & Hall, 1993. 506
[157] Chris C. Paige. Properties of Numerical Algo-
rithms Related to Computing Controllability. IEEE
transactions on Automatic Control, AC–26(1):130–
138, 1981. 361
[158] Sudharkar Madhavrao Pandit and Shien-Ming
Wu. Time Series and System Analysis with Applica-
tions. Wiley, 1983. 311
APPENDIX E. USEFUL TEST MODELS
[159] Rajni V. Patel, Alan J. Laub, and Paul M. van
Dooren. Numerical Linear Algebra Techniques for
Systems and Control. IEEE Press, 1994. A selected
reprint volume. 62, 361, 362
[160] Linda Petzold. Differential/Algebraic Equations
are not ODE’s. SIAM J. Sci. Stat. Comput., 3(3):367–
384, 1982. 123
[161] C.L. Phillips and H.T. Nagle. Digital Control System
Analysis and Design. Prentice–Hall, 2 edition, 1990.
200
[162] Jack W. Ponton. Neural Networks: Some Ques-
tions and Answers. J. Process Control, 2(3):pp163–
165, 1992. 514
[163] W.H. Press, B.P. Flannery, S.A Teukolsky, and W.T.
Vetterling. Numerical Recipes: The Art of Scien-
tific Computing. Cambridge University Press, 1986.
110, 220, 221, 229, 409
[164] S. Joe Qin and Thomas A. Badgwell. A sur-
vey of industrial model predictive control tech-
nology. Control Engineering Practice, 11(7):733–764,
July 2003. 469, 489
[165] W. Fred Ramirez. Process Control and Identification.
Academic Press, 1994. 407, 415, 419
[166] C. V. Rao, S. J. Wright, and J. B. Rawlings. Applica-
tion of interior-point methods to model predictive
control. Journal of Optimization Theory and Applica-
tions, 99(3):723–757, 1998. 475
[167] T. Rautert and E.W. Sachs. Computational design
of optimal output feedback controllers. Techni-
cal Report Nr. 95-12, Universität Trier, FB IV, Ger-
many, June 1995. 466
[168] James B. Rawlings. Tutorial overview of Model
Predictive Control. IEEE Control Systems Magazine,
20(3):38–52, June 2000. 469
[169] W. Harmon Ray and Julian Szekely. Process Opti-
mization. John Wiley & Sons, 1973. 407, 410
[170] W.H. Ray. Advanced Process Control. McGraw–Hill,
NY, 1 edition, 1981. 407
[171] John R. Rice. Numerical Methods, Software, and
Analysis. McGraw–Hill, 1983. 120
[172] C. Magnus Rimvall and Christopher P. Jobling.
Computer Aided Control System Design. In W.S.
Levine, editor, The Control Handbook, chapter 23,
pages 429–442. CRC Press, 1995. 2
[173] William H. Roadstrum and Dan H. Wolaver. Elec-
trical Engineering for all Engineers. John Wiley &
Sons, 2 edition, 1994. 13
E.2. AIRCRAFT MODEL
[174] E.R. Robinson. Time Dependent Chemical Processes.
Applied Science Publishers, 1975. 407
[175] J. A. Rossiter. Model-Based Predictive Control: A
Practical Approach. CRC Press, 2003. 489
[176] Hadi Saadat. Computational Aids in Control Systems
using Matlab. McGraw–Hill, 1993. 50
[177] S.P. Sanoff and P.E. Wellstead. Comments on: im-
plementation of self-tuning regulators with vari-
able forgetting factors. Automatica, 19(3):345 – 346,
1983. 299
[178] Robert Schoenfeld. The Chemist’s English. VCH
Verlagsgesellschaft mbH, D-6940 Weinheim, Ger-
many, 2 edition, 1986. 220
[179] J. Schoukens and R. Pintelon. Identification of Linear
Systems. Pergamon, 1991. 110
[180] Tobias Schweickhardt and F. Allgower. Linear
control of nonlinear systems based on nonlinear-
ity measures. Journal of Process Control, 17:273–284,
2007. 129
[181] Dale E. Seborg, Thomas F. Edgar, and Duncan A.
Mellichamp. Process Dynamics and Control. Wiley,
2 edition, 2005. 152, 155
[182] D.E. Seborg, T.F. Edgar, and D.A. Mellichamp. Pro-
cess Dynamics and Control. Wiley, 1989. 15, 31, 104,
105, 138, 151, 153, 254
[183] D.E. Seborg, T.F Edgar, and S.L. Shah. Adaptive
Control Strategies for Process Control: A survey.
American Inst. Chemical Engineers J, 32(6):881–913,
1986. 322
[184] Warren D. Seider, J.D. Seader, and Daniel R.
Lewin. Process Design Principles: Synthesis, Anal-
ysis and Evaluation. John Wiley & Sons, 1999. 105
[185] Bahram Shahian and Michael Hassul. Control Sys-
tem Design Using Matrixx . Prentice–Hall, 1992. 67
[186] Kermit Sigmon. Matlab primer, 3rd edition. Dept.
of Mathematics, University of Florida, 1993. 3, 4
[187] Jonas Sjöberg, Qinghua Zhang, Lennart Ljung, Al-
bert Benveniste, Bernard Delyon, Pierre-Yves Glo-
rennec, Håkan Hjalmarsson, and Anatoli Judit-
sky. Nonlinear black-box modeling in system
identification: a unified overview. Automatica,
31(12):1691–1724, 1995. Trends in System Identi-
fication. 238, 283
[188] S. Skogestad. Dynamics and Control of Distilla-
tion Columns– A Critical Survey. In DYCORD+,
pages 11–35. Int. Federation of Automatic Control,
1992. 95
535
[189] Sigurd Skogestad. Simple analytic rules for model
reduction and PID controller tuning. Journal of Pro-
cess Control, 13(4):291–309, 2003. 134, 153
[190] Jean-Jacques E. Slotine and Weiping Li. Applied
Nonlinear Control. Prentice–Hall, 1991. 74, 75, 386
[191] T. Söderström and P. Stocia. System Identification.
Prentice–Hall, 1989. 236, 314
[192] Harold W. Sorenson, editor. Kalman filtering: The-
ory and Application. Selected reprint series. IEEE
Press, 1985. 439
[193] Henry Stark and John W. Woods. Probability, Ran-
dom Processes, and Estimation Theory for Engineers.
Prentice–Hall, 2 edition, 1994. 440
[194] George Stephanopoulos. Chemical Process Control:
An introduction to Theory and Practice. Prentice–
Hall, 1984. 105, 145
[195] Dominick Stephens. The Reserve Bank’s Fore-
casting and Policy System. Technical report, Eco-
nomics Department, The Reserve Bank of New
Zealand, Wellington, NZ, 2004. 470
[196] O. Taiwo. Comparison of four methods of on-
line identification and controller tuning. IEE
Proceedings-D, 140(5):323–327, 1993. 159, 166
[197] Les Thede. Analog and Digital Filter Design using C.
Prentice–Hall, 1996. 204
[198] J. Villadsen and M.L. Michelsen. Solution of differ-
ential equation models by polynomial approximation.
Prentice–Hall, 1978. 86
[199] Stanley M. Walas. Modeling with Differential
Equations in Chemical Engineering. Butterworth–
Heinemann, 1991. 86, 123, 409
[200] Liuping Wang. Model Predictive Control System De-
sign and Implementation using Matlab. Springer,
2009. 489
[201] Liuping Wang and William R. Cluett. From Plant
Data to Process Control. Taylor and Francis, 11 New
Fetter Lane, London, EC4P 4EE, 2000. 260
[202] Ya-Gang Wang, Zhi-Gang Shi, and Wen-Jian Cai.
PID autotuner and its application in HVAC sys-
tems. In Proceedings of the American Control Confer-
ence, pages 2192–2196, Arlington, VA, 25–27 June
2001. 177
[203] P. E. WELLSTEAD and S. P. SANOFF. Extended
self-tuning algorithm. International Journal of Con-
trol, 34(3):433–455, 1981. 299
[204] P.E. Wellstead and M.B. Zarrop. Self-tuning Sytems:
Control and Signal Processing. John Wiley & Sons,
1991. 245, 299, 304, 306, 312, 314, 316, 318, 326, 461
536 APPENDIX E. USEFUL TEST MODELS

[205] David I. Wilson. Introduction to Numerical Anal-

ysis with Matlab or what’s a NaN and why do I
care? Auckland University of Technology, Auck-
land, New Zealand, July 2007. 465pp. 3, 120, 197,
205, 283, 399, 490, 493

[206] R.K. Wood and M.W. Berry. Terminal composition

control of a binary distillation column. Chemical
Engineering Science, 28(9):1707–1717, 1973. 95

[207] Minta Yuwana and Dale E. Seborg. A New

Method of On-line Controller Tuning. American
Inst. Chemical Engineers J, 28(3):434–440, May 1982.
159

[208] J.G. Ziegler and N.B. Nichols. Optimum Settings

for Automatic Controllers. Trans. ASME, 64:759–
768, 1942. 151
I NDEX

⊗, see Kronecker product Cayley-Hamilton theorem, 367

ceiling function, 204
A/D, 13 Chebyshev
Ackermann, 367, 371 design, 206
adaptive control Chebyshev filter, see filter
applications, 321 Cholesky decomposition, 76, 77, 80, 461
classification, 323 Cholesky matrix square root, 461
general scheme, 322 closed loop, 46
suitable, 322 coloured noise, 305
affine, 387 Computer design aids, 3
AIC, see Akaike Information Criteria confidence ellipses, 118
Akaike Information Criteria, 278 confidence intervals, 116
algebraic Riccati equation, 424 controllability, 358
are, 424 computing, 361
solving, 423, 425 covariance
alias, 17 wind-up, 298
detecting, 18 covariance matrix, 116, 287, 293, 441
ARMAX model, 262 reset, 296
arms race, 318 cover-up rule, 26
ARX model, 261 curve fitting, 511
ATV tuning method, 167
auto-tuners, 325 D/A, 13
autonomous systems, 46 DAEs, 123
dare, 451
back propagation, 511 data acquisition, 3
balance arm, 7, 138 dead time compensation, 185
Bernoulli’s equation, 89 deadbeat control, 371
bilinear transform, 37 deadtime, 280
black-box, 175 difference equations, 20
blackbox, 6 differentiation
Bode noise, 196
diagram, 39, 72 digital filter, 212
Bristol, 103 aliasing, 17
Bromwich integral, 31 diophantine
bursting, 340 convmtx, 332
bvp4c, 410, 413 Diophantine equation, 330
Dirac delta function, 21
c2d.m, 61 direct method, see Lyapunov
canon, 54, 359 discrete algebraic Riccati equation, 451
canonical form, 49, 54 distillation
controllable, 48 interaction, 102
diagonal, 48 model, 98
Jordan, 48 simulation, 101
observable, 48 dlqe, 451
Cauchy form, 47 DMC, see dynamic matrix control

537
538 INDEX

double integrator, 88 gradient, 257

DSP, 209
dynamic matrix control, 476 Hammerstein models, 283
Hankel matrix, 270, 478
etfe, 252 Hardamard product, ⊗, 104
Euler helicopter, 8
relations, 222 hidden layers, 510
exact feedback linearisation, 386
cautions, 389 identification, 235
design procedure, 388 black and white, 237
exhaustive search, 402 closed loop, 312
expert systems, 499 continuous models, 254
languages, 500 differentiation, 256
process control, 503 frequency response, 250
extended Kalman filter, 463 frequency testing, 245
graphical, 239, 254
FFT, see Fourier transform, 246 model, 236
fft, 228 noise models, 304
filter, 193, 209, 257 nonlinear, 283
bi-quad, 217 offline, 239
Butterworth, 201 online, 283
characteristics, 204 parameters, 236
cascade, 199 PRBS, 243
Chebyshev, 205 real-time, 238
DFI, 215 state-space, 275
DFII, 216, 219 subspace, 277
digital, 212 step response, 240
elliptic, 201, 208 system, 236
hardware, 215 time, 239
ideal, 198 two-dimensional, 114
minimum order, 204 initial value theorem, 23
proto-type, 200 integral states, 435
quantisation, 218 integral-only control, 135
SOS, 217 inter-sample ripple, 371
final value theorem, 23 interaction, 102
finite differences, 19 Internal Model Control, 153
Euler, 20 inverse response, 181
first-order plus deadtime, 240 invfreqs, 246
fixed-position, 467 Jordan form, 61, 62
flapper, 8 Jury test, 71
FOPDT, see first-order plus deadtime
forgetting factor, 295, 328 Kaczmarz, 310
influence, 297 Kalman filter, 446, 450
varying, 299 algorithm, 450
Fourier transform, 219, 250 current estimation type, 452
definitions, 221 extended, 463
fast, 225 prediction type, 452
slow, 221 steady-state, 450
frequency pre-warping, 213 Kaos, 268
function approximation, 220 Kronecker delta function, 25
Kronecker product, 77, 80, 425, 446
gain scheduling, 323 Kronecker tensor product, 77
Gauss, 107
Gilbert’s criteria, 359 Laguerre functions, 260
INDEX 539

Laplace transform CSTR, 92

inverting distillation, 95
numerically, 31 high purity, 101
least-squares, 107 Wood-Berry, 95
improvements, 110 double integrator, 88
nonlinear, 112 evaporator, 93
partial, 111 order, 280
recursive, 284 pendulum, 86
recursive estimation, 287 stirred heated tank, 90
solution, 109 validation, 277
level control, 88 model predictive control, 469
Liapunov, see Lyapunov model reference control
Lie algebra, 386 GMC, 381
limit oscillation, 168 modelling, 85, 86
linear program, 490 cautions, 91, 259
linear quadratic regulator, 417 evaporators, 93
continuous, 417 extrapolation, 91
discrete, 426 models
steady-state, 423 discrete, 260
time varying, 419 ARMAX, 262
linearise, 125 ARX, 260
finite difference, 127 BJ, 265
linmod, 127 family, 265
tank model, 127 FIR, 265
Lissajou, 248 general, 265
load disturbance, 158 OE, 264
loss function, see performance index linear, 237
LQR, see linear quadratic regulator nonlinear, 283
LTI object, 36 monotonicity index, 181
Lyapunov, 74, 76, 467 MPC, see model predictive control
continuous stability, 75 algorithm, 471
discrete, 446 horizons, 475
discrete stability, 75 Toolbox, 472
function, 74 toolbox, 489
matrix equation, 77 MuPad, 523
solving, 80
Nader, Ralph, 4
Maple, 3, 523 Nelder-Mead, 113
mapping, 20 neural networks, 505
Matlab, 2 activation function, 507
Matlab alternatives architecture, 507
Octave, 3 neurons, 507
SciLab, 3 toolbox, 511
matrix exponential, 58 training, 510
matrix inversion lemma, 286 Newell-Lee evaporator, see modelling
maximum likelihood, 283 Newton-Rhapson, 157
Maxwell, 1 Nilpotent matrix, 371
measurement noise, 116 noise
Method of Areas, 240 coloured, 305
micro-controllers, 13 industrial, 194
minimum variance control, 327 models, 304
dead-time, 346 non-minimum phase, 181, 340
MIT rule, 302 nonlinear measure, 128
model nonlinear models, 127, 283
540 INDEX

nonlinearities discrete, 143

types, 125 drawbacks, 180
Nyquist parallel, 132, 134
diagram, 39, 40 position form, 145
frequency, 16, 17, 204, 210, 212 programming, 143
dimensionless, 17 standard, 133
Nyquist diagram, 72 textbook, 132
tuning, 150
observability, 358 pidtool, 153
OCP, 409 pidtune, 153
ODEs, 120 closed loop, 155
numerical solutions, 120 Ziegler-Nichols, 155
two-point boundary value problems, 122 velocity form, 145
OE model, see output-error model pole excess, see time delay
Opti toolbox, 3, 112, 113 pole-placement, 329, 371
optimal control, 395 acker, 368
generic problem, 405 partial cancellation, 342
Hamiltonian, 408 place, 368
linear programming, 489 practical issues, 340
PID tuning, 396 Simulink, 339
Pontryagin, 407 unstable zeros, 340
TPBVP, 409 polyfit, 110
optimal control problem, 407 polynomial
optimal prediction, 348 addition, 325
optimisation, 113, 301, 395 multiplication, 325
cost function, 396 polynomial controller design, 325
nonlinear, 112 poorly damped, 341
Opti, 112 positive definite, 418
parametric, 396 check, 461
output optimal control, 466 lost, 460
output variance, 5 power spectral density, 226, 228
output-error model, 264 process control
overfitting, 277 economics, 4
psd, see power spectral density
packed matrix notation, 53 pseudo-inverse, 111
Padé approximation, 38
parameter optimisation, 300 quadratic form, 418
parametric adaption, 303 quadratic program, 475
Parseval, 221 quantisation error, 14
partial fractions, 25, 27
Matlab, 27 random inputs, 243
partial least-squares, 111 receding horizon control, 469
PDEs, 120 recurrent networks, 510
pendulum, 86 recursive
simulation, 120 mean, 285
penicillin, 391 variance, 285
performance index, 107 recursive estimation
IAE, 396 extended, 305
periodogram, 227 SI toolbox, 309
persistent excitation, 243, 312 simplified, 310
PI-D, 138 Kaczmarz, 310
PID, 131 starting, 289
cascade, 134 recursive extended least-squares, 306
classical, 132 recursive least squares, see least squares
INDEX 541

regression gun, 268 spectral plot, 226

regression vector, 266 spline, 197
relative gain array, 103 stability, 68
dynamic, 105 BIBO, 69
relative order, 388 closed loop, 72
relay, 167, 175 continuous time, 69
autotuning, 170 definition, 68
example, 170 discrete time, 73
relay feedback, 166 Jury test, 71
noise, 173 Lyapunov, 74
RELS, see recursive estimation nonlinear, 74
remembering factor, see forgetting factor Routh array, 70
residue, 25 unit circle, 73
RGA, see relative gain array stacking, 81
Riccati equation state, 45
algebraic, 424 state reconstruction, 362
computing solution, 427 state space, 45
differential, 419 conversions, 49
time varying, 425 definitions, 46
ringing, 341 pole-placement, 365
ripple, 205 state-space
RLS, 284 conversions, 47
robust fitting, 282 statical networks, 510
Routh table, 70 statistic, t, 116
Statistics toolbox, 3
safety, 4 steady-state models, 86
sample and hold steepest descent, 301
zoh, 35 step tests, 239
sample time selection, 15 Stixbox, 3
sampling, 14 stochastic processes, 445
sampling theorem, 16 subspace identification, 277
alias, 16 Sylvester equation, 77, 81
SattControl, ECA400, 167 Sylvester matrix, 331
self tuning controllers symbolic algebra, 3, 23, 24, 60
relay feedback, 166 Maple, 3
self tuning regulators, 326 MuPad, 3
problems, 352 symbols, 517
sensitivity function, 188 System Identification toolbox, 307
shift register, 293 system type, 435
Simulink
real-time, 10 tapped delay, 215
single precision, 219 target set, 414
singular value decomposition, 110 Technical support notes, 226
SISO, single input/single output, 44 tides, 513
Slyvester’s criteria, 78, 80 time delay
Smith predictor, 185 discrete, 67
Simulink, 187 time series analysis
smooth project, 311
input, 257 Toeplitz matrices, 270
smoothing trace of a matrix, 299
fourier, 228 transfer function, 54
SOS, 217 time constant, 54
spectral analysis, 224 transition matrix, 58
experimental, 245 Tustin’s approximation, 38
542 INDEX

Tustin’s method, see bilinear transform

unstable zeros, 340, 345, 349

vectorisation, 81

waypoints, 32
Weierstrass approximation theorem, 510
well damped, 345
white noise, 289
Wiener models, 283
Wood-Berry column model, 95

X-15, 321

Yuwana-Seborg tuning, 159

z-transform, 21
geometric series, 23
inversion, 24
computational, 29
long division, 27
partial fractions, 25
symbolic, 24
zero-pole-gain, 55
zeroth-order hold, 39
Ziegler-Nichols
closed loop, 155
example, 156
openloop, 151
zoh, see zeroth-order hold