Computers

& Chemical Engineering

Vol. 9. No. 4. pp. 359-365.

1985

009s1354/85

$3.00 + .a0

Printed I Great Britain.

Pcrgamon Press Ltd.

SENSITIVITY ANALYSIS OF INITIAL VALUE PROBLEMS WITH MIXED ODES AND ALGEBRAIC EQUATIONS
MAKIS CARACOTSIOS and WARREN E. STEWART? Department of Chemical Engineering, University of Wisconsin, Madison, WI 53706, U.S.A.
(Received 2 May 1984; revision received 1 October 1984; receivedfor Abstract-An publication 23 October 1984)

efficient method is described for sensitivity analysis of nonlinear initial value problems, which may include algebraic equations as well as ordinary differential equations. The linearity of the sensitivity equations is utilized to solve them directly via the local Jacobian of the state equations. The method is implemented with the implicit integrator DASSL and is demonstrated on a stiff industrial reaction model. Scope-With the rapid development of digital computers, increasingly realistic mathematical models are being used to investigate chemical phenomena. New mechanistic features, however, call for new physicochemical parameters whose values may not be accurately known. Consequently, there is an increasing need for parametric sensitivity analysis of proposed differential and algebraic models. Parametric sensitivity analysis is a very active research area. Extensive reviews can be found in Rabitz et al [l] and in Tilden et al [2]. Applications occur in every engineering and scientific discipline. Potential areas of application in chemical engineering include optimization, parameter estimation, model simplification, process sensitivity and multiplicity and experimental design. In this article we address the problem of numerical computation of sensitivity functions for systems of ordinary differential and algebraic equations. We develop a simple, efficient algorithm for this purpose by extension of a standard implicit integrator.
Conclusions and Significance-An efficient method has been developed here for the calculation of parametric sensitivities for mixed systems of ordinary differential and algebraic equations. The method has been tested successfully on many problems, ranging from simple ordinary differential equations to large stiff systems of differential and algebraic equations. The scheme has been implemented for a robust implicit integrator (DASSL) and is applicable to any implicit integrator.

PROBLEM

STATEMENT

Consider a dynamic system, described by the following set of differential and algebraic equations:
Eu(t) = f(t, u(t); 0) (la)

u(t = to) = u,(8).

(lb)

where I is the identity matrix of order s. Ifs = n, the system in eqn (la) consists of purely differential equations. If 1 5 s < n, the system in eqn (la) consists of ordinary differential and algebraic equations. The latter case arises, for instance, in analysis of reaction schemes where equilibria give algebraic constraints on the concentrations. We define the (n, m) matrix W(t) of sensitivity functions as

Here u is an n-dimensional vector of state variables, 8 is an m-dimensional vector of time-independent parameters and E is an (n, n) matrix of constant coefficients. Most frequently in chemical kinetics calculations, the matrix E assumes the form

W(t): = ---g

au(t)

(3)

This matrix satisfies a set of differential/algebraic equations that can be derived by partial differentiation of eqns (la) and (1 b) with respect to the parameter vector 9:
E=

EW(t)

- J(r)W(t)

= $

f(t, u(t); 63)

(da)

TAuthor to whom correspondence

should be addressed.

W(t = to) =

au, (0)

(4b)

CACE 9:4-D

359

360

MAKIS CARACOTSIOS and WARREN

E. STEWART

where the matrix J(t) (shorthand defined as

for J(t, u(t); e)) is

J(t): = t

f(t, u(t); e).

Various properties of eqns (4a) and (4b) are described in Tomovic and Vukobratovic [3]. The most striking feature of these sensitivity equations is that they are linear, regardless of the linearity or nonlinearity of the state eqns (1 a) and (1 b). The problem studied here is the numerical computation of the matrix W(t) from eqns (4a) and (4b).

LITERATURE

REVIEW

ANDTHEORETICALBACKCROUND

Before describing the new algorithm, we review a few known facts about the solution of mixed systems of ordinary differential and algebraic equations. Several investigators [4-61 have considered this subject, and recently Petzold [7] published an algorithm called DASSL for the solution of such systems, Not all systems of differential/algebraic equations are solvable. The reader is encouraged to consult, in the literature, Petzold and Gear [8] and Campbell and, Petzold [9] on this peculiar feature of mixed systems. However, for the systems that we are considering, where the matrix E assumes the form shown in eqn (2), sufficient conditions are known [8] for the solvability of eqns (la) and (1 b). Let the function f(t, u(t); e) be continuously differentiable with respect to u(t). Now consider the Jacobian matrix J(t) defined by eqn (5) for the system shown in eqn (la). If we partition the Jacobian matrix according to the partition of E, i.e.

methods are for systems of ordinary differential equations (ODES) only; that is, for systems with E = I. The available algorithms include the Fourier amplitude test [ 121, direct differential methods [ 131, Greens function methods [ 141, the analytically integrated Magnus method [l] (a modification of the Greens function method) and finite difference methods. The Greens function method and its variations exploit the fact that the sensitivity equations are linear inhomogeneous with time-varying coefficients; consequently they can be solved by first calculating the solution of the homogeneous part and then determining the particular solution corresponding to each parameter. Several authors have proposed solving the sensitivity equations by extending known solution algorithms for the state equations. This idea is based on the identity of the coefficient matrices in the sensitivity equations with those in the locally linearized form of the state equations on the locus u(t). Stewart and S$rensen [I I], Vemuri and Raefsky [ 151, Lojek [ 161 and Hwang [ 171 have developed various aspects of this method; nevertheless the idea is still under development. In the present work, we fully exploit the similarity of the sensitivity and state equations by building the sensitivity analysis into a robust differential/algebraic equation solver. Then we illustrate the algorithm by solving a stiff industrial kinetics problem.

MATHEMATICAL

DEVELOPMENT

OFTHESENSITIVITY

ANALYSIS ALGORITHM

J(t) ==

. .

where J,,(t) is an (s, s) matrix, then the system in eqns (1 a) and (1 b) is solvable if

det J**(f) # 0

for all t.

(7)

One of the most important steps in developing the sensitivity analysis algorithm was the selection of the integrator. For mixed systems of differential and algebraic equations, there are several codes [4-71 designed to perform the integration. These codes are primarily based on an idea developed by Gear [4]; specifically, the derivative u(t,, ,) is approximated by a backward difference formula with adaptable order and step size, and the resulting system of nonlinear equations is solved for u(t,, ,) via a modified Newton scheme. We chose for this work the package DASSL developed by Petzold [7]. This package is portable, robust and easy to use. We tested the code successfully on a wide variety of stiff problems, both differential and mixed, before adding the sensitivity analysis algorithm. First, consider the solution of state and sensitivity eqs (1 a), (1 b), (4a) and (4b) as a single system. In this approach, one needs the Jacobian matrix of the total system in eqns (la) and (4a). If we partition the sensitivity matrix W(t) into column vectors as

Under this condition, the solution obtained by a k-step backward differentiation formula algorithm with k < 7 and fixed step size h converges to O(h) if all initial values are correct. Further aspects of eqns (la) and (lb) and their numerical treatment are discussed in Petzold [lo]. I et us now review some of the methods used for the computation of the sensitivity matrix W(t). With one exception (Stewart and S$rensen [l l]), the known

W(r) = [w,(t) I W?(t)I

where

I W,,(f)1

(8)

() w,(t): = dQ,

i = 1 2

.m

(9)

Sensitivity analysis of initial value problems

361

then the Jacobian matrix J*(t) [shorthand for J* (t, u(t), W(t); e)] of the total system in eqns (la) and (4a) is

in which GiP(t) is the predicted value of 9,(t) via a kth-order predictor formula. The vectors Ei(t) can be calculated at the current t as follows: G,(t) = iv!@(t) + Ait, I = 1, 2,.

J(t) j,(t)
J*(t) =

0 0

. . . . . -. - - - - - - -

0
0

. m.

(15)

J(t)

j,(t)
. . . .

0 J(t) . . . . . . . .
0 0

j,,,(t)

. --. . -.

J(t)

(10)
where
ii(t): = f$

On completion of the update, the local truncation error is tested and, if necessary, the step size h and approximation order k are adjusted to achieve the specified accuracy for u(t) and W(t). Numerical tests show that stringent tolerances on u(t) normally lead to a good solution for W(t) as well, provided J(t) is updated before W(t) is computed. On the other hand, if the iteration matrix is only occasionally updated (as is usual in implicit integrators), then the local tolerances on W(t) are essential to control the calculation.

aJ(r) w,(t) + F

i= i . 2 1. .m.

COMPUTER IMPLEMENTATION

(11)
The computer analysis algorithm implementation of the sensitivity was done as follows:

The evaluation of J*(t) is a formidable calculation; however, it is a natural requirement of Newton iteration on the total equation system; eqns (la) and (4a). A simpler and quicker approach is to solve eqn (la) before eqn (4a) at each time step, as shown below; then the matrices j,(t) are not required. Let i(t) be the local interpolant of u(t) obtained in r Newton iterations of a kth-order integrator within a given time step. Then the next iteration will give the interpolant i(t) + AC(t), which satisfies the following linearized form of eqn (la): E(ii((t) + Aii(t)) = f(t, tic) (t); 9) (12)

The working arrays used by DASSL were modified to provide storage allocation for the sensitivity functions. An algorithm was written for the automatic formation of the sensitivity equations. The integrator in DASSL was properly extended to include the solution for the sensitivities. The following definitions were adopted:

1. lteration matrix D(t): = cE - J(t), where c is a constant that depends on step-size history. 2. Residual of the state equations: R,(t): = Eu(t) - f(t, u(t); 0). equations:

+ J(t, ii(t); 8)AO + O(hL). Hence the correction EAu - J(t)A,P Ai? satisfies = f(t, ii(t); 8) - E@(t) + O(h)

3. Residual of the sensitivity

Ri(t): = E[w;@(t) - ~wi~(t)]

& f(t, u(t); 0)

(13) The calculation flow chart. sequence is outlined

i=

1,2,...m.

in the adjoining

when the standard Newton method [with J(t) updated for each iteration] is used. If A@)(t) converges toward zero with increasing r, then J(t) converges toward J(t), and eqn (13) becomes formally similar to eqn (4a). Therefore, we can defer consideration of eqn (4a) until P(t) has converged to i(t) at the current value of t. Then we can update the sensitivity solution W(t) directly by the use of eqn (4a), which has the same coefficients as eqn (13) but a different, now computable, right-hand function. More specifically, the corrections AC,(t) are computed in a single iteration by solving EAG;(t) - J(t) A%,(t) == -Eti;P(t) + J(t) %-+)(t)

NUMERICAL EXAMPLE

The algorithm was tested on a wide variety of problems, ranging from linear and nonlinear systems of differential equations to large systems of differential and algebraic equations. We present here a batch reactor example given by the Dow Chemical Company [ 181. Figure 1 shows the proposed mechanism for the reaction system. The time-dependent concentrations are modeled by the following system of differential and algebraic equations:

u;(t) = -k,u,(t)u,(t) u;(t) = -k,u2(t)ue(f) +

k_,u,,(t)

+ &f(t.
Vi

ti(t);e)

-t O(P)

i = 1,2,.

.m

(14)
-

k+,(t) ug(t)

362

MAKIS CARACOTSIOS WARRENE. STEWART and

lnilialize arrays for new time step

P
6

Compute coefficients for backward differentiation formulas

Predict

~ltl,~~tl.~~tl.~(t)

SOIVS D(1) I

&* _.,

-6 .,0(t)

I Convsrpsncs ? NO

FLOW CHART C

FLOW CHART A

Slow

Kinetic

React ions

13
Solve o(t) ii = Estimate for local truncation u(t).w(t) _ w

0
5

%4- + 20M
A- +
error

k1- IoMBMk-l
k2 -ABM-

-Fii (11

BM

M- + 4AB Rapid 5MBMH HA 3HABM

k3 _

k-3
KI _

ABM-

Acid-Base

React ions MBMA+ + 7HS H+

K2

K3

ABM- +

H+

FLOW CHART B

Fig. I. Chemical reaction model for the numerical example [IS]. The chemical species are numbered as in eqn (16).

Sensitivity

analysis

of initial value problems

363

u;(t) = -k,u,(t)u,(t)

+ k~,u,(t)

u;(t) = k,uz(r)u,(t) ~ k ,u,,(t) d(f) = -k,u,(t)u,(t) ~ k+q(f)ug(r)

0

+ k-,u,o(t) + k A(Z)
1
1- tlrs3 4 5 6

u,(t)

= -0.0131 + u,o(t)

+ u,(t)

+ u,(r)

+ U,(f) Fig. 2. Concentration functions u,(r) in gmole/kg.

43(t) = u,(t) = ho(t) =

with initial
u,(O)

Kzu,(t) K, + u,(r) K,u,(t) K, + u,(t) K, us(t) 4 + u,(t)

conditions

0.015 ~
6

/_.C.----

AA- _ _..._.-

...--R

(16)
0.00

= 1.5776

,/ ,/ - / .\ 5,/ l ..<
--A. 0.0

A--+__

u*(O) = 8.32 u,(O) &(O) = 0 j = 3, 4, 5,9, 10

0.000,

, . I . ( , , I . , I I , I , I , I I 0.5 I.0 2.0 1.5

t-hrs
Fig. 3. Concentration functions u,(t) in gmole/kg.

= 0.013

u,(O) = 0.5 i--K, + JK; + 4K,u,}

u,(O) = u,(O). (17)

The following values of rate and equilibrium were used [ 191: k, = 21.893 k_, = 2.14 E09 k, = 32.318 k, = 21.893 k 3 = 1.07 E09 K, = 7.65 E-18 K, = 4.03 E-l I K, = 5.32 E-18 hr Kg gmole hr- hr Kg gmole hr- Kg gmole hr- gmole Kg gmole Kg gmole Kg .
Table

constants

kl

-0.66
(18)

t-hrs

Fig. 4. Elements of the sensitivity vector &s(t)/alnk,. The number I denotes the sensitivity coefficient u,(t) with respect to Ink,. Numbers not shown correspond to sensitivity functions indistinguishable from zero at all times.

1. Computational

effort for the solution of the Dow problem Sensitivity P = l.OE-6 185 377 18 not exceed rtol equations e _ l.OE-7 283 571 27

State equations et = l.OE-6 Time steps Function evaluations CPU set I91 418 4.0 c = l.OE-7 265 587 5.5 that abs (local error)

ILocal error test at each time step which requires where rtol = atol = e.

. abs (u) + atol

364

MAKIS CARACOTSIOS and WARREN E. STEWART

The natural logarithms of these constants make up the parameter vector 8. Our goal is to estimate the sensitivity matrix W(t) ==&i(t)/*. The combined system of state and sensitivity functions consists of 90 equations, 54 of which are differential and 36 are algebraic. This problem presents a severe test for the DASSL integrator and the sensitivity analysis algorithm. Table 1 summarizes the computational effort for the solution of the above problem on a VAX I l/780 computer. All calculations were carried out in double precision. A mixed local truncation error control provided by Petzold [7] was used. The tolerances for the sensitivities were equated with the tolerances of the state variables. The total reaction time considered was 53 hr. Figures 2 and 3 show the evolution of the concentration profiles as functions of time, while Figs 4 through 11 show the dynamic behavior of the sensitivity functions.

Sensitivity plots, like those in Figs 4 to 11, can be of considerable use to the theoretician as well as to the experimentalist. From Figs 4 through I1 we see that all of the rate and equilibrium constants have comparable effects on the concentrations. and therefore we

-0.4

I 0

, 2

t!hrs

, 6

, 8

I 10

Fig. 8. Elements

of the sensitivity

vector ~?u(t)/dlnk_,

-0.4.0

t-hrs
Fig. 5. Elements of the sensitivity vector du(t)/dlnk_,

-.4~
Fig. 9. Elements

IO t-hrs
vector du(t)/dlnK,

of the sensitivity

k2

-0.75

0
Fig. 6. Elements

t4hrs

IO

o.4iY-TT7+0

t-hrs
of the sensitivity vector &~(t)/dlnk,. Fig. IO. Elements of the sensitivity vector du(t)/alnK,.

0.4 k3

0.4

0.2-

*0.0 K3

dK,
-0.2-

-0.4.0

t-hrs
Fig. 7. Elements of the sensitivity vector du(t)/~Ylnk,. Fig.

I I. Elements of the sensitivity vector du(t)/alnK,.

Sensitivity cannot

analysis

of initial value problems 6. J. W. Starrier, A numerical algorithm

365 for the solution of

eliminate any step from the proposed mechanism in Fig. 1. However, a close inspection of the results reveals the following linear relations:
k

implicit algebraic-differential systems of equations. Technical Report 3 18. Department of Mathematics and Statistics, University of New Mexico (1976).
7. L. R. Petzold, A description of DASSL: a differential/ algebraic system solver. Sandia Tech. Rep. 82-8637 (1982). 8. L. R. Petzold and C. W. Gear, ODE methods for the solution of differential/algebraic systems. Sandia Tech. Rep. 82-8051 (1982). 9. S. L. Campbell and L. R. Petzold, Canonical forms and solvable singular systems of differential equations. Sandia Tech. Rep. 82-8891 (1982). equations are not IO. L. R. Petzolh. Differential/algebraic ODEs, SIAM J. Sci. Siat. Comout. 3. 367-384 (19821. II. W. E. Stewart and J. P. S&ensen; Sensitivity and regression of multicomponent reactor models. Fourth International Symposium on Chemical Reaction Engineering, DECHEMA, Frankfurt, I-12 to I-20 (1976). 12. R. I. Cukier, H. B. Levine and K. E. Shuler, Nonlinear sensitivity analysis for multiparameter model systems. J. Comput. Phys. 26, 1-42 (I 978). 13. R. P. Dickinson and R. J. Gelinas, Sensitivity analysis of ordinary differential equations-a direct method. J. Compuf. Phys. 21, 1233143 (1976). 14. J. T. Hwang, E. P. Dougherty, S. Rabitz and H. Rabitz, The Greens function method of sensitivity analysis in chemical kinetics. J. Chem. Phys. 69, 5180-5191 (1978). 15. V. Vemuri and A. Raefsky, On a new approach to parameter estimation by the method of sensitivity functions. Inf. J. Syst. Sci. 10, 3955407 (1979). 16. B. Lojek, Sensitivity analysis of nonlinear circuits. IEEE Proc. 1296(3), 85-88 (1982). 17. J. T. Hwang. Sensitivity analysis in chemical kinetics by the method of polynomial approximations. In?. J. Chem. Kin&. IS,3955407 (1983). 18. G. Blau, L. Kirkby and M. Marks, An industrial kinetics problem for testing nonlinear parameter estimation algorithms. Process Math Modeling Department, The Dow Chemical Company (1981). 19. M. Caracotsios, W. E. Stewart and J. P. Sdrensen, Nonlinear parameter estimation for an industrial kinetics problem. In preparation.

au(t) au(t)=K_
ak.,

dK,

(19)

au(t) _

_K

au(t)

-3 ak-~,

aK,

(20)

Thus one cannot estimate all parameters of the model from data on u(t) for the range of conditions shown. This information proved useful in fitting the model in eqns ( 16) and (17) to the published experimental data for this system [ 191. The software package for the present algorithm is available from the authors.
work was supported by the National Science Foundation through Grant CPE-8308748 and by the United States Army under Contract DAAG2980-C-0041. We also express our appreciation to Linda Petzold for helpful discussions regarding the use of her integration package. DASSL.

AcknowledgPmenr~This

REFERENCES
I.

H. Rabitz, M. Kramer and D. Dacol, Sensitivity analysis

in chemical kinetics. .4nn. Rev. Phys. Chem. 34,419-461 (1983). J. W. Tilden, V. Constanza, G. J. McRae and J. H. Seinfeld. In: Modeling of Chemical Reaction Systems, K. H. Ebert, P. Deuflhard and W. Jager, eds. SpringerVerlag, Berlin, pp. 69-91 (1981). R. Tomovic and M. Vukobratovic, General Sensitivity Theory. American Elsevier, New York (I 972). C. W. Gear, Simultaneous numerical solution of differential-algebraic equations. lEEE Trans. Circuit Theory CT-18. 89-95 (1971). A. C. Hindmarsh, LSODE and LSODI, two initial value ordinary differential equation solvers. ACM SIGNUM Newsletter 15, IO-1 1 (1980).

2.

3. 4.

5.