Three-Dimensional Air Combat:

Numerical Solution of Complex Dierential Games

R. Lachner, M. H. Breitnery, and H. J. Peschz
May 16, 1996

Abstract
Complex pursuit-evasion games with complete information under state
variable inequality constraints are investigated. By exploitation of Isaacs'
minimax principle, necessary conditions of rst and second order are de-
rived for the optimal trajectories. These conditions give rise to multipoint
boundary-value problems, which yield open-loop representations of the op-
timal strategies along the optimal trajectories. The multipoint boundary-
value problems are accurately solved by the multiple shooting method.
The computed open-loop representations can thereafter be used to syn-
thesize the optimal strategies globally.
As an illustrative example, the evasion of an aircraft from a pursuing
missile is investigated. The ight of the aircraft is restricted by various
control variable inequality contraints and by a state variable inequality
constraint for the dynamic pressure. The optimal trajectories exhibit
boundary arcs with regular and singular constrained controls. The in-
uence of various singular surfaces in the state space including a low-
dimensional universal surface is discussed.
Key Words. Dierential games, pursuit-evasion games, singular sur-
faces, multipoint boundary-value problems, multiple shooting method.

1 Introduction
Dierential games are useful to describe many aspects of human interactions.
Unlike optimal control problems, they deal with the competitive character of
those interactions. Even unpredictable disturbances can be investigated, if
\nature" is taken into account as an additional player. The solution of dif-
ferential games is much more involved than the solution of optimal control
problems. This is due to the fact that only optimal feedback-type strategies
 Assistant Professor. E-mail: lachner@math.tu-clausthal.de.
y Assistant Professor. E-mail: breitner@math.tu-clausthal.de.
z Professor of Mathematics. E-mail: pesch@math.tu-clausthal.de.
All authors' address: Institute of Mathematics, Clausthal University of Technology, Erzstr. 1,
D{38678 Clausthal-Zellerfeld, Germany.
Formerly, Department of Mathematics, Munich University of Technology, D{80290 Munich,
Germany.

1
are suitable to describe the global behavior of the players, especially if one of
them acts nonoptimally. A global computation of these optimal strategies is
generally impossible, since the state space is split up by singular surfaces even
for simple dierential games. Another reason is that direct optimization meth-
ods, which are easy to deal with, do not exist so far for dierential games. Direct
optimization methods for optimal control problems (see, e. g., von Stryk [30])
are incompatible with the information structures usually associated with dier-
ential games. Moreover, feedback control schemes that can be applied in real
time, have been developed even for complicated optimal control problems (see
the survey in [25]) whereas only a few real-time feasible and generally applicable
methods can be found which have already been applied to dierential games;
see [3], [4], [5], and [27].
In the present paper, optimal evasive maneuvers of an aircraft against an
optimally guided missile are investigated. The results of [21], where the missile-
versus-aircraft scenario has been restricted to a common vertical plane, are ex-
tended here to pursuit-evasion maneuvers in three space dimensions. Thereby,
the maneuverability of the aircraft is restricted by several control and state vari-
able inequality constraints. Various initial conditions are investigated including
the head-on shoot.
The application of Isaacs' minimax principle yields complicated optimal con-
trol histories including constrained and singular subarcs. These control histories
coincide with the optimal strategies along the optimal trajectories. A combin-
ation of all the necessary conditions from the minimax principle gives rise to
multipoint boundary-value problems with interior point conditions and jump
conditions for the state and the adjoint variables. These multipoint boundary-
value problems are then solved by means of the multiple shooting method; see
Bulirsch [9], Stoer and Bulirsch [29], and the references cited in [25]. For the
most recent code used in this paper, see Hiltmann et. al. [15].
By means of homotopy techniques, the optimal trajectories and the associ-
ated open-loop representations of the optimal strategies can be computed in the
entire capture zone. These open-loop representations provide the information
by which optimal strategies can be approximated globally, e. g., by a successive
solution of neighboring multipoint boundary-value problems (see [2]), by Taylor
series expansions around many optimal trajectories (see [3], [4]), or by neural
networks (see [27]). Therefore, this work can be considered as a rst step to-
wards the real-time computation of optimal strategies for realistic dierential
games of pursuit-evasion type.

2 Pursuit-Evasion Game
The following pursuit-evasion game arises as a subproblem for an air combat
scenario between two aircraft. Similar problems have been treated recently with
various solution methods; see, e. g., Fink [10], Greenwood [11], Guelman, Shinar,

2
Green [13], Gutman, Katz [14], and the papers [6], [7], [19], [20], and [26] of the
authors. At time t = 0 the pursuing aircraft launches a medium-range air-to-air
missile P , which carries on pursuing the evading aircraft E . The missile P is
much more maneuverable then the aircraft E , but can accelerate only a short
time due to its limited fuel supply. As a result, E can always escape if the
initial distance between the opponents is large enough. Therefore, only initial
constellations for which P can enforce capture against all possible maneuvers
of the aircraft are considered here. This aerial scenario can be formulated as
a pursuit-evasion game with the missile P identied with the pursuer and the
aircraft E with the evader.
Note that the restriction of the ight paths to a common vertical plane as
in [6], [7], [19], [20], [21], and [26] is discarded here. Therefore, the computation
of optimal ight paths for almost arbitrary initial constellations including the
so-called head-on shoot can be performed. As a by-product, the two-dimensional
optimal trajectories computed in the above-mentioned papers, will turn out to
be optimal in the three-dimensional space, too, if the initial velocity vectors lie
in a common vertical plane.
The state of P and E is described by their position vectors and their veloctity
vectors. The equations of motion for the missile P and the aircraft E then read
as follows (see, e. g., Miele [24]),
x_ P = fxP = vP cos P cos P ;
x_ E = fxE = vE cos E cos E ;
y_P = fyP = vP cos P sin P ;
y_E = fyE = vE cos E sin E ;
h_ P = fhP = vP sin P ;
h_ E = fhE = vE sin E ;
v_ P = fvP = TP;max(t) ? D0;P (hP m ; vP ) ? nP2 DI;P (t; hP ; vP )
P (t)
?g sin P ; (1)
v_ = f = E TE;max(hE ; vE ) ? D0;E (hE ; vE ) ? nE DI;E (hE ; vE )
2
E vE mE
?g sin E ;
_ P = f P = vg (nP cos P ? cos P ) ;
P
g
_ E = f E = v (nE cos E ? cos E ) ;
E
_ P = fP = v nPcossin P ;
g
P P
g
_ E = fE = v cos E :
n E sin
E E

3
Subscripts P and E refer here to the missile P and the aircraft E . The state
variables x, y, h, v, , and  denote the two coordinates of the horizontal
position, the altitude, the velocity, and the vertical and horizontal ight path
angle of P and E , respectively. The thrust TE = E TE;max of the aircraft
is controlled by the throttle E 2 [E;min; E;max]. The thrust TP = TP;max
of the missile is not controllable. It is assumed to be piecewise constant with
two discontinuities at t = 3 [s] and t = 15 [s] indicating the transitions from
the boost phase to the sustain phase and from the sustain phase to the coast
phase. The control variables nP and nE denote the load factors, which gov-
ern the ight path angles. The bank angles P and E denote control vari-
ables peculiar to the three-dimensionality of the ight scenario. They determ-
ine the radii of curvature. Since only ight times less then 90 seconds occur,
the mass mE of the aircraft is assumed to be constant neglecting the eects
of fuel consumption (
mE < 0:1 mE ). The mass mP of the missile decreases
linearly during the boost and the sustain phase and is constant during the coast
phase. The constant g denotes the gravitational acceleration. The complic-
ated functions TE;max, D0;P , D0;E , DI;P , and DI;E are based on data for the
F15E-Strike Eagle multi-role combat aircraft and the medium-range air-to-air
missile AMRAAM AIM-120A. These data provide realistic approximations for
the maximum thrust of the aircraft, the zero-lift drag, and the induced drag.
The coecients for the approximations have been determined by the method of
least squares.
At time t = 0 the game starts with the initial conditions
xP (0) = 0 [m] ; xE (0) = xE;0 [m] ;
yP (0) = 0 [m] ; yE (0) = yE;0 [m] ;
hP (0) = hP;0 [m] ; hE (0) = hP;0 [m] ; (2)
vP (0) = vP;0 [m/sec] ; vE (0) = vE;0 [m/sec] ;
P (0) = 0 [deg] ; E (0) = E0 [deg] ;
P (0) = 0 [deg] ; E (0) = E;0 [deg] :
These initial conditions are abbreviated by z (0) = z0 with
z = (xP ; xE ; yP ; yE ; hP ; hE ; vP ; vE ; P ; E ; P ; E )> :
Note that xP (0) = 0, yP (0) = 0 and P (0) = 0 can be prescribed without loss of
generality, since these initial conditions can always be obtained by a translation
and rotation of the coordinate system. The dimension of the state space can be
reduced from 12 to 9 dening the new variables xE ? xP , yE ? yP , and E ? P .
The condition P (0) = 0 is due to the technical reason that the launch of a
missile is dangerous for the carrying aircraft when 6= 0.
By choosing admissible values for the control variables of both players in
Eqs. (1), a trajectory is determined uniquely in the state space. The terminal
time tf of the game is determined by the capture condition
?x (t ) ? x (t )
2 + ?y (t ) ? y (t )
2 + ?h (t ) ? h (t )
2 ? d2 = 0 ; (3)
P f E f P f E f P f E f

4
where the capture radius d is chosen as 50 [m]. Equation (3) denes a hyper-
manifold in the state space called the terminal manifold. If P is not able to
enforce capture against all possible maneuvers of E , the outcome of the game
is \E not captured" and tf is set to innity.
The players choose their strategies ?P and ?E under the aspect of minim-
aximizing the objective functional
J (?P ; ?E ; t = 0; z = z0 ) = tf : (4)
The pursuer P engages in driving the state z from the initial state z0 to the
terminal manifold in minimal time, whereas the evader E tries to avoid capture
or, if this is impossible, tries to maximize the capture time tf .
For a realistic modelling, several constraints have to be taken into account.
The most important constraint for the evader, which shall particularly be dis-
cussed here, is the limit of the dynamic pressure qE ,
Q(hE ; vE ) = qE (hE ; vE ) ? qE;max  0 (5)
with qE = 21 (hE ) vE 2 , qE;max = 80 [kPa], and  denoting the air density. This
rst-order state variable inequality constraint keeps the aircraft away from the
utter boundary and limits the static load. For certain initial conditions of the
state, it is advantageous for the aircraft to descent, in order to maximize thrust,
to gain additional kinetic energy, and to force the missile to follow into regions
of high drag. In this case the altitude constraint
hE  0 ; (6)
a second-order state variable inequality constraint, may become active. How-
ever, optimal trajectories with altitude-constrained subarcs will not be presented
here; see [19] instead. Finally, the control variables are bounded by
nP 2 [nP;min; nP;max] = [0; 20] ;
nE 2 [nE;min; nE;max] = [0; 7] ; (7)
E 2 [E;min; E;max ] = [0:23; 1] :
Note that, unlike the case in the two-dimensional version of this game in [6], [7],
[19], [20], [21], and [26], the nE;max -constraint is more restrictive, since sharp
turns of E are optimal for many initial constellations; see also [10]. The Mach
limit at high altitudes and the maximum lift coecient for low speeds are of
secondary importance and are not taken into account. These constraints can,
however, be included in a similar way. Also a visibility condition for the missile
to keep the aircraft in its radar cone, can be included analogously.

5
3 Necessary Conditions for Optimality
3.1 Dierential Game Approach
In the sequel it is assumed that the game is started with an arbitrary initial
state z0 for which capture is guaranteed against all admissible strategies of E ,
provided that P plays optimally. The set of all optimal trajectories emanating
from these initial states span a subset in the state space, the capture zone.
It is separated from the remainder of the state space, the escape zone, by a
submanifold, the barrier. Note that the derivation of necessary conditions in
this section is valid only for initial states in the capture zone.
Only admissible strategies
 n (t; z) 
uP (t; z ) = P = ?P (t; z ) ;
P (t; z )
0 n (t; z) 1 (8)
uE (t; z ) = @ EE (t; z) A = ?E (t; z )
E (t; z )
are considered; see, e. g., Basar and Olsder [1]. The players P and E have
perfect information about the actual state, but no information about the actual
or future control of the opposite player. A strategy ?P  is called optimal for P
if
J (?P  ; ?E ; t; z )  J (?P ; ?E ; t; z ) (9)
holds for all admissible strategies ?E , for all t, and for all z within the capture
zone. Vice versa, a strategy ?E  is called optimal for E if
J (?P ; ?E ; t; z )  J (?P ; ?E  ; t; z ) (10)
holds for all admissible strategies ?P , for all t, and for all z within the capture
zone.
The optimal value V  of the objective function as a function of the actual
time t and the actual state z is dened by
V  (t; z ) = J (?P  ; ?E  ; t; z ) : (11)
A trajectory generated by optimal strategies will be called an optimal tra-
jectory. Along optimal trajectories, the optimal strategies yield optimal con-
trols uP  (t) := uP  (t; z (t)) and uE  (t) := uE  (t; z (t)). Henceforth, we will
speak about optimal controls if we have the values of the optimal strategies
along optimal trajectories in mind.
The minimax principle of Isaacs [16] and [17] yields local necessary conditions
which must be satised by the optimal controls uP  and uE  , i. e., for all t along

6
the optimal trajectories. Dening auxiliary functions by
u~P (t; z; Vt (t; z ); Vz (t; z ); uE ) = arg min d V  (t; z )
uP ?dt

= arg min
uP t
V  (t; z ) + Vz (t; z ) f (t; z; uP ; uE ) ;
u~E (t; z; Vt (t; z ); Vz (t; z ); uP ) = arg max d V  (t; z )
uE ?dt
= arg min V  (t; z ) + V  (t; z ) f (t; z; uP ; uE )
;
uE t z
(12)
the optimal controls must satisfy
uP  (t; z; Vt (t; z ); Vz (t; z ))
= arg min
?V (t; z) + V (t; z) f (t; z; u ; u~ (t; z; V ; V  ; u ))
;
uP t z P E t z P
   (13)
uE (t; z; Vt (t; z ); Vz (t; z ))
= arg max
?V (t; z) + V  (t; z) f (t; z; u~ (t; z; V ; V ; u ); u )
;
uE t z P t z E E
for all t.
In Eqs. (12) and (13), the minimization and maximization must be per-
formed only for controls uP and uE which are admissible in t. Because of the
separability of dV  =dt, the auxiliary functions u~P and u~E coincide with the op-
timal controls uP  and uE  . Therefore, one need not to distinguish upper and
lower values of the game. Since the game has a terminal payo [see Eq. (4)],
the optimal value V  satises the nonlinear partial dierential equation of rst
order
Vt + Vz f (t; z; uP  (t; z; Vt ; Vz ) ; uE  (t; z; Vt ; Vz )) = 0 (14)
for all (t; z ) in the capture zone. This equation is known as Isaacs equation
(see [16] and [17]). Unfortunately, Eq. (14) cannot be used directly for com-
puting a pair of optimal strategies, since it is a nonlinear partial dierential
equation for 13 independent variables here. A state space reduction to the
essential variables can decrease the dimension only to 10. Moreover, the func-
tion V  is not continuously dierentiable in the entire capture zone. However,
an equivalent system of ordinary dierential equations along a characteristic
curve can be formulated identifying the gradient (Vt  ; VxP  ; : : :)> at (t; z ) with
the merely time-dependent adjoint functions ( ; xP ; : : :)> . For this purpose,
the system (1) is rstly rewritten in autonomous form using  = t as a new state
variable,   
d  = 1

dt z f (; z; u ; u ) :
P E
(15)
The new initial condition for  is (0) = 0. Secondly, the system for the adjoint

7
variables is given, according to [17], by
   >
d
dt

 = ? @ (@H
; z )
u P  ; uE  )
= ? > @f (; @z;(; z)
 u  !>
+ @ (u@H; u ) @ (;@ z ) P
uE  ; (16)
P E
where the Hamiltonian is dened by
H (; z;  ; ; uP ; uE ) =  + > f (; z; uP ; uE ) : (17)
Note that there holds
H (; z;  ; ; uP  ; uE  ) = ddt V  (; z ) : (18)
Following this procedure, the optimal controls uP  and uE  in Eq. (13) can now
be identied along optimal trajectories through
?
?
uP  (t) = uP  ?t; z (t) = uP  t; z
(t); Vt (t; z (t)); Vz (t; z (t))


= uP  t; z (t);  (t); (t) ;
?
?
uE  (t) = uE  ?t; z (t) = uE  t; z
(t); Vt (t; z (t)); Vz (t; z (t))

(19)
= uE  t; z (t);  (t); (t) ;
and the optimal strategies ?P  and ?E  are obtained along optimal trajectories
from the optimal control histories uP  and uE  via
?P  (t; z (t)) = uP  (t) ; ?E  (t; z (t)) = uE  (t) : (20)
These equations provide an open-loop representation of the optimal strategies;
for details see [1].
3.2 Open-Loop Representations of Optimal Strategies for
the Missile
In this subsection, it is assumed that the nP;max-constraint is not active during
the entire maneuver. This assumption is valid for all numerically computed
optimal trajectories presented in Section 4. Furthermore, it can be shown that
nP  (t) = nP;min holds only for isolated t; see Grimm [12]. From
H (; z;  ; ; nP ; P ; nE ; E ; E ) = H1 (; z; ) nP 2
+H2 (z; ) nP cos P + H3 (z; ) nP sin P
+H4 (; z;  ; ; nE ; E ; E ) ; (21)

8
the optimal controls of the missile P are computed using the minimax principle,
@ H (; z;  ; ; n  ;   ; n ;  ;  ) = 0;
@nP  P P E E E
@ H (; z;  ; ; n  ;   ; n ;  ;  ) = 0;
@P  P P E E E
(22)
@ 2 H (; z;  ; ; n  ;   ; n ;  ;  )  0;
@nP 2  P P E E E
@ 2 H (; z;  ; ; n  ;   ; n ;  ;  )  0:
@P 2  P P E E E

This yields
nP  = 2 DmP gvUP ; (23)
I;P P vP
cos P = ? U P ;
 (24)
P
sin P  = ? U cos
P
(25)
P P
with q
UP =  P 2 + (P = cos P )2
and the necessary sign condition vP  0.

3.3 Open-Loop Representations of Optimal Strategies for

the Aircraft
The dierent optimal control laws for E depend on the function
q
UE =  E 2 + (E = cos E )2 : (26)
During the numerical computations it turned out that the singular case UE  0
can occur only on dynamic-pressure-constrained subarcs. Hence, we will have
to distinguish between regular and singular constrained subarcs. However, we
rst consider state-unconstrained subarcs.
3.3.1 Dynamic Pressure Constraint Inactive
The optimal controls of E can be obtained by maximizing the Hamiltonian
H (; z;  ; ; nP ; P ; nE ; E ; E ) = H 1 (z; ) nE 2
+H 2 (z; ) nE cos E + H 3 (z; ) nE sinE
+H 4(z; ) E + H 5 (; z;  ; ; nP ; P ) (27)

9
over the set of admissible controls [nE;min; nE;max]] ? ; ]  [E;min; E;max].
This yields
8 nE;max ; if vE  0,
>
< 

nE = >  (28)
: min 2 DmE gvUE ; nE;max ; if vE > 0,
I;E E vE

cos E  = U E ; (29)
E
sin E  = E
8 UE cos E ; (30)
>
< E;min ; if vE < 0,

E = > undened ; if vE = 0, (31)
: E;max ; if vE > 0.

3.3.2 Dynamic Pressure Constraint Active

Unlike the situation in the case of optimal control problems, state variable in-
equality constraints or, more precisely, state constraints of order greater than
zero, are not well understood in dierential games. This obstacle can generally
be remedied by strengthening the state constraint by a mixed control-state con-
straint; see, e. g., [2], and [21]. Their theoretical treatment is well understood.
Unfortunately, this procedure leads to optimal solutions with a chattering con-
trol here due to the nonconvexity of the hodograph. An extended class of
strategies would have been permitted then which, however, cannot be imple-
mented for aircraft. Therefore, the well-known necessary conditions of optimal
control theory for state constraints of higher order (see, e. g., Bryson, Denham,
and Dreyfus [8], Jacobson, Lele, and Speyer [18], and Maurer [23]) are applied
to the present dierential game despite the lack of a rigorous proof in the con-
text of dierential games. This approach can be justied according to [21] as
follows. The state-constrained dierential game problem can be imbedded into
a one-parameter family of more stringent mixed-state-control-constrained prob-
lems. By weakening these control-state-constraints, numerical convergence of
the associated one-parameter family of solutions has been observed in [21] to-
wards the solution associated with those necessary conditions. For more details,
see [21].
If the dynamic pressure constraint holds, the set of admissible controls is
restricted by the condition

(hE ) vE 21 @ ln(@h (hE )) v 2 sin ? g sin
E E E
E
1 ?

+  T
mE E E;max ?D ?n D
0;E E 2
I;E  0 : (32)

10
This is implied by the rst derivative of the dynamic pressure constraint (5).
The aircraft E again attains its optimal controls by maximizing the Hamilto-
nian. Since the set of admissible controls is restricted by Eq. (32), this condition
is, according to [8], adjoined to H by a Lagrange multiplier qE which necessarily
must be nonpositive. The optimal controls are then given by
"  1 @ ln((h
nE  = 1 E )) m v 2 sin
DI;E 2 @hE E E E
#1=2
?mE g sin E + TE;max ? D0;E ; (33)

cos E  = U E ; (34)
E
sin E  = U cos
E ;
E (35)
E
E  = E;max ; (36)
provided that vE > 0 and UE > 0. From a discussion of the Hamiltonian, there
follows that vE < 0 cannot occur on dynamic-pressure-constrained subarcs;
compare [21]. Note that the adjoint equations dier from those on unconstrained
subarcs, denoted by the superscript \free",

@ d Q(z ) > :

_ = _ free ? qE @z dt (37)

Furthermore, the interior point condition qE (tentry ) = qE;max at the junction

point tentry induces discontinuities in the adjoint variables, namely
@ >
(tentry + ) = (t ?
entry ) ?  @z Q(z ) (38)
with a necessarily nonpositive parameter ,
 = qE (tentry+ ) : (39)
If UE  0 holds on a certain subinterval, the Hamiltonian has no unique
maximum on the set of admissible controls. From E  0 and _ E  0, it can
be derived E = const yielding
sin E  = 0 ; (40)
cos E  2 f?1; 1g : (41)
Note that on singular subarcs, the optimal ight path of E is in a vertical plane.
The control that keeps  E  0 can be obtained by dierentiating this identity

11
twice with respect to time. Then there holds
 
nE  cos E  = cos E 1 + 21g @ ln(@h
(hE )) v 2 :
E (42)
E
Since the activity of the dynamic pressure constraint must be maintained,
Eq. (32) with the equality sign yields

E  = T 1 D0;E + mE g sin E
E;max 
? 12 @ ln(@h(hE )) mE vE 2 sin E + DI;E nE  2 : (43)
E
An additional necessary condition on singular subarcs is
@ 
@ (nE cos E )  E  0 : (44)
As long as this condition holds, the singular subarc cannot be left without
contradiction to the minimax principle. For, one obtains two local maxima
of the Hamiltonian for all nE cos E diering from the respective value (42)
on the singular subarc. None of these maxima can be chosen as candidate
optimal, since the other immediately yields a larger value of the Hamiltonian;
see [21] for more details. Equation (44) is always satised here on singular
subarcs. Hence, optimal trajectories always terminate on singular subarcs, if
they include a singular subarc and if no other constraint is violated by the
singular control. If the game terminates on a singular subarc, it can be easily
shown from  _ E (tf ) = 0 that the velocity vector of E at the nal time tf is
parallel to the line of sight.
Finally, it should be mentioned that singular subarcs are located in a singular
surface of universal type. Entering the universal surface, the adjoint variables
are continuous.

3.4 Jump Conditions and Interior Point Conditions

The following conditions are related to the transition points (i) between the dif-
ferent thrust phases of the missile, (ii) between nE;max-constrained subarcs with
E  = E;min and nE;max-constrained subarcs with E  = E;max , (iii) between
nE;max-constrained subarcs and free subarcs, (iv) between free subarcs and
qE;max-regular-constrained subarcs, (v) between qE;max-regular-constrained
subarcs and free subarcs, and (vi) between qE;max-regular-constrained subarcs
and qE;max -singular-constrained subarcs.
(i) The discontinuities in the right-hand side of the dynamic system (1)
induced by the dierent thrust phases of P , create singular surfaces in the state
space which lead to jumps in the adjoint vector when the optimal trajectory

12
penetrates these surfaces. The two transition points t1 = 3 [s], and t2 = 15 [s]
between the dierent thrust phases must be treated as interior point conditions.
The continuity of the Hamiltonian implies that  has discontinuities at ti ,
i = 1; 2,
?

 (ti ? ) ?  (ti + ) = mvP ((tti)) TP;max(ti + ) ? TP;max(ti ? ) ; (45)
P i
see Leitmann [22].
(ii) The switching point tE associated with that junction is determined by
the interior point condition
vE (tE ) = 0 : (46)

At this point, a bang of E from the minimal admissible value to the maximal
admissible value takes place.
(iii) The optimal load factor nE  maximizing the Hamiltonian H moves
from the nE;max -boundary into the interior of [nE;min; nE;max]. Therefore, the
associated switching point tnE is determined by [compare (28)]
mE g UE 

2 DI;E vE vE t=tnE = nE;max : (47)

(iv) It can be shown that the control variables of the aircraft E are continu-
ous at the transition point tentry into a dynamic-pressure-constrained subarc.
The adjoint vector is discontinuous, the jump is given by Eq. (38). Note that
the adjoint equations must be modied according to Eq. (37) on those subarcs.
The switching point tentry is determined by the interior point condition
1 2
2 (hE (tentry )) vE (tentry) = qE;max : (48)
(v) The optimal load factor nE  maximizing the Hamiltonian does no longer
satisfy the condition (d=dt)Q = 0. Therefore, the optimal trajectory leaves
the dynamic pressure constraint. The exit point texit is determined by [com-
pare (28)]
mE g UE 

2D v   = nE  (texit ? ) : (49)
I;E E vE t=texit
Note that this condition is equivalent to qE (texit? ) = 0.
(vi) The optimal control variables nE  and E  are discontinuous at the
transition point tsing into a singular subarc. The following conditions can be
shown,
UE (tsing ? ) = 0 ; (50)
( sin  (t ?) = 0 ;
E sing
_ E (tsing ? ) = 0 ) (51)
cos E  (tsing ? ) 2 f?1; 1g ;

13
 _ E (tsing ? ) = 0 ; (52)
_ E (tsing ? ) = 0 : (53)

3.5 Two-Point Boundary Conditions

The initial conditions are given by Eqs. (2). In order to obtain conditions
at t = tf for the adjoint variables, the terminal manifold described by Eq. (3)
is parameterized; see [17]. Partial dierentiation of the value V  ((tf ); z (tf ))
with respect to the parameters yields a system of linear equations for (tf ).
One obtains, for trajectories not ending on the dynamic pressure limit,
xE (tf ) [yE (tf ) ? yP (tf )] ? yE (tf ) [xE (tf ) ? xP (tf )] = 0 ;
xE (tf ) [hE (tf ) ? hP (tf )] ? hE (tf ) [xE (tf ) ? xP (tf )] = 0 ;
xP (tf ) + xE (tf ) = 0 ; yP (tf ) + yE (tf ) = 0 ;
hP (tf ) + hE (tf ) = 0 ; (54)
vP (tf ) = 0 ; vE (tf ) = 0 ;
 P (tf ) = 0 ;  E (tf ) = 0 ;
P (tf ) = 0 ; E (tf ) = 0 ;
 (tf ) = 1 ;
and for those which do,
xE (tf ) [yE (tf ) ? yP (tf )] ? yE (tf ) [xE (tf ) ? xP (tf )] = 0 ;
xE (tf ) [hE (tf ) ? hP (tf )] ? hE (tf ) [xE (tf ) ? xP (tf )] = 0 ;
xP (tf ) + xE (tf ) = 0 ; yP (tf ) + yE (tf ) = 0 ;
hP (tf ) + hE (tf ) ? 21 vE (tf ) @ ln(@h (hE (tf ))) v (t ) = 0 ;
E f (55)
E (tf )
vP (tf ) = 0 ; vE (tf ) = arbitrarily nonnegative ;
 P (tf ) = 0 ;  E (tf ) = 0 ;
P (tf ) = 0 ; E (tf ) = 0 ;
 (tf ) = 1 :
Together with the Isaacs equation

H (; z;  ; ; nP  ; P  ; nE  ; E  ; E  )t=tf = 0 ; (56)
the adjoint variables are determined at t = tf . An additional terminal condition,
which determines the nal time tf is given by the capture condition (3).

14
3.6 Singular Subarcs
On state-constrained subarcs
qE (hE ; vE )  qE;max ; (57)
the optimal controls may become singular indicated by UE  0. This equation
is equivalent to
 E  0 ; (58)
E  0 : (59)
These three identities dene a universal-type singular surface of codimension
three.
For the 12 state variables, the 13 adjoint variables, and the parameters tentry,
tsing , and tf , there are 28 interior and boundary conditions to be satised,
namely the 12 initial conditions (2), the capture condition (3), the 13 terminal
conditions (55) and (56), the entry condition (48) into the state-constrained
subarc, and the entry condition (50) into the singular subarc.
It should be mentioned, that Eq. (50) itself cannot be chosen for numerical
reasons. Instead, one of the conditions
 E (tsing ) = 0 ; (60)
E (tsing ) = 0 : (61)
is prescribed. The other one is checked afterwards and can always be found to
be fullled with an absolute error of at most 10?6.
If additional constraints become active here, e. g., the altitude constraint (6),
the switching structure might become even more complicated; see [12] and Sey-
wald, Cli, and Well [28] for similar problems in optimal control where more
complicated switching structures occur due to prescribed terminal conditions.
This section completes now the derivation of well-dened multipoint bound-
ary-value problems for all switching structures that have been obtained during
the numerical computations by means of the multiple shooting method.

4 Numerical Results
Although superior with respect to accuracy and reliability, the multiple shoot-
ing method suers from the fact that an initial estimate of all variables must
be provided. This diculty is surmounted by using a trajectory from [21] as
initial guess where ight scenarios in a vertical plane have been investigated.
For, if the initial positions and the velocity vectors of P and E are placed in a
common vertical plane, the optimal trajectories remain in this plane. The pro-
cedure in [21] is based on the following idea: Firstly, an optimal control problem

15
is established which is related to a family of dierential games including the dif-
ferential game to be solved. Secondly, the optimal control problem is solved by
means of a robust direct collocation method due to [30] which is not so exacting
with respect to an appropriate initial guess. This method additionally yields an
approximation for the adjoint variables and a hypothesis of the switching struc-
ture. Thereafter, the multiple shooting method is employed in connection with
homotopy techniques to solve the dierential games. Finally, the trajectories
for arbitrary initial conditions in the capture zone are computed by means of
homotopy techniques, too.

hP ;hE = [km]

xP ;xE = [km]

yP ;yE = [km]

Figure 1: Optimal ight path of P (gray) and E (black) and projections into
the coordinate planes.
To illustrate this, the following set of initial conditions is chosen
xP;0 = 0 [m] ; xE;0 = 12500 [m] ;
yP;0 = 0 [m] ; yE;0 = 1000 [m] ;
hP;0 = 5000 [m] ; hE;0 = 6000 [m] ;
(62)
vP;0 = 250 [m/s] ; vE;0 = 400 [m/s] ;
P;0 = 0 [deg] ; E;0 = 0 [deg] ;
P;0 = 0 [deg] ; E;0 = 120 [deg] :
Figure 1 shows the optimal ight paths of P and E in a (x; y; h)-coordinate
system. Additionally, the projections of the ight paths into the coordinate
planes are drawn (thin dashed lines). It can be seen that the optimal ight paths
of P and E enter a common vertical plane. The missile P ascends immediately

16
after the launch in order to exploit the low drag in the higher regions of the
atmosphere. Since the initial angle E;0 = 120 [deg] is quite unfavourable,
the aircraft E is forced to turn right as sharp as possible. Hence, a nE;max -
constrained subarc occurs during the rst 7 seconds of the maneuver; see Fig. 3.

hP ; hE = [km]
ight envelope

vP ; vE = [m/s]

Figure 2: History of h and v for P (gray) and E (black).

Figure 2 shows the history of the optimal trajectory in an altitude-velocity
diagram. The shaded area marks the set of all pairs (hE ; vE ) for which E can
maintain a stationary horizontal ight. The boundary of this set is called the
ight envelope of the aircraft E . Note that the lower right part of the ight en-
velope is determined by the dynamic pressure constraint. It should be mentioned
that only the mid course guidance of the missile is modelled here appropriately.
The nal approach immediately before the hit must be left to special guidance
laws. The corners in the curve of P mark the two transitions between the dif-
ferent thrust phases of the missile. The eects of these transition points govern
also the history of the optimal load factor nP  in Fig. 3. The corner in the
optimal load factor nE  at t = 27 [s] marks the junction point between inactive
and active dynamic pressure constraint. At t = 29 [s] the entry point of the sin-
gular subarc is reached. Note that nE  as well as E  are discontinuous at tsing ;
see Fig. 4. Figure 5 shows the history of the optimal throttle E  which is also
discontinuous at tsing . However, the jump in E  is below drawing accuracy.

17
nP  ; nE 

t = [s]

Figure 3: Optimal load factors for P (gray) and E (black).

P  ; E  = [deg]

t = [s]

Figure 4: Optimal bank angles for P (gray) and E (black).

18
 E 

t = [s]

Figure 5: Optimal throttle for E .

In Fig. 6 eight optimal ight paths are depicted in the (x; y; h)-space. For
the sake of visibility, only the projections into the (x; y)-plane are drawn. These
ight paths belong to the initial conditions
xP;0 = 0 [m] ;
yP;0 = 0 [m] ; yE;0 = 0 [m] ;
hP;0 = 5000 [m] ; hE;0 = 10500 [m] ;
(63)
vP;0 = 250 [m/s] ; vE;0 = 300 [m/s] ;
P;0 = 0 [deg] ; E;0 = 0 [deg] ;
P;0 = 0 [deg] ; E;0 = 90 [deg] ;
xE;0 = [m] 2 f3000; 4500; 6000; 7500; 9000; 10500; 12000; 13500g :
The dynamic pressure constraint becomes active only for the trajectory with
the largest initial distance. The two optimal trajectories related to the smallest
and the largest value of xE;0 lie close to the barrier. The dierences of the
optimal ight paths, depending on the initial distance between P and E , are
considerable.

19
 hP ;hE = [km]

xP ;xE = [km]

yP ;yE = [km]

Figure 6: Optimal ight paths for dierent initial constellations. Flight paths
of P are gray and those of E are black. Initial conditions dier only in xE;0 .

Finally, Fig. 7 shows seven optimal ight paths related to the following set
of initial conditions
xP;0 = 0 [m] ; xE;0 = 15000 [m] ;
yP;0 = 0 [m] ; yE;0 = 0 [m] ;
hP;0 = 10000 [m] ; hE;0 = 10000 [m] ; (64)
vP;0 = 250 [m/s] ; vE;0 = 500 [m/s] ;
P;0 = 0 [deg] ; E;0 = 0 [deg] ;
P;0 = 0 [deg] ;

E;0 = [deg] 2 f0; 30; 60; 90; 120; 150; 177g :

As in Fig. 6, the turns of the optimal paths into common vertical planes can
be seen. Each trajectory terminates on a singular subarc. Note that the almost
head-on initial constellation for E;0 = 177 [deg] is close to a dispersal surface
in the state space beyond which a symmetric situation can be found. For the
tail-chase initial constellations, the missile P again ascends into the regions of
low drag. This behavior is optimal for P if the duration of the game is not too
short.

20
 hP ;hE = [km]

xP ;xE = [km]

yP ;yE = [km]

Figure 7: Optimal ight paths for dierent initial constellations. Flight paths
of P are gray, those of E are black. Initial conditions dier only in E;0 .

5 Conclusions
Optimal trajectories and their associated open-loop representations of the op-
timal strategies can be computed for complicated pursuit-evasion games in the
entire capture zone. These open-loop representations provide information by
which optimal strategies can be approximated globally, e. g., by the subsequent
solution of neighboring boundary-value problems [2], by Taylor series expansions
around many optimal trajectories ([3], and [4]), or by neural networks [27]. The
last two of these methods are real-time feasible. This idea is generally applicable
and not conned to the air-combat problem investigated in this paper.
This air-combat problem includes a rst-order state variable inequality con-
straint for the dynamic pressure of the aircraft. The numerical solutions of this
dierential game have been obtained by applying the known necessary condi-
tions of optimal control theory. In addition, the controls may become singular
on state-constrained subarcs. The associated singular surface is of universal type
and has codimension three. All the necessary conditions have been examined for
not contradicting the minimax principle. More detailed and rigorous investiga-
tions concerning dierential games with state constraints of order greater than
zero are still pending.

21
Acknowledgements
The authors would like to thank Professor R. Bulirsch of Munich University
of Technology, Department of Mathematics, for his encouragement. The au-
thors are also indebted to the Reviewers for their helpful comments. This work
has been partly granted by the German National Science Foundation within
the Project \Applied Optimization and Control", and by FORTWIHR, the
Bavarian Consortium on High Performance Scientic Computing.

REFERENCES
[1] Basar, T. , and Olsder, G. J., Dynamic Noncooperative Game Theory ,
Academic Press, London, Great Britain, 1982, 2nd ed., 1994.
[2] Breitner, M. H., Construction of the Optimal Feedback Controller for
Constrained Optimal Control Problems with Unknown Disturbances , in:
R. Bulirsch, and D. Kraft (Eds.), Computational Optimal Control, Inter-
national Series of Numerical Mathematics 115, Birkhauser, Basel, Switzer-
land, pp. 147{162, 1994.
[3] Breitner, M. H., Real-Time Capable Approximation of Optimal
Strategies in Complex Dierential Games , in: M. Breton, G. Zaccour
(Eds.), Proceedings of the Sixth International Symposium on Dynamic
Games and Applications, St-Jovite, Quebec, GERAD, E cole des Hautes
E tudes Commerciales, Montreal, Canada, pp. 370{374, 1994.
[4] Breitner, M. H., Robust-optimale Ruckkopplungssteuerungen gegen un-
vorhersehbare Ein usse: Dierentialspielansatz, numerische Berechnung
und Echtzeitapproximation , PhD thesis, Department of Mathematics, Mu-
nich University of Technology, Munich, Germany, in preparation.
[5] Breitner, M. H., and Pesch H. J., Reentry Trajectory Optimization
under Atmospheric Uncertainty as a Dierential Game , in: T. Basar, and
A. Haurie (Eds.), Advances in Dynamic Games and Applications, An-
nals of the International Society of Dynamic Games 1, Birkhauser, Basel,
Switzerland, pp. 70{88, 1994.
[6] Breitner, M. H., Grimm, W., and Pesch, H. J., Barrier Trajectories
of a Realistic Missile/Target Pursuit-Evasion Game , in: R. P. Hamalainen,
and H. K. Ehtamo (Eds.), Dierential Games | Developments in Model-
ling and Computation, Lecture Notes in Control and Information Sciences
156, Springer, Berlin, Germany, pp. 48{57, 1991.
[7] Breitner, M. H., Pesch, H. J., and Grimm, W., Complex Dierential
Games of Pursuit-Evasion Type with State Constraints, Part 1: Necessary

22
 Conditions for Optimal Open-Loop Strategies, Part 2: Numerical Compu-
tation of Optimal Open-Loop Strategies , Journal of Optimization Theory
and Applications 78, pp. 419{441, 443{463, 1993.
[8] Bryson, A. E., Denham, W. F., and Dreyfus, S. E., Optimal Pro-
gramming Problems with Inequality Constraints, I: Necessary Conditions
for Extremal Solutions , AIAA Journal 1, pp. 2544{2550, 1963.
[9] Bulirsch, R., Die Mehrzielmethode zur numerischen Losung von Rand-
wertproblemen und Aufgaben der optimalen Steuerung , Report of the Carl-
Cranz-Gesellschaft, DLR, Oberpfaenhofen, Germany, 1971; Reprint, De-
partment of Mathematics, Munich University of Technology, Munich, Ger-
many, 1985 and 1993.
[10] Fink, G., Numerische Berechnung optimaler Strategien fur ein realitats-
nahes Dierentialspiel einer Verfolgung im dreidimensionalen Raum , Dip-
loma thesis, Department of Mathematics, Munich University of Techno-
logy, Munich, Germany, 1994.
[11] Greenwood, N. J. C., Applied Dierential Games in the Aerial Scen-
ario , PhD thesis, Department of Mathematics, University of Queensland,
Brisbane, Australia, 1994.
[12] Grimm, W., Lenkung von Hochleistungs ugzeugen. Vergleich von opti-
maler Steuerung und fastoptimaler Echtzeitsteuerung , PhD thesis, Depart-
ment of Mathematics, Munich University of Technology, Munich, Germany,
1992.
[13] Guelman, M., Shinar, J., and Green, J., Qualitative Study of a Planar
Pursuit Evasion Game in the Atmosphere , in: Proceedings of the AIAA
Guidance, Navigation and Control Conference, Minneapolis, Minnesota,
1988.
[14] Gutman, S., and Katz, D., On Guaranteed-Cost, Closed-Form Guidance
via Simple Linear Dierential Games , in: Proceedings of the 27th IEEE
Conference on Decision and Control, Austin, Texas, 1988.
[15] Hiltmann, P., Chudej, K., and Breitner, M. H., Eine modizierte
Mehrzielmethode zur Losung von Mehrpunkt-Randwertproblemen | Ben-
utzeranleitung , Report No. 14, Deutsche Forschungsgemeinschaft, Sonder-
forschungsbereich 255 \Transatmospharische Flugsysteme", Department
of Mathematics, Munich University of Technology, Munich, Germany,
1993.
[16] Isaacs, R., Games of Pursuit , Paper No. P-257, RAND Corporation,
Santa Monica, California, 1951.

23
[17] Isaacs, R., Dierential Games , Wiley, New York, New York, 1965;
Krieger, New York, New York, 1975.
[18] Jacobson, D. H., Lele, M. M., and Speyer, J. L., New Necessary
Conditions of Optimality for Control Problems with State Variable Inequal-
ity Constraints , Journal of Mathematical Analysis and Applications 35,
pp. 255{284, 1971.
[19] Lachner, R., Realitatsnahe Modellierung eines Dierentialspieles mit
Zustandsbeschrankungen und numerische Berechnung optimaler Trajekt-
orien , Diploma thesis, Department of Mathematics, Munich University of
Technology, Munich, Germany, 1992.
[20] Lachner, R., Breitner, M. H., and Pesch, H. J., Optimal Strategies of
a Complex Pursuit-Evasion Game , in: R. Fritsch, and M. Toepell (Eds.),
Proceedings of the 2nd Gauss Symposium, Munich, Germany, 1993; to
appear in Journal of Computing and Information, 1995.
[21] Lachner, R., Breitner, M. H., and Pesch, H. J., Ecient Numerical
Solution of Dierential Games with Application to Air Combat , Report
No. 466, Deutsche Forschungsgemeinschaft, Schwerpunkt \Anwendungs-
bezogene Optimierung und Steuerung", Department of Mathematics, Mu-
nich University of Technology, Munich, Germany, 1993.
[22] Leitmann, G., Einfuhrung in die Theorie optimaler Steuerung und der
Dierentialspiele , R. Oldenburg, Munich, Germany, 1974.
[23] Maurer, H., Optimale Steuerprozesse mit Zustandsbeschrankungen , Ha-
bilitationsschrift, University of Wurzburg, Wurzburg, Germany, 1976.
[24] Miele, A., Flight Mechanics I, Theory of Flight Paths , Addison Wesley,
Reading, Massachusetts, 1962.
[25] Pesch, H. J., Oine and Online Computation of Optimal Trajectories in
the Aerospace Field , in: A. Miele and A. Salvetti (Eds.), Applied Math-
ematics in Aerospace Science and Engineering, Plenum, New York, New
York, pp. 165{219, 1994.
[26] Pesch, H. J., Solving Optimal Control and Pursuit-Evasion Game Prob-
lems of High Complexity , in: R. Bulirsch, and D. Kraft (Eds.), Compu-
tational Optimal Control, International Series of Numerical Mathemat-
ics 115, Birkhauser, Basel, Switzerland, pp. 43{64, 1994.
[27] Pesch, H. J., Gabler, I., Miesbach, S., and Breitner, M. H., Syn-
thesis of Optimal Strategies for Dierential Games by Neural Networks , in:
G. J. Olsder (Ed.), Advances in Dynamic Games and Applications, Annals
of the International Society of Dynamic Games, this volume, Birkhauser,
Basel, Switzerland, 1995.

24
[28] Seywald, H., Cliff, E. M., and Well, K. H., Range Optimal Traject-
ories for an Aircraft Flying in the Vertical Plane , Journal of Guidance,
Control, and Dynamics 17, pp. 389-398, 1994.
[29] Stoer, J., Bulirsch, R., Introduction to Numerical Analysis , Springer,
New York, New York, 2nd ed., 1993.
[30] von Stryk, O., Numerische Losung optimaler Steuerungsprobleme:
Diskretisierung, Parameteroptimierung und Berechnung der adjungier-
ten Variablen , Fortschritt-Berichte VDI, Reihe 8, Nr. 441, VDI-Verlag,
Dusseldorf, Germany, 1995

25