A Metric in Angularly Quantized Polar

Domain for Pole-Based Classification

of Stochastic Models and Dynamic
Systems EGNiYA Teknik Rapor
(Ingilizce)

Agustos 2016

EGNiYA
 A Metric in Angularly
Quantized Polar Domain for
Pole-Based Classification of
Stochastic Models and
Dynamic Systems
Abstract Autoregressive (AR) models are
used in a variety of applications and an AR
model can be represented by the poles
corresponding to describing AR coefficients.
Similarly, the behavior of linear dynamic single-
input-single-output (SISO) systems can be
described as a function of the system poles,
which are directly estimated from the given data
and represent a system as a set of poles without
any identities, which is analogous to the nature
of association-free multi-target tracking and
corresponding application of set distances
known as optimal subpattern assignment
(OSPA) distance. In this work, we define a new
metric in terms of sums and differences of
magnitude and angles of poles in pole-space and
provide a measure of the distance between AR
processes, or linear dynamic systems
represented by poles.
Index TermsAutoregressive (AR) models,
classification, stochastic models, metric, distance
measures, poles, linear dynamic systems
I. INTRODUCTION
In many applications, autoregressive (AR)
models are increasingly popular. In radar signal
processing, the properties of the Gaussian clutter
can be analyzed by modeling it by an AR process.
In biomedical applications, AR models can be
used to classify signals of patients with a specific
pathology from signals recorded with healthy
people. In speech processing such as speech
analysis and Kalman-filter based enhancement,
sets of AR parameters { } estimated by
different methods, are often compared one
another. [10] Another area is classification of
systems described by poles which includes
several similar but slightly different topics.
Change detection considers the problem of
finding modifications to the system based on
observations. Anomaly detection is concerned
with finding elements or behavioral patterns
deviating from elements or patterns that are
defined as normal. Outliers detection addresses
the problem of finding elements of a set that
deviate markedly from other set members. Fault
detection is concerned with the problem of
recognizing system behavior or states that
exceed a permitted region of operation. Concept
drift addresses the very slow and gradual
changes in processes. Novelty detection emerged
from the field of machine learning and has its
focus on discerning incoming information with
respect to its coverage by already learned
models and pattern. In all those areas, a form of
model/system comparison (based on either
dissimilarity/distance/divergence or contrast)
and consequent classification is required. [9]
Commonly used dissimilarity measures between
AR-model pairs include the Itakura divergence,
the Itakura-Saito divergence, the log-spectral
distance or Jeffreys divergence (JD) which are
based on spectral characteristics of the
processes. [9],[6] One common distance
measure between systems is based on the
spectral characteristics, namely a distance
between the cepstra, the inverse Fourier
transform of the logarithm of the power
spectrum. The ARMA distance, a metric for
ARMA processes as the distance of two cepstra,
can be calculated by a function of the system
poles. [4] Another distance measure between
systems is association-free distance measure,
namely the optimal subpattern assignment
(OSPA) distance (and a variant of it called as
MAX-OSPA), the system distance as a distance
between their sets of poles representing the
spectral characteristics of the system. [1], [2], [9]
The key to comparison and classification of
models or systems represented by poles is a set
of representative and intuitive parameters, (in
order to define a distance measure as a basis for
decision-making) which can be the set of the AR
parameters or system coefficients, the set of
poles or the corresponding PSDs. [9], [10] In this
work, the main contribution is a new approach
and derivation of a novel metric based on poles
which can be used for classification of AR
processes, or classification, identification or
change detection in linear dynamic systems
represented by system poles.
The remainder of this paper is structured as
follows. First, in Sec. II we give a formal
description of the problem along with
parametric models, pole estimation and existing
approaches. Sec. III introduces the classification
of stochastic models and linear dynamic systems
in pole-space. We derive an efficient novel
metric in pole-space in Sec. IV. A numerical
example along with a comparison of the
proposed metric is presented in Sec. V. Finally,
Sec. VI concludes the work.
1
 II. PROBLEM STATEMENT D. Existing Approaches

A. Mathematical Model There are several proposals in literature using

The distance between a reference model
() ()
and a test model calculated on basis of
() ()
their parameter vectors and , which are
usually estimated from the input and observed
output data,
() () ()
= ( , ) = ( , )
where the superscripts () and () represents
the reference and test models respectively,
which are either two models in AR models space
or two systems in linear time-varying systems
space. [9]
B. Parametric Models
Autoregressive (AR), moving-average (MA) and
autoregressive moving average (ARMA) models
are useful time-domain models for the
representation of discrete-time signals. The time
series is called as an ARMA(, ) model if
= =0 + =0 (2)
where is uncorrelated noise of variance 2 . In
domain the system function is
information on the positions of poles,
amplitudes of characteristic frequencies and
subspace formulations [10], which constitutes
different metrics and distance measures used for
classification of models and systems. We are
briefing some popular ones and shortly
mentioning about similarities and limitations of
()
others next in Section III.
(1)
III. POLE-BASED DISTANCE MEASURES
A. Base Distance of Poles
The base distance (. , . ), with : 0+ as a
function satisfying identity, symmetry, and the
triangle inequality. The base distance is the
() ()
distance between two complex poles and
The plane, in which the poles of a discrete-
time system are defined, can be interpreted as a
specific Mobius transformation from the more
intuitive plane of a continuous-time system
that transforms the Cartesian coordinate system
of the plane into a polar coordinate
system.[9] Let and be two poles of a
continuous-time system in the plane. The
base distance (, ) between and is defined
as

(1 /)
() = 2 =0 = 2 =1
(3) (, ) = [() ( )]2 + [() ( )]2

=0
(1 /)
=1
(4)
which is the Euclidian distance between and
where 0 = 0 = 1, the s are the AR in the complex plane. Let and be two poles of
coefficients, s are the MA coefficients, and a discrete-time system in the plane. The base
are the poles and zeros of the model distance (, ) between and is defined as
respectively. [4]
1
(, ) = [() ( )]2 + [ ]2 (5)
C. Pole Estimation

where is the sampling period, , and , are

There are different approaches for the
polar coordinates of and respectively. [9]
estimation of the poles of a system but our main
focus in this work is providing a metric for a
given set of system or model poles rather than B. OSPA and MAX-OSPA Distance
estimation or statistical properties of the
estimator, so we assume we have different sets The optimal subpattern assignment (OSPA)
of poles obtained using a proper pole estimation distance between two sets of poles
() () () () () ()
method. {1 , 2 , , } and {1 , 2 , , } each
comprising elements and given by the ordered
However, in systems with real valued output, () () () ()
vectors = [1 , 2 , , ] and
poles occur either real or in complex conjugate
() () () ()
pairs and a known proportion of real and = [1 , 2 , , ] is defined by
complex poles can be assumed. Since an a prior () ()
( , )
estimate is easy to attain from a small number of 1

data samples, this assumption is not unrealistic. 1 () ()
= [ min ( ( , () ) )]

=1

2
 (6)
() ()
with 1 , ( , ) is the base distance
()
between the two elements and
and
describes all permutations of the set {1,2, , }.
()
The notation () represents the -th element of
()
the permutation of which is generated by
reordering the vector. Given two sets of poles by
() ()
the two vectors and , the MAX-OSPA
distance is defined by
() ()
( , ) = max=1,2,.. ( ( , () ))
(7)
where is the optimal OSPA permutation given
by
()
= min [max=1,2,.. ( ( , () ))]
()
So, the maximum base distance between two
() ()
single elements of a pair and is chosen,
while using the optimal subpattern assignment
over the maximum norm ( ) resulted in
minimized maximal base distance.
The OSPA distance finds the optimal assignment
by minimizing over the sum of distances
between poles. The MAX-OSPA chooses an
optimal assignment by minimizing the maximum
distance between two poles.
In OSPA and MAX-OSPA a weighting scheme is
generally used (usually as the exponential
weighting) in order to take closeness of poles to
unit circle (or imaginary axis in domain) into
account [9]
E. Limitations & Requirements
()
Most of the existing or proposed metrics or
measures use either norms for distances
between poles in domain or transformed
poles in domain assisted with some
weighting schemes. The pole representation in
either in domain or in domain is a two
dimensional representation and any distance
measure based solely on this two dimensional
representation (which corresponds to a pattern
with two features forming a two dimensional
() () feature vector) is far from measuring the
distance between corresponding
models/systems. Besides a distance measure in
pole-space must consider not only the distance
between poles of reference and test models but
also the distance among the poles (intra-pole
(8) distance) of each model/system. So the distance
among poles of a model is a factor as well, which
might be as important as the closeness of a pole
to the unit circle. In other words, the metrics or
distance measures based solely on two
dimensional representation might suffer from
loss of available information. ARMA metric can
be thought as a nonlinear transformation of two
dimensional space into a higher dimensional
space and might not have suffering from loss of
information.
The main requirement can be defined as reliable,
analytically tractable, physically meaningful and
computationally efficient metric to measure
distance between models or systems in their
associated spaces. We are outlining a new
approach and a new metric based on this
requirements avoiding majority of the
limitations next in Section IV.

C. ARMA Metric IV. PROPOSED APPROACH & METRIC

ARMA metric is one of the popular metrics A. Preliminaries
whose derivation is based on cepstrum domain
processing or homomorphic processing. For two A model/system can be represented by poles,
()
stable AR models a reference model and a coefficients or PSD. Although there is not a one-
()
test model with orders and two sets of to-one correspondence, pole representation can
() () () () ()
poles {1 , 2 , , } and {1 , 2 , , } the
() be thought as a vector quantized form, or low
rank approximation or projection onto a lower
ARMA metric is defined as; dimensional space, in other words it contains all
() ()
the information contained by PSD in a lower
( , ) dimensional space. Our aim is extracting this
1/2
=1 =1(1 () () () () available information contained in poles to
) =1 =1(1 )
= [ ] measure the distance between two
=1 =1(1 () () () ()
) =1 =1(1 ) models/systems.
(9)
B. Approach
which is in the form of finite products in pole
domain. [4]

3
 () ()
Consider two 1 (as pole vectors, consisting of (or ) as components in the
poles as components of the vector) respective diagonal elements. (and also provides
() () () ()
= [1 , 2 , , ] a link to cepstrum, FFT and PSD domains)
() () () ()
= [1 , 2 , , ] (10)

() ()
where and represents the pole vectors
of reference and test models/systems
() ()
respectively. and are the poles which can
be written as
() () ()
=
() () ()
= (11)
in polar domain. The key idea is extracting
information contained in poles by using a
transformation to a higher dimensional space in
such a way that each pole (actually magnitude of
each complex pole) corresponds to a coordinate Figure 1. Selection of angular
in the new transformed space and each quantization parameter (number of
model/system can be represented as a point angular sectors) based on minimum
(which is defined by the coordinates of the angular separation between poles
model/system) in this high dimensional space,
(which can be thought as a model/system space Although we will not use angular sectorization
where magnitudes of poles forms the and high dimensional space in final metric
coordinates as aligned vectors with a subgroup definition, it is useful in derivation. Selection of
of the axes of new space) . is important and one condition on that comes
from representability of each model/system as a
Assume that polar domain is quantized angularly point in the new high dimensional space, which
(with angular sectors, each angular sector means each angular sector must contain only
width is 2/, = 2/ and they are one pole of either reference or test model
numbered from 1 to in counterclockwise otherwise none. This can be satisfied by
direction starting from (0,1) point of complex selecting considering the minimum angular
plane) as shown in Fig.1, each non-empty (not separation between poles of either test or
containing any poles) angular sector contains reference model, whichever is smaller. (This
either one pole of reference model or one pole of operation takes the distance among poles of
test model. model/system (intra-poles distance) argument
in previous section into account)
If we think each angular sector as an axis of the
new space (which is the high dimensional C. Proposed Metric
transform space with dimension ), then each
model/system can be uniquely represented as Since we have -dimensional vector space and
dimensional vectors () () models/systems represented as points in this
and which are
space as

()

() () ()
= [0, 0, . . 1 , 2 , , ,0,0]
()

() () ()
= [0, 0, . . 1 , 2 , , ,0,0]

()

() () ()
= [0, 0, . . 1 , 2 , , ,0,0] (12)
()

() () ()
= [0, 0, . . 1 , 2 , , ,0,0] (13)
now model/system vectors for reference and
test models/systems respectively.(note that, the then we can use familiar vector space properties
() () () and define the metrics to measure distance
positions of both and components in between the models/systems.
and ()
is based on angular quantization and the
value of ). Although usual vector space metrics (1 and 2
or -norms for example) can be used to measure
This operation (angular quantization and distance between two models represented by
obtaining model/system vectors) can be thought ()
vectors and ()
, this would be a coarse
as transforming the original 1pole vectors to distance measure (does not take angular sector
1 vectors by a rank matrix which contains index or angular separation into account, and

4
results in virtual equidistant models) and space formed by radial quantization and weights
requires formation of those -dimensional as pole angle differences are the absolute value
vectors implying realization of angular metrics on this new high dimensional space
sectorization by search methods or matrix constructed by radial quantization. Then
multiplication which we avoid due to weighted distance measure can be written as;

computational efficiency considerations. To be
able to take effect of angular separation into 1 ()
= 1 ( () () ()
, ) = | () ()|
account and in fact to be able to provide a solid =1
link to cepstrum, FFT and PSD domains a form (16)
of weighting should be applied as well. where is the diagonal weighting matrix
Weighting can be applied either in complex with each diagonal element is in the form of
domain as pre-weighting (which is a transform multiples of ,
() ()
of 1vectors and by a real valued
0 0
diagonal weighting matrix) or in high
0 2 . .
dimensional transform space as post weighting =[ ] (17)
.
(which is a transform of 1 vectors ()
and 0 .
()
by a real valued diagonal weighting

matrix). In first case since original poles move In limiting case when any two poles are too
through radial directions the distances between close, 0 and , which allows us to
models/systems are preserved to a possible write Eq.16 as

scale factor. In latter case, the post weighting
matrix transforms dimensional linear vector ()
1 ( ()
, )
()
= (| | | |)(
() ()
+ )
()

space and distances are defined on =1

hyperellipsoids, in other words the distances can (18)
be called as well-known Mahalonabis distances. However, although defined for 0 , the
() ()
Before going back to dimensional pole space distance 1 is valid for to satisfy
and defining the metric in that space we initial assumptions that each angular sector
continue with derivations in dimensional contains only one pole of either test or reference
space. model or not any poles. Using similar arguments
for radial quantization as used in angular
When the minimum angular separation is sectorization gives the final form of the metric in
chosen as minimum angular separation between pole space which we call as (PSM stands for
the angularly nearest two poles, the respective pole space metric) and defined as;
() () ()
positions of both and components in
() () ()
and is different and absolute value metric = ( , )
()
can be written as sum of components and () () () ()
() = |( )(| | + | |)
.
=1

()
1 = 1 ( () () ()
, ) = [| () ()|]
() () () ()
+ (| | | |)( + )|
=1
=1
() () (19)
= ( + ) In vector notation, it can be rewritten as;
=1
(14) () ()
As mentioned, this would be a coarse distance ( , ) = | + |
measure producing virtual equidistant models where
() () () ()
and to take effect of angular separation a form of = [(1 1 ), , ( )]
weighting is applied as each weight defined as () () () ()
the angular separation between corresponding = [(1 + 1 ), , ( + )]
() () () ()
poles, = [(| | | |), , (| | | |)]
() ()
= | | | | (15) () () () ()
= [[(| | + | |), , (| | + | |)]]
Actually this weighting scheme can be (20)
considered as the corollary of angular It should be noted that this form of the metric is
sectorization as radial quantization, for each association-free or valid for unlabeled or non-
radial quanta each pole angle represents one ordered pole pairs of test and reference models
coordinate in the axes of the high dimensional which avoids permutations and also performs
well for both higher order models/systems or

5
applications requiring more than two model () ()
phases and changes between 0 and 2)
classification. in second scenario.

As seen from Eq.19, computational load is In first scenario, since the reference model has
relatively low, can be computed by 2 () ()
poles at = . / and = . /
multiplications and (4 + 1) additions, and can
when the radially moving complex pole pair of
be used for high order model classification
test model coincides with those then the
problems as well. We are dealing with a
distance becomes zero, as the complex poles pair
numerical example next in Section V.
moves away from that point the distance
between models increase as expected, as shown
V. NUMERICAL ANALYSIS in Fig 2.
There might be some applications where pole
dominancy or movement of some poles might be
important as in change detection or system
identification problems. We use pole movement
to illustrate the numerical evaluation of the
metric and provide a comparison with ARMA
and OSPA metrics.

() ()
The ARMA metric ( , ) used in
comparison simulations and graphs are actually
squared form of the ARMA metric given in Eq (9)
so can be written as;

() ()
( , )
= =( () () () () Figure 2. Behavior of Proposed Metric PSM
) = =( )
= [ () () () ()
] and comparison with ARMA & OSPA metrics
= =( ) = =( ) for radially moving complex pole pair
() ()
The PSM pole space metric ( , ) is
In second scenario, since the reference model
used as it is; () ()
has poles at = . / and =
() () . / when the angularly moving complex
( , ) = | + |
pole pair of test model coincides with those then
the distance becomes zero, as the complex poles
and the OSPA metric is used without number of pair moves away from that point the distance
poles normalization with = as; between models increase as expected, as shown

in Fig 3.
() () () ()
( , ) = [ ( ( , () ))]

=

Consider a reference model with

() ()
= . / = . /
() ()
= . / = . /
() ()
= . / = .
() ()
= . / = .

()
and a test model with same poles except and
()
complex pole pair, which are the only
difference between reference and test models
Figure 3. Behavior of Proposed Metric PSM
and those two complex pole pair move radially
() () and comparison with ARMA & OSPA metrics
(while phase kept same, magnitudes and for an angularly moving complex pole pair
changes between 0 and 1) in first scenario and
moves angularly (while magnitude kept same,

6
In both scenarios, the proposed metric
behaves properly as expected and this behavior
is very similar to ARMA metric, those two
scenarios summarize the overall picture of
classification of models or systems based on
their pole characterization, test model/system
poles move away or get closer to reference
model/system poles in time varying systems,
and at a distance in static model/system
classification, so the PSM metric can be
used in a great range of problems, especially for
the ones with computational sensitivities.
Figure 4. Computational requirements of
ARMA and PSM metrics in terms of
multiplications
Figure 5. Computational requirements of
ARMA and PSM metrics in terms of additions
The computational comparison in terms of
number of multiplications and number of
additions are given in Figure 4 and Figure 5 for
PSM and ARMA metrics. The reason OSPA metric
is not included in that plots is due to huge
number of both multiplications and additions
required by OSPA metric due to permutation
operation which makes computational load
proportional to ! . Just to show effect of this
permutation operation on computational load
we provided another plot which shows the
number of multiplications required by each
metric for number of poles up to ten in Figure 6.
In all those three graphs the axis showing the
required number of computations is logarithmic.
Figure 6. Computational requirements of
ARMA,OSPA and PSM metrics in terms of
multiplications
As seen from the graphs, the PSM metric is the
least demanding in terms of both multiplications
and additions and does not require
permutations for non-ordered set of poles,
which makes it suitable for
classification/identification of state changes in
linear dynamic systems as well as applications
involving high model orders and more than two
model classification requirements.
VI. CONCLUSIONS
We have described a new approach for pole
based distance measure and derived a
computationally efficient ( multiplications
and (( + ) additions), analytically tractable
and physically meaningful pole-based metric.
The metric can be used in classification of
models and systems which has a pole-space
representation and also in detection, estimation
and classification of change in model parameters
or system state changes in time varying AR
models or linear time varying dynamic systems.
Our main focus was on stable AR models and
systems represented by poles, however the
approach can be extended to involve other
models and might work for non-stable non-
minimum phase systems or the systems
represented by entities both inside and outside
of the unit circle of complex plane.
7
REFERENCES
[1] D.Schuhmacher, B.-T. Vo, and B.-N. Vo, A
Consistent Metric for Performance Evaluation of
Multi-Object Filters, IEEE Trans. Signal
Processing, Vol. 56, No. 8, pp. 3447-3457, Aug.
2008

[2] B. Ristic, B.-N. Vo,D. Clark and B.-T. Vo, A

Metric for Performance Evaluation of Multi-
Target Tracking Algorithms, IEEE Trans. Signal
Proc., Vol. 59, No. 7, pp. 3452-3457, July. 2011

[3] H. W. Kuhn, The Hungarian Method for the

Assignment Problem, Naval Research Logistics
Quarterly, 2:8397, 1955..

[4] R.J. Martin, A metric for ARMA processes,

IEEE Trans. Signal Process. 48 (2000) 1164
1170.

[5] P. Van Overschee and B. De Moor., Subspace

identification for linear systems: Theory
Implementation-Applications. Kluwer Academic
Publishers, 1996.

[6] M. Basseville, Distance measures for signal

processing and pattern recognition, Signal
Processing 18 (4) (December 1989) 349369.

[7] K. De Cock, B. De Moor, Subspace angles

between ARMA models, Systems Control Lett. 46
(4) (July 2002) 265270.

[8] G.H. Golub, C.F. Van Loan, Matrix

Computations, The Johns Hopkins University
Press, Baltimore, 1996.

[9] C. Chlebek and U. D. Hanebeck, Pole-based

Distance Measure for Change Detection in Linear
Dynamic Systems, in Proc. 2014 European
Control Conference (ECC 2014), Strasbourg,
France, June 2014.

[10] C. Magnant, E. Grivel, A. Giremus, L. Ratton,

B. Joseph, Classifying Autoregressive Models
Using Dissimilarity Measures: A Comparative
Study, in 23rd European Signal Processing
Conference(EUSIPCO)