A comparison between non-linear optimization methods of Bayesian

inversion and genetic algorithm for inverting spectral induced polarization

data for Cole-Cole parameters
Ahmad GHORBANI1, Gonca OKAY2, Philippe COSENZA3

1) Mining and Metallurgical Engineering department, Yazd University, Yazd, Iran, aghorbani@yazduni.ac.ir
2) UMR Sisyphe, Universit Paris 6, Paris, France
3) FRE CNRS 3114 HydrASA, Universit Poitiers, Poitiers, France

ABSTRACT
The induced polarization (IP) method has been used in mining prospecting and
increasingly in environmental investigations because IP measurements are very sensitive to
the low frequency capacitive properties of rocks and soils. Cole-Cole model parameters
widely use to interpret both of time and frequency domain induced polarization data.
Among many studies in which Cole-Cole parameters are estimated from SIP
measurements on soils and rocks, the majority use least squares methods. In this work, we
developed a Bayesian method with simulated annealing sampling algorithm to invert for
double Cole-Cole parameters from SIP data. We also reproduced the genetic algorithm
developed by Cao et al. and compared performance of simulated annealing method with
genetic algorithm method through inversion of synthetic data.
Both of two methods are provides a global approach for inverting SIP data for Cole-Cole
parameters; the obtained estimates are independent of initial values. Our results show that for
the SIP synthetic data with random noises up to 10%, the inversed parameters obtained from
simulated annealing method in comparison with genetic algorithm method are more close to
the real parameters.

Keywords: Bayesian inference, Simulated annealing, Genetic algorithm, Cole-Cole model,

SIP, inversion.

1
1- INTRODUCTION
The induced polarization (IP) method has been used in mining prospecting and increasingly
in environmental investigations because IP measurements are very sensitive to the low
frequency capacitive properties of rocks and soils.
Many different models have been proposed for the description of the dispersive behaviour
of the IP, but the most widely used seems to be the Cole-Cole model to describe resistivity
dispersion observed in field data from areas with metallic mineral content. It is also used to
estimate various subsurface properties of non metallic soil and rocks in IP frequency domain
investigations (SIP). A multiple Cole-Cole model typically is a more general and proper
model than a single Cole-Cole model for describing IP data with various dispersion ranges,
caused either by multiple-length scales in sediments or by coupling effects in the IP
measurements.
Among many studies in which Cole-Cole parameters are estimated from SIP measurements
on rocks, the majority use least squares methods which are based on linear inversion theory.
They use derivatives of the forward model. One of the limitations of the least squares based
method is that convergence to the global minimum is not guaranteed, and estimation results
depend on the choice of starting values.
Due to their respective special character in the function expression, the inversion process is
usually instable and even sometimes not convergent. Non-linear optimization methods as
Bayesian inversion and genetic algorithms (GA) have been used to invert the non-linear
equation of Cole-Cole model (Cao et al., 2005; Ghorbani et al., 2007 and Chen et al., 2008).
Ghorbani et al. (2007) developed a Bayesian model to invert time and frequency domain IP
data for parameters in a single Cole-Cole model. They showed that, even for synthetic and
laboratory data sets, the correct estimation of the Cole-Cole parameters using classical least
squares methods is difficult (Ghorbani et al., 2007).
Chen et al. (2008) developed a Bayesian model to invert SIP data for Cole-Cole
parameters using Markov-chain Monte Carlo (MCMC) sampling methods. They used the
Gibbs sampler to draw samples from the joint posterior distribution (Chen et al., 2008).
Cao et al. (2005) developed GA to invert the parameters of double Cole-Cole model. The
results of their algorithm on the synthetic data showed that GA has good stability and high
precision and permits a few observed data error (Cao et al., 2005).
In this paper, we develop a Bayesian method to invert SIP data for double Cole-Cole
parameters using simulated annealing (SA) method in order to find an approximation to the
global minimum. We evaluate the performance of the sampling-based Bayesian model by
applying it to synthetic SIP data and comparing the inversion results with those obtained from
GA method reproduced by author and developed by (Cao et al., 2005).

2- THE COLE-COLE MODEL

The empirical ColeCole is a model that widely uses to describe the resistivity dispersion
observed in field data from areas with metallic mineral content and estimate the hydraulic
conductivity of sediments. The Cole-Cole model is given by
1 (1)
() 0 1 m 1
c
1 i

where 0 is the resistivity at the DC, m the chargeability, the time constant, and c the
frequency dependence. These four parameters make the ColeCole model sufficiently
flexible to adapt itself to a large number of cases. For rock with a unimodal distribution of
length scales (i.e., a unimodal grain or pore size distribution), a phase spectrum with a single
peak, as reported repeatedly in the literature, can be expected. However, for more complex
distributions of length scales such as bimodal distributions, phase spectra with more than one
phase peak can be observed. A multiple and/or additive form of Cole-Cole equation is to
simulate the Electromagnetic (inductive and/or capacitive) coupling effects associated with

2
instrumentation and cable layout (i.e., Pelton et al, 1978). In this case, however, the Cole-
Cole parameters themselves normally are not of interest, but only the response of the
parameter set with an objective of removing it from the measured data.
We use a double Cole-Cole model as used by (Cao et al., 2005) in our inversion
algorithms to allow for analysis of phase spectra with more than one dispersion range caused
either by the multiple modality of the rock or by coupling effects in the measurements.
1 1 (2)
(k ) 0 1 m1 1
c m2 1

c

1 i1

k 1

2
1 i
k 2

Where m1, m2, 1, 2 and c1, c2 represent chargeability, time constant, and dependence
factor for the first and second dispersion terms in the double Cole-Cole model, respectively
and k=1,2, , N. N number of frequencies which IP data are measured.

3- BAYESIAN METHOD
The Bayesian approach to inverse problems, describes the a priori information we
may have on the model vector m, by a probability density fa(m). Then, it combines this
information with the information provided by the measurement of the data vector and with
the information provided by the physical theory, as described by equation:
f p ( m ) t f a m L m (3)
in order to define a probability density fp(m) representing the a posteriori information.
This a posteriori probability density describes all the information we have. t is an appropriate
normalization constant. L(m) is likelihood function which, crudely speaking, measures the
fit between observed data and data predicted from the model m. When we describe
experimental results by a vector of observed values d iobs with Gaussian experimental
uncertainties, then
d cal d iobs
2

L( m ) t.exp- i (4)

i i
or

Lm t.exp -
g m d obs
i i
2

(5)

i i

where d ical is the calculated values obtained from the forward problem d cal=g(m), i are the
estimated uncertainties.
Parameters: We develop Bayesian model to estimate parameters in the double Cole-Cole
model given by equations 2. Real and imaginary parts of the complex resistivity are used for
this model. The unknown parameters vector is m=[ 0, m1, m2, log(1), log(2), c1, c2]T.
Likelihood function: we suppose data have a distribution close to the normal distribution.
With this error distribution, the likelihood function is given by
1 (6)
L m t.exp - S m
2
1 1 (7)
S m 2 Re k Re obs k 2 Im k Im obs k
2 2

Re k Im k
where Re((k)) and Im((k)) the real and imaginary components of the complex resistivity
collected at frequency k (k=1,2, ,N) and Re and Im the inverse standard deviations of
measurement errors in the real and imaginary parts of complex resistivity.
A priori information: We use noninformative prior distributions for the parameters. Then,
the parameters have uniform distributions over given ranges. For synthetic case studies
presented here, the prior ranges of parameters 0, m, and c are given as (1, 1000) ms, (0,
1), (10-5, 10 5) s and (0, 1), respectively.
Sampling of a posteriori probability: A posteriori information may well be multimodal,

3
not have a mathematical expectation, have infinite variances, or some other pathologies, but it
constitutes the complete solution to the inverse problem (Mosegaard and Tarantola, 1995).
We obtain estimates of unknown parameters by drawing many samples from the joint
posterior using Monte Carlo Marcov chain (MCMC) methods. A generalized form of MCMC
method for examining the equations of state and frozen states of n-body systems is SA
method. By analogy the generalization of this Monte Carlo approach to combinatorial
problems is straight forward (Kirkpatrick et al., 1983). The original Metropolis scheme was
that an initial state of a thermodynamic system was chosen at energy E and temperature T,
holding T constant the initial configuration is perturbed and the change in energy dE is
computed. If the change in energy is negative the new configuration is accepted. If the
change in energy is positive it is accepted with a probability given by the Boltzmann factor
exp -(dE/T). This processes is then repeated sufficient times to give good sampling statistics
for the current temperature, and then the temperature is decremented and the entire process
repeated until a frozen state is achieved at T=0.
There are different algorithms on the rate of decrease of the temperature, so that the SA
algorithm converges to the set of global maxima (i.e., Geman and Geman, 1984 and Duflo,
1996). For example, if Ti=/logi, there is convergence to a global maximum if and only if
S. Duflo (1996) illustrated that the form Ti=/log(i+1) are satisfactory for large enough.
In this paper, we use a perturbed model as followed
1 if S i mnew Si mold
(8)
Pi accept S i
exp - T if Si m new S i mold
i
where Ti=/log(i+1), and i is iteration number. We used 5% of the real and imaginary part
of the complex resistivity as standard deviation of the synthetic data. In this case, the suitable
value of is found 10 -5 in our algorithm.
Monitoring convergence: We run three chains by starting from different sets of initial
values for the total number of k iterations. We calculate a criterion, referred to as the scale
reduction score in (Gelman, 1992), based on the three Markov chains. With that approach, if
the scale reduction score is less than 1.2, the Markov chain is considered to be converged;
otherwise, more runs are needed.
4- GENETIC ALGORITHM METHOD
We reproduce the GA that is developed by (Cao et al., 2005) in order to fitting double
Cole-Cole model to SIP data. During a GA, they gave a data set; the evolution starts with a
population of randomly selected individuals (models). The initial population should cover the
entire range of possible solutions. Subsequent generations are constructed by the action of
three reproduction operators: Selection (based on misfit), Cross-over (swapping information)
and mutation (Changes in individual genes). Therefore, it is optimized the objective function
and population of GA to dynamic fitness value function and varying population. In GA,
fitness value function should be positive and should be compared with each other, on the
basis of which selection probability can be calculated. SIP inversion solution is a minimum
value problem, so the pth populations fitness value function is:
Cmax hp m when hp m Cmax
f p m (9)
0 others

k 2
hp m g i m d i obs
(10)
i 1
Where p=1, 2,, P, with P the population size; Cmax is the given constant. Equation (12)
calculates the objective function. Cao et al. (2005) used Cmax as a dynamic variable. Because

4
if Cmax is a constant value and too large then fp(m) Cmax, which means the selection is nearly
random and it is very difficult to converge to objective value; if it is constant and too small
then fp(m)0, which means the eliminating rate is high and the selection pressure is large to
lead to prematurity with ease (Cao et al., 2005). Adjusting the objective function to
dynamic fitness value function effectively overcomes the prematurity phenomena and
assures the search speed. They selected Cmax as
1 K
Cmax A g p m (11)
K p 1
weight coefficient A is A = a/(c + t1) where t1 is the evolution generation variable; a and c
are constants. A varies between 2 and 5.
Inversion algorithm with the synthetic data illustrate better population size is 39. In the
GA, however, population size is not altered.Too small population size will probably result in
GA converging to local extreme value.
5- SYNTHETIC DATA INVERSION
We first demonstrate the use of the sampling-based Bayesian method for Cole-Cole
parameter estimation using a synthetic SIP data set. We then compare the results obtained
from the SA method with those obtained from the GA method developed by (Cao et al.,
2005). We choose a synthetic case with a double Cole-Cole model.
The synthetic Cole-Cole model parameters are the same as those used by (Cao et al., 2005)
(table 1). We generated synthetic SIP data using logarithmic scale of frequencies ranging
(Hz) from equation f=12000/2n where n changes between 0 and 23. Consequently, the
frequency changes between 1 mHz to 12 kHz as is typical of SIP measurements. We added
different relative random noises 5 and 10% to the amplitude and phase of the generated
resistivity data.
Table 1 and Figure 1 show that both SA and GA methods return real parameters when
random noises are less than 5%. For the data with random noises up to 10%, the inversed
parameters obtained from SA method are more close to the real parameters. However, when
random errors are greater than 10%, the inversion error is relatively large and the results of
inversion become even worse. Two Cole-Cole models have six parameters needed to be
solved; the solution space increases and equivalence phenomenon become to arise. Especially
when errors rise, the equivalence phenomena become complicated which cause that inversion
error increases relatively large when the random errors are added up largely.
Table 1: Comparison of estimates from inversion of the SIP data using simulated Annealing (SA) and genetic
algorithm (GA) methods.
SA method GA method
Model Real
Added 5% Added 10% Added 5% Added 10%
Parameters parameter
random noises random noises random noises random noises
0 (m) 25.00 25.025 26.703 25.15 27.032
m1 0.50 0.503 0.510 0.505 0.515
1 10.00 10.000 9.997 9.067 8.458
c1 0.40 0.404 0.405 0.410 0.421
m2 0.01 0.012 0.0097 0.0078 0.0065
2 1.00 0.9999 0.9999 0.947 0.888
c2 0.98 0.982 0.97999 0.998 0.9999

5
 Figure 1: Argand diagram of the Synthetic SIP data with 5 and 10% relative noise and obtained fits using the SA
method and GA method for a double Cole-Cole model.

6- CONCLUSIONS
We developed a Bayesian method with simulated annealing (SA) sampling algorithm to
invert for double Cole-Cole parameters from SIP data. We also reproduced the genetic
algorithm developed by (Cao et al., 2005) and compared performance of SA method with GA
method through inversion of synthetic data.
We use noninformative priors in the SA method; the estimates of Cole-Cole parameters.
Both of two methods are provides a global approach for inverting SIP data for Cole-Cole
parameters; the obtained estimates are independent of initial values. Our results show that for
the SIP synthetic data with random noises up to 10%, the inversed parameters obtained from
SA method in comparison with GA method are more close to the real parameters.

7- REFERENCES
Cao, Z., Y. Chang, and Luo, Y., 2005, Inversion study of spectral induced polarization
based on improved genetic algorithm: Progress, in Electromagnetics Research
Symposium_PIERS_Online, 1, 266270; http://piers.mit.edu/piersonline/piers.php; doi:
10.2529/piers04120094920.
Chen, J., Kemna, A., and Hubbard, S., 2008, A comparison between Gauss-Newton and
Markov-chain Monte Carlobased methods for inverting spectral induced-polarization data
for Cole-Cole parameters, Geophysics, 73 (6).
Duflo, 1996, Applications of Mathematics. I, vol. 34. Springer-Verlag, Berlin, New York.
Gelman, A., and Rubin, D. B., 1992, Inference from iterative simulation using multiple
sequences, Statistical Science, 7, 457-511.
Geman, S., and Geman, D., 1984, Stochastic relaxation, Gibbs distribution, and Bayesian
restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6,
721741.
Ghorbani, A., Camerlynck, C., Florsch, N., Cosenza, P., and Revil, A., 2007, Bayesian
inference of the Cole-Cole parameters from time and frequency domain induced polarization,
Geophysical Prospecting, 55, 589605.
Kirkpatrick, S., Gelatt, C. D., and Vecchi, Jr., M. P., 1983, Optimization by Simulated
Annealing, Science, 220 (4598).
Mosegaard, K., and Tarantola, A., 1995, Monte Carlo sampling of solutions to inverse
problems, Journal of Geophysical Research, vol. 100, no. B7, p.p. 12,43112,447.
Pelton, W. H., Ward, S. H., Hallof, P. G., Sill, W. R., and Nelson, P.H., 1978, Mineral
discrimination and removal of inductive coupling with multifrequency IP, Geophysics, 43,
588609.