INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING

Int. J. Adapt. Control Signal Process. 2006; 20:213223

Published online 23 March 2006 in Wiley InterScience (www.interscience.wiley.com) DOI:10.1002/acs.896

A new blind source separation method based on

fractional lower-order statistics
Daifeng Zha1,n,y and Tianshuang Qiu2
1

College of Electronic Engineering, Jiujiang University, Jiujiang 332005, China

School of Electronic and Information Engineering, Dalian University of Technology, Dalian 116024, China

SUMMARY
We proposed neural network structures related to multilayer feed-forward networks for performing blind
source separation (BSS) based on fractional lower-order statistics. As alpha stable distribution process has
no its second- or higher-order statistics, we modied conventional BSS algorithms so that their capabilities
are greatly improved under both Gaussian and lower-order alpha stable distribution noise environments.
We analysed the performances of the new algorithm, including the stability and convergence performance.
The analysis is based on the assumption that the additive noise can be modelled as alpha stable process.
The simulation experiments and analysis show that the proposed class of networks and algorithms is more
robust than second-order-statistics-based algorithm. Copyright # 2006 John Wiley & Sons, Ltd.
KEY WORDS:

alpha stable distribution; blind source separation; independent component analysis; neural
networks; second-order statistics; higher-order statistics; fractional lower-order statistics
(FLOS); non-Gaussian noise

1. INTRODUCTION
In some applications, such as underwater acoustic signal processing, radio astronomy,
communications, radar system, etc., and for most conventional and linear-theory-based
methods, it is reasonable to assume that the additive noise is usually assumed to be Gaussian
distributed with nite second-order statistics (SOS). In some scenarios, it is inappropriate to
model the noise as Gaussian noise. Recent studies [1,2] show that alpha stable distribution is
better for modelling impulsive noises, including underwater acoustic, low-frequency atmospheric, and many man-made noises, than Gaussian distribution in signal processing, which has
some important characteristics and makes it very attractive. Stable process arises as limiting
processes of sum of independent, identically distributed random variables via the generalized
central limit theorem. This kind of physical process with sudden and short endurance high
impulse in real world, called lower-order alpha stable distribution random process, has no
second- or higher-order statistics. It has no close-form probability density function (p.d.f.) so

Correspondence to: Daifeng Zha, College of Electronic Engineering, Jiujiang University, Jiujiang 332005, China.
E-mail: zhadaifeng@163.com

Copyright # 2006 John Wiley & Sons, Ltd.

Received 14 December 2004

Revised 24 July 2005
Accepted 27 December 2005

214

D. ZHA AND T. QIU

that we can only describe it by its characteristic function [1,2]:

jt expfjmt
gjtja 1 jb sgntot; ag
where
ot; a

8
ap
>
>
< tan 2

a=1

>
2
>
: logjtj a 1
p

15x51; d > 0; 05a42;
14b41

a is the characteristic exponent. It controls the thickness of the tail in the distribution. The
Gaussian process is a special case of stable processes with a 2: The dispersion parameter g is
similar to the variance of Gaussian process and b is the symmetry parameter. If b 0; the
distribution is symmetric and the observation is referred to as the SaS (symmetry a-stable)
distribution, i.e. it is symmetrical about m. m is the location parameter. When a 2 and b 0;
the stable distribution becomes the Gaussian distribution.
Conventional blind source separation (BSS) [3] is optimal in approximating the input data in
the mean-square error sense, describing some second-order characteristics of the data. Nonlinear BSS [4,5] method which is related to higher-order statistical techniques is a useful
extension of standard BSS that has been developed in context with blind separation of
independent sources from their linear mixtures. Such blind techniques are needed, for example,
in various applications of array processing, communications, medical signal processing, and
speech processing. In BSS, the data are represented in an orthogonal basis determined merely by
the SOS (covariance) of the input data [610]. Conventional BSS methods are based on secondorder statistics (SOS) or higher-order statistics (HOS) and often utilized in blind separation of
unknown source signals from their linear mixtures. In this paper, we proposed neural network
structures related to multilayer feed-forward networks for performing BSS based on fractional
lower-order statistics (FLOS). As lower-order alpha stable distribution process has no its
second- or higher-order statistics, we modied conventional algorithms so that their capabilities
are greatly improved under both Gaussian and fractional lower-order alpha stable distribution
noise environments.
2. DATA MODEL AND NETWORK STRUCTURES
In the following, we present the basic data model used in the source separation problem, and
discuss the necessary assumptions. Assume that there exist P zero-mean source signals
si n; i 1; 2; . . . ; P that are scalar-valued and mutually statistically independent for each
sample value or, in practice, as independent as possible. The independence condition is formally
dened so that the joint probability density of the source signals must be the product of the
marginal densities of the individual sources. More concretely, the source signals could be
sampled discrete time waveforms. We assume that the original sources are unobservable, and all
that we have are a set of noisy linear mixtures Xn x1 n; x2 n; . . . ; xM nT ; n 1; 2; . . . ; N:
We can write the signal model in matrix form as follows:
X AS V

Here, X is M  N observation matrix, A is M  P constant mixing matrix with full column

rank, For the sake of convenience, we can assume that the number of sources is equal to the
Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223

NEW BLIND SOURCE SEPARATION METHOD

215

Figure 1. The linear feed-forward neural network structure.

dimension of Xn; i.e. the number of sources is known. S is P  N independent source signals
matrix, V denotes M  N possible additive lower-order alpha stable distribution noise matrix,
mixture matrix A is unknown. The task of source separation is merely to nd the estimation of
the sources, knowing only the data X: Each source signal is a stationary zero-mean stochastic
process.
Now, let us consider a two-layer feed-forward neural network structure [3,8] shown in
Figure 1. The inputs of the network are components of the observation matrix X (not counted as
a layer). In the hidden layer there are P neurons, and the output layer consists again of P
neurons. Let B denote P  M the pre-processing matrix between the inputs and the hidden
layer, and WT ; the P  P weight matrix between the hidden and output layers, respectively.
Based on above network structure, the BSS can be done in two subsequent stages as follows: (a)
Obtain a pre-processing matrix that orthogonalizes the mixture matrix. (b) Learn a weight matrix,
which maximizes the p-order moment of every elements of Yn:
max Efjyi njp g EfjWTi Znjp g

05p5a42

In the meantime, the sources are separated.

3. PRE-WHITENING BASED ON NORMALIZED COVARIANCE MATRIX

Generally, it is impossible to separate the possible noise in the input data from the source
signals. In practice, noise smears the results in all the separation algorithms [7]. If the amount of
noise is considerable, the separation results are often fairly poor. Here, we introduce a two-step
separation method that achieves the BSS. The rst step is a whitening procedure that
orthogonalizes the mixture matrix. Here, we search for a matrix B; which transforms mixing
matrix A into a unitary matrix. Classically, for a nite variance signal, the whitening matrix is
computed as the inverse square root of the signal covariance matrix. In our case, alpha stable
impulsive noise has innite variance. However, we can take advantage from the normalized
covariance matrix.
Theorem 1 (Sahmoudi et al. [12])
Let X X1; X2; . . . ; XN be a stable process data matrix, then normalized covariance
matrix of X
Cx

Copyright # 2006 John Wiley & Sons, Ltd.

XXT
N  TraceXXT =N

Int. J. Adapt. Control Signal Process. 2006; 20:213223

216

D. ZHA AND T. QIU

Converges asymptotically to the nite matrix when N ! 1; i.e.

lim Cx ADAT

N!1

where di limN!1 Di =

PP

j1

D diagd1 ; d2 ; . . . ; dM
2

Dj kaj k ; aj is column of A; Di

6
PN

n1

x2i n=N:

Theorem 2
Let we have eigendecomposition of Cx as Cx UX2 UT and we can get its eigenvalues and
corresponding eigenvectors lm, em ; m 1; 2; . . . ; M and whitening matrix B X
1 UT ; then the
following equation is orthogonal:
Z BX

Proof
ZZT BXXT BT BCx N  TraceXXT =NBT
N  TraceXXT =N  B  UX2 UT  BT
N  TraceXXT =N  X
1 UT  UX2 UT  U  X
1 T
N  TraceXXT =N  X
1  I  X2  I  X
1 T
N  TraceXXT =N  I
So we can write
BCx BT BADAT BT BAD1=2 BAD1=2 T I

8
&

4. SEPARATING ALGORITHMS
The core part and most dicult task in BSS is learning of the separating matrix WT : During
recent years, many neural blind separation algorithms have been proposed. In the following, we
discuss and propose separation algorithms, which are suitable for alpha stable noise
environments in BSS networks. Let us consider ith output weight vector Wi ; i 1; 2; . . . ; P;
standard BSS is based on SOS and maximize the output variance (power) Efjyi nj2 g
EfjWTi Znj2 g subject to orthogonal constraints Wi WTi IP : As lower-order alpha stable
distribution noise has no second-order moment, we must select appropriate optimal criterion.
So the BSS problem corresponding to p-order moment maximization is solution to optimization
problem for each Wi ; i 1; 2; . . . ; P subject to orthogonal constraints Wi WTi IP :



1 T
opt
p
jZ nWi j
Wi arg max E
9
Wi WTi IP
Wi
p
Let objective function be

1 T
jZ nWi jp
JWi E
p
Copyright # 2006 John Wiley & Sons, Ltd.

P
1
1 X
lij WTi Wj
lii WTi Wi
1
2
2 j1;j=i

10

Int. J. Adapt. Control Signal Process. 2006; 20:213223

217

NEW BLIND SOURCE SEPARATION METHOD

Here, the Lagrange multiplier is lij, imposed on the orthogonal constraints Wi WTi IP : For
each neuron, Wi is orthogonal to the weight vector Wj ; j=i:
The estimated gradient of JWi with respect to Wi is
p
2
T
#
rJW
conjZT nWi g
i EfZnjZ nWi j

P
X

11

lij Wj

j1

At the optimum, the gradients must vanish for i 1; 2; . . . ; P; and WTi Wj dij : These
can be taken into account by multiplying (11) by WTj from left. We can obtain lij

WTj EfZnjZT nWi jp
2 conjZT nWi g: Inserting these into (11) we get
"
#
P
X
T
#
rJWi I

Wj W EfZnjZT nWi jp
2 conjZT nWi g
12
j

j1

A practical gradient algorithm for optimization problem (9) is now obtained by inserting (12)
#
into Wi n 1 Wi n
mnrJW
i n; where m(n) is the gain parameter. The nal algorithm
is thus
"
#
P
X
T
Wi n 1 Wi n
mn I

Wj nWj n ZnjZT nWi njp
2 conjZT nWi n 13
j1

As yi n ZT nWi n; (13) can be written as follows:

"
Wi n 1 Wi n
mnjyi nj

p
2

conjyi n Zn

P
X

#
yj nWj n

14

j1

Let gt jtjp
2 conjt; then g(t) is an appropriate network non-linear transform function for
lower-order alpha stable distribution impulse noise.
P
Considering that during the iteration error item of gradient I
Pj1 Wj nWTj n might be
zero instantaneously, we modify (14) to improve robustness of algorithm as
"
#
P
X
gyi nWj n
15
Wi n 1 Wi n
mngyi n Zn

j1

Thus, W1 ; W2 ; . . . ; WP can be obtained.

Let Yn y1 n; y2 n; . . . ; yP nT ; W W1 ; W2 ; . . . ; WP ; for whole network, solution to W
and optimization problem is



P
X
1 T
p
jZ nWi j
E
Wopt arg max
16
W
p
i1
According to the above derivation, by using gt jtjp
2 conjt; the algorithm for learning
W is
Wn 1 Wn
mnZn
WngYngYT n
Copyright # 2006 John Wiley & Sons, Ltd.

17

Int. J. Adapt. Control Signal Process. 2006; 20:213223

218

D. ZHA AND T. QIU

5. PERFORMANCES ANALYSIS
Dierent non-linear functions can be applied to dierent blind signal separation problems.
Some popular functions are gt signt and gt tanht corresponding to the double
exponential p.d.f. 12 exp
jxj and the inverse-cosine-hyperbolic p.d.f. 1=p1=coshx;
respectively. For the class of symmetric normal inverse Gaussian (NIG), it is straightforward
to obtain according to Kidmose [11]
p

at K2 a d2 t2
p
gt p
d2 t2 K1 a d2 t2
where K1 : and K2 : are the modied Bessel functions of the second kind with indices 1 and 2.
As lower-order alpha stable distribution noise has no second- or higher-order moments, we
must select appropriate non-linear function gt jtjp
2 conjt p5a: If t is real data,
gt jtjp
1 signt; if p 1; gt signt: Figure 2 shows the non-linear functions of alpha
stable distribution for dierent a.
We start from the learning rule (17), and we assume that there exists a square separating
matrix HT such that Un HT Zn: The separating matrix HT must be orthogonal. To make
the analysis easier, we multiply both sides of the learning rule (17) by HT : We obtain
HT Wn 1 HT Wn mnHT Zn
HT WngWnT ZngZnT Wn

18

For the sake of HHT IP ; we can get

HT Wn 1 HT Wn mnHT Zn
HT WngWnT HHT ZngZnT HHT Wn
T

19

T
1

Dene Qn H Wn; Wn H Qn; (19) is written as

Qn 1 Qn mnUn
QngQnT UngQnUnT

20

Figure 2. The non-linear functions of alpha stable distribution for dierent a.

Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223

NEW BLIND SOURCE SEPARATION METHOD

219

Figure 3. The stability and convergence of the algorithms based on SOS and FLOS.

Geometrically, the transformation multiplying by the orthogonal matrix HT simply means a

rotation to a new set of co-ordinates such that the elements of the input vector expressed in these
co-ordinates are statistically independent.
Analogous dierential equation of (20) is obtained as matrix form:
dQ=dt EfUgUT Qg
EfgQT UgUT Qg

21

According to Karhumen et al. [3], we can easily prove that (21) has stable solution. For the
sake of Q HT W; thus W HT
1 Q is asymptotic stable solution of (17). Figure 3 shows the
stability and convergence of the algorithms based on SOS and FLOS. From Figure 3, we know
the algorithm based on FLOS has better stability and convergence performances than the
algorithm based on SOS.

6. EXPERIMENTAL RESULTS
The assumption that signals are of nite variances is commonly made in the real world and is
reasonable for array signal processing. The array under consideration is a uniform linear array
(ULA) with interelement spacing equal to half of a wavelength. Since an alpha stable random
variable with a52 is of innite variance, we can use generalized SNR (GSNR) [1,2] which is the
ratio of the signal power over the noise dispersion gv
!
N
Ps
1 X
2
10 log
GSNR 10 log
jsnj
22
gv  N n1
s
Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223

220

D. ZHA AND T. QIU

For a nite sample realization, the GSNR can be computed by

!
PN
2
jsnj
n1
GSNR  10 log PN
2
n1 jvnj

23

Since an SaS random variable is characterized by two parameters a and g in each experiment,
we use the GSNR to describe the signal-to-noise ratio. The simulations here implemented the
algorithms based on FLOS and conventional SOS, respectively.
Experiment 1: Suppose that an linear microphone array with ve sensors, random audio time
domain signals of a piano and a bird enter the array from dierent directions. Whats more,
alpha stable impulsive noises with a 1:7 exist in the array at the same time. GSNR is 20 dB.
Two algorithms are used in the experiment, including: (1) SOS with non-linear function
gt tanht; and (2) FLOS with gt jtjp
2 conjt; respectively. We can get signal waveforms

Figure 4. The source and separate signals in time domain.

Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223

NEW BLIND SOURCE SEPARATION METHOD

221

in time domain as shown in Figure 4, where (a) and (b) are source signals, (c) and (d) are
separate signals based on SOS algorithm, (e) and (f) are separate signals based on FLOS
algorithm.
Experiment 2: We repeat simulations when GSNR is 20 dB. Two independent sources are
linearly mixed. One is the periodical noise free brain evoked potential (EP) signal, the period is
128 points and the sampling frequency is 1000 Hz. The other is an alpha stable non-Gaussian
noise with a 1:7: Two algorithms are used in the experiment, including: (1) SOS with
non-linear function gt tanht; (2) FLOS with gt jtjp
2 conjt; respectively. We can get
signals in time domain as shown in Figure 5, where (a) and (b) are source signals, (c) and (d) are
separated signals based on SOS, (e) and (f) are separated signals based on FLOS. For FLOS
algorithm, the correlation coecient between the separated and source EP signals is 0.9213,
and the correlation coecient between the separated and source alpha stable non-Gaussian
noises is
0.9098.
Experiment 3: Separate the mixed EP signal and noise again with the new FLOS algorithm and
conventional SOS algorithm, respectively. And the results of 10 independent experiments are
shown in Figure 6 and Table I. The correlation coecients of EP and of the noise are calculated

Figure 5. Separating results: (a), (b) are the source signals; (c), (d) are the separated signals with SOS; and
(e), (f) are the separated signals with FLOS.
Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223

222

D. ZHA AND T. QIU

Figure 6. The correlation coecients of EP and noise.

Table I. Comparison between the two algorithms.

Correlation coecient (FLOS)
Iteration n times
50
100
150
200
250
300
350
400
450
500

Correlation coecient (SOS)

EP

Noise

EP

Noise

0.1244

0.3450
0.4378
0.6766

0.9291

0.9287

0.9293
0.9295
0.9299

0.9501

0.1044

0.3050
0.4378
0.7706

0.9091

0.9107

0.9113
0.9195
0.9292

0.9593

0.0044

0.0050
0.1378
0.1716

0.1711

0.3937

0.4993
0.3945
0.2935

0.2804

0.0004

0.0063
0.1072
0.1212

0.1451

0.2231

0.2923
0.3045
0.1935

0.1904

at some iteration times. From Table I, we get that the performance of the new algorithm is
better than the Conventional algorithm.

7. CONCLUSION
This paper briey introduced the statistical characteristics of stable distribution and proposed a
blind source separation method based on fractional lower-order statistics in non-Gaussian
Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223

NEW BLIND SOURCE SEPARATION METHOD

223

impulsive noise environments. Furthermore, we have analysed its stability and convergence
performances.
In our simulations, we compared the performances of FLOS algorithm with those of SOS
algorithm. From above simulations, we can easily obtain the following conclusions: the
proposed class of network and BSS algorithm based on FLOS is more robust than conventional
algorithm based on SOS so that its separation capability is greatly improved under both
Gaussian and fractional lower-order stable distribution noise environments.

REFERENCES
1. Nikias CL, Shao M. Signal Processing with Alpha-Stable Distributions and Applications. Wiley: New York, 1995.
2. Shao M, Nikias CL. Signal Processing with fractional lower order moments: stable processes and their applications.
Proceedings of IEEE 1993; 81(7):9861010.
3. Karhumen J, Oja E, Wang L, Vigario R, Joutsensalo J. A class of neural networks for independent component
analysis. IEEE Transactions on Neural Networks 1997; 8(3):6778.
4. Zhang Y, Ma Y. CGHA for principal component extraction in the complex domain. IEEE Transactions on Neural
Networks 1997; 8(5):105117.
5. Karhunen J, Joutsensalo J. Nonlinear generalizations of principal component learning algorithms. Proceedings of
International Joint Conference on Neural Networks, vol. 3. 1993.
6. Wang L, Karhunen J, Oja E. A bigradient optimization approach for robust BSS, MCA, and source separation.
Proceedings of IEEE International Conference on Neural Networks, vol. 4. 1995.
7. Winter S, Sawada H, Makino S. Geometrical understanding of the BSS subspace method for over-determined blind
source separation. IEEE Transactions on Acoustics, Speech, and Signal Processing 2003; 13(6):112126.
8. Mutihac R, Van H. BSS and ICA neural implementations for source separation}a comparative study. Proceedings
of the International Joint Conference on Neural Networks, vol. 1. 2003; 2024.
9. Szu H, Hsu C. Unsupervised neural network learning for blind sources separation. Proceedings of 5-th Brazilian
Symposium on Neural Networks, 1998.
10. Diamantaras KI. Asymmetric BSS neural networks for adaptive blind source separation. Proceedings of the 1998
IEEE Signal Processing Society Workshop, 1998; 103112.
11. Kidmose P. Blind separation of heavy tail signals. IMM-Phd, LYNGBY, Technical University of Denmark, 2001.
12. Sahmoudi M, Abed-Meraim K, Benidir M. Blind separation of impulsive alpha-stable sources using minimum
dispersion criterion. IEEE Signal Processing Letters 2005; 12(4):281284.

Copyright # 2006 John Wiley & Sons, Ltd.

Int. J. Adapt. Control Signal Process. 2006; 20:213223