NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez

Introduction
In this work we describe the importance of the kronecker product as a
mathematical language for the formulation of fast Fourier transform
(FFT) algorithms as well as its use as a tool aid in their analysis,
design, implementation and modification. We first present a brief
introduction of the FFT as an efficient algorithm for computing the
discrete Fourier transform (DFT). The DFT is presented in matrix form
and the FFT is presented as a factorization of this DFT matrix into
various factors. Since this factorization is not unique, we can get
many factorizations. In principle, each factorization can be associated
with an FFT algorithm.
This process of factoring the DFT matrix may become very tedious and
cumbersome and a methodology for producing this factorization is
always useful. In their tutorial work, J. Johnson, R. Johnson, D.
Rodriguez, and R. Tolimieri [1], presented a methodology for the
factorization of the DFT matrix using kronecker product algebra and an
associated set of permutation operations.
We describe this
methodology in this work. We then proceed to describe the work of J.
Granata, M. Conner, and R. Tolimieri presented in [2] where they
explain how to use the kronecker product as a mathematical language
to aid in the programming and implementation of FFTs and other
digital signal processing (DSP) operations.
The main objective of our work is to describe, in a more simplified and
concise manner, how the kronecker product can assist in the
implementation of the FFTs. We finish our work by presenting some
examples of FFTs in kronecker product form and show how we can
obtain other formulations using basic algebraic properties. One of the
advantages of obtaining different mathematical formulations is that
some of the formulations are better matched to certain computer
architectures than others.

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez
The Fast Fourier Transform

The FFT is an efficient algorithm for computing the DFT. The DFT of an
arbitrary discrete signal x(n), of length N, is given by
N 1

X ( k ) = x ( n )e

j 2nk
N

k = 0,1,..., N 1;

j = 1

(1)

n=0

The DFT can be obtained from the Fourier transform X() of x(n) by
uniformly sampling X() at N points = k, k=0,1,,N-1. The DFT of
the signal x(n) can also be expressed in matrix form, after expanding
the summation operation given in (1) as X=FNx. Here, the signal x(n)
and its transform X(k)have been written in column vector form, with
their entries in the natural order, starting their indexation at 0 and
ending at N-1 for both the k and n variables. The matrix FN is then
given by
FN = [W Nkn ]
WN = e

j 2
N

(2)
;

k , n = 0,1,..., N 1

We take the case of N = 4 to provide an example of the DFT operation

expressed in matrix form.
3

X (k ) = x(n)W 4kn

(3)

n=0

X (k ) = x(0) + x(1)W 4k + x(2)W 42 k + x(3)W 43k

X (0) 1 1
X (1) 1 W
4

=
X (2) 1 W 42

3
X (3) 1 W 4

1
2
4

W
1

W 42

1 x ( 0)
W 43 x(1)

W 42 x(2)

W 4 x(3)

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez

Thus, the matrix F4 is given by

[ ]

F4 = W 4kn

k , n = 0 ,1, 2 , 3

1
1
1 1
1 j 1 j

=
1 1 1 1

1 j 1 j

(4)

Kronecker Product Algebra

In this section we present the basic definition of the kronecker product
of two matrices and describe some basic properties of the kronecker
product operation. We then proceed to describe the DFT matrix as a
composition of sparse matrices, some of them expressed in kronecker
product form. This property of having sparse matrices as factors in
the decomposition of the Fourier matrix FN is one of the fundamental
reasons for obtaining efficient implementations. Here, we loosely
define a matrix, as a sparse matrix of the entries of the matrix are
mostly zeros. We will use square matrices to define the kronecker
product, but the definition generally applies to matrices of any
dimensions, as it is define in [3].
Let A and B be two matrices of order R and S, respectively. The
kronecker product of A and B is a binary operation which results in a
new matrix of order N=RS denoted by C=AB, given by

A B = [a kl B] k ,l = 0,1,..., R 1

(5)

a
As an example, take R=S=2. Let A = 00
a10
a B a 01 B
C = A B = [a kl B ] = 00

a10 B a11 B

a 00 b00
a b
C = 00 10
a10 b00

a10 b10

a 00 b01

a 01 b00

a 00 b11

a 01 b10

a10 b01
a10 b11

a11 b00
a11 b10

a 01 b01
a1b11
a11 b01

a11 b11

a 01
b
, B = 00

a11
b10

b01
b11

(6)

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez

If we let A=I2, the identity matrix, and set B=F2, the DFT matrix of
order 2, we then get C= AB= I2F2.
Thus
1 0 1 1
C=

0 1 1 1

F
C= 2
0

1 1
0 1 1
=
F2 0 0

0 0

(7)

0 0
0 0
1 1

1 1

The kronecker product is not a commutative operation as it can be

seen

1
0
by noticing that I2F2 F2I2 =
1

0 1 0
1 0 1
0 1 0

1 0 1

(8)

In general, an expression of the form IRFS is seen as a parallel

operation since the nonzero elements, the matrices Fs, appear along
the diagonal.
The matrix IRFS is a sparse matrix and its
implementation favors a parallel architecture.
This can be
demonstrated with a little example. Take R=3 and S=2. If we
compute b = (I R FS )a , this matrix-vector multiplication operation
becomes
b0
b
1 F
b2 2
=0
b 3 0
b4

b5

0
F2
0

a 0
a
0 1
a
0 2
a
F2 3
a 4

a 5

(9)

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez
b0 1 1
b 1 1
1
b2 0 0
=
b3 0 0
b4 0 0

b5 0 0

1 0
1 1 0
0

0 a 0 a 0 + a1
0 a1 a 0 a1
0 a 2 a 2 + a 3

=
0 a 3 a 2 a 3
1 a 4 a 4 + a 5

1 a 5 a 4 a 5

(10)

If the column vector a is divided into 3 sections, of length 2 each, the

computation b = (I 3 F2 )a could be perform by computing three
simultaneous DFTs of length 2. To this effect, operation b = ( I R FS )a
can be performed by computing R simultaneous FFTs of length S each.
If we had a computer architecture with R processors, then these
operations could be performed concurrently.
This
The kronecker product FSIR also has special properties.
expression favors an architecture with vector processing capabilities.
The operation b = ( FS I R )a can be computed at a vector level instead
of at a scalar level as can be demonstrated by the following example.
Let S=2 and R=4. Then, the operation b = (F2 I 4 )a becomes
b0
b
1
b2
1

b3 = 1 1 0
b4 1 1 0

b5
0
b
6
b7

0 0
1 0
0 1
0 0

a 0
a
1
0 a 2

0 a 3
0 a 4

1 a 5
a
6
a 7

(11)

After performing the kronecker product we set

b0
b
1
b2

b 3 = I 4
b4 I 4

b 5
b
6
b7

a 0
a
1
a 2

I 4 a3
I 4 a 4

a 5
a
6
a 7

(12)

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez

The entries in the square matrix are themselves identity matrices of

order 4 each. Thus, rewriting, we get

b0 1
b 0
1
b2 0

b3 = 0
b4 1

b5 0
b 0
6
b7 0

0 0 0
1 0 0
0 1 0
0 0 1
0 0 0
1 0 0
0 1 0
0 0 1

0 a 0
0 1 0 0 a1
0 0 1 0 a 2

0 0 0 1 a 3
1 0 0 0 a 4

0 1 0 0 a 5
0 0 1 0 a 6

0 0 0 1 a 7
1

(13)

This matrix-vector multiplication operation can be in terms of

submatrix-vector segment multiplication as follows.
We thus get
b0 a 0
a 4
b

1 I a1 + I a 5
b2 4 a 2 4 a 6


a 7
b3 = a 3
b4 a 0
a 4


b5 I a1 I a 5
b 4 a 2 4 a 6
6

a 7
b7 a 3

(14)

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez

Since we are just multiplying the vector segments by the identity

matrix, we get the following final result
b0 a 0 a 4
b
1 a1 + a 5
b2 a 2 a 6

b 3 = a 3 a 7
b4 a 0 a 4

b 5 a 1 a 5
b a 2 a 6
6
b7 a 3 a 7

(15)

In general, the operation b = (FS I R )a can be computed in a machine

with vector processing capabilities. The input vector a could be divided
into S segments of length R each.
The expressions IRFS and FSIR are prevalent in the kronecker
product formulations of FFT algorithms and become instrumental in
obtaining efficient FFT implementations, as will be shown in the next
section.

Formulating FFTs Using the Kronecker Product

In this section we present mathematical formulations of FFT algorithms
in terms of the kronecker product. We proceed to describe in a
concise manner, how the DFT matrix can be decomposed into a
sequence of sparse matrices, all of the same order. We first set the
order of the DFT matrix to be N in order to simplify the discussion. It
is very desirable that N be a highly composite number, being N=2M the
best composite number for the kronecker formulation.
Take N=RS, where R and S may or may not be composite numbers.
When we are to compute the DFT of an Npoint discrete signal we
perform the operation

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez
N 1

X (k ) = x(n)e

j 2nk
N

k = 0,1,..., N 1;

j = 1

(16)

n =0

The essence in obtaining a kronecker formulation for this operation is

to express the indexing variables k and n in terms of two new
variables. For instance, one form of doing this is to set k=k0+Rk1,
where k0=0,1,,R-1 and k1=0,1,,S-1, and to set n=n0+Sn1, where
n0=0,1,,S-1 and n1=0,1,,R-1. This change of variables will turn the
single summation operation (16) of the DFT computation into a double
summation operation. To arrive at the kronecker product formulation,
the double summation operation must be expanded using the new
variables k0, k1, n0 , n1.
The input data, which was first in the form of one-dimensional array in
the variable n, becomes a two dimensional array in the variables n0,
n1. The same happens with the output array in the variable k. These
two-dimensional arrays, the input and output data, must be presented
in a single column vector form in order to get the kronecker
formulation after rearranging the system of equations which results
when the double summation operation is expanded.
As presented in [3], if N=RS, then, following the procedure described
above, the DFT computation of an N-point discrete signal x(n) can be
expressed in kronecker product form as follows

X = FN x = ( FR I S )T N , S ( I R FS ) PN , S x

(17)

Here TN,S is a diagonal matrix of order N, called twiddle or phase factor

and PN,S is a permutation matrix, also of order N, called the stride by S
permutation matrix.
This permutation operation, present in all
kronecker product formulation of the FFT, reorganizes the input data
following a decimation procedure.
The importance of this formulation is that it can be used in an iterative
fashion; for instance, if S is also a composite, say S=LM, we can
repeat the process, inserting the new formulation in the formula (17)
above and using properties of kronecker product algebra to arrive at a
desired expression.

NOTES ON KRONECKER PRODUCTS

Prof. Domingo Rodriguez
Concluding Remarks

We have seen that the kronecker product is a powerful language to

formulate FFT algorithms. We also have described how the kronecker
product factors that appear any given kronecker product formulation of
an FFT algorithm can be adapted to machines with parallel
architectures, vector architectures, or both.
This important fact,
combined with the iteration process used for highly composite values
of N, the length of the input signal in an FFT computation, become
instrumental in obtain efficient FFT implementations.
Properties of kronecker product algebra, and linear algebra in general,
also assist in obtaining good implementations. For instance, the
matrix FN is symmetric, so FNT= FN and another formulation for the FFT
of an N-point sequence is simply the expression
X = FN x = PNT, S ( I R F S ) T T N , S ( FR I S ) T x

(18)

This transposition operation, which turns equation (17) into equation

(18), demonstrates that the kronecker product is indeed a very useful
and powerful programming language tool to assist in the analysis,
design, implementation, and modification of FFT algorithms.

References
[1] R. J. Johnson, J. J. Johnson, D. Rodriguez, R. Tolimieri, A
Methodology for Designing, Modifying, and Implementing Fourier
Transform Algorithms on Various Architectures, Journal of Circuits,
Systems, and Signal Processing, Vol.9, No. 4, Birkhuser, 1990.
[2] J. Granata, M. Conner, and R. Tolimieri, The Tensor Product: A
Mathematical Programming Language for FFTs and other Fast DSP
Operations, IEEE SP Magazine, 1992, Vol. 9, No. 1, page 40-48.
[3] R. Tolimieri, M. An, C. Lu, Algorithms for Discrete Fourier
Transform and Convolution, Second Edition, Springer-Verlag, 1997.