Comments on Fast Transforms: Banded

Matrices with Banded Inverses

The Narrator
November 08, 2015

Theory behind Banded Matrices with Banded Inverses

In general, the aim of a communication engineer is to be able to get the information as
correct as possible by using the noisy (distorted) signal as it is shown in Fig.1. Sometimes,
the original signal is expressed using different basis (domain) to make use of properties of the
transformed signal. An important property, e.g., can be the increased number of zeros in the
transformed signal vector so that less number of bits is sent. We can introduce a transform
by using the equation y = Ax where A Rnn and y, x Rn . Obviously, such a transform
should be invertible to be able to reverse back the transformation. For this reason, A is
non-singular. Thus, the inverse transform can be written as x = A1 y, and this one-to-one
mapping does not introduce any information loss.
According to the properties of the applications and signals, one may need different transformations. Some applications (e.g. HD Broadcasting) need fast forward and inverse transformation to be able to send or decode (compressed) data on time. This idea makes the banded
matrices with banded inverses very special class of the matrices as they provide very fast
forward and inverse transformation thanks to their special structure.
The question, now, is how to generalize an identifying property about the banded matrices
with banded inverses. According to the theorem in the paper, these matrices can be expressed
as a product of N block-diagonal matrices, namely A = F1 F2 ...FN , where Fi is block-diagonal
and non-singular for any i = 1, 2, .., N . At this point, one may argue that the product of

Source X

y = Ax

Black Box

A1

Figure 1: An example of a typical transformation scheme

1

x= A1 y

block-diagonal matrices gives another block-diagonal matrix. That seems true, but the idea
is arranging the matrices in a way that the blocks in one of the matrices in the product
should be offset from the blocks in others (this will be clearer with the example below). In
addition, each block-diagonal matrix composed of 2 by 2 and 1 by 1 blocks (not always a
constant size).1 Once this idea is understood, one can easily deduce that the inverse of this
banded matrix will also be banded, leading to a fast inverse transformation.2
Another very important property of this kind of matrices is as follows: Let A be a banded
matrix with a banded inverse A1 . The bandwidths are w and W, respectively. Then,
the number of block-diagonal matrices can be upper bounded by a function C(w3 + W 3 ),
independent of the matrix dimension n. This independence of n implies that the blockdiagonal matrices with invertible blocks are generator matrices for the singly infinite banded
matrices with banded inverses as n goes to infinity.
Example 1

1
1 1
1
1 2

1 2

, F2 =

3 1
F1 =
1 1

> A = F1 F2 = 1

2 1
2 1
1 2
2

1
2
2
0

0
6
3
4
2

1 0

2 2
1 4

A1

2
1 2

1 1 2
0

1
2
1 1 3 3
=

2 2
2 1
0 16 31

As it is shown in the example, the inverse of the banded matrix that is constructed in the
way explained above is also banded (offset is coming from the scalar in the first entry of
F1 ). If we chose a random banded matrix, with high probability the inverse would be a
full matrix instead of banded. Therefore, we can make a more general conclusion regarding
this properties: If one can design a banded transformation matrix by using the product of
non-singular block-diagonal matrices (or equivalently, one can show that a banded matrices
can be factorized into a product non-singular block-diagonal matrices), then s/he makes
sure that the inverse transformation will also be requires less calculation, hence, quicker
execution.3
1

The reason for limiting block-diagonal matrices to be 2 by 2 or 1 by 1 matrices is to reduce the bandwidth
of matrix A.
2
Note that the inverse of a block-diagonal matrix is block-diagonal, and A1 = FN1 FN11 ...F11 .
3
3
According to Prof. Strang, for a band matrix, ordinary elimination requires w2 n instead of n3 for a full
matrix, where n is the dimension of the matrix and w is the bandwidth of banded matrix.

Practical Implications
In this part, we will give a few practical examples, in which banded matrices with banded
inverses can be of great importance. The first example can be from Polar Codes. The
idea of polar coding is to introduce a one-to-one mapping between the source symbols and
the channel input symbols, which polarizes the channel.4 By channel polarizing, we mean
forming generic channels between input and output symbols by using this transformation.
Polar codes achieve capacity with a decoding complexity of O(nlogn). The idea of using a
banded matrix as a transformation matrix can be of great benefit in decoding complexity
because of the less number of calculations. However, one should be careful at choosing a
suitable transformation to polarize the channel.
Another example is broadcasting in High Definition TV (HD TV).5 In this scheme, if one is
dealing with problems offline, then delay is not very important. However, the main challenge
comes into the picture when the high quality content is broadcasted online (e.g., live HD
broadcasting). Assume we have very long sequence of video stream x to be transmitted. As
we do not have enough resources, e.g. time, we need to compress data. If we can find a
banded matrix with a banded inverse and if the resulted matrix y = Ax has less number of
data bits to be transmitted (i.e., more zeros in the vector y) with the same entropy, then
the process would be faster, resulting in smaller delay. As the mapping is one-to-one we do
not lose any information by doing this transformation.
Lastly, banded matrices can be useful for the Big Data or Machine Learning applications,
where one has huge data matrices and there is no obvious structure to be used. For example,
in deep learning algorithms, we have very big data matrices without structure. Then, we
can try to find a one-to-one mapping by introducing a banded transformation matrix with
a banded inverse. Thanks to this invertible transformation, we may need less number of
computations if the dimensions are very big, and the bandwidths of the banded matrices are
small.

4
5

Note that one can also think about source polarization.

In Image and Video Compression algorithms, this matrices can be of the great importance.