MOTION ESTIMATION AND VIDEO COMPRESSION
ABSTRACT of change of this arrangement. Some have considered the
We propose a method for video compression using 2-D
motion estimation by using optical flow equation and
quantization techniques. Optical flow cannot be computed
locally, as only one independent measurement is available
from the image sequence at a point. However flow velocity
has 2 components in either direction. Thus we need a second
constraint to calculate it. We assume that the images are
composed of moving blocks. Thus we divide the image frame
into blocks of equal size, perform block motion estimation on
each block and obtain motion vectors. We start by taking a
video sequence, extract frames and divide each frame into
blocks of equal size. Further we divide each block into 9
equal sub-blocks and calculate the mean square error
between the center pixel of each block in frame k to all the
pixels in the respectively blocks in frame k+1. By repeating
this procedure for all the blocks, we obtain the motion
vectors, which are in turn are used to generate an estimate of
the next frame. We quantize the difference between the
original successive frame and the predicted frame. Adding
this quantized error to the predicted frame gives estimated
reconstructed video.
Index Terms- Block motion, displacement vector, intensity
gradient, motion compensation, motion estimation, occlusion,
optical flow, quantization and relative motion.
1. INTRODUCTION
Video sequence is a projection of 2-D images. However 2-D
images are the perspective views of a scene. Thus it is hard to
determine the 3-D structure of the object. Moreover it is also
difficult to determine the actual distance between the objects
in a scene. This becomes more challenging when we need to
estimate the motion using the 2-D images. Our aim is to
formulate a methodology to accurately estimate the 2-D
motion field from the time-varying images sampled on a
lattice and apply it to video compression.
Optical flow is the distribution of apparent velocities of
movement of brightness patterns in an image. Optical flow
can arise from the relative motion of objects and the viewer.
Consequently, optical flow can give important information
about the spatial arrangement of objects viewed and the rate
problem of recovering the motions of objects relative to the
viewer from the optical flow [4, 5]. In some cases
information about the shape of an object may also be
recovered. Related work has been done to formulate a model
for the short range motion detection processes in human
vision [6, 7]. However this algorithm is computational
complex and consumes more time to execute. It is also
sensitive to changes in lighting conditions and occlusion.
Block motion estimation and compensation is a widely
used algorithm for video compensation and compression. It is
applied in various video compression standards, such as
MPEG 1-2 and H.261. In this paper, we assume that images
are composed of moving blocks. We divide 2 consecutive
video frames into blocks of equal size and perform motion
estimation by three step algorithm technique and extend it to
video compression.
We start with an overview of the proposed algorithm for
video compression using block motion estimation in Section
2, followed by an overview on the three step search technique
in Section 3. In Section 4 we discuss describe how we use the
motion vectors obtained in previous section to compress the
video. In Section 5, we discuss the results obtained using the
algorithm proposed in Section 2. In Section 6 we give the
concluding remarks and some important references in Section
7.
2. VIDEO COMPRESSION USING BLOCK MOTION
ESTIMATION TECHNIQUE
Two dimensional motions are also known as “projected
motion”, which refers to the perspective or the orthographic
velocity projection of 3-D motion onto the image plane. We
perceive motion when a changing picture is projected onto a
stationary screen. Thus a moving object may give rise to a
constant brightness pattern. For our convenience, we tackle a
particular simple world where the apparent velocity patterns
can be directly identified with the movement of surfaces in
the scene. To avoid variations in the brightness due to
shading effects we assume the surface plane to be flat. We
further assume that the illumination distribution is uniform
along the surface. Eventually we may assume that at a point
in an image plane, the brightness is directly proportional to
 Input Extract 3-step Obtain Forward Predicted
Color frames ‘k’ motion motion motion video
Video and ‘k+1’ estimation vectors estimation frame

-
Predicted Original +
video frame
frame ‘k+1’

+
Reconstructed Quantized Prediction
frame ‘k+1’ error error

Fig. 1: Block representation for video compression using block motion estimation technique

The reflectance of the surface at the corresponding point on

the object. 3-D motion can be characterized in terms of Input RGB video sequence
either point.
Fig. 1 gives a diagrammatic representation of the Extract frames and resize each frame
compression technique using block motion estimation
technique. As shown in the figure, we start by extracting Divide each frame into blocks of size 16 by 16
frames from a color video and apply it to a three step block
motion estimator. The block motion estimator considers Divide each block into 9 equal parts
only two successive video frames, ‘k’ and ‘k+1’ at a time,
estimates the motion of frame ‘k+1’ with respect to frame
‘k’ and generates motion vectors. The block motion Calculate the MSE of a pixel in frame k with
estimation technique is described in Section 3. We predict Step respect to al the pixels in frame k+1
the next frame ‘k+1’ using the present frame ‘k’ and motion
vectors. This predicted frame is the prediction for frame
Find block with lowest MSE/MAD
‘k+1’. Thus there is an error in prediction. We calculate this
error by subtracting the predicted frame from the original
Divide this block into 9 equal parts
frame. This error cannot be transmitted directly. Thus we
quantize its values, combine them with the motion vectors
and transmit them to the decoder. Calculate the MSE of a pixel in frame k with
The decoder on the other hand, adds this quantized Step respect to al the pixels in frame k+1
error to the predicted error to generate the next frame ‘k+1’.

3. THREE-STEP BLOCK MOTION ESTIMATION Find block with lowest MSE/MAD

In this method, we divide two successive image frames into Divide this block into 9 equal parts
blocks of size N1 X N2. We consider one block at a time and
divide the block into 9 sub-blocks of equal size and
calculate the MSE of point (n1,n2) in each block in frame k Step Calculate the MSE of a pixel in frame k with
with respect to every pixel (n1,n2) in the respective sub- respect to al the pixels in frame k+1
block in frame k+1. We then consider the block with
minimum MSE and further divide the block into 9 equal Find block with lowest MSE/MAD
sized blocks. Then we decrease the window size and repeat
the procedure till the distance between the center pixel and
the neighboring pixel is 1 pixel. We repeat this procedure Draw a line connecting the center of the frame
for every block in the image. Fig. 4 shows the diagrammatic to the this point
representation of this method.
The matching of the blocks can be quantified according
to various criteria including the maximum cross-correlation, Fig. 2: Flowchart of 3 step motion estimation method

2
the minimum mean square error, minimum mean absolute
difference (MAD) and maximum matching pel count
(MPC). Here we use minimum mean square error criteria to
find the best match between two blocks in the successive
frames. It is defined as:
∑[s(n , n , k) − s(n +1, n2 +1, k +1)] (1)
1
MSE =
2
1 2 1
N1 N 2 (n1 , n2 )∈B
where, s(n1 , n2 , k ) corresponds to a pixel in frame k,
s(n1 + 1, n2 + 1, k + 1) corresponds to a pixel in frame k+1 and B
denotes an N1 X N2 block, for a set of candidate motion
vectors (d1,d2) . The estimate of the motion vector is taken
to be the value of (d1,d2) which minimizes the MSE.
Mathematically we may express it as,
∧ ∧ 
= arg . min MSE ( d 1 , d 2 )
T
 d 1 , d 2 
(d1 ,d 2 )
The displaced frame difference can be given by,
dfdk ,k+1 (n1, n2 , k) = s(n1, n2 , k) − s(n1 +1, n2 +1, k +1)
However implementing a squared term in hardware is not a
trivial task. Thus the MSE criterion is not generally
preferred to implement in hardware. Thus we use minimum
mean absolute difference criterion, defined as
∑| s(n1, n2 , k ) − s(n1 + d1, n2 + d2 , k + 1) |
1
MAD =
N1N2 ( n1 , n2 )∈B
(4)
The estimate of the motion vector to minimize the MAD is
given by,
∧ ∧
[ d 1 , d 2 ]T = arg . min MAD( d1 , d 2 )
( d1 , d 2 )
4. TECHNIQUE TO ACHIEVE VIDEO
COMPRESSION USING MOTION VECTORS
Motion vectors are achieved by using the technique
described in Section 3. These motion vectors denote the
direction of displacement along which the mean square
errors are minimum. Thus we move the center of each N1 X
N2 block to the co-ordinates given by the respective motion
vector. By repeating this for every block in the video frame,
we get an estimate of the next frame. However these
estimates are not accurate. Thus reconstructing a video
sequence using these blocks gives inaccurate results. The
resultant video is not continues and the block are very
evident. Thus, we formulate a technique to smoothen the
video.
This can be achieved by calculating the difference
between the predicted frame and the original frame ‘k+1’
and sending it to the decoder, as shown in Fig.3. However
sending this raw information is not feasible in digital world.
Thus we pass it through a quantizer which assigns a
particular value to each quantization level and transmits it
along with the motion vector to the input of the decoder.
Frame ‘k’ Frame ‘k+1’
+ Prediction
error
(2)
(3)
Fig. 3: Calculate prediction error
At the decoder’s end, the predicted frame is
reconstructed using the previous frame ‘k’ and the motion
vectors and added to the quantized error. This gives an
approximate of the reconstructed frame. Repeating this for
the entire video sequence will achieve compression.
5. EXPERIMENTAL RESULTS
The proposed algorithm was tested on a database of videos
(5) and was found to work satisfactorily with natural as well as
synthetic videos. For illustration, we take a natural video
shown in Fig. 4. This video is composed of 166 video
frames. In the first step we extract all these frames. Then we
consider 2 successive video frames ‘k’ and ‘k+1’ at a time
and apply them to three step block estimation algorithm. In
Fig. 2 we show extracted frame 60 and 61. The block
motion estimator gives motion vectors, which in turn gives
an estimate of where the block has moved in the successive
frame. By using forward block estimation technique and the
motion vectors obtained, we predict the next frame ‘k+1’.
However this prediction is not optimal and has errors. Thus,
we find the difference between the predicted and the
original frame ‘k+1’ and find the prediction error. This
prediction error cannot be transmitted in the form it is. Thus
it needs to be quantized into steps. This error varies in the
range of -1 to +1. Here we use an 8 bits uniform quantizer to
divide this range into 256 equal levels. This quantized error
along with the motion vectors and the previous frame ‘k’ to
the decoder.
At the decoder’s end, it receives the motion vectors,
previous frame ‘k’ and the quantized error as input. It
generates a predicted frame ‘k+1’ using the motion vectors
3
 3-step
motion
estimation
Extracted frames ‘k’
Color video
and ‘k+1’
Predicted frame
Reconstructed
video frame
Fig. 4: Results of video compression using block motion estimation algorithm
and the previous frame. The resultant is added with the
quantized error to reconstruct the video frame. The
reconstructed video is not exactly the same as the original
frame due to quantization loss. Thus it is a lossy
compression technique. This algorithm when applied to
entire video sequence, achieves compression.
6. CONCLUSION
In this paper a new approach to video compression using
block-matching estimation has been introduced that uses
general geometric transformations of pixel coordinates. The
main advantages of the proposed method are their simplicity
and ease to implement in hardware. They can be widely
used for video compression and in several digital video
processing applications like motion-compensated filtering
for standard conversions.
However these algorithms fail for zoom, rotational
motion, and under local deformations, which results in
serious artifacts, especially for very low bit rate
applications. They also fail to estimate motion when a
circular object with uniform surface intensity is rotated
about its axis. In such cases, estimating motion becomes
difficult. Thus much of the regions in motion estimation
remain untouched. Eventually adds constraint to video
compression. However there is lot of scope for development
of this algorithm to augment video compression process to
any arbitrary motion of objects in a video frame.
Forward
motion
estimation
Predicted frame
Motion Vectors
-
+
Frame ‘k+1’
Prediction error
+
+
Quantized
error
7. REFERENCES
[1] H. Gharavi and M. Mills, “Block-matching motion
estimation algorithms: New results,” IEEE Trans. Circ.
and Syst., vol. 37, pp. 649-651, 1990.
[2] V. Seferidis and M. Ghanbari, “General approach to
block-matching motion estimation,” Optical Engineering,
vol. 32, pp. 1464-1474, July 1993.
[3] M. Bierling, “Displacement estimation by hierarchical
block-matching,” Proc. Visual Comm. and Image Proc.,
SPIE vol. 1001, pp. 942-951, 1988.
[4] B. K. P. Horn and B. G. Schunck, “Determining
Optical Flow,” Artif. Intell., vol. 17, pp. 185-203, 1981.
[5] S. V. Fogel, “Estimation of velocity vector fields from
time varying image sequences,” CVGIP: Image
Understanding, vol. 53, pp. 253-287, 1991.
[6] T. S. Huang, ed., Image Sequence Analysis, Springer
Verlag, 1981.
[7] A. V. Oppenheim and R. W. Schafer, “Discrete - Time
Signal Processing,” Prentice Hall Signal Processing
Series, 1989.
[8] A. M. Tekalp, “Digital Video Processing,” Prentice
Hall Signal Processing Series, 1995.
[9] D. E. Dudgeon, “Multidimensional Digital Signal
Processing,” Prentice Hall Signal Processing Series, 1996.
[10] K. Sayood, “Introduction to Data Compression,”
Morgan Kaufmann Publishers, 2006.