Robust Super-Resolution

Presented By:
Assam Daitchi

Project Goals
Understanding & simulation of
Dr. Assaf Zomet, et.al. paper :
Robust Super-Resolution [1]
Comparing the results obtained by this
method to other methods
Enhancing the method introduced in: [1]

Super-Resolution Objective
Generation of a high-resolution image
from multiple low-resolution moving
frames of a scene.

Super-Resolution Formulation

bk Dk Ck Fk X nk
b1 D1C1 F1


b p D p C p F

b HX n

1 k p

n1

n p

Solutions for S-R Problem

No Noise:
1

X H b
With Noise

min b HX 2 Lx , L I
2

X ( H H I ) H b
T

Problem with These Solutions

In the presence of outliers (error in
motion estimation, inaccurate blur
model, pepper & salt noise, ), these
methods do not work accurately.
Robust S-R methods can help in these
situations.

Robust S-R Formulation

bk Dk Ck Fk X nk
p

1 k p

1
L( X ) bk Dk Ck Fk X
2 k 1

Bk F C D ( Dk Ck Fk X bk )
T
k

T
k

T
k

Robust S-R Formulation

p

L( X ) Bk

n 1

k 1
n

X * L ( X )

Robustness

L(X)(x, y) median{B k (x, y)}

p
k 1

B1 B2 B3 B4 B5 B6 B7 B8 B9

median

Why Median?
Median is an estimate of mean.
Unlike regularization method only one of low
resolution frames contributes to reconstruct
each pixel in the high-resolution frame. So
outliers in other frames are discarded in the
reconstruction process.
Claim: In the absence of additive noise to all
frames median method works better than
regularization method.

What if noise is added to all

frames?
Claim: If considerable amount of
additive noise is present in all frames
regularization method works as good or
even better than median method.

Median-Average
Reconstruction
Instead of using

L(X)(x, y) median{B k (x, y)}

p
k 1

We can combine average and median

operators to get better results.

B1 B2 B3 B4 B5 B6 B7 B8 B9

average

Bias Detection Procedure

It is useful to detect the outliers in the
low resolution frames.
We can omit those outlier pixels in our
procedures.
The difficulty is to differentiate
between aliasing and outlier effects.

Bias Detection
Formulation

e yk Dk Ck Fk X
j
k

After thresholding non zero values are

due to aliasing or outliers.

Bias Detection Result

Due to outlier

1
1

1 1 1 1

1 1 1 1
1 1 1 1

1 1 1 1

Due to aliasing

0 0
0 0

1 -1

- 1 1

1 -1 0

-1 1 0
1 -1 0

0 0 0

Bias detection Procedure

1: compute e y k Dk Ck Fk X
2: Threshold
3: Filter the result with a LPF
4: Threshold
5: Omit corresponding pixels from
super-resolution procedure
j
k

Bias Detection
B-D method works only for uniform
gray level outliers.
In many situations median operation in
robust super-resolution eliminates the
bias of the estimator.

Original L-R Frame

Robust S-R Reconstructed H-R

Frame mse=0.0017

Median Reconstruction after

adding noise 0.0131

Regularized S-R Reconstructed

H-R Frame mse=0.0131`

Regularized Reconstruction
after adding noise mse=.0125

Error Due to Outlier

Error Due to Aliasing

Conclusion
Robust super-resolution method is quite
effective in the presence of outliers,
and produces better results in
comparison with regularization method.
In the presence of additive noise in all
low resolution frames this method
loses its superiority to the regularization
method.

Conclusion & Results

Combination of mean and median
operators can help us in this situation.
Proposed bias detection algorithm is an
effective method to detect outliers.
If outliers are the only source of error in
the L-R frames(no additive noise), more
iterations we use smaller mse we will
achieve.

Suggestions for Future

Research
Combining regularization and robust
super-resolution methods.
Using bias detection results in
regularization method.
Using robust super-resolution method in
frequency domain.

Acknowledgment
Thanks to Dr.Assaf Zomet, Dr.Michael
Elad, Dirk Robinson and Dr. Peyman
Milanfar for their valuable advices &
suggestions.

references
"Robust Super Resolution",
A. Zomet, A. Rav-Acha and S. Peleg
Proceedings of the International Conference
on Computer Vision and Pattern Recognition
(CVPR), Hawaii, December 2001.
"Efficient Super-Resolution and Applications to
Mosaics",
A. Zomet and S. Peleg,
Proceedings of the International Conference
on Pattern Recognition (ICPR), Barcelona,
September 2000.

A Computationally effective Image superresolution Algorithm, Nguyen, N., P. Milanfar,

G.H. Golub, IEEE Transactions on Image
Processing, vol. 10, no. 4, pp. 573-583, April
2001
A Fast Super-Resolution Reconstruction
Algorithm for Pure Transnational Motion and
Common Space Invariant Blur, M. Elad and Y.
Hel-Or, the IEEE Trans. on Image Processing,
Vol.10, no.8, pp.1187-93, August 2001.

Thank You All

Additional Simulatins

Original H-R Frame

Blured median

Projected L-R frame

L-R Frame

Regularization with high noise

mse=.0481

Median with high noise

mse=.0693

Composite Median & Average

MSE=0.0592

Original H.R. Frame

L.R. Frame Before Adding

Noise

L.R. Frame After Adding

Noise

Regularized Reconstruction
MSE=.0216

Median Reconstruction
MSE=.0077