You are on page 1of 5

International Journal of Engineering Research-Online

A Peer Reviewed International Journal Vol.2., Issue.6, 2014


Articles available online http://www.ijoer.in

RESEARCH ARTICLE ISSN: 2321-7758

A SURVEY ON ADAPTIVE EDGE-ENHANCED COLOR INTERPOLATION PROCESSOR


FOR VLSI ARCHITECTURE

JAYA PRAKA V.M*1, Mr. RAMJI D.R2, Dr. SREEJA MOLE S.S3
1
PG Student, ECE Department, Narayanaguru College of Engineering, Manjalumoodu, Tamil Nadu,
KK District, India.
2
Assistant Professor, ECE Department, Narayanaguru College of Engineering, Manjalumoodu, Tamil
Nadu, KK District, India.
3
Head of the Department ECE, Narayanaguru College of Engineering, Manjalumoodu, Tamil Nadu,
KK District, India.

Article Received: 20/12/2014 Article Revised on: 28/12/2014 Article Accepted on:30/12/2014

ABSTRACT
The digital cameras are developed by Charge Coupled Device (CCD) or
Complementary Metal Oxide Semiconductor (CMOS) image sensor that can capture
images by Color Filter Array (CFA) technique. A CFA used in digital camera is a mosaic
of spectrally selective filters, which allow only one color component to be sensed at
each pixel. The missing two components of each pixel have to be estimated by
methods known as Demosaicing. In this survey, different types of interpolation
algorithm and their hardware architecture have been analyzed and compared. The
algorithms are implemented for different types of field programmable gate array
(FPGA) and/or by different types of complementary metal oxide semiconductor
JAYA PRAKA V.M SARANYA.K
(CMOS) technologies like TSMC 0.18 and TSMC 0.13. Here, we propose a low
complexity adaptive edge-enhanced color interpolation algorithm for the VLSI
implementation in real time video applications. The goal of this proposed work is to
reduce computational complexity by reducing the number of components used for
interpolation. This novel algorithm consists of an edge detector, an anisotropic
weighting model and a filter based compensator. This proposed work is tries to
reduce the number of line buffers and combinational elements such as adder and
multiplexer of the existing architecture. In addition, the hardware cost was reduced
by using hardware sharing and reconfigurable techniques.

Key words- Camera, Charge coupled device (CCD), Demosaicing, Color filter array
(CFA), Edge detector, Interpolation, FPGA.

©KY Publications

254 JAYA PRAKA V.M


International Journal of Engineering Research-Online
A Peer Reviewed International Journal Vol.2., Issue.6, 2014
Articles available online http://www.ijoer.in

INTRODUCTION requirement. Furthermore, these algorithms are not


Digital cameras have become popular and easy to be realized using VLSI technique. For this
many people are choosing to take their pictures with reason, the VLSI architecture of low complexity and
digital cameras instead of film cameras. When a low memory requirement color interpolation
digital image is recorded, the camera needs to algorithms were conducted.
perform a significant amount of processing to COLOR DEMOSAICING METHODS
provide the user a viewable image. This processing Several demosaicing methods has been published in
includes white balance adjustment, gamma the past decade. Demosaicing methods are divided
correction, compression and more. A very important into three groups:
part of this image processing chain is color filter 1) Heuristic approaches
array interpolation or demosaicing. A color image 2) Restoration techniques
requires at least three color samples at each pixel 3) Image formation modeling
location. A camera would need three separate Heuristic approaches do not try to solve a
sensors to make these measurements. To reduce mathematically defined problem, but they are based
the size and cost, many cameras use a single sensor on reasonable assumptions about color images.
array with a color filter array. The color filter array Heuristic approaches are spatially adaptive, and they
allows only one part of the spectrum to pass to the may exploit correlation among the color channels.
sensor so that only one color is measured at each The second group of algorithms makes some
pixel. This means the camera must estimate the assumptions about the inter-channel correlations,
missing two color values at each pixel. This process and solves a mathematical problem based on those
is known as demosaicing. The most frequently used assumptions.
color filter array pattern is the Bayer CFA pattern The last group of methods models the image
shown in figure 1. formation process and formulates the demosaicing
problem based on this model.
Low power High quality Color Interpolation
Processor for CCD Camera [10]
VLSI implementation of low power high
quality color interpolation is proposed for CCD
camera. This method provides a color interpolation
technique for a single-chip charge coupled device
(CCD) with color filter array format. The
performance of this design was improved by a
pipeline schedule and time sharing technique.
Although the local gain that is obtained by the edge
direction weighting information, it can efficiently
Figure 1: Bayer CFA pattern improve the quality of interpolated images. Based
In the above figure, half of the pixels are allocated to on this algorithm, we arrange a suitable timing
the green color; other half is shared by the red and schedule to compute various color components for
blue color. The green image is measure at higher real time chip implementation using a parallel
sampling rate because the peak sensitivity of the architecture. It is necessary to use two division and
human visual system lies in the medium wavelength, three multiplication operations to obtain the edge
corresponding to the green portion of the spectrum. direction weighting and local gain information.
Many efficient high quality algorithms have been Figure 2 shows the interface of the color
proposed for reconstructing the full RGB color from interpolation processor.
CFA images. But these algorithms have the
characteristics of high complexity and high memory

255 JAYA PRAKA V.M


International Journal of Engineering Research-Online
A Peer Reviewed International Journal Vol.2., Issue.6, 2014
Articles available online http://www.ijoer.in

The control unit provides the control signals to other


units and handles the whole interpolation flow.

Figure 2: Interface of the color interpolation


processor
In this figure, the CCD signal is sampled with an
analog/ digital converter (ADC), and then the Figure 3: Block diagram of demosaicing chip
sampling data is sent to a color interpolation Adaptive homogeneity directed demosaicing
processor. Meanwhile the color interpolation algorithm [6]
processor reads the vertical and horizontal The demosaicing algorithm estimates
synchronous signals to control the operators. The missing pixels by interpolating in the direction with
color interpolation algorithm also employs three fewer color artifacts. To identify three inherent
scanning lines, so real time implementation requires problems often associated with directional
two line buffers. The line buffers are used to store interpolation approach to demosaicing algorithms:
the scanning lines. misguidance color artifacts, interpolation color
An Area Efficient Color Demosaicing Scheme [11] artifacts, and aliasing. The level of misguidance color
VLSI implementation of area efficient artifacts present in two images can be compared
demosaicing scheme is proposed for digital still using metric neighborhood modeling. The aliasing
camera (DSC). This design improved the quality of problem is solved by applying filter bank techniques
interpolated images by using edge information and to directional interpolation. The interpolation
inter- channel correlations. By using pipeline artifacts are reduced using a nonlinear iterative
architecture, the performance was greatly procedure.
improved. Since the parameters of the edge- Demosaicing algorithm with a directionality
information and inter-channel correlations were selection approach suffers from two types of color
divided by 2, 4, or 8, it is unnecessary to use any artifacts. The first type is called misguidance color
divider to realize this design. With this artifact. The misguidance occurs when the direction
implementation technique, the chip area of interpolation is erroneously selected. The second
significantly reduced. The reconstructed pixels are type of color artifact is associated with limitations in
interpolated along the edge instead of across the the interpolation. That is, even with a perfect
edge, so, may obtain better image quality. Figure 3 directional selector, the interpolation algorithm may
shows the block diagram of demosaicing chip. The not reconstruct the color image perfectly.
VLSI architecture consist of four main parts: the Interpolation artifacts are reduced using iterative
interpolating unit, the line buffer, the register bank algorithm. The following iterative procedure,
and the control unit. The interpolating unit performs performed m times, suppresses small variation in
the interpolation process row by row to generate color while preserving edges:
the missing R, G or B components of each pixel. A repeat _m_ times
line buffer is used to store the pixel values of one 1. R = median (R-G) + G
row in the image. The register bank is employed to 2. B = median (B-G) + G
store the neighboring pixel values temporally and 3. G = ½ (median (G-R) + median (G-B) + R
provide the needed data to the interpolating unit. + B)

256 JAYA PRAKA V.M


International Journal of Engineering Research-Online
A Peer Reviewed International Journal Vol.2., Issue.6, 2014
Articles available online http://www.ijoer.in

end _ repeat. the four nearest CFA samples of the same color. The
Wavelet based color filter array demoasicing [17] estimates from one particular direction is obtained
To employ the higher spatial resolution of by exploiting the spectral correlations along that
the luminance image, by merging it with the directions, the correlation that states that the
interpolated color image. For this merging, difference between the green and red/blue values
multiresolution techniques using the wavelet within a local neighborhood are well correlated with
transform. Employ the wavelet transform for the constant offsets. The post processing step aims to
merging of an interpolated luminance image with an suppress visible artifacts residing in the
interpolated color image, where the conventional demosaicked images obtained from the
interpolation techniques are applied. In a first step, interpolation step. There are two main types of
a luminance image is formed. This can be done in demosaicking artifacts, namely false colors and
one of the conventional ways (bilinear interpolation zipper effect. The false colors are those artifacts
of the green pixels as in the Bayer reconstruction, corresponding to noticeable color errors as
the alternative Bayer reconstruction or the gradient compared to the non-mosaicked original image. The
reconstruction). Also the red, blue, and green zipper effect refers to abrupt or unnatural changes
component images are interpolated in a in color or intensity between neighboring pixels,
conventional ways. Then the luminance image and manifesting as an “on-off” pattern.
each band of the RGB image is wavelet transformed PROPOSED METHOD
separately. Two merging rules are investigated: the Here, a low complexity adaptive edge
‘replace’ rule replaces the coefficients from the enhanced color interpolation algorithm is proposed
color band by the coefficients from the luminance for the VLSI implementation in real time video
image. The ‘max’ rule replaces the coefficients from applications. The proposed novel algorithm consists
the color band by the maximum of the coefficients of an edge detector, an anisotropic weighting model
from the luminance and the color band. Finally the and a filter-based compensator. The anisotropic
obtained color wavelet coefficients are inversely weighting model is designed to catch more
wavelet transformed to obtain the demosaicked information in horizontal direction than vertical
RGB image. directions. A low complexity edge detector is
Adaptive CFA demosaicing using Directional spatial created to enhance the edge information in the
and spectral correlations [14] images. It used only addition, subtraction, and
To develop a new and improved CFA absolute operations to obtain the edge information.
demosaicing method for producing high quality The hardware cost of the edge detector is much less
color images from CFA samples. This method than the previous designs such as [8] and [10], which
consists of two successive steps: utilize dividers and multipliers to obtain the edge
1) Interpolation step and gain information. The filter-based compensation
2) Post processing step methodology includes a Laplacian and spatial
An interpolation step estimating the missing color sharpening filters which are developed to improve
values by exploiting spatial and spectral correlations the edge information and reduce the blurring effect.
among neighboring pixels in different directions, and The hardware cost is reduced by using hardware
a post processing step suppressing noticeable sharing and reconfigurable techniques. Compared
artifacts by using adaptive mean filtering. The with previous low complexity techniques, this work
interpolation step fills in missing color values in reduces the gate counts or power consumption, also
sequentially, the green plane is the first to be fully improves the average CPSNR quality.
populated and (once fully populated) used to assist CONCLUSION
the subsequent red and blue plane interpolation. In this paper different types of
Each missing color value is interpolated by properly interpolation algorithm and their hardware
combining the estimates obtained from four architectures are discussed. Also a novel color
interpolation directions which are defined based on interpolation algorithm is proposed to develop low

257 JAYA PRAKA V.M


International Journal of Engineering Research-Online
A Peer Reviewed International Journal Vol.2., Issue.6, 2014
Articles available online http://www.ijoer.in

cost, low power, high performance, and high quality Trans. Image Process., vol. 14, no. 2, pp.
color interpolation processor for real time video 267-278, Feb. 2005.
applications. This algorithm consists of an [10]. S. C. Hsia, M. H. Chen, and P. S. Tsai, “VLSI
anisotropic weighting model, an edge detector, and implementation of low-power high-quality
filter-based compensation methods which are used color interpolation processor for CCD
to reduce the memory requirements and improve camera,” IEEE Trans. Very Large Scale
the quality of images. The number of arithmetic Integration (VLSI) Systems, vol. 14, no. 4,
elements in proposed architecture is much less than pp. 361-369, Apr. 2006..
the other interpolation methods. [11]. Y. H. Shiau, P. Y. Chen, and C. W. Chang,
REFERENCES “An area-efficient color demosaicing
[1]. H. A. Chang, and H. H. Chen, “Stochastic scheme for VLSI architecture,” International
color interpolation for digital cameras”, Journal of Innovative Computing
IEEE Transaction on Circuits and Systems Information and Control, vol. 7, no. 4, pp.
for Video Technology, Vol. 17, no. 8, pp. 1739-1752, Apr. 2011.
964-973, Aug. 2007. [12]. J. Marial, M. Elad, and G. Sapiro, “Sparse
[2]. R. Lukac, K. N. Plataniotis, and D. H. representation for color image
Atzinakos, “ Color image zooming on the restoration,” IEEE Trans. Image Process.,
Bayer pattern,” IEEE Transaction on Circuits vol. 17, no. 1, pp. 53-69, Jan 2008.
and Systems for Video Technology, Vol. 15, [13]. N. X. Lian, L. Chang, Y. P. Tan, and V.
no. 8, pp. 1475-1492, Nov. 2005. Zagorodnov, “Adaptive filtering for CFA
[3]. S. C. Pei, and I. K. Tam, “Effective color demosaicing,” IEEE Trans. Image Process.,
interpolation in CCD color filter arrays using vol. 16, no. 10. pp. 2515-2525, Oct. 2007.
signal correlation,” IEEE Transaction on [14]. L. Chang, and Y. P. Tan, “Effective use of
Circuits and Systems for Video Technology, spatial and spectral correlations for CFA
Vol. 13, no. 6, pp. 503-513, Jun.2003. demosaicking,” IEEE Trans. Consumer
[4]. B. E. Bayer, “Color Imaging Array,” U. S. Electronics, vol. 50, no. 1, pp. 355-365. Feb.
patent 3 971 065, Jul. 1976. 2004.
[5]. D. Menon, S. Andriani, and G. Calvagno, [15]. D. Menon and G. calvagno, “Regularization
“Demosaicing with directional filtering and Approaches to Demosaicking,” IEEE Trans.
a posteriori decision,” EEE Trans. Image Image Process., vol. 18, no. 10, pp. 2209-
Process., vol. 16, no. 1, pp. 132-141, Jan. 2220, Oct 2009.
2007. [16]. K. Jensen and D. Anastassiou, “Subpixel
[6]. K. Hirakawa and T. W. Parks, “Adaptive edge localization and the interpolation of
homogeneity-directed demosaicing still images,” IEEE Trans. Image Process.,
algorithm,” IEEE Trans. Image Process., vol. vol. 4, no.3, pp. 285-295, Mar. 1995.
14, no. 3, pp. 360-369, Mar. 2005. [17]. J. Driesen and P. Scheunders, “Wavelet
[7]. D. Alleysson, S. Susstrunk, and J. Herault, based color filter array demosaicing”,
“Linear demosaicing inspired by the human International Conference on Image
visual system,” IEEETrans. Image Process., processing (ICIP), 2004.
vol. 14, no. 4, pp. 439-449, Apr. 2005. [18]. B. K. Gunturk, Y. Altunbasak, and R. M.
[8]. S. C. Hsia, and P. S. Tsai, ”VLSI Mersereau, “Color plane interpolation
implementation of camera digital signal using alternating projection”, IEEE Trans.
processor for document projection,” in Image Process., vol. 11, no. 9, pp. 997-1013,
Proc. IEEE Int. Conf. Signal Processing Sep.2002.
System (ICSPS), Jul. 2010, pp. 657-660.
[9]. D. D. Mmuresan and T. W. Parks, “
Demosaicing using optimal recovery,” IEEE

258 JAYA PRAKA V.M