IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO.

5, MAY 2011

1249

A Generalized Unsharp Masking Algorithm

Guang Deng
AbstractEnhancement of contrast and sharpness of an image is required in many applications. Unsharp masking is a classical tool for sharpness enhancement. We propose a generalized unsharp masking algorithm using the exploratory data model as a unied framework. The proposed algorithm is designed to address three issues: 1) simultaneously enhancing contrast and sharpness by means of individual treatment of the model component and the residual, 2) reducing the halo effect by means of an edge-preserving lter, and 3) solving the out-of-range problem by means of log-ratio and tangent operations. We also present a study of the properties of the log-ratio operations and reveal a new connection between the Bregman divergence and the generalized linear systems. This connection not only provides a novel insight into the geometrical property of such systems, but also opens a new pathway for system development. We present a new system called the tangent system which is based upon a specic Bregman divergence. Experimental results, which are comparable to recently published results, show that the proposed algorithm is able to signicantly improve the contrast and sharpness of an image. In the proposed algorithm, the user can adjust the two parameters controlling the contrast and sharpness to produce the desired results. This makes the proposed algorithm practically useful. Index TermsBregman divergence, exploratory data model, generalized linear system, image enhancement, unsharp masking.

I. INTRODUCTION NHANCING the sharpness and contrast of images has many practical applications. There has been continuous research into the development of new algorithms. In this section, we rst briey review previous works which are directly related to our work. These related works include unsharp masking and its variants, histogram equalization, retinex and dehazing algorithms, and generalized linear systems. We then describe the motivation and contribution of this paper. A. Related Works 1) Sharpness and Contrast Enhancement: The classical unsharp masking algorithm can be described by the equation: where is the input image, is the result of a linear low-pass lter, and the gain is a real scaling is usually amplied to factor. The signal increase the sharpness. However, the signal contains 1) details of the image, 2) noise, and 3) over-shoots and under-shoots
Manuscript received May 27, 2010; revised August 31, 2010; accepted November 01, 2010. Date of publication November 15, 2010; date of current version April 15, 2011. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Yongyi Yang. The author is with Department of Electronic Engineering, La Trobe University, Bundoora, Victoria 3086, Australia (e-mail: d.deng@latrobe.edu.au). Color versions of one or more of the gures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TIP.2010.2092441

in areas of sharp edges due to the smoothing of edges. While the enhancement of noise is clearly undesirable, the enhancement of the under-shoot and over-shoot creates the visually unpleasant halo effect. Ideally, the algorithm should only enhance the image details. This requires that the lter is not sensitive to noise and does not smooth sharp edges. These issues have been studied by many researchers. For example, the cubic lter [1] and the edge-preserving lters [2][4] have been used to replace the linear low-pass lter. The former is less sensitive to noise and the latter does not smooth sharp edges. Adaptive gain control has also been studied [5]. Contrast is a basic perceptual attribute of an image [6]. It is difcult to see the details in a low contrast image. Adaptive histogram equalization [7], [8] is frequently used for contrast enhancement. The retinex algorithm, rst proposed by Land [9], has been recently studied by many researchers for manipulating contrast, sharpness, and dynamic range of digital images. The retinex algorithm is based upon the imaging model in which the observed image is formed by the product of scene reectance and illuminance. The task is to estimate the reectance from the observation. Many algorithms use the assumption that the illuminance is spatially smooth. The illuminance is estimated by using a low-pass lter or multiresolution [10] or formulating the estimating problem as a constrained optimization problem [11]. To reduce the halo effect, edge-preserving lters such as: adaptive Gaussian lter [12], weighted least-squares based lters [13] and bilateral lters [11], [14] are used. Recently, novel algorithms for contrast enhancement in dehazing applications have been published [15], [16]. These algorithms are based upon a physical imaging model. Excellent results have been demonstrated. An important issue associated with the unsharp masking and retinex type of algorithm is that the result is usually out of the range of the image [12], [17][19]. For example, for an 8-bit image, the range is [0, 255]. A careful rescaling process is usually needed for each image. A histogram-based rescaling process and a number of internal scaling processes are used in the retinex algorithm presented in [19]. 2) Generalized Linear System and the Log-Ratio Approach: In his classic book, Marr [20] has pointed out that to develop an effective computer vision technique one must consider: 1) why the particular operations are used, 2) how the signal can be represented, 3) what implementation structure can be used. Myers [21] has also pointed out that there is no reason to persist with particular operations such as the usual addition and multiplication, if via abstract analysis, more easily implemented and more generalized or abstract versions of mathematical operations can be created for digital signal processing. Consequently, abstract analysis may show new ways of creating systems with desirable properties. Following these ideas, the generalized linear system, shown in Fig. 1, is developed. The generalized addi-

1057-7149/$26.00 2010 IEEE

1250

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Fig. 1. Block diagram of a generalized linear system, where () is usually a nonlinear function.

tion and scalar multiplication operations denoted by are dened as follows:

and (1)

and (2) where and are signal samples, is usually a real scalar, and is a nonlinear function. The log-ratio approach [17] was proposed to systematically tackle the out of range problem in image restoration. The logratio can be understood from a generalized linear system point of view, since its operations are implicitly dened by using (1) and (2). A remarkable property of the log-ratio approach is that the gray scale set is closed under the new operations. Deng [22] used the log-ratio in a generalized linear system context for image enhancement. In their review papers, Cahill and Deng [23] and Pinoli [24] compared the log-ratio with other generalized linear system-based image processing techniques such as the multiplicative homomorphic lters [25] and the logarithmic image processing (LIP) model [18], [26]. B. Issues Addressed, Motivation, and Contributions In this paper we address the following issues related to contrast and sharpness enhancement. 1) Enhancement of the overall contrast and the sharpness of the image are two related tasks. However, contrast enhancement does not necessarily leads to sharpness enhancement. 2) Classical unsharp masking techniques aiming at enhancing sharpness of the image usually suffers from the halo effect. 3) When enhancing the sharpness of an image, minute details are usually enhanced at the cost of enhancing noise as well. Part of the difculty in dealing with this issue is rooted in the denition of image details. (4) Recently published algorithms, although very successful in contrast and sharpness enhancement, have a rescaling process which must be performed very carefully for each image to ensure the best result. In this paper, we address issues 1)3) using an exploratory data model and issue 4) using the log-ratio operations and a new generalized linear system proposed in this paper. This work is partly motivated by a classic work in unsharp masking [1], an excellent analysis of the halo effect [27], and the requirement of the rescaling process [12], [19]. In [17], only the log-ratio operations were dened. Motivated by the elegant theory of the LIP model [24], we study the properties of these operations as well as the generalized linear system. The major contribution and the organization of this paper are as follows. In Section II, we rst present an exploratory data model as a unied framework for developing a generalized unsharp masking algorithm. We then outline the proposed algorithm and show how the previous issues are dealt with in terms of the model. In Section III, the log-ratio operations are

dened and their properties are discussed. This is followed by a study of the connection between the generalized linear systems (including the log-ratio, multiplicative homomorphic system (MHS) and the LIP model) and the Bregman divergence. This connection not only provides us with a new insight into the generalized linear system, but also opens a new path way for developing such systems from different Bregman divergences. As an example, we propose a new generalized linear system called the tangent system based upon a specic Bregman divergence. We show that the new system can also be used to systematically solve the out-of-range problem. To the best of our knowledge, the connection between the Bregman divergence and the generalized linear system has not been published before. Thus, the study reported in this section constitute a novel contribution of this paper. In Section IV, we describe the details of each building block of the proposed algorithm which includes the iterative median lter (IMF), the adaptive gain control, and the adaptive histogram equalization. In Section V, we present simulation results which are compared to recently published results. Conclusion and future work are presented in Section VI. In Appendix I, we discuss the log-ratio and tangent operations from a vector space point of view. II. EXPLORATORY DATA ANALYSIS MODEL FOR IMAGE ENHANCEMENT A. Image Model and Generalized Unsharp Masking A well known idea in exploratory data analysis is to decompose a signal into two parts. One part ts a particular model, while the other part is the residual. In Tukeys own words the PLUS residuals ([28] pp.208). From data model is: this point of view, the output of the ltering process, denoted , can be regarded as the part of the image that ts the model. Thus, we can represent an image using the generalized operations (not limited to the log-ratio operations) as follows: (3) where is called the detail signal (the residual). The detail signal where is the generalized subtraction is dened as operation. Although this model is simple, it provides us with a unied framework to study unsharp masking algorithms. A general form of the unsharp masking algorithm can be written as (4) and could where is output of the algorithm and both be linear or nonlinear functions. This model explicitly states that the part of the image being sharpened is the model residual. This will force the algorithm developer to carefully select an appropriate model and avoid models such as linear lters. In addition, this model permits the incorporation of contrast enhancement by such as adaptive means of a suitable processing function histogram equalization. As such, the generalized algorithm can enhance the overall contrast and sharpness of the image. B. Outline of the Proposed Algorithm The proposed algorithm, shown in Fig. 2, is based upon the previous image model and generalizes the classical unsharp

DENG: A GENERALIZED UNSHARP MASKING ALGORITHM

1251

denitions are presented in Appendix I. We then study properties of these new operations from an image processing perspective. We show the connection between the log-ratio, generalized linear systems and the Bregman divergence. As a result, we not only show novel interpretations of two existing generalized linear systems, but also develop a new system. A. Denitions and Properties of Log-Ratio Operations
Fig. 2. Block diagram of the proposed generalized unsharp masking algorithm.

TABLE I COMPARISON OF THE CLASSICAL UNSHARP MASKING (UM) ALGORITHM WITH THE PROPOSED GENERALIZED UNSHARP MASKING (GUM) ALGORITHM. LPF: LOW-PASS FILTER. EPF: EDGE PRESERVING FILTER. ACE: ADAPTIVE CONTRAST ENHANCEMENT

1) Nonlinear Function: We consider the pixel gray scale of . For an -bit image, we can rst add a very an image small positive constant to the pixel gray value then scale it by such that it is in the range . The nonlinear function is dened as follows: (6) To simplify notation, we dene the ratio of the negative image to the original image as follows: (7) 2) Addition and Scalar Multiplication: Using (1), the addition of two gray scales and is dened as (8) and where gray scale by a real scalar using (2) as follows: . The multiplication of a is dened by

masking algorithm by addressing issues stated in Section I-B. We address the issue of the halo effect by using an edge-preserving lter-the IMF to generate the signal . The choice of the IMF is due to its relative simplicity and well studied properties such as the root signals. Other more advanced edge preserving lters such as the nonlocal means lter and wavelet-based denoising lters can also be used. We address the issue of the need for a careful rescaling process by using new operations dened according to the log-ratio and new generalized linear system. Since the gray scale set is closed under these new operations and scalar multiplication formally dened in (addition Section III), the out-of-range problem is systematically solved and no rescaling is needed. We address the issue of contrast enhancement and sharpening by using two different processes. The image is processed by adaptive histogram equalization . The detail image is processed and the output is called where is the adaptive gain and is a by function of the amplitude of the detail signal . The nal output of the algorithm is then given by (5) We can see that the proposed algorithm is a generalization of the classical unsharp masking algorithm in several ways which are summarized in Table I. In the following, we present details of the new operations and enhancement of the two images and . III. LOG-RATIO, GENERALIZED LINEAR SYSTEMS AND BREGMAN DIVERGENCE In this section, we rst dene the new operations using the generalized linear system approach. We use (1) and (2) to simplify presentation. Note that these operations can be dened from the vector space point of view which is similar to that of the development of the LIP model [26]. The vector space related

(9) This operation is called scalar multiplication which is a terminology derived from a vector space point of view [29]. We can dene a new zero gray scale, denoted , as follows: (10) . This denition is consistent with It is easy to show that . As the denition of scalar multiplication in that and a result, we can regard the intervals as the new denitions of negative and positive numbers, respectively. The absolute value, denoted by , can be dened in a similar way as the absolute value of the real number as follows: (11) 3) Negative Image and Subtraction Operation: A natural extension is to dene the negative of the gray scale. Although this can be dened in a similar way as those described in (8) and (9), we take another approach to gain deeper insights into the operation. The negative of the gray scale , denoted by , is obtained by solving (12) The result is which is consistent with the classical denition of the negative image. Indeed, this denition is also consistent with the scalar multiplication in that

1252

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Fig. 3. Effects of the log-ratio addition y

= x 8 a (top row) and scalar multiplication operations y = 

x (bottom row).
TABLE II ORDER RELATIONS OF THE LOG-RATIO ADDITION

. Following the notation of the classical denition of negative numbers, we can dene the negative gray scale as: . We can now dene the subtraction operation using the addition operation dened in (8) as follows:

(13) where we can easily see that . Using the denition of the negative gray scale, we also have a clear understanding of the scalar multiplication for
Fig. 4. Enhancement of the dark areas of an image. Images from left to right are: original, enhanced by gamma correction and enhanced by log-ratio addition. Only the center part of the image is shown due to space limitation.

(14) Here we have used two real scalars and and the distributive law for (15) The distributive law can be easily veried by using (9). Other laws such as the commutative and associative laws can be easily proved and are presented in Appendix I. 4) Properties: What would be the result if we add a constant to an image such that ? Since , we only consider two cases: and the zero is . The order relations for the addition operation are shown in Table II. When is xed, the result is a function of . In the top row of Fig. 3, we show several cases in order to visualize the

effects. We can see that when the dynamic range of the higher gray scale values are expanded at the cost of com, the pressing that of lower gray scale values. When effect is just the reverse of that of the previous case. Therefore, has a similar effect as that of the gamma the operation correction operation dened as . For example, to enhance the dark areas of the cameraman image (8-bit/pixel), we could use for the log-ratio and for gamma correction. The results are shown in Fig. 4. For the subtraction operation, we can derive similar results as those of the addition operation by considering . , we already know three For scalar multiplication . In Table III, we list the order special cases when

DENG: A GENERALIZED UNSHARP MASKING ALGORITHM

1253

TABLE III ORDER RELATIONS OF THE LOG-RATIO SCALAR MULTIPLICATION

B. Log-Ratio, the Generalized Linear System and the Bregman Divergence We study the connection between the log-ratio and the Bregman divergence. This connection not only provides new insights and a geometrical interpretation of the log-ratio, but also suggests a new way to develop generalized linear systems. 1) Log-Ratio and the Bregman Divergence: To motivate our discussion, we study the following problem: since the classical weighted average can be regarded as the solution of the following optimization problem: (20) what is the corresponding optimization problem (if it exists) that leads to the generalized weighted average stated in (19)? To study this problem, we need to recall some recent results in the Bregman divergence [30], [31]. The Bregman divergence of , is dened as follows: two vectors and , denoted (21) where is a strictly convex and differentiable funcis the tion dened over an open convex domain and gradient of evaluated at the point . The centroid of a set of vectors denoted in terms of minimizing the sum of the Bregman divergence is studied in a recent paper [31]. The weighted left-sided centroid1 is given by

relations for . The corresponding relationships for the case can be easily derived by using (14). In the bottom row of of Fig. 3, we show several cases which help us to understand the characteristics of the scalar multiplication. We can see from , the dynamic ranges of the the gure that when gray scale values close to 0 or 1 are expanded and that near the is compressed. When , the dynamic range middle is expanded and those near 0 of the gray scale values near or 1 are compressed. In fact, when is set a large number, say for 10, the effect is just like a thresholding process, i.e., and for . 5) Computation: Computations can be directly performed using the new operations. For example, for any real numbers and , the weighted summation is given by (16) The subtraction operation can be regarded as a weighted sumand . In image processing the mation with is classical weighted average of a block of pixels given by

(22) (17) where . Similarly, the generalized weighted averaging operation is dened as Comparing (19) with (22), we can see that when is a scalar, the generalized weighted average of the log-ratio is a special . case of the weighted left-sided centroid with In fact, it is easy to show that

(18) where and are the weighted geometric means of the original and the negative images, respectively. An indirect computational method is through the nonlinear function (6). For example, the generalized weighted average can be calculated as follows:

(23) where the constant of the indenite integral is omitted. The previous function is called the bit entropy and the corresponding Bregman divergence is dened as

(24) which is called the logistic loss.

(19) Although this computational method may be more efcient than the direct method [see (18)] in certain applications, the direct method provides us more insights into the operation. For example, (18) clearly shows the connection between the generalized weighted average and the geometric mean.

1Since

(ykx), there is also a righted-sided centroid

the Bregman divergence is not symmetric, i.e.,

(xky)

6=

= arg min 1

A
x

(x

jjc)

where

A=

1254

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Therefore, the log-ratio has an intrinsic connection with the Bregman divergence through the generalized weighted average. This connection reveals a geometrical property of the log-ratio which uses a particular Bregman divergence to measure the generalized distance between two points. This can be compared with the classical weighted average which uses the Euclidean distance. In terms of the loss function, while the log-ratio approach uses the logistic loss function, the classical weighted average uses the squared loss function. 2) Generalized Linear Systems and the Bregman Divergence: Following this approach, we can derive the connections between the Bregman divergence with other well established generalized linear systems such as the MHS with where and the LIP model [26] with where . The corresponding Bregman divergences are the KullbackLeibler (KL) divergences for the MHS [31] (25) and the LIP (26) respectively. In our past work [32], we demonstrated that the LIP model has an information-theoretic interpretation. The relationship between the KL divergence and the LIP model reveals a novel insight into its geometrical property. 3) A New Generalized Linear System: Because for a specic Bregman divergence, there is a corresponding generalized weighted average (the left-sided centroid), we can dene a new . generalized linear system (see Fig. 1) by letting For example, the log-ratio, MHS and LIP can be developed from their respective Bregman divergences. This approach of dening a generalized linear system is based upon using the Bregman divergence as a measure of the distance of two signal samples. The measure can be related to the geometrical properties of the signal space. A new generalized linear system for solving the out-of-range problem can be developed by using the following Bregman divergence (called Hellinger-like divergence in Table I in [31]): (27) which is generated by the convex function whose domain is . The nonlinear function for the corresponding generalized linear system is as follows: (28) In this paper, the generalized linear system is called the tangent system2 and the new addition and scalar multiplication operagiven in (28)] are called tions [dened in (1) and (2) with tangent operations. For image processing applications, we can rst linearly to a new interval map the pixel value from the interval
2The term tangent comes directly from a geometrical interpretation of the function (x) = x= 1 x as the tangent of an angle, i.e., tan( ) = (x) where  is an angle determined uniquely by x.

Fig. 5. Properties of the tangent addition operation y = x constant.

8 a where a is a

. Then the image is processed by using the tangent operations. The result is then mapped back to the interval through the inverse mapping. We can easily verify that the is closed under the tangent operations. signal set As such the tangent system can be used as an alternative to the log-ratio to solve the out-of-range problem. The properties and applications of the tangent operations can be studied in a similar way as those presented in Section III-A. We can dene the negative image and the subtraction operation, and study the order relations for the tangent operations. As an example, we show in Fig. 5 the results of adding a constant to an N-bit using the tangent addition. In this simulation, image we use a simple function to map the image to . We can see that the effect is similar to from that of the log-ratio addition (see Fig. 4). 4) Summary: In this section, we show a novel connection between the generalized linear systems and the Bregman divergence. We also propose a new system. In Table IV, we summarize key components of some generalized linear systems which can be developed by using the Bregman divergence. However, a detailed comparison of these systems is out of the scope of this paper. Pinoli [24] presented a comparison of some generalized linear systems from a vector space point of view. IV. PROPOSED ALGORITHM A. Dealing With Color Images We rst convert a color image from the RGB color space to the HSI or the LAB color space. The chrominance components such as the H and S components are not processed. After the luminance component is processed, the inverse conversion is performed. An enhanced color image in its RGB color space is obtained. The rationale for only processing the luminance component is to avoid a potential problem of altering the white balance of the image when the RGB components are processed individually. B. Enhancement of the Detail Signal 1) The Root Signal and the Detail Signal: Let us denote the which maps median ltering operation as a function the input to the output . An IMF operation can be represented as: where is the iteration index . The signal is usually called the root signal of and

p 0

DENG: A GENERALIZED UNSHARP MASKING ALGORITHM

1255

TABLE IV KEY COMPONENTS OF SOME GENERALIZED LINEAR SYSTEM MOTIVATED BY THE BREGMAN DIVERGENCE. THE DOMAIN OF ; M ). IN THIS TABLE, IT IS NORMALIZED BY M TO SIMPLIFY NOTATION THE LIP MODEL IS (

01

Fig. 6. Mean squared difference of images between two iterations H (y ; y ) = (1=N ) y y where N is the number of pixels in the image. Results using two settings of the median lter and using the cameraman image are shown.

jj 0

jj

the ltering process if as follows: root signal subject to

. It is convenient to dene a (29)

where is a suitable measure of the difference between the two images and is a user dened threshold. For natural images, it is usually the case that the mean squared differ( is the ence, dened as number of pixels), is a monotonic decreasing function of . An example is shown in Fig. 6. It can be easily seen that the denition of the root signal depends upon the threshold. For example, it is possible to set a large value to such that is the root signal. Indeed, after about the difference changes only ve iterations very slightly. As such, we can regard or as the root signal. Of course, the number of iterations, the size and the shape of the lter mask have certain impacts on the root signal. The properties of the root signal has been extensively studied [33]. Here we use an example to illustrate the advantage of the proposed algorithm over the classical unsharp masking algorithm.

The original signal is shown in Fig. 7, which is the 100th row of the cameraman image. The root signal is produced by an IMF lter with a (3 3) mask and three iterations. The signal is produced by a linear low-pass lter with a uniform mask of (5 5). The gain for both algorithms is three. Comparing the enhanced signals (last row in the gure), we can clearly see that while the result for the classical unsharp masking algorithm suffers from the out of range problem and halo effect (under-shoot and over-shoot), the result of the proposed algorithm is free of such problems. 2) Adaptive Gain Control: We can see from Fig. 3 that to enhance the detail signal the gain must be greater than one. Using a universal gain for the whole image does not lead to good results, because to enhance the small details a relatively large gain is required. However, a large gain can lead to the saturation of the detailed signal whose values are larger than a certain threshold. Saturation is undesirable because different amplitudes of the detail signal are mapped to the same amplitude of either 1 or 0. This leads to loss of information. Therefore, the gain must be adaptively controlled. In the following, we only describe the gain control algorithm for using with the log-ratio operations. Similar algorithm can be easily developed for using with the tangent operations. To control the gain, we rst perform a linear mapping of the detail signal to a new signal

(30) such that the dynamic range of is . A simple idea is to set the gain as a function of the signal and to gradually when decrease the gain from its maximum value to its minimum value when . More specically, we propose the following adaptive gain control function (31) where is a parameter that controls the rate of decreasing. The two parameters and are obtained by solving the equations:

1256

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Fig. 8. Illustrations of adaptive gain control for four combinations of and  . The results of using a xed gain are also shown.

Fig. 7. Illustration of the difference between the proposed generalized unsharp masking algorithm (left panel) and the classical unsharp masking algorithm (right panel). Comparing the two enhanced signal shown in the bottom row of the gure, we can clear see that while the classical unsharp masking algorithm (right) produces over- and under-shoots around sharp edges (halo-effects), the proposed algorithm (left) is almost free of these effects.

and . For a xed , we can easily determine the two parameters as follows: (32) and (33) Although both and could be chosen based upon each individual image processing task, in general it is reasonable . This setting follows the intuition that when the to set amplitude of the detailed signal is large enough, it does not need further amplication. For example, we can see that (34) As such, the scalar multiplication has little effect. We now study the effect of and by setting . by using In Fig. 8, we show the nonlinear mapping function for the four combinations adaptive gain control with . We also compare these functions to that using of and a xed gain . We can see that while the xed gain leads to saturation when the amplitude of the detail signal is large, the adaptive gain does not suffer from this problem. For the adaptive gain control (dashed-line), a larger value of leads to a larger amplication of the amplitude of the

detail signal around 1/2. This is evident when we compare the gradients of those dash lines around 1/2. Recall that 1/2 is the is equivalent to ) zero (smallest amplitude, i.e., helps enof the detailed signal. Therefore, large value of hance the minute details of the image. The role of the parameter is to control the amplication behavior of the nonlinear mapis relatively large. Comparing the two ping function when , plots at the bottom row of Fig. 8, we can see that when the nonlinear mapping function is close to saturation when is large. As such, the setting of and (bottom and (bottom right). We also left) is better than is relatively small such as observe that when the effect of the adaptive gain control is not signicant. This is also observed in our experiment with images. As such, adaptive must be used gain control is useful when a relatively large to enhance the minute details and to avoid the saturation effect. C. Contrast Enhancement of the Root Signal For contrast enhancement, we use adaptive histogram equalization implemented by a Matlab function in the Image Processing Toolbox. The function, called adapthisteq, has a parameter controlling the contrast. This parameter is determined by the user through experiments to obtain the most visually pleasing result. In our simulations, we use default values for other parameters of the function. V. RESULTS AND COMPARISON All test images and Farbmans results (called combined.png for each test image) are downloaded from the Internet: www.cs.huji.ac.il/~danix/epd/. We use the canyon image called: Hunts Mesa (shown in top-left of Fig. 12) to study the proposed algorithms. We rst show the effects of the two contributing parts: contrast enhancement and detail enhancement. As shown in Fig. 9, contrast enhancement by adaptive histogram equalization does remove the haze-like effect of the original image and contrast of the cloud is also greatly enhanced.

DENG: A GENERALIZED UNSHARP MASKING ALGORITHM

1257

Fig. 9. Comparison of individual effects of contrast enhancement and detail enhancement. Images from left to right: original, only with contrast enhancement using adaptive histogram equalization, only with detail enhancement, and with both enhancements.

Fig. 10. Results of the proposed algorithm using (3 3) mask with different shapes. Top left: square shape. Top right: diagonal cross. Bottom left: horizontalvertical cross. To illustrate the halo-effect, a linear lter with a (3 3) uniform mask is used to replace the median lter in the proposed algorithm. The result is shown in the bottom right. The halo-effects are marked by red ellipse.

enhanced. However, the minute details on the rocks are not sharpened. On the other hand, only using detail enhancement does sharpen the image but does not improve the overall contrast. When we combine both operations both contrast and details are improved. Next, we study the impact of the shape of the lter mask of the median lter. In Fig. 10, we show results of the proposed 3 lter mask with shapes of square, diagalgorithm using 3 onal cross and horizontal-vertical cross . For comparison, we also show the result of replacing the median lter with a linear lter having a (3 3) uniform mask. As we can observe from these results, the use of a linear lter leads to the halo effect which appears as a bright line surrounding the relatively dark mountains (for an example, see the bottom right gure in Fig. 10). Using a median lter, the halo effect is mostly avoided, although for the square and diagonal cross mask there are still

a number of spots with very mild halo effects. However, the result from the horizontal-vertical cross mask is almost free of any halo effect. In order to completely remove the halo effect, adaptive lter mask selection could be implemented: the horizontal-vertical cross mask for strong vertical/horizontal edge, the diagonal cross mask for strong diagonal edge and the square mask for the rest of the image. However, in practical application, it may be sufcient to use a xed mask for the whole image to reduce the computational time. We have also performed experiments by replacing the logratio operations with the tangent operations and keeping the same parameter settings. We observed that there is no visually signicant difference between the results. The results shown in Fig. 11 were obtained by the following settings: 1) a median lter having a (3 3) uniform mask and three iterations, 2) xed gain , and 3) the contrast enhancement factor of 0.005.

1258

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Fig. 11. Original image is shown on the left. Results of the proposed algorithm using the log-ratio (middle) and the tangent (right) operations. We can see there is no visually signicant difference between the middle and right images.

Fig. 12. Comparison with other published results. Top left: original image. Top right: produced by the proposed algorithm. Bottom left: Farbmans result (combined.pgn). Bottom right: produced by Meylans algorithm using the authors source code.

Finally, we compare the proposed algorithm with that of two other published results. In the proposed algorithm, the following settings are used: 1) a median lter having a (3 3) horizontalvertical mask and ve iterations, 2) adaptive gain with , and 3) the contrast enhancement factor of 0.005. The source code for Meylans paper [19] on a retinex-based adaptive lter is available on: ivrgwww.ep.ch/supplementary_material. We compare our result with Farbmans result [13] because it is a multiresolution tone manipulation method which identied the problem of halo effect and proposed to use edge preserving lters to replace linear lters. The use of an IMF in our work is inspired by Farbmans work. We compare our result with

Meylans result because it is a state-of-the-art retinex-type of algorithm which requires a sophisticated rescaling process with carefully tuned parameters for every image. The use of the logratio approach in our method to systematically eliminate the rescaling process is clearly an advantage. Comparing these images, we can see that the processed images are free of haze-type effect in the original image. They are also free of halo-effects. The contrast and details are signicantly enhanced in our result and Farbmans result. This can be clearly seen in the sky part of the image where the contrast of the clouds are revealed. The contrast of Meylans result is not as good as our result and Farbmans result. We believe the reason is that the default set-

DENG: A GENERALIZED UNSHARP MASKING ALGORITHM

1259

Fig. 13. Comparison using the badger, door, and rock images. Images from top to bottom are: original (rst row), Farbmans results (second row), results of the proposed algorithm using contrast parameter of 0.001 (third row) amd using contrast parameter 0.02 (fourth row).

ting of parameters for the rescaling processes is not optimal for this image. When these parameters are tuned to this image, comparable result could be obtained. This highlights the importance of the rescaling process in the retinex type of algorithm in particular and other image enhancement algorithms using classical addition and scalar multiplication operations in general. In In Fig. 13, we present more examples showing that the performance of the proposed algorithm (third row) is similar to that of Farbmans algorithm (second row). We do not present results for Meylans algorithm because using default parameter settings, the results are not as good as those of Farbmans. In this paper, we do not attempt to quantitatively measure the performance of the proposed algorithm. This is a difcult task. Part of the difculty comes from the fact that image enhancement such as the results reported in this paper is a subject for human evaluation. For example, it is a matter of subjective assessment to compare the result of our algorithm (top right of Fig. 12) and that of Farbmans (bottom left of Fig. 12). We believe that there is no denite answer to the question of which one is better than the other. We note that objective assessment of image enhancement is a very important topic for further research. Interested readers can nd some recent works in [34], [35]. In the following, we present some results obtained in an attempt to understand the contribution of the contrast parameter to the average contrast of the processed image. There are a number

of ways to measure the average contrast [6]. We use the following measure due to its simplicity: (35) is the normalized gray scale of the th pixel and is the mean . In Fig. 14, we show the average contrast as a function of the contrast parameter for the four test images. The average contrast is calculated using the I component (in the HSI color space) of the processed color image. We can clearly see that as the parameter increases, the average contrast will increase and will then be saturated. However, when the average contrast reaches its saturation value the image appears to be overly enhanced. This is shown in the fourth row of Fig. 13. For example, the smooth and out-of-focus background of the original badger image has been greatly enhanced. This is not a desirable outcome for emphasizing the subject which is the animal. The color of the door in the door image appears unnaturally bright. In the rock image a black dot appears in the middle of the image. The black dot, which may be due to a tiny dust on the camera sensor, is hardly visible in the original image. We nd that setting the contrast parameter such that the average contrast is not saturated generally produces visually good results. For this purpose, the contrast parameter is between 0.001 and 0.005. Results are shown in the second row of Fig. 13. where

1260

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 5, MAY 2011

Fig. 14. Average contrast of the proposed image as a function of the contrast parameter.

It is interesting to note that the default setting of the contrast parameter is 0.005 for the Matlab Image processing Toolbox func. tion: The other major parameter of the proposed algorithm is which controls the sharpness of the maximum gain the image. However, the sharpness of the image is difcult to measure. For example, when we use the average absolute gradient values as a measure, we face at least two problems: (a) it does not necessarily correlate with human perception, and (b) it seems to increase with the increase of the gain . Therefore, a practical approach, taken by developers of image processing software, is to let the user to experiment with different settings of the parameter and to choose the one that produces the desired result for the user. We now comment on the computational complexity of the proposed algorithm which is implemented using Matlab. In a personal computer with Intel Core2 CPU (2.40 GHz), it takes about 0.88 s for the proposed algorithm to process a (600 800) color image. It takes about 98 s for Meylands software (using default parameter settings) to process the same image. The running time for Farbmans algorithm is not available. However, according to [13], the running time for the weighted least squared (WLS) lter in Farbmans algorithm is about 3.5 s per one mega pixel. Accordingly, an estimate of the running time for the WLS lter for a (600 800) image is about 1.7 s. Therefore, the proposed algorithm is relatively fast compared to these two algorithms. We note that the previous discussion could only provide a very rough comparison of the computational complexity. The running time of an algorithm depends upon factors such as: hardware platforms and levels of software optimization. VI. CONCLUSION AND FURTHER WORK In this paper, we use an exploratory data model as a unied framework for developing generalized unsharp masking algorithms. Using this framework, we propose a new algorithm to address three issues associated with classical unsharp masking algorithms: 1) simultaneously enhancing contrast and sharpness by means of individual treatment of the model component and the residual, 2) reducing the halo-effect by means of an edge-preserving lter, and 3) eliminating the rescaling process

by means of log-ratio and tangent operations. We present a study of the log-ratio operations and their properties. In particular, we reveal a new connection between the Bregman divergence and the generalized linear systems (including the log-ratio, MHS and the LIP model). This connection not only provides us with a novel insight into the geometrical property of generalized linear systems, but also opens a new pathway to develop such systems. We present a new system called the tangent system which is based upon a specic Bregman divergence. Experimental results, which are comparable to recently published results, show that the proposed algorithm is able to signicantly improve the contrast and sharpness of an image. In the proposed algorithm, the user can adjust the two parameters controlling the contrast and sharpness to produce the desired results. This makes the proposed algorithm practically useful. Extensions of this work can be carried out in a number of directions. In this work, we only test the IMF as a computationally inexpensive edge preserving lter. It is expected that other more advanced edge preserving lters such as bilateral lter/non-local means lter, the least squares lters and waveletbased denoising can produce similar or even better results. The proposed algorithm can be easily extended into multiresolution processing. This will allow the user to have better control of the detail signal enhancement. For contrast enhancement, we only use adaptive histogram equalization. It is expected that using advanced algorithms such as recently published retinex and dehazing algorithm can improve the quality of enhancement. We do not consider the problem of avoiding enhancement of noise. This problem can be tackled by designing an edge preserving lter which is not sensitive to noise. The idea of the cubic lter can be useful. It can also be tackled by designing a smart adaptive gain control process such that the gain for noise pixels is set to a small value. We have shown that the proposed the tangent system is an alternative to the log-ratio for systematically solving the out-of-range problem. It is an interesting task to study the application of the tangent system and compare it with other generalized systems. APPENDIX I We present the log-ratio and the tangent operations from a vector space point of view [29]. These operations are dened in Table IV. We denote as a set of real numbers in the interval . The interval is and for the log-ratio and the tangent operations, respectively. In the following, we use the and with the understanding that they general notation could be applied to both the log-ratio and the tangent cases, respectively. We can verify that the set together with the binary operation on is an albelian group such that the following axioms are satised: the group is closed under the operation, i.e., for ; the operation is associative and commutative, i.e., for and ; such that for there is an identity element ; all of such that there is an inverse element for all .

DENG: A GENERALIZED UNSHARP MASKING ALGORITHM

1261

For the log-ratio operation, the identity element is which is the new denition of zero [see (10)] and the inverse . element [see (12)] is given by the inverse image For the tangent operations, the identity element is and the . inverse element is given by the inverse image We can also verify that the previous dened albelian group and a eld , together with an operation of scalar multiplication of each element of by each element of on the left, is a and the following vector space such that for conditions are satised: ; ; ; ; such that for all . there is an element For both log-ratio and the tangent operations, it is easy to see . It is now clear that set is closed under the log-ratio that and the tangent operations. REFERENCES
[1] G. Ramponi, A cubic unsharp masking technique for contrast enhancement, Signal Process., pp. 211222, 1998. [2] S. J. Ko and Y. H. Lee, Center weighted median lters and their applications to image enhancement, IEEE Trans. Circuits Syst., vol. 38, no. 9, pp. 984993, Sep. 1991. [3] M. Fischer, J. L. Paredes, and G. R. Arce, Weighted median image sharpeners for the world wide web, IEEE Trans. Image Process., vol. 11, no. 7, pp. 717727, Jul. 2002. [4] R. Lukac, B. Smolka, and K. N. Plataniotis, Sharpening vector median lters, Signal Process., vol. 87, pp. 20852099, 2007. [5] A. Polesel, G. Ramponi, and V. Mathews, Image enhancement via adaptive unsharp masking, IEEE Trans. Image Process., vol. 9, no. 3, pp. 505510, Mar. 2000. [6] E. Peli, Contrast in complex images, J. Opt. Soc. Amer., vol. 7, no. 10, pp. 20322040, 1990. [7] S. Pizer, E. Amburn, J. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. Romeny, J. Zimmerman, and K. Zuiderveld, Adaptive histogram equalization and its variations, Comput. Vis. Graph. Image Process., vol. 39, no. 3, pp. 355368, Sep. 1987. [8] J. Stark, Adaptive image contrast enhancement using generalizations of histogram equalization, IEEE Trans. Image Process., vol. 9, no. 5, pp. 889896, May 2000. [9] E. Land and J. McCann, Lightness and retinex theory, J. Opt. Soc. Amer., vol. 61, no. 1, pp. 111, 1971. [10] B. Funt, F. Ciurea, and J. McCann, Retinex in MATLAB, J. Electron. Imag., pp. 4857, Jan. 2004. [11] M. Elad, Retinex by two bilateral lters, in Proc. Scale Space, 2005, pp. 217229. [12] J. Zhang and S. Kamata, Adaptive local contrast enhancement for the visualization of high dynamic range images, in Proc. Int. Conf. Pattern Recognit., 2008, pp. 14. [13] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, Edge-preserving decompositions for multi-scale tone and detail manipulation, ACM Trans. Graph., vol. 27, no. 3, pp. 110, Aug. 2008. [14] F. Durand and J. Dorsey, Fast bilateral ltering for the display of highdynamic-range images, in Proc. 29th Annu. Conf. Comput. Graph. Interactive Tech., 2002, pp. 257266. [15] R. Fattal, Single image dehazing, ACM Trans. Graph., vol. 27, no. 3, pp. 19, 2008.

[16] K. M. He, J. Sun, and X. O. Tang, Single image haze removal using dark channel prior, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 19561963. [17] H. Shvaytser and S. Peleg, Inversion of picture operators, Pattern Recognit. Lett., vol. 5, no. 1, pp. 4961, 1987. [18] G. Deng, L. W. Cahill, and G. R. Tobin, The study of logarithmic image processing model and its application to image enhancement, IEEE Trans. Image Process., vol. 4, no. 4, pp. 506512, Apr. 1995. [19] L. Meylan and S. Ssstrunk, High dynamic range image rendering using a retinex-based adaptive lter, IEEE Trans. Image Process., vol. 15, no. 9, pp. 28202830, Sep. 2006. [20] D. Marr, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco, CA: Freeman, 1982. [21] D. G. Myers, Digital Signal Processing Efcient Convolution and Fourier Transform Technique. Upper Saddle River, NJ: Prentice-Hall, 1990. [22] G. Deng and L. W. Cahill, Image enhancement using the log-ratio approach, in Proc. 28th Asilomar Conf. Signals, Syst. Comput.s, 1994, vol. 1, pp. 198202. [23] L. W. Cahill and G. Deng, An overview of logarithm-based image processing techniques for biomedical applications, in Proc. 13th Int. Conf. Digital Signal Process., 1997, vol. 1, pp. 9396. [24] J. Pinoli, A general comparative study of the multiplicative homomorphic log-ratio and logarithmic image processing approaches, Signal Process., vol. 58, no. 1, pp. 1145, 1997. [25] A. V. Oppenheim, R. W. Schafer, and T. G. Stockham, Jr., Nonlinear ltering of multiplied and convolved signals, Proc. IEEE, vol. 56, no. 8, pp. 12641291, Aug. 1969. [26] M. Jourlin and J.-C. Pinoli, A model for logarithmic image processing, J. Microsc., vol. 149, pp. 2135, 1988. [27] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, Edge-preserving decompositions for multi-scale tone and detail manipulation, ACM Trans. Graph., vol. 27, no. 3, pp. 67:167:10, 2008. [28] J. W. Tukey, Exploratory Data Analysis. Reading, MA: AddisonWesley, 1977. [29] J. B. Fraleigh, A First Course in Abstract Algebra. Reading, MA: Addison-Wesley, 1978. [30] A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, Clustering with bregman divergences, J. Mach. Learn. Res., vol. 6, pp. 17051749, Oct. 2005. [31] F. Nielsen and R. Nock, Sided and symmetrized bregman centroids, IEEE Trans. Inf. Theory, vol. 55, no. 6, pp. 28822904, Jun. 2009. [32] G. Deng, An entropy interpretation of the logarithmic image processing model with application to contrast enhancement, IEEE Trans. Image Process., vol. 18, no. 5, pp. 11351140, May 2009. [33] J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., Root properties and convergence rates of median lters, IEEE Trans. Acoust. Speech, Signal Process., vol. 33, no. 1, pp. 230240, Feb. 1985. [34] S. Agaian, B. Silver, and K. Panetta, Transform coefcient histogrambased image enhancement algorithms using contrast entropy, IEEE Trans. Image Process., vol. 16, no. 3, pp. 741758, Mar. 2007. [35] M. Cadk, M. Wimmer, L. Neumann, and A. Artusi, Evaluation of HDR tone mapping methods using essential perceptual attributes, Comput. Graph., vol. 32, no. 3, pp. 330349, Jun. 2008. Guang Deng is a Reader and Associate Professor in electronic engineering at La Trobe University, Victoria, Australia. His current research interests include Bayesian signal processing and generalized linear systems.