Professional Documents
Culture Documents
t phi(s)
K S
Fig. 6. A choice of the function 4 (. ) that leads to edge enhancement.
V. EXPERIMENTAL RESULTS
W e choose c to be a function of the gradient of I: c ( X,
Our anisotropic diffusion, scale-space, and edge detec-
Y, t> = g(L,(x, Y, t>> as in (4). Let 4(L) * s(L) * L
denote the flux c * I,. tion ideas were tested using a simple numerical scheme
Then the 1-D version of the diffusion equation (3) be- that is described in this section.
Equation (3) can be discretized on a square lattice, with
comes
brightness values associated to the vertices, and conduc-
tion coefficients to the arcs (see Fig. 7). A 4-nearest-
4 = t 4(Zx> = 4’(4> * 41. (5) neighbors discretization of the Laplacian operator can be
used:
W e are interested in looking at the variation in time of
the slope of the edge: a/&( Z,). If c( * ) > 0 the function I;,;’ = z:,j + X[c, * v,z + cs * v,z
I( * ) is smooth, and the order of differentiation may be
inverted: + CE . VEz + Cw ’ OWZ]:,j (7)
where 0 I X I l/4 for the numerical scheme to be sta-
ble, N, S, E, W are the mnemonic subscripts for North,
= 4" - zzx+ 4' * z,,,. South, East, West, the superscript and subscripts on the
(6) square bracket are applied to all the terms it encloses, and
Suppose the edge is oriented in such a way that Z, > 0. the symbol V (not to be confused with V, which we use
At the point of inflection I,, = 0, and I,,, << 0 since the for the gradient operator) indicates nearest-neighbor dif-
point of inflection corresponds to the point with maximum ferences:
slope (see Fig. 5). Then in a neighborhood of the point
of inflection a/&( I,) has sign opposite to 4' (Z,). If O,Zi,j E Zi - 1.j - Zi,j
4’ (I,) > 0 the slope of the edge will decrease with time;
OSzi,j E zi + 1.j - ‘i,,
if, on the contrary 4' (I,) < 0 the slope will increase with
time. oEz;,j E zi,j+l - zi,j
Notice that this increase in slope cannot be caused by a
scaling of the edge, because this would violate the max- O,Z;,j G Zi,j-1 - Zi,j* (8)
imum principle. The edge becomes sharper.
The conduction coefficients are updated at every itera-
There are several possible choices of 4 ( . ), for exam-
tion as a function of the brightness gradient (4):
ple, g(Z,) = C/( 1 + (IX/K)‘+“) with (Y > 0 (see Fig.
6). Then there exists a certain threshold value related to
K, and (Y, below which 4 ( * ) is monotonically increasing, cx., = ~(ll(v~):+(,,2,,JIl~
+ cE * VEz + cw ’ &z];j
+ xz,:,(cN + cs + CE + cp+Jj
=
zM:, (12)
Fig. 8. The structure of a network realizing the implementation of aniso-
tropic diffusion described in Section V, and more in detail in [19]. The and, similarly :
charge on the capacitor at each node of the network represents the bright-
ness of the image at a pixel. Linear resistors produce isotropic linear z;,j’l 1 I&, ( 1 - X( CA’ + CS + CE + cW>f,j)
diffusion. Resistors with an I-V characteristic as in Fig. 6 produce an-
isotropic diffusion.
+ XZrnfj(CN + cs + CE + cw,; j = zm;,,. (13)
The value of the gradient can be computed on different The numerical scheme used to obtain the pictures in this
neighborhood structures achieving different compromises paper is the one given by equations (7), (8), and (lo),
between accuracy and locality. The simplest choice con- using the original image as the initial condition, and adi-
sists in approximating the norm of the gradient at each arc abatic boundary conditions, i.e., setting the conduction
location with the absolute value of its projection along the coefficient to zero at the boundaries of the image. A con-
direction of the arc: stant value for the conduction coefficient c (i.e., g ( * ) =
ciV,., = g( ( ONzi,j-() 1) leads to Gaussian blurring (see Fig. 3).
Different functions were used for g ( . ) giving percep-
C&J = g( 1 O.Sz:,j() tually similar results. The images in this paper were ob-
tained using
G,., = g( ( vEzi,j I) g(vz) = ,c-tllwm
'iVi,j = g( ( oWzi,jj)* (10)
(Fig. 9), and
This scheme is not the exact discretization of (3), but
of similar diffusion equation in which the conduction ten-
sor is diagonal with entries g ( ] Z, ( ) and g ( ] ZY( ) instead
of g ( ]I VI]] ) and g ( )I VI (I). This discretization scheme
preserves the property of the continuous equation (3) that
the total amount of brightness in the image is preserved. (Figs. 12-14). The scale-spaces generated by these two
Additionally the “flux” of brightness through each arc of functions are different: the first privileges high-contrast
the lattice only depends on the values of the brightness at edges over low-contrast ones, the second privileges wide
the two nodes defining it, which makes the scheme a nat- regions over smaller ones.
ural choice for analog VLSI implementations [ 191. See The constant K was fixed either by hand at some fixed
Fig. 8. Less crude approximations of the gradient yielded value (see Figs. 9-14), or using the “noise estimator”
perceptually similar results at the price of increased com- described by Canny [4]: a histogram of the absolute val-
putational complexity. ues of the gradient throughout the image was computed,
634 IEEE TRANSACTIONS O N PATTERN ANALYSIS AND .MACHINE INTELLIGENCE. VOL 12. N O 7. JULY 1990
Fig. 10. Edges detected using (a) anisotropic diffusion and (b) Gaussian
smoothing (Canny detector).
Fig. I I Scale space obtained with anisotropic diffusion. The dltfusion was
performed in 2-D on the Canaletto image of wjhich one line (the hori-
zontal line number 400 out of 480-just above the gondola) IS shown.
Fig. 9. Effect of anisotropic ditfusion (b) on the Canaletto image (a) [3]. Notice that the edges remain sharp until their disappearance.
Fig. 1.5. Scale-space using linear convolution. The edges are distorted and Fig. 16. Edges detected by thresholding the gradient in Fig. 14. Linking
the junctions disappear. Images generated using the Canny detector and is not necessary. Thinning is only for the finer scales. Compare to Fig.
smoothing Gaussian kernels of variance (top left to bottom right) l/2, 17 where thinning and linking have been used.
1, 2, 4, 8, 16 pixels. Compare to Fig. 17 where anisotropic diffusion
preserves edge junctions, shape, and position.
where the function U( * ) has the form of (14) [6], [14]. Fig. 17. Edges detected in Fig. 14 using a thinning and linking stage [4].
Because the exponential function is monotonic the max-
ima of the probability distribution and the minima of the
energy function coincide, and we can limit our attention approximation of anisotropic diffusion that we suggest in
to the schemes based on minimizing the energy. Section V may be seen as a gradient descent of the a priori
part of the energy function
The energy function (14) is the sum of two terms: the
a priori term (the sum of the “clique” functions I/ con-
taining the a priori knowledge about the image space-
(16)
see any one of [6], [16], [2] for a complete discussion),
and a term depending on the data available (the sum of The steepest descent strategy for finding minima of a
the functions Wi). V( * , * ) is typically an even function function consists of starting from some initial state, and
depending only on the value of the difference of its ar- then changing iteratively the state following the opposite
guments (with abuse of notation V(zi, zj) = V(zj - zj) ). of the gradient vector. The gradient of the energy func-
It has minimum at zero and it is monotonic on the positive tion, which may be computed from (16) differentiating
and negative semilines assigning higher energy ( e lower with respect to the zj, is the vector of components
probability) to the pairs i, j of lattice nodes whose bright-
ness difference I( zi - zi I( is bigger. We will show that the 07)
PERONA AND MALIK: SCALE-SPACE AND EDGE DETECTION 637
Cc) (4
of images at different scales since shape and position are
preserved at every single scale.
In images in which the brightness gradient generated by
Fig. 18. (a) The local energy function proposed by [6], [2], [14] typically the noise is greater than that of the edges, and the level
is equal to the square of the nearest-neighbor brightness difference, and of the noise varies significantly across the image the
saturates at some threshold value. (b) The first derivative of the energy
function (a). (c), (d) The anisotropic diffusion conduction coefficient and scheme that we have described proves insufficient to ob-
flux function as a function of the brightness gradient magnitude, propor- tain a correct multiscale segmentation. In this case a global
tional to the nearest neighbor brightness difference in the discrete case.
(b) and (d) have the same role.
noise estimate does not provide an accurate local esti-
mate, and the local value of the gradient provides too par-
tial a piece of information for distinguishing noise-related
therefore the gradient descent algorithm is
and edge-related gradients. Moreover, the abscissa K of
a the peak of the flux function 4 ( * ) has to be set according
zZi= -A . je$i, v(Zi - Zj) (18) to the typical contrast value, if this changes considerably
through the image the value of K has to be set locally. To
where A is some “speed” factor. tackle these difficulties anisotropic diffusion should be
Suppose that V( . ) is differentiable in the origin and implemented using local contrast and noise estimates.
define+(*) = -p.SinceI/(*)iseven,+(*)isanodd
function and 4(O) = 0. Then we may write 4(s) = s * APPENDIX
c(s) for some function c( * ) even and positive. Substi- PROOF OF THE MAXIMUM PRINCIPLE
tuting into (18) we obtain
Call A an open bounded set of W ” (in our case A is the
a plane of the image, a rectangle of a*), and T = (a, b)
z Z i = A * jE$j, C(zj - zi) * (2, - Z-i) (19) an interval of R. Let D be the open cylinder of R!“”
formed by the product D = A x T = { (x, t) : x E A, t E
which is exactly the anisotropic diffusion algorithm de- T}. Call aD the boundary of D, D its closure, and &D,
fined by (7), (8), and (10) if the neighborhood structure a,D, and aBD the top, side, and bottom portions of aD:
is given by natural nearest-neighbor cliques of a square
lattice. The flux functions obtained by differentiating the aTD = {(x, t):x E A, t = cz]
local energy functions V( * ) of [6], [15], [2] are similar
to the shape of flux function that the analysis in Section a,D = ((x, t):x l aA, t E T}
IV-B suggests. See Fig. 18.
aBD = {(x, t):x E A, t = b}
To summarize: anisotropic diffusion may be seen as a
gradient descent on some energy function. The data (the and, for convenience, call a,D the side-bottom bound-
original image) are used as the initial condition. In the ary:
energy-based methods [6], [ 161, [2] the closedness of the
solution to the data is imposed by a term in the energy assD = a,o u aBD.
function. This makes the energy function nonconvex and
The following theorems hold.
more complicated optimization algorithms than gradient
Theorem: Consider a functionf : W ”+ ’ + R that is con-
descent are necessary. Most of the algorithms that have
tinuous on 0, and twice differentiable on D U a,D. If f
been proposed (simulated annealing for example) appear
satisfies the differential inequality
too slow for vision applications. Perhaps the only excep-
tion is the GNC algorithm proposed by Blake and Zisser- C(x, t)fr - c(x, t) Af - vc . Vf I 0 (20)
man [2] which does not guarantee to find the global op-
timum for generic images, but appears to be a good on D, with C : R” + ’ + R+ continuous on 0, and differ-
compromise between speed and accuracy. entiable on D U aTD, then it obeys the maximum prin-
ciple, i.e., the maximum off in b is reached on the bot-
VII. CONCLUSION tom-side boundary assD of D:
W e have introduced a tool, anisotropic diffusion, that
we believe will prove useful in many tasks of early vision. mFf = maxf.
asBD
638 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL. 12, NO. 7. JULY 1990
Corollary: Consider a function f satisfying the hy- Notice that the maximum principle also guarantees that
potheses of the previous theorem, and such that f is twice there are no local maxima off in D U &D. The same
differentiable on a,D, and V, f = 0 (where V, indicates technique used in the proof restricting D to be a cylinder
the gradient operator along the x direction). Then contained in the neighborhood where the local maximum
is a strict maximum may be used to see that the existence
m?f = maxf. of one at p E D U &D would violate the differential in-
alSo
equality.
The following proof is adapted from John [lo]. The corollary may be proven along the same lines: since
Proof First consider f satisfying the stricter condi- f is, by hypothesis, differentiable on 8, D one can use (2 1))
tion and (22) for any p E a,D, with u in an appropriate hemi-
sphere so that p + cu E D.
C(x, t)ft - c(x, t)Af - Vc * Vf < 0. (21) If a function f satisfies the differential equation
By hypothesis f is continuous on D, a compact set, hence C(x, t)ff - c(x, t)Af - vc * O f = 0 (23)
it has a maximum in it. Call p = ( y, T) this maximum. with the hypotheses already stated on the functions C( * )
Suppose that p E D. Since f is twice continuously dif- and c ( * ), the arguments above can be run for f and h =
ferentiable in D we can write the first three terms of the
-f proving that both a maximum and minimum principle
Taylor expansion off about p: have to be satisfied.
f(p + EU) =f(p) + EVf% + E2VTXfU The diffusion equation (3) is a special case of (23) (set
C(x, t) = 1, and f = I), hence the scale-space brightness
+ WE31 5 f(P) P-4 function Z(x, y, t) obeys the maximum principle provided
that the conduction coefficient c never takes negative value
where v E R”+‘, E E some neighborhood of zero, and Xf (in fact the condition that c does not take negative value
indicates the n + 1 X n + 1 Hessian matrix off. For the where f has a maximum is sufficient) and is differentiable.
sake of compactness, unlike in the rest of the paper, Vf If adiabatic (V, f = 0) boundary conditions are used then
in (22) indicates the gradient off with respect to the space the hypotheses of the corollary are satisfied too, and the
coordinates and the time coordinate. Since p is a point maxima may only belong to the initial condition.
where f has a maximum, the gradient Vf in the first order Solutions f of (3) have an additional property if the con-
term of the expansion (22) is equal to zero therefore the duction coefficient is constant along the space axes: c =
second term cannot be positive, Vu E Rn ’ ’ : o TXfu I c(t). In this case, all spatial derivatives off are solutions
0; the Hessian matrix is therefore negative semidefinite, of (3), and therefore satisfy the hypotheses of the maxi-
which implies that the entries on its diagonal are either mum principle. So the causality criterion is satisfied by
equal to zero or negative. The Laplacian is a sum of en- all such functions: the components of the gradient, the
tries on the diagonal and therefore Af I 0. This would Laplacian, etc. It is important to notice that this is not
mean that at p true in general for solutions of (3) when the conduction
coefficient varies in scale and space. W e show in Section
C(x, t)f, - c(p)Af- Vc * Af2 0 IV-B that in fact anisotropic diffusion can increase the
contrast (i.e., the magnitude of the gradient) of edges in
contradicting the hypothesis.
the image.
Similarly, if p E a,D the first derivative with respect to
t off could only be positive or equal to zero, while the ACKNOWLEDGMENT
first derivatives with respect to the x variables would have W e are grateful to L. Semenzato, A. Casotto, P. Kube,
to be equal to zero, and the second derivatives with re-
and B. Baringer who gave very friendly assistance in set-
spect to the x variables could only be equal to zero or ting up the software simulations, and taking the pictures.
negative, giving the same inequality at p as above. This
R. Brodersen kindly provided the photographic equip-
would again contradict the hypothesis. So, if f satisfies ment. B. Hummel pointed to us the result by Niremberg.
(21), then it obeys the maximum principle.
If f satisfies the weak inequality (20) the function g de- REFERENCES
fined as g = f - X (t - a) satisfies the strict inequality [I] J. Babaud, A. Witkin, M. Baudin, and R. Duda, “Uniqueness of the
(21), and therefore the maximum principle, for any h > gaussian kernel for scale-space filtering,” IEEE Trans. Pattern Anal.
0. Observe thatf = g + A(t - a) I g + X(b - a) on Machine Intell., vol. PAMI-8, Jan. 1986.
[2] A. Blake and A. Zisserman, Visual Reconstruction. Cambridge,
0, and because of this MA:‘MIT Press, 1987.
[3] Canaletto, “View in Venice,” National Gallery of Art, Washington,
m?f 5 rn2 (g + h(b - u)) DC, circa 1740.
[4] J. Canny, “A computational approach to edge detection,” IEEE
Trans. Pattern Anal. Machine Intell., vol. PAMI-8, pp. 679-698,
=;$+X(h-u))=&f+h(h-a)). 1986.
[S] J. Clark, “Singularity theory and phantom edges in scale space,”
IEEE Trans. Pattern Anal. Machine Intell., vol. 10, no. 5, pp. 720-
Letting X + 0 we obtain the thesis. 0 727. 1988.
PERONA AND MALIK, SCALE-SPACE AND EDGE DETECTlOi% 639
161S. Geman and D. Geman. “Stochastic relaxation. Gibbs distribu- [?I] A. Witkin. “Scale-space tiltering.” in IIII. Jor~/ Cmj. ArtificiuI In-
tions, and the Bayesian restoration of images.” IEEE Trtrr~s. Purrrrrr re//igetlc,c, Karlaruhe. West Germany. 1983. pp. 1019-1021,
Anuf. h4ochine Inrrll.. vol. PAMI-6. pp. 72 I-741. Nov. 1984. (221 A. Yuille and T. Poggio. “Scaling theorems for zero crossings.”
171 A. Hummel. “Representations based on zero-crossings in scale- IEEE Trrrrrs. Ptrrtem Awl. Mnchirie Inrrll.. vol. PAMI-8. Jan. 1986.
space, ’’ in Proc. IEEE Computer Visit and Pattrm Rrco,qnirim
Conf.. June 1986. pp. 204-209: reproduced in: Rw~dings in Cornp~rrer
Ksim: I.\suP.s. Problems. Principles crud Parc~drgm. M, Fischler and
0. Firschein. Eds. Los Altos, CA: Morgan Kaufmann, 1987. Pietro Perona was horn in Padua. Italy. on Sep-
181-, “The scale-space formulation of pyramid data structures.” in tember 3. 1961. He received the Doctor degree in
Parallrl Computer Vi&ion, L. Uhr. Ed. New York: Academic. 1987. electrical engineering cum laude from the Univer-
pp. 187-223. sity of Padua m 1985 wtth a thesis on dynamical
1% A. Hummel. B. Kimia, and S. Zucker. “Deblurring Gaussian blur.“ systems theory.
Comnpur. Vision, Graphics. Image Pruceshing. vol. 38, pp. 66-80. He received the Ph.D. degree from the De-
1987. partment of Electrical Engineering and Computer
[lOI F. John. Pnrrial Diflerentiol Equafions. New York: Springer-Ver- Science of the University of California at Berke-
lag, 1982. ley in 1990. His research interests are in compu-
IllI J. Koenderink. “The structure of images.” Biol. C&m.. vol. 50. tational and biological vision.
pp. 363-370. 1984.
1121J. Malik, “Interpreting line drawings of curved object\.” 1,n. J.
Compuf. Vision. vol. I, no. 1, pp. 73-103. 1987.
r131 D. Marr, Vision. San Francisco, CA: Freeman, 1982.
1141 J. Marroquin, “Probabilistic solution of inverse problems.” Ph.D. Jitendra Malik (A-88) was born in Mathura, In-
dissertation, Massachusetts Inst. Technol., 1985. dia, on October 1 I. 1960. He received the B.Tech
I151 -, “Probabilistic solution of inverse problems,” Artificial Intell. degree from Indian Institute of Technology. Kan-
Lab., Massachusetts Inst. Technol., Tech. Rep. AI-TR 860, 1985. pm. in 1980 where he was awarded the gold medal
1161 D. Mumford and J. Shah. “Optimal approximation of piecewise for the best graduating student in electrical engi-
smooth functions and associated variational problems.” Commun. neering. He received the Ph.D. degree in com-
Pure Appl. Math., vol. 42, pp. 577-685, 1989. puter science from Stanford University, Stanford.
]I71 L. Nirenbarg. “A strong maximum prmciple for parabolic equa- CA. in 1986.
tions,” Cotnmur~. Purr Appl. Mtrth., vol. VI. pp. 1677177. 1953. Since January 1986. he has been an Assistant
[I81 P. Perona and J. Malik, “Scale space and edge detection using an- Professor in the Computer Science Divtsion. De-
isotropic diffusion.” in Proc. IEEE Compur. Sot. Workshop Com- partment of EECS, University of California at
puter Vision. Miami. FL. 1987, pp. 16-27. Berkeley. Since October 1988 he has also been a member of the group in
1191 - “A network for edge detection and scale space,” in Proc. IEEE Physiological Optics at UC Berkeley. His research interests are in machine
Int. iymp. Circuits and Svsrems, Helsinki. June 1988, pp. 2565-2568. vision and computational modeling of early human vision. These include
1201 A. Rosenfeld and M. Thurston. “Edge and curve detection for vtsual work on edge detection. texture segmentation, line drawing interpretation,
scene analysts,” fEEE Trans. Compur., vol. C-20, pp. 562-569. May and 3-D object recognition.
1971. Dr. Malik received a Presidential Young Investigator Award in 1989.