ΗΥ 590-71 - presentation

HY-590.
71 Final Presentation 8/7/2011
Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

Color data modeling Segmentation by iterative energy minimization User Interaction and incomplete trimaps Border Matting Foreground estimation
Transparency
Results
HY-590.71 Final Presentation
8/7/2011

Transparency
Results
8/7/2011
Magic Wand
User specifies a point or a region to compute a region of connected pixels All the selected pixels within some adjustable tolerance of the color statistics of the specified region are considered foreground
Issue
Tolerance level is hard to be defined
8/7/2011
Intelligent Scissors
User chooses a minimum cost contour by roughly tracing the objects boundary with the mouse. As the mouse moves, the minimum cost path from the cursor position back to the last seed point is shown. If the computed path deviates from the desired one, additional user-specified seed points are necessary.
Issue
Many user interactions are often required
8/7/2011
Bayes Matting
The user specifies a trimap T = {TB, TU, TF} in which background and foreground regions TB and TF are marked, and alpha values are computed over the remaining region TU. Models color distributions are calculated probabilistically to achieve full alpha mattes.
Issues
Considerable degree of user interaction is required Results are not satisfying when
TU is too large Foreground and background color have similar color distribution
8/7/2011
Knockout 2
Plug-in for Photoshop which is driven from a user-defined trimap. Approach is like Bayes matting, and its results are sometimes similar
8/7/2011
Graph Cut
Optimization technique that can be used in a setting similar to Bayes Matting. Includes trimaps and probabilistic color models, to achieve robust segmentation, when foreground and background color distributions are not well separated.
8/7/2011
Grab Cut
Matting tool to produce continuous alpha values [0,1] Hard segmentation of the image (using iterative graph cut) Border matting
Innovations
Iterative estimation
Incomplete labeling
Iterative minimization of energy TF = 0, TU = NOT TB
8/7/2011

Transparency
Results
8/7/2011
Graph Cut
[Boykov and Jolly, 2001]
Image is expressed as an array z = (Z1,.,ZN) of gray values The segmentation of the image is expressed as an array of opacity values a = (a1, . . . ,aN) at each pixel. Generally 0 a 1, but for hard segmentation an {0,1} with 0 for foreground and 1for background The parameters describe image foreground and background grey-level distributions, i.e. a pair of histogram of gray values = {h(z;a), a = 0,1}
8/7/2011
Energy E
Energy function E is defined so that its minimum corresponds to a good segmentation Background/foreground are coherent parts. E(,,z) = U(,,z) + V(,z)
Where
U is Unary Potential V is Pairwise Potential
Segmentation (transparency values) are calculated as a global minimum

Min Cut algorithm
8/7/2011
Unary U :
evaluates the fit of opacity distribution a to data z, given
Pairwise V (Smoothness term)

C is the set of pairs of neighboring pixels dis refers to the Euclidean distance of pixels Constant > 0, = 50
8/7/2011

Transparency
Results
8/7/2011
Use of Gaussian Mixture Models (GMMs) instead of color histograms GMMs

One for the foreground, one for the background Each Gaussian Variable consists of K=5 components
N total pixels vector k={k1,,kN}
Each pixel belongs to a GMM component ki {1,,K} Pixel i ai = {0,1}
8/7/2011
The Energy for segmentation
Unary Potential
where
D(an, kn, ,zn) = log p(zn|an, kn, ) log(an, kn), p() is a Gaussian probability distribution () are mixture weighting coefficients
8/7/2011
The parameters of the model are
The smoothness term V is basically unchanged from the monochrome except that the contrast term is computed using Euclidean distance in color space
8/7/2011

Transparency
Results
8/7/2011
Iterative image segmentation in GrabCut
8/7/2011
Algorithm converges Reduce of user interaction Simpler initial interaction
Convergence of iterative minimization for the data of the figure next to (a) The energy E for the llama example converges over 12 iterations. (b) The GMM in RGB color space (side-view showing R,G) at initialization (c) after convergence K = 5 mixture components were used for both background (red) and foreground (blue).
8/7/2011

Transparency
Results
8/7/2011
Incomplete trimaps
In place of the full trimap T, the user needs only to specify the background region TB, leaving TF = 0.
Further user editing

Marking roughly with a foreground brush (white) and a background brush (red) is sufficient to obtain the desired result (bottom row).
8/7/2011
8/7/2011
Foreground brush
Background brush
8/7/2011

Transparency
Results
8/7/2011
Hard Segmentation
Pixels of an image either belong or not to background and foreground respectively (alpha values = {0,1})
Soft Segmentation
Pixels of an image may partially belong to foreground or background (alpha values = [0,1])
Full transparency is allowed in a narrow strip around the hard segmentation boundary.
The goal is to compute the map an where n TU.
8/7/2011
A dynamic programming (DP) algorithm for estimating a throughout TU. Aim is to compute (center) and (width)
8/7/2011
nergy function using DP over t:
V is a smoothing regularizer
1 = 50 2 = 1000
8/7/2011
The data term D is defined as
where N(z; , ) denotes a Gaussian probability density for z with mean and covariance .
The Gaussian parameters t (a), t (a), a = 0,1 for foreground and background are estimated as the sample mean and covariance from each of the regions Ft and Bt Regions Ft and Bt defined as Ft = St \TF and Bt = St \TB, where St is a square region of size LL pixels centered on the segmentation boundary C at t (and we take L = 41).
8/7/2011

Transparency
Results
8/7/2011
The aim is to estimate foreground pixel colors without colors bleeding in from the background of the source image First the Bayes matte is applied to obtain an estimate of foreground color fn on a pixel n TU. Then, from the neighborhood Ft(n), the pixel color that is most similar to fn is stolen to form the foreground color fn . Finally, the combined results of border matting, using both regularized alpha computation and foreground pixel stealing, are illustrated below.
8/7/2011

Transparency
Results
8/7/2011
GrabCut obtains foreground alpha mattes of good quality for moderately difficult images with a rather modest degree of user effort. Combines hard segmentation by iterative graph-cut optimization with border matting to deal with blur and mixed pixels on object boundaries.
8/7/2011
8/7/2011
8/7/2011
8/7/2011

ΗΥ 590-71 - presentation

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ΗΥ 590-71 - presentation

Uploaded by

Copyright:

Available Formats

HY-590.

71 Final Presentation 8/7/2011

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

Iterative minimization of energy TF = 0, TU = NOT TB

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

[Boykov and Jolly, 2001]

HY-590.71 Final Presentation

Segmentation (transparency values) are calculated as a global minimum

HY-590.71 Final Presentation

evaluates the fit of opacity distribution a to data z, given

Pairwise V (Smoothness term)

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

Use of Gaussian Mixture Models (GMMs) instead of color histograms GMMs

N total pixels vector k={k1,,kN}

Each pixel belongs to a GMM component ki {1,,K} Pixel i ai = {0,1}

HY-590.71 Final Presentation

The Energy for segmentation

HY-590.71 Final Presentation

The parameters of the model are

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

Iterative image segmentation in GrabCut

HY-590.71 Final Presentation

Algorithm converges Reduce of user interaction Simpler initial interaction

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

Further user editing

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

nergy function using DP over t:

HY-590.71 Final Presentation

The data term D is defined as

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

HY-590.71 Final Presentation

Approaches Image segmentation by Graph Cut The GrabCut segmentation algorithm

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

HY-590.71 Final Presentation

You might also like