Professional Documents
Culture Documents
CS 4620 Lecture 13
Reconstruction Filtering
Cornell CS4620 Fall 2008 Lecture 13 • 3 Cornell CS4620 Fall 2008 Lecture 13 • 4
Roots of sampling Sampling in digital audio
Cornell CS4620 Fall 2008 Lecture 13 • 5 Cornell CS4620 Fall 2008 Lecture 13 • 6
Undersampling Undersampling
• What if we “missed” things between the samples? • What if we “missed” things between the samples?
• Simple example: undersampling a sine wave • Simple example: undersampling a sine wave
unsurprising result: information is lost unsurprising result: information is lost
surprising result: indistinguishable from lower frequency surprising result: indistinguishable from lower frequency
also was always indistinguishable from higher frequencies also was always indistinguishable from higher frequencies
aliasing: signals “traveling in disguise” as other frequencies aliasing: signals “traveling in disguise” as other frequencies
Cornell CS4620 Fall 2008 Lecture 13 • 7 Cornell CS4620 Fall 2008 Lecture 13 • 7
Undersampling Undersampling
• What if we “missed” things between the samples? • What if we “missed” things between the samples?
• Simple example: undersampling a sine wave • Simple example: undersampling a sine wave
unsurprising result: information is lost unsurprising result: information is lost
surprising result: indistinguishable from lower frequency surprising result: indistinguishable from lower frequency
also was always indistinguishable from higher frequencies also was always indistinguishable from higher frequencies
aliasing: signals “traveling in disguise” as other frequencies aliasing: signals “traveling in disguise” as other frequencies
Cornell CS4620 Fall 2008 Lecture 13 • 7 Cornell CS4620 Fall 2008 Lecture 13 • 7
Undersampling Undersampling
• What if we “missed” things between the samples? • What if we “missed” things between the samples?
• Simple example: undersampling a sine wave • Simple example: undersampling a sine wave
unsurprising result: information is lost unsurprising result: information is lost
surprising result: indistinguishable from lower frequency surprising result: indistinguishable from lower frequency
also was always indistinguishable from higher frequencies also was always indistinguishable from higher frequencies
aliasing: signals “traveling in disguise” as other frequencies aliasing: signals “traveling in disguise” as other frequencies
Cornell CS4620 Fall 2008 Lecture 13 • 7 Cornell CS4620 Fall 2008 Lecture 13 • 7
Preventing aliasing Preventing aliasing
Cornell CS4620 Fall 2008 Lecture 13 • 8 Cornell CS4620 Fall 2008 Lecture 13 • 8
• Transformations on signals; e.g.: • basic idea: define a new function by averaging over a sliding window
bass/treble controls on stereo
• a simple example to start off: smoothing
blurring/sharpening operations in image editing
smoothing/noise reduction in tracking
• Key properties
linearity: filter(f + g) = filter(f) + filter(g)
shift invariance: behavior invariant to shifting the input
• delaying an audio signal
• sliding an image around
Cornell CS4620 Fall 2008 Lecture 13 • 9 Cornell CS4620 Fall 2008 Lecture 13 • 10
Convolution warm-up Convolution warm-up
• basic idea: define a new function by averaging over a sliding window • basic idea: define a new function by averaging over a sliding window
• a simple example to start off: smoothing • a simple example to start off: smoothing
Cornell CS4620 Fall 2008 Lecture 13 • 10 Cornell CS4620 Fall 2008 Lecture 13 • 10
• basic idea: define a new function by averaging over a sliding window • basic idea: define a new function by averaging over a sliding window
• a simple example to start off: smoothing • a simple example to start off: smoothing
Cornell CS4620 Fall 2008 Lecture 13 • 10 Cornell CS4620 Fall 2008 Lecture 13 • 10
Convolution warm-up Discrete convolution
each sample gets its own weight (normally zero far away)
Cornell CS4620 Fall 2008 Lecture 13 • 11 Cornell CS4620 Fall 2008 Lecture 13 • 12
• Sequence of weights a[j] is called a filter • Can express sliding average as convolution with a box filter
• Filter is nonzero over its region of support • abox = […, 0, 1, 1, 1, 1, 1, 0, …]
usually centered on zero: support radius r
a box filter
Cornell CS4620 Fall 2008 Lecture 13 • 13 Cornell CS4620 Fall 2008 Lecture 13 • 14
Example: box and step Example: box and step
Cornell CS4620 Fall 2008 Lecture 13 • 15 Cornell CS4620 Fall 2008 Lecture 13 • 15
Cornell CS4620 Fall 2008 Lecture 13 • 15 Cornell CS4620 Fall 2008 Lecture 13 • 15
Example: box and step Convolution and filtering
Cornell CS4620 Fall 2008 Lecture 13 • 15 Cornell CS4620 Fall 2008 Lecture 13 • 16
Cornell CS4620 Fall 2008 Lecture 13 • 16 Cornell CS4620 Fall 2008 Lecture 13 • 17
Discrete convolution Discrete filtering in 2D
Cornell CS4620 Fall 2008 Lecture 13 • 18 Cornell CS4620 Fall 2008 Lecture 13 • 19
And in pseudocode…
[Philip Greenspun]
original!!!!|!!!!box blur sharpened!!!!|!!!!gaussian blur
Cornell CS4620 Fall 2008 Lecture 13 • 20 Cornell CS4620 Fall 2008 Lecture 13 • 21
Optimization: separable filters
[Philip Greenspun] this is a useful property for filters because it allows factoring:
original!!!!|!!!!box blur sharpened!!!!|!!!!gaussian blur
Cornell CS4620 Fall 2008 Lecture 13 • 21 Cornell CS4620 Fall 2008 Lecture 13 • 22
Cornell CS4620 Fall 2008 Lecture 13 • 23 Cornell CS4620 Fall 2008 Lecture 13 • 23
Continuous convolution: warm-up Continuous convolution: warm-up
• Can apply sliding-window average to a continuous function just as • Can apply sliding-window average to a continuous function just as
well well
output is continuous output is continuous
integration replaces summation integration replaces summation
Cornell CS4620 Fall 2008 Lecture 13 • 24 Cornell CS4620 Fall 2008 Lecture 13 • 24
• Can apply sliding-window average to a continuous function just as • Can apply sliding-window average to a continuous function just as
well well
output is continuous output is continuous
integration replaces summation integration replaces summation
Cornell CS4620 Fall 2008 Lecture 13 • 24 Cornell CS4620 Fall 2008 Lecture 13 • 24
Continuous convolution: warm-up Continuous convolution
• Can apply sliding-window average to a continuous function just as • Sliding average expressed mathematically:
well
output is continuous
integration replaces summation note difference in normalization (only for box)
Cornell CS4620 Fall 2008 Lecture 13 • 24 Cornell CS4620 Fall 2008 Lecture 13 • 25
• Continuous–discrete convolution
Cornell CS4620 Fall 2008 Lecture 13 • 26 Cornell CS4620 Fall 2008 Lecture 13 • 27
Continuous-discrete convolution Continuous-discrete convolution
Cornell CS4620 Fall 2008 Lecture 13 • 27 Cornell CS4620 Fall 2008 Lecture 13 • 27
Cornell CS4620 Fall 2008 Lecture 13 • 27 Cornell CS4620 Fall 2008 Lecture 13 • 27
Resampling Resampling
Cornell CS4620 Fall 2008 Lecture 13 • 28 Cornell CS4620 Fall 2008 Lecture 13 • 29
Resampling Resampling
Cornell CS4620 Fall 2008 Lecture 13 • 29 Cornell CS4620 Fall 2008 Lecture 13 • 29
Resampling Resampling
Cornell CS4620 Fall 2008 Lecture 13 • 29 Cornell CS4620 Fall 2008 Lecture 13 • 29
Cornell CS4620 Fall 2008 Lecture 13 • 30 Cornell CS4620 Fall 2008 Lecture 13 • 31
Cont.–disc. convolution in 2D Separable filters for resampling
Cornell CS4620 Fall 2008 Lecture 13 • 32 Cornell CS4620 Fall 2008 Lecture 13 • 33
A gallery of filters
• Box filter
Simple and cheap
• Tent filter
[Philip Greenspun]
Linear interpolation
• Gaussian filter
Very smooth antialiasing filter
• B-spline cubic
Very smooth
• Mitchell-Netravali cubic
Good for image upsampling
Cornell CS4620 Fall 2008 Lecture 13 • 34 Cornell CS4620 Fall 2008 Lecture 13 • 35
Box filter Tent filter
Cornell CS4620 Fall 2008 Lecture 13 • 36 Cornell CS4620 Fall 2008 Lecture 13 • 37
−3(1 − |x|)3 + 3(1 − |x|)2 + 3(1 − |x|) + 1 −1 ≤ x ≤ 1,
1
fB (x) = (2 − |x|)3 1 ≤ |x| ≤ 2,
6
0 otherwise.
Cornell CS4620 Fall 2008 Lecture 13 • 38 Cornell CS4620 Fall 2008 Lecture 13 • 39
Catmull-Rom cubic Michell-Netravali cubic
−3(1 − |x|)3 + 4(1 − |x|)2 + (1 − |x|) −1 ≤ x ≤ 1, 1 2
1 fM (x) = fB (x) + fC (x)
fC (x) = (2 − |x|)3 − (2 − |x|)2 1 ≤ |x| ≤ 2, 3 3
2
−21(1 − |x|)3 + 27(1 − |x|)2 + 9(1 − |x|) + 1 −1 ≤ x ≤ 1,
0 otherwise. 1
= 7(2 − |x|)3 − 6(2 − |x|)2 1 ≤ |x| ≤ 2,
18
0 otherwise.
Cornell CS4620 Fall 2008 Lecture 13 • 40 Cornell CS4620 Fall 2008 Lecture 13 • 41
• For some filters, the reconstruction process winds up implementing a • Degree of continuity
simple algorithm
• Impulse response
• Box filter (radius 0.5): nearest neighbor sampling
• Interpolating or no
box always catches exactly one input point
it is the input point nearest the output point • Ringing, or overshoot
so output[i, j] = input[round(x(i)), round(y(j))]
x(i) computes the position of the output coordinate i on the input grid
Cornell CS4620 Fall 2008 Lecture 13 • 42 Cornell CS4620 Fall 2008 Lecture 13 • 43
Ringing, overshoot, ripples Constructing 2D filters
• Ripples
constant in,
non-const. out
ripple free when:
Cornell CS4620 Fall 2008 Lecture 13 • 44 Cornell CS4620 Fall 2008 Lecture 13 • 45
• What about near the edge? • What about near the edge?
the filter window falls off the edge of the image the filter window falls off the edge of the image
need to extrapolate need to extrapolate
methods: methods:
• clip filter (black) • clip filter (black)
• wrap around • wrap around
• copy edge • copy edge
• reflect across edge • reflect across edge
• vary filter near edge • vary filter near edge
[Philip Greenspun]
[Philip Greenspun]
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 46
Yucky details Yucky details
• What about near the edge? • What about near the edge?
the filter window falls off the edge of the image the filter window falls off the edge of the image
need to extrapolate need to extrapolate
methods: methods:
• clip filter (black) • clip filter (black)
• wrap around • wrap around
• copy edge • copy edge
• reflect across edge • reflect across edge
• vary filter near edge • vary filter near edge
[Philip Greenspun]
[Philip Greenspun]
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 46
• What about near the edge? • What about near the edge?
the filter window falls off the edge of the image the filter window falls off the edge of the image
need to extrapolate need to extrapolate
methods: methods:
• clip filter (black) • clip filter (black)
• wrap around • wrap around
• copy edge • copy edge
• reflect across edge • reflect across edge
• vary filter near edge • vary filter near edge
[Philip Greenspun]
[Philip Greenspun]
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 46
Yucky details Yucky details
• What about near the edge? • What about near the edge?
the filter window falls off the edge of the image the filter window falls off the edge of the image
need to extrapolate need to extrapolate
methods: methods:
• clip filter (black) • clip filter (black)
• wrap around • wrap around
• copy edge • copy edge
• reflect across edge • reflect across edge
• vary filter near edge • vary filter near edge
[Philip Greenspun]
[Philip Greenspun]
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 46
• What about near the edge? • What about near the edge?
the filter window falls off the edge of the image the filter window falls off the edge of the image
need to extrapolate need to extrapolate
methods: methods:
• clip filter (black) • clip filter (black)
• wrap around • wrap around
• copy edge • copy edge
• reflect across edge • reflect across edge
• vary filter near edge • vary filter near edge
[Philip Greenspun]
[Philip Greenspun]
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 46
Yucky details Yucky details
• What about near the edge? • What about near the edge?
the filter window falls off the edge of the image the filter window falls off the edge of the image
need to extrapolate need to extrapolate
methods: methods:
• clip filter (black) • clip filter (black)
• wrap around • wrap around
• copy edge • copy edge
• reflect across edge • reflect across edge
• vary filter near edge • vary filter near edge
[Philip Greenspun]
[Philip Greenspun]
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 46
Cornell CS4620 Fall 2008 Lecture 13 • 46 Cornell CS4620 Fall 2008 Lecture 13 • 47
Resampling example Resampling example
Cornell CS4620 Fall 2008 Lecture 13 • 48 Cornell CS4620 Fall 2008 Lecture 13 • 48
Cornell CS4620 Fall 2008 Lecture 13 • 48 Cornell CS4620 Fall 2008 Lecture 13 • 48
Resampling example Resampling example
Cornell CS4620 Fall 2008 Lecture 13 • 48 Cornell CS4620 Fall 2008 Lecture 13 • 48
Cornell CS4620 Fall 2008 Lecture 13 • 48 Cornell CS4620 Fall 2008 Lecture 13 • 48
Resampling example Resampling example
Cornell CS4620 Fall 2008 Lecture 13 • 48 Cornell CS4620 Fall 2008 Lecture 13 • 48
Cornell CS4620 Fall 2008 Lecture 13 • 49 Cornell CS4620 Fall 2008 Lecture 13 • 50
[Philip Greenspun]
[Philip Greenspun]
by dropping pixels gaussian filter
box reconstruction filter bicubic reconstruction filter
Types of artifacts
• Garden variety
what we saw in this natural image
fine features become jagged or sparkle
• Moiré patterns
Cornell CS4620 Fall 2008 Lecture 13 • 53 Cornell CS4620 Fall 2008 Lecture 13 • 54
Types of artifacts
• Garden variety
what we saw in this natural image
fine features become jagged or sparkle
• Moiré patterns
caused by repetitive patterns in input
produce large-scale artifacts; highly visible
• These artifacts are aliasing just like in the audio example earlier
Cornell CS4620 Fall 2008 Lecture 13 • 55 Cornell CS4620 Fall 2008 Lecture 13 • 56