43 views

Uploaded by John Gkanatsios

remote sensing

- Lec 6BUE
- AutoCAD Commands
- NumericalMethods g
- 2012,Wilde & Deutsch Kriging and Simulation in Presence of Stationary Domains.pdf
- Cad.doc
- En.wikipedia
- Homogeneous Coordinates en Slides
- pgp 2
- Multivariable Calculus
- Numerical Analysis ( Approximation of Functions)
- Nr-10202- Computers and Information Technology
- USCIP
- yanrMLS
- Lecture 18
- plugin-teclas_atalho
- Pd Kernels
- Matrix Sol
- 3D - Important Mathematical Functions-1
- SimbioTools.pdf
- Matrix Rotation

You are on page 1of 48

Warping, Registration,

Morphing

Yao Wang

Polytechnic University, Brooklyn, NY 11201

With contribution from Zhu Liu, Onur Guleryuz, and

Partly based on

A K.

A.

K Jain,

Jain Fundamentals of Digital Image Processing

Lecture Outline

Introduction

Image deformation model

Image warping

Image registration

Image morphing

Geometric Transformation

So far, the image processing operations

we have discussed modify the color values

of pixels in a given image

With geometric transformation, we modify

the positions of pixels in a image, but keep

their colors unchanged

To create special effects

To register two images taken of the same

scene at different times

To morph one image to another

Geometric Transformation

Let (u, v) represent the image coordinate in an original

image, and (x, y) in a deformed (or warped) image. We

use a function pair to relate corresponding pixels in the

two images:

Forward mapping:

x x(u, v)

, or

y

(

u

,

v

)

Inverse mapping:

u u ( x, y )

, or u u (x)

v

(

x

,

y

)

x x(u)

g(x) the deformed image. Then they are related by:

g ( x, y ) f (u ( x, y ), v( x, y ))

g (x) f (u (x))

,

or

f

(

u

,

v

)

g

(

x

(

u

,

v

),

y

(

u

,

v

))

f (u) g ( x(u))

Geometric Transformation

Mapping Functions

y

v

Forward

x(u,v),

( ) y(u,v)

( )

(u, v)

v

u

Geometric Transformation

x

u

Inverse

u(x,y), v(x,y)

(x, y)

Translation

Translation is defined by the following

v

mapping functions:

x u tx

y v ty

and

u x tx

v y ty

In matrix notation

x u t, u x t

where

(tx,ty)

t x

x

u

x , u , t .

y

v

t y

Geometric Transformation

Scaling

Scaling is defined by

x sxu

and

y

s

v

y

u x / s x

v y / s y

v

sx=2,sy=1/2

Matrix notation

x Su, u S 1x

where

sx

S

0

0

s y

If sx < 1 and

d sy < 1,

1 thi

this represents

t a minification

i ifi ti

or shrinking, if sx >1 and sy > 1, it represents a

magnification or zoom.

zoom

Geometric Transformation

Rotation

Rotation by an angle of is defined by

x u cos v sin

i

y u sin v cos

and

i

u x cos y sin

v x sin y cos

In matrix format

v

=/4

x Ru, u R T x

where

cos

R

sin

sin

cos

(x,y)

(u,v)

Geometric Transformation

By Onur Guleyuz

Geometric Transformation

Geometric Transformation

A geometric transformation refers to a

combination of translation,

translation scaling,

scaling and

rotation, with a general form of

x RS(u t ) Au b,

u A 1 (x b) A 1x c,

with A RS, b RSt, c t.

operations will lead to different results.

Geometric Transformation

10

Affine Mapping

All possible geometric transformations are special cases

pp g

of the Affine Mapping:

x a0 a1u a2v

or x Au b

b

u

b

v

0

1

2

a a

A 1 2 ,

b1 b2

a

b 0

b0

matrix it corresponds to a

rotation matrix, and the corresponding affine mapping

reduces to a geometric mapping.

Geometric Transformation

11

Mapping straight lines to straight lines

The mapping between two arbitrary triangles

can be captured by an affine mapping

Affine mapping coefficients can be uniquely

determined from displacements of 3 vertices

v 3

y

2

1

2

x

12

from Vertex Correspondences ?

Go through in class

v 3

y

2

1

2

Geometric Transformation

13

Matlab Functions

two-dimensional affine transformation that maps each row of U

to the corresponding row of X

X. U and X are each 3

3-by-2

by 2 and

define the corners of input and output triangles. The corners

may not be collinear.

E ample

Example

------Create an affine transformation that maps the triangle with vertices

(0 0) (6

(0,0),

(6,3),

3) ((-2,5)

2 5) tto th

the ttriangle

i

l with

ith vertices

ti

((-1,-1),

1 1) (0

(0,-10),

10)

(4,4):

u = [ 0 6 -2]';

2]'

v = [ 0 3 5]';

x = [-1 0 4]';

y = [-1

[ 1 -10

10 4]'

4]';

tform = maketform('affine',[u v],[x y]);

Geometric Transformation

14

G = MAKETFORM('affine',T) builds a TFORM struct G for an Ndimensional affine transformation. T defines a forward

transformation such that TFORMFWD(U,T),

TFORMFWD(U T) where U is a 1

1-by-N

by N

vector, returns a 1-by-N vector X such that X = U * T(1:N,1:N) +

T(N+1,1:N).T has both forward and inverse transformations. N=2 for

2D image transformation

In MATLAB notation

a1

T a2

a0

Geometric

Geometric

Transformation

Transformation

b1

b2

b0

0

T

A

0 T

EL512EL512

ImageImage

Processing

Processing

15

15

according to the 2-D spatial transformation defined by TFORMB;

INTERP specifies the interpolation filter

Example 1

--------Apply a horizontal shear to an intensity image.

I = imread('cameraman.tif');

(

)

tform = maketform('affine',[1 0 0; .5 1 0; 0 0 1]);

J = imtransform(I,tform);

figure imshow(I),

figure,

imshow(I) figure

figure, imshow(J)

Show in class

Geometric Transformation

16

tform = maketform(

maketform('affine'

affine ,[1

[1 0 0; .5

5 1 0; 0 0 1]);

In MATLAB, affine transform is defined by:

[a1,b1,0;a2,b2,0;a0,b0,1]

With notation

t ti usedd in

i this

thi lecture

l t

t

note

1 0.5

A

,

0

1

0

b

0

Note in this example, first coordinate indicates horizontal position, second coordinate indicate vertic

Geometric Transformation

17

Bilinear Mapping

x a0 a1u a2v a3uv

quadrangle

More generally mapping a quadrangle to another

quadrangle.

8

8 parameters.

t

Can be completely determined from how the four

corners moved.

Geometric Transformation

18

coefficients from the vertex

correspondences?

3

v 3

y

2

2

Geometric Transformation

1 u

1

EL512 Image Processing

x

19

Non-chirping models

(Original)

(Affine)

Chirping models

(Bilinear)

(Projective)

(Relativeprojective)

(Pseudoperspective)

(Biquadratic)

Chirping: increasing perceived spatial frequency for far away objects

Converging (Keystone): parallel lines converge in distance

20

Polynomial Warping

The polynomial warping includes all deformations that

can be modeled by polynomial transformations:

x a0 a1u a2 v a3uv a4u 2 a5v 2

b5v 2

y

b

b

u

b

v

b

uv

b

u

0

1

2

3

4

Geometric Transformation

21

x(u, v), y(u, v).

F

Forward

d Mapping

M

i

For each point (u, v) in the original image, find the corresponding

position (x, y) in the deformed image by the forward mapping function,

and let g(x

g(x,y)=f(u,v).

y)=f(u v)

What if the mapped position (x,y) is not an integer sample in the desired

image?

Warping points

are often noninteger samples

Many integer

samples o

are not assigned

Values

Geometric Transformation

22

For each point (x, y) in the image to be obtained, find its

corresponding point (u, v) in the original image using the

inverse mapping function, and let g(x, y) = f(u, v).

What if the mapped point (u,v) is not an integer sample?

Interpolate from nearby integer samples!

P1

P P3

P2

P4

P will be interpolated

from P1, P2, P3, and P4

Geometric Transformation

23

Interpolation Method

Nearest neighbor:

Round (u

(u,v)

v) to the nearest integer samples

Bilinear interpolation:

find four integer samples nearest to (u,v),

apply bilinear interpolation

O

Other

h higher

hi h order

d iinterpolation

l i methods

h d

can also be used

Requiring more than 4 nearest integer

samples!

Geometric Transformation

24

Invert the forward mapping

If x Au b

Then u A 1 x b

correspondence

Geometric Transformation

25

ZI = INTERP2(X,Y,Z,XI,YI, METHOD) interpolates to find

ZI, the values of the underlying 2-D function Z at the

points in matrices XI and YI.

Matrices X and Y specify the points at which the data Z is given

given.

METHOD specifies interpolation filter

'linear'

linear - bilinear interpolation

'spline' - spline interpolation

'cubic' - bicubic interpolation as long as the data is uniformly spaced,

otherwise the same as 'spline'

Geometric Transformation

26

warping

Use inverse mapping

Step

p 1: For all p

possible p

pixels in output

p image

g ((x,y),

,y), find

corresponding points in the input image (u,v)

(X,Y)=meshgrid(1:M,1:N)

Apply inverse mapping function to find corresponding (u

(u,v),

v) for

every (x,y), store in (UI,VI)

Can use tforminv( ) function if you derived the transformation using maketform().

Or write your own code using the specified mapping

image at (UI,VI) from their values at regularly sampled

points (U,V)

(U,V)=meshgrid(1:M,1:N)

Outimg=interp2(U,V,inimg,UI,VI,linear);

Geometric Transformation

27

according to the 2-D spatial transformation defined by TFORM

INTERP specifies

ifi the

th interpolation

i t

l ti filter

filt

Example 1

--------Apply a horizontal shear to an intensity image.

I = imread(

imread('cameraman.tif');

cameraman.tif );

tform = maketform('affine',[1 0 0; .5 1 0; 0 0 1]);

J = imtransform(I,tform);

fi

figure,

i h (I) figure,

imshow(I),

fi

i h (J)

imshow(J)

Geometric Transformation

28

tform = maketform(

maketform('affine'

affine ,[1

[1 0 0; .5

5 1 0; 0 0 1]);

In MATLAB, affine transform is defined by:

[a1,b1,0;a2,b2,0;a0,b0,1]

With notation

t ti usedd in

i this

thi lecture

l t

t

note

1 0.5

A

,

0

1

0

b

0

Geometric Transformation

29

Two-Pass Mapping

The idea is to transform each row of f(u, v) first, to obtain

an intermediate image, f(x,y), and then each column of

this intermediate image is transformed to obtain the final

image g(x, y).

(u,v)

f(u,v)

xx=u

=u, y=y(u

y =y(u,v)

v)

i u

x cos sin

Ex: rotation

y sin cos v

Step 1: Fixed each row x=u, find y=usin+vcos

f( y)=f(u,v)

f(x,

) f( )

Step2: Fix each column y=y find x=ucos-vsin

y 'u sin y ' x' sin

cos

cos

u x'

Geometric Transformation

x x' cos

(u,y(u,v))

f(x,y)

x=x(x,y)=x(u,v)

y=y

(x(u,v),y(u,v))

sin

( y ' x' sin )

cos

cos

30

WAVE1

WAVE2

wave1:x(u,v)=u+20sin(2v/128);y(u,v)=v;

wave1:x(u,v)

u 20sin(2v/128);y(u,v) v;

wave2:x(u,v)=u+20sin(2u/30);y(u,v)=v.

By Onur Guleyuz

Geometric Transformation

31

WARP

SWIRL

WARP

SWIRL

y (u , v) (u x0 ) sin( ) (v y0 ) cos( ) y0 ;

r ((u x0 ) 2 (v y0 ) 2 )1/ 2 , r / 512.

By Onur Guleyuz

Geometric Transformation

32

Image Registration

Suppose we are given two images taken at

different times of the same object. To observe

the changes between these two images, we

need to make sure that they are aligned properly.

To obtain this goal, we need to find the correct

mapping function between the two. The

determination of the mapping functions between

two images is known as the registration problem.

Once the mapping function is determined

determined, the

alignment step can be accomplished using the

warping methods.

Geometric Transformation

33

Assume the mapping function is a polynomial of

order N

Step 1: Identify KN corresponding points

between two images,

g , i.e.

(ui , vi ) ( xi , yi ), i 1,2..., K .

0,,N-1 by solving

x(ui , vi ) a0 a1ui a2 vi xi ,

i 1,2,..., K

y (ui , vi ) b0 b1ui b2 vi yi ,

Geometric Transformation

34

Equations?

Convert to matrix equation:

Aa x, Ab y

where

1 u1

1 u

2

A

1 u K

v1

v2

vK

a0

b0

x1

y1

a

b

x

y

1

1

2

, b

, x , y 2

,a

a

b

x

K

yK

N 1

N 1

a A 1x, b A 1y

If K > N,

N then

th we can use a least

l

t square solution

l ti

a ( A T A) 1 A T x, b ( A T A) 1 A T y

If K < N,

N or A is singular

singular, then more corresponding feature points

must be identified.

Geometric Transformation

35

Examples

If we want to use an affine mapping to

register to images

images, we need to find 3 or

more pairs of corresponding points

If we have only 3 pairs

pairs, we can solve the

mapping parameters exactly as before

If we have more than 3 pairs, these pairs

may not all be related by an affine

mapping. We find the least squares fit by

solving an over-determined system of

equations

Geometric Transformation

36

Example

Geometric Transformation

37

TFORM=CP2TFORM(INPUT_POINTS,BASE_POINTS,TRANSFORM

TYPE)

returns

t

a TFORM structure

t t

containing

t i i a spatial

ti l transformation.

t

f

ti

INPUT_POINTS is an M-by-2 double matrix containing the X and Y

coordinates of control points in the image you want to transform.

BASE_POINTS is an M-by-2 double matrix containing the X and Y

coordinates of control points in the base image.

TRANSFORMTYPE can be 'nonreflective similarity', 'similarity',

'affine', 'projective', 'polynomial', 'piecewise linear' or 'lwm'.

Geometric Transformation

38

two images?

Which points to select in one image (image 1)?

Ideally choose interesting

interesting points:

points : corners, special

features

Can also use points over a regular grid

image (image 2)?

Put a small block around the point in image 1

Find a block in image 2 that matches the block

pattern the best

Exhaustive search within a certain range.

Geometric Transformation

39

Algorithm (EBMA)

Target frame

Rx

Anchor frame

Ry

dm

Bm

Best match

Search region

Bm

Current block

40

MATLAB Example

unregistered = imread('westconcordaerial.png');

figure, imshow(unregistered)

figure,

g

imshow('westconcordorthophoto.png')

(

p

p g)

load westconcordpoints % load some points that were already

picked

t concord = cp2tform(input_points,base_points,

t_concord

cp2tform(input points,base points,'projective');

projective );

info = imfinfo('westconcordorthophoto.png');

registered = imtransform(unregistered,t_concord,...

'XD t ' [1 info.Width],

'XData',[1

i f Width] 'YData',[1

'YD t ' [1 info.Height]);

i f H i ht])

figure, imshow(registered)

Geometric Transformation

41

Image Morphing

Image morphing has been widely used in

movies and commercials to create special visual

effects. For example, changing a beauty

gradually into a monster.

The fundamental techniques behind image

morphing is image warping.

Let the original image be f(u) and the final image

be g(x). In image warping, we create g(x) from

f( ) b

f(u)

by changing

h

i itits shape.

h

IIn iimage morphing,

hi

we use a combination of both f(u) and g(x) to

create a series of intermediate images

images.

Geometric Transformation

42

Cross

Dissolve

I(t) = (1-t)*S+t*T

Mesh

based

George Wolberg,

Wolberg Recent

Recent Advances in Image Morphing

Morphing ,

Computer Graphics Intl. '96, Pohang, Korea, June 1996.

Geometric Transformation

43

Suppose the mapping function between the two

end images is given as x

x=u+d(u).

u d(u). d(u) is the

displacement between corresponding points in

these two images.

In image morphing, we create a series of images,

starting with f(u) at k=0, and ending at g(x) at

k=K. The intermediate images are a linear

combination of the two end images:

hk (u sk d) (1 sk ) f (u) sk g (u d(u)),

)) k 0,1,..., K ,

where sk k / K .

Geometric Transformation

44

control poins

INPUT is the image that needs to be warped to bring it into the

coordinate system of the BASE image

image.

Example

cpselect('westconcordaerial.png','westconcordorthophoto.png')

Geometric Transformation

45

Demo

Show work by senior students

Geometric Transformation

46

Homework

You are given two pictures of the same scene, taken at different

times. In order to align the two pictures, you need to find a

mapping function between the two pictures based on some

common feature

f t

points.

i t Suppose

S

you were able

bl to

t extract

t tN

(N>=3) feature points in both images that correspond to the same

set of object features, with image coordinates given as (u_k,v_k)

and (x_k,y_k),k=1,2,,N. Also, suppose you want to use an affine

mapping

i tto approximate

i t th

the actual

t l unknown

k

mapping

i ffunction.

ti

How would you determine the affine mapping parameters?

Suppose you want make a panoramic picture of a wide landscape

p

y captured

p

p

pictures with a certain overlap.

p

out of two separately

Propose an algorithm for stitching up the two images to make a

panorama. (list the steps involved).

Computer assignment: Write a matlab program that implements

rotation of an image by a certain angle, and apply it on a selected

image. The rotation center should be the image center. Please

note that you should not use the imrotate() or imtransform()

function in MATLAB. You should write your own program, which

can call interp2().

interp2() .

Geometric Transformation

47

Reading

Prof. Yao Wangs Lecture Notes, Chapter

10.

10

R. Gonzalez, Digital Image Processing,

Section 5

5.11

11

George Wolberg, Digital Image Warping,

Wiley-IEEE Computer Society Press, 1990

Geometric Transformation

48

- Lec 6BUEUploaded byTarek Hassan
- AutoCAD CommandsUploaded byA
- NumericalMethods gUploaded bydwight
- 2012,Wilde & Deutsch Kriging and Simulation in Presence of Stationary Domains.pdfUploaded byruribev2000
- Cad.docUploaded byius
- En.wikipediaUploaded bypietisc
- Homogeneous Coordinates en SlidesUploaded byeisbergeisberg
- pgp 2Uploaded bySergio Alfonso Nuñez
- Multivariable CalculusUploaded byJohn
- Numerical Analysis ( Approximation of Functions)Uploaded byYogesh Rana
- Nr-10202- Computers and Information TechnologyUploaded bySrinivasa Rao G
- USCIPUploaded byGabreal
- yanrMLSUploaded bydignor.sign3941
- Lecture 18Uploaded byAnil Yvs
- plugin-teclas_atalhoUploaded bybagecosta
- Pd KernelsUploaded byJyotsna Reddy
- Matrix SolUploaded byAyie Khairie
- 3D - Important Mathematical Functions-1Uploaded byAli Pliego
- SimbioTools.pdfUploaded byingjojeda
- Matrix RotationUploaded byRodel Mansilla Opiana
- 4Uploaded byHitesh Soni
- VishUploaded byAli Gh
- homework helper-grade 3 module 1Uploaded byapi-420469579
- Numerical_Analysis_Lecture_NotesUploaded byZhenhuan Song
- Matlab TutorUploaded byali adnan
- lab sheet 1Uploaded bySashindran Viliam
- 1-s2.0-S0047272710001660-mainUploaded byHernán Felipe Herrera Velásquez
- Chapter 2 (John H. Mathews)Uploaded byAlan Wang
- Frequency responses for sampled-data systemsUploaded byrcloudt
- assessmentcommentsUploaded byapi-375234253

- inSAR processing for DEM generation.pdfUploaded byJohn Gkanatsios
- sea Ice Classification MethodUploaded byJohn Gkanatsios
- Slant to GroundUploaded byJohn Gkanatsios
- A PhD is not enoughUploaded byAndrew Hall
- Stp ManualUploaded byJohn Gkanatsios
- Integrated InSAR n LevellingUploaded byJohn Gkanatsios
- Hyperspectral Imaging Carbon IdentificationUploaded byJohn Gkanatsios
- C Band Measuremetns of Sea Ice in GreenlandUploaded byJohn Gkanatsios
- Can Tho Vietnam Resilient Cities ProgramUploaded byJohn Gkanatsios
- Dyke System ImplementationUploaded byJohn Gkanatsios
- Señales y sistemas Sung KimUploaded byBryan Cuervo Arias
- Automated Approach for Sea Ice ThicknessUploaded byJohn Gkanatsios
- MSc Thesis GuidelinesUploaded byJohn Gkanatsios
- FFTandMatLab Wanjun HuangUploaded byDiego Purizaga
- CanThoResilienceactivitiesStratergy-15-30FUploaded byJohn Gkanatsios
- inSAR processing for DEM generation.pdfUploaded byJohn Gkanatsios
- dfdgshgdUploaded byBryan Torres
- dfdgshgdUploaded byBryan Torres
- dfdgshgdUploaded byBryan Torres
- SarUploaded byJohn Gkanatsios
- Height Ambiguity_flat Earth_ ExmplesUploaded byJohn Gkanatsios
- asterUploaded byJohn Gkanatsios
- asterUploaded byJohn Gkanatsios
- lecture12_ImageWarpingUploaded byJohn Gkanatsios
- Polarimetry for Ice DiscriminationUploaded byJohn Gkanatsios
- SarUploaded byJohn Gkanatsios
- GIToolsV2 TutorialUploaded byJohn Gkanatsios
- GIToolsV2 TutorialUploaded byJohn Gkanatsios
- LeastSquares Fit in MatlabUploaded byJohn Gkanatsios

- 3c3-LinearDiffEqns_StuUploaded byMukesh Purbey
- mso203bUploaded byAnshu Kumar Gupta
- Sample chapter from Oxford A Level Mathematics for Edexcel Further Pure FP2Uploaded byOxford University Press Children and Schools
- Hadamard Codes 2Uploaded byVô Thường
- C ProgramsUploaded bygouse1210
- E-noteUploaded byPriteshShah
- Howell - 04 Starting_PointUploaded bymtichyscribd
- Griffiths Ch 3 Selected Solutions in Quantum Mechanics Prob 2,5,7,10,11,12,21,22,27Uploaded byDiane Beamer
- Anharmonic Oscillator In Quantum MechanicsUploaded byRobert Ringstad
- 35675966 EMT Electromagnetic Theory MODULE I (1)Uploaded bytearsome
- NLBeamUploaded bymustaphaxa
- c++ programs for chemical engineeringUploaded byGokul Kadam
- ch25[1]Uploaded bysmprabu24317
- 0012032v1Uploaded byBayer Mitrovic
- Chapter 11 Conic SectionsUploaded byAnonymous tANmvTTMp9
- conicsUploaded byPiyKakde
- Edexcel June 2012 C2Uploaded byReginald A Mattia
- Numerical Methods in Physics and AstrophysicsUploaded byEndless Love
- basic trig identitiesUploaded byapi-285179261
- book.pdfUploaded byMohammad
- A Radical Approach to Real Analysis Second Edition Classroom ResourceUploaded byjavedral
- Aod Assign ObjUploaded byRaju Sharma
- Digital Signal Processing- A Computer Based Approach_S. K. Mitra.pdfUploaded byGiry Makond
- Scilab ControlUploaded byFernando Montenegro Diaz
- C3 June 2014Uploaded byKareem Mohamed
- CS202_LAP2Uploaded byAncient Hermit
- Structure and Randomness in the Prime NumbersUploaded byokie-wulan-9887
- Functions Domain RangeUploaded byEymardMoisesMabutasEstalilla
- Master's TheoremUploaded bySudip Prasad
- Alg Puzzles for Practice SwbUploaded byMarcia Cristina R. Angeles-Lopez