Computer Vision For Robotic Navigation and Augmented Reality Applications

Introduction
Computer vision by phase encoding methods

Developed phase-based measurement system
Conclusions
References
Computer vision for robotic navigation and augmented

reality applications
Dr. Rigoberto Juarez Salazar
1
Instituto Tecnologico
Superior de Zacapoaxtla
de Ingeniera Informatica
Division
Congreso Multidisciplinario Ciencia y Tecnologa para el

Desarrollo Sustentable
October 15, 2014
1 / 40
Dr. Rigoberto Juarez

Salazar
rjuarezsalazar@gmail.com
Computer vision in robotic navigation and augmented reality applications
Introduction
Conclusions
References
Content
2 / 40
Introduction

de patrones de franjas
Normalizacion
Fourier fringe-normalized analysis
Inhomogeneous phase-shifting
Desenvolvimiento de fase
Conclusions
References

Salazar
Introduction
Conclusions
References
Introduction
Figura : Metrology is a very important task in most of human task. It increases our senses,
particularly, vision sense. Vision is about discovering from images what is present in the scene and
where it is. It is our most powerful sense.
3 / 40

Salazar
Introduction
Conclusions
References
What is computer vision?

In computer vision a camera (or several cameras) is linked to a computer. The
computer automatically interprets images of a real scene to obtain useful information
(e.g., 3D reconstruction) and then acts on that information (e.g. for navigation or
manipulation).
4 / 40

Salazar
Introduction
Conclusions
References
What is computer vision?

In computer vision a camera (or several cameras) is linked to a computer. The
computer automatically interprets images of a real scene to obtain useful information
(e.g., 3D reconstruction) and then acts on that information (e.g. for navigation or
manipulation).
It is not:
Image processing. Image enhancement, image restoration, image compression. Take
an image and process it to produce a new image which is, in some
way, more desirable.
Pattern recognition. Classifies patterns into one of a finite set of prototypes.
4 / 40

Salazar
Introduction
Conclusions
References
Applications of computer vision

Automation of industrial processes: object recognition, visual inspection, robot
hand-eye coordination, robot navigation.
Space and military: remote sensing, surveillance, target detection and tracking
(traffic, aircraft, etc.), UAV localization.
Human-computer interaction: Face detection and recognition. Mobile-phone
applications, augmented reality.
3D modeling.
5 / 40

Salazar
Introduction
Conclusions
References
A proposed computer vision-based robotic navigation
6 / 40

Salazar
Introduction
Conclusions
References
Computer vision-based robotic navigation
7 / 40

Salazar
Introduction
Conclusions
References
Approaches in computer vision
There are two main approaches to obtain three-dimensional information of a scene.

Amplitude or intensity modulation
The information is obtained from variations of intensity of the object, for example,
shadows, illumination conditions, surface color, etc. It is simple to implement and easy
the experimental setup. However, it is very sensitive to noise sources.
Phase modulation
The information of interest is encoded into a phase distribution, for example, by phase
shifting, or frequency carrier. The experimental setup is lightly more complex than
amplitude modulation approach. The data processing is more complex. The main
advantages is that phase modulation have a very high robustness to noise sources. It
allows to reach a high resolution.
8 / 40

Salazar
Introduction
Conclusions
References
Phase-based measurement systems
Experimental setup
Inputs
Physical Outputs
parameter
Sensing
device
Fringe-
pa+erns
Fringe-
pa9erns
Pre-
processing
Wrapped
phase
extrac7on
Intermediate
processing
Fringe analysis
S1
S2
S3
S4
S5
Mesurand
Mesurand
Post-
processing
Phase-
unwrapping
Figura : General scheme of a phase-based measurement system.
The experimental setup generates fringe-patterns where the encoded phase is

associated with the physical parameter of interest.
The fringe analysis block extracts the physical information from the given
fringe-patterns.
9 / 40

Salazar
Introduction
Conclusions
References

By the good properties of using phase modulation, we are focused on reconstruction of
3D objects by phase-shifting as well as Fourier fringe analysis to robotic navigation and
augmented reality applications.
The phase encoding approach consist on the following two procedures.
1. Wrapped phase extraction
One or more fringe-patterns are processed to obtain the encoded phase in a wrapped
format. For this, the two main methods are
Phase-shifting method.
Advantages. Full resolution is reached, algorithms are simple and very robust.
Disadvantages. More than one image is required. This make it no much available
for real-time applications.
Fourier fringe analysis.
Advantages Only one image is sufficient to work, it is appropriate for real-time
applications.
Disadvantages. Reduced resolution, algorithms are complex. Filtering is difficult.
10 / 40

Salazar
Introduction
Conclusions
References

By the good properties of using phase modulation, we are focused on reconstruction of
3D objects by phase-shifting as well as Fourier fringe analysis to robotic navigation and
augmented reality applications.
The phase encoding approach consist on the following two procedures.
2. Phase unwrapping
The synthetic phase jumps induced by the phase extraction must be removed before
that any useful information can be reached. For this, a phase unwrapping process is
applied in the pipeline.
11 / 40

Salazar
Introduction
Conclusions
References
Phase demodulation process
(a)
Experimental setup
Inputs
Physical Outputs
parameter
Sensing
device
(b)
Fringe-
pa+erns
Fringe-
pa9erns
Pre-
processing
(c)
Wrapped
phase
extrac7on
Intermediate
processing
Fringe analysis
S1
S2
S3
S4
S5
Mesurand
Mesurand
Post-
processing
Phase-
unwrapping
Figura : General scheme of a phase-based measurement system.

12 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Fringe-pattern normalization
...
An efficient and automatic processing algorithm.
Ik
13 / 40
LS
ak

Salazar
LS
bk2 2
2u
bk
uv
Ik
sat
Ik
Introduction
Conclusions
References
Normalizacion
Let be a set of K fringe-patterns of the form
Ik (p) = ak (p) + bk (p) cos k (p), k = 0, K 1.
(1)
Then, the background and modulation lights can be recovered by1

ak = Aa Aa Ik ,
(2)
bk2 = 2Ab Ab (Ik ak )2 .

Finally, the normalization is carried out by the function sat() as

Ik = sat Ik ak = cos k (p) bk (p) 6= 0.
bk
Ik
LS
ak
u2
LS
bk2 2
2u
bk
uv
Ik
(3)
sat
Ik
Figura : Block diagram of the proposed fringe-pattern normalization method.

1
Juarez-Salazar, R., Robledo-Sanchez, C., Meneses-Fabian, C., Guerrero-Sanchez, F., and Aguilar, L. A.
Generalized phase-shifting interferometry by parameter estimation with the least squares method. Optics and
Lasers in Engineering, 51(5):626 632 (2013).
14 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Example for the two-dimensional case

Video...
15 / 40

Salazar
Introduction
Conclusions
References
(a)
Espectrum
FT
(b)
Filtered spectrum Remaning

zero order
FT
IFT
IFT
Filtered spectrum
Espectrum
Normalizacion
Wrapped phase extraction

...
Advanced algorithms for the extraction of wrapped phase by

using spatial and temporal carriers.
16 / 40

Salazar
(i)
N
LS
LS
cA
cS
sw cos
(ii)
cos1
LS
I
Introduction
Conclusions
References
Normalizacion
17 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

The classical approach considers fringe-patterns of the form
I(p) = a(p) + b(p) cos[(p) + 2f p].
(4)
The respective spectrum is

I() = A() + C( f ) + C ( + f ).
(5)
If a fringe-pattern normalization is previously applied, both background and modulation

lights are removed and the respective spectrum is suppressed. Thus, the filtering
procedure is less critical
(a)
Espectrum
FT
(b)
Filtered spectrum Remaning

zero order
FT
IFT
Filtered spectrum
Espectrum
IFT
Figura : (Top) Analysis scheme with the Fourier method. (Down) Proposed scheme.2
2
Casco-Vasquez, J. F., Juarez-Salazar, R., Robledo-Sanchez, C., Rodriguez-Zurita, G., Sanchez, F. G.,
Arevalo
Aguilar, L. M., and Meneses-Fabian, C. Fourier normalized-fringe analysis by zero-order spectrum
suppression using a parameter estimation approach. Optical Engineering, 52(7):074109074109 (2013).
18 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Numerical example
Casco-Vasquez, J. F., Juarez-Salazar, et. al., Fourier normalized-fringe analysis by zero-order spectrum
19 / 40
suppression
using
estimation
approach. Optical Engineering,
(2013). reality applications
a parameter
Dr. Rigoberto
Juarez
Salazar
Computer vision52(7):074109074109
in robotic navigation and augmented
Introduction
Conclusions
References
Normalizacion
Numerical and experimental examples

Wrapped phase extraction by using the proposed Fourier fringe-normalized analysis.
3
Arevalo
suppression using a parameter estimation approach. Optical Engineering, 52(7):074109074109 (2013).
20 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Improved efficiency and robustness

Video...
21 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Inhomogeneous generalized
phase-shifting algorithm
22 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Inhomogeneous generalized phase-shifting algorithm

For phase-shifting, it are considered fringe-patterns of the form
Ik (p) = ak (p) + bk (p) cos[(p) + k (p)],
(6)
where the phase shift k (p) is, in general, inhomogeneous and unknown.
Video...
Phase shift
(d)
Phase shift
(c)
Phase shift
(b)
Phase shift
(a)
Figura : Generalized phase shift. Homogeneous: (a) linear, and (b) nonlinear in k. Inhomogeneous:
(c) nonlinear in k but linear in p, and (d) nonlinear in both k and p.
23 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Algoritmo de corrimiento de fase generalizado inhomogeneo

del algoritmo
Descripcion
1
2
de los patrones de franjas.

Normalizacion
De las cantidades Ak = Ik 1 + Ik y Sk = Ik1 Ik de dos patrones adyacentes,
el coseno del paso de fase entre esos patrones se obtiene mediante
cAk = Ac Ac (A2k 1),
(7a)
Ac Ac (1
(7b)
Sk2 ).
cSk =
() [ver Fig. 8(b)] se usa para seleccionar entre cAk y cSk como
La funcion
cos k = cSk (ck ) + cAk (ck ),
(8)
donde el paso de fase k se obtiene calculando el coseno inverso de (8).

(b)
(a)
3
2
1
3
0
0.6
( c k)
0.4
( c k)
0
1
2
0
/4
3/4
/2
Phase step k
0.2
Equivalent data
cos( )
0.8
(c)
A k2 1
2
1 Sk
cos( k)
0.5
0
ck
0.5
3
0
/4
3/4
/2
Phase step k
de cos k . Para k [0, /2] es

Figura : (a) Los datos A2k 1 y 1 Sk2 para estimacion
conveniente elegir 1 Sk2 porque la amplitud del ruido es baja, y viceversa para k [/2, ].
24 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

4
El corrimiento de fase es recuperado mediante la sumatoria:

k = 0 +
k
X
` ,
k = 1, 2, , K 1.
(9)
`=1
5
finalmente, la fase envuelta es obtenida mediante

w = arctan(/),
(10)
donde = sin y = cos son obtenidos mediante:

T
= A I0 I1 IK 1 ,
A = cos k
(i)
N
LS
LS
cA
cS
sw cos
(ii)
cos1

sin k .
LS
(11)
Figura : Diagrama de bloques del algoritmo de corrimiento de fase generalizado inhomogeneo

propuesto.4
4
R. Juarez-Salazar, et. al., Generalized phase-shifting algorithm for inhomogeneous phase shift and
spatio-temporal fringe visibility variation. Opt. Express, 22(4):47384750 (2014).
25 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
rapida,
Extraccion
automatica
y robusta de fase envuelta

Video...
Corrimiento de fase generalizado
26 / 40

Salazar
Visibilidad espacio-temporal
Solo dos patrones
Introduction
Conclusions
References
Normalizacion
rapida,
Extraccion
automatica
y robusta de fase envuelta

Video...
Corrimiento de fase no-lineal inhomogeneo

patrones
27 / 40

Salazar
Visibilidad espacio-temporal
Solo dos
Introduction
Conclusions
References
Normalizacion
...
Desenvolvimiento de fase rapido

y exacto
28 / 40
1
2

Salazar
round
LS
k round k
2 k +
Introduction
Conclusions
References
Normalizacion
Un mapa de fase envuelto (p) puede ser descrito como
(p) = (p) 2k(p),
k() Z,
(12)
de valor entero)
donde (p) es el mapa de fase continuo y 2k(p) (con k una funcion
son los saltos de fase.
Enfoque espacial de desenvolvimiento de fase
de
En general, los algoritmos de desenvolvimiento de fase se basan en la ecuacion
Itoh:
(p) = W[(p)].
(13)
(13) se puede resolver para mediante los metodos
La ecuacion
de seguimiento de
trayectoria o de norma mnima:
Z

(p) = 0 (p) +
W{(p)} dp,
o
mn W[(p)] (p) .
(14)
29 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Usualmente, el desenvolvimiento de fase se realiza removiendo los saltos de fase y
reconstruyendo el mapa de fase continuo.
(p) = (p) + 2k(p),
(15)
Alternativamente, es posible reconstruir los saltos de fase y entonces agregar la

de fase envuelta para obtener un mapa de fase continuo.
informacion
(p) = (p) + 2k(p).
(16)
Algunas ventajas del metodo

alternativo
() <,
k() D =
(
{1, 0, 1}
{, 0, }
Caso discreto,
Caso continuo.
(17)
simples y robustos debido a que es directo y estable

Esto sugiere algoritmos mas
aproximar a un elemento de tres posibles.
30 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Desenvolvimiento de fase por mnimos cuadrados y redondeo
Se puede mostrar que el gradiente k(p) satisface la ecuacion

1
k(p) = round
,
2
(18)
de redondeo. Entonces, la funcion

k se puede
donde round() indica la operacion
estimar desde k mediante el problema de optimizacion:

2
2

mn kx k LTx + ky Ly k
,
(19)
k
que aproxima a los datos k.

donde k es la funcion
(19) es equivalente a resolver la ecuacion
de Lyapunov
El problema de optimizacion
[?] Ak + k B = C, con A = LTy Ly , B = LTx Lx , y C = LTy ky + kx Lx .
1
2
round
LS
k round k
2 k +
Figura : Desenvolvimiento de fase por mnimos cuadrados y redondeo.5

5
R. Juarez-Salazar, et. al., Phase-unwrapping algorithm by a rounding-least-squares approach. Opt.
Engineering, 53(2):074109 (2013).
31 / 40

Salazar
Introduction
Conclusions
References
Normalizacion
Desenvolvimiento de fase rapido,

automatico
y exacto
Video...
32 / 40
Funcionamiento del algoritmo de desenvolvimiento de fase propuesto

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto
Video...
33 / 40
Factibilidad del algoritmo propuesto

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto
Figura : Ejemplos de desenvolvimiento de fase. (1ra columna) Datos sinteticos,

(2da y 3ra
columna) Datos experimentales obtenidos por interferencia, (4ta columna) Datos experimentales
de franjas.
obtenidos por proyeccion
34 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto
de objetos 3D mediante proyeccion

de franjas usando el algoritmo de
Reconstruccion
desenvolvimiento de fase propuesto.
35 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto

Reconstruccion
35 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto

Reconstruccion
35 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto

Reconstruccion
35 / 40

Salazar
Introduction
Conclusions
References
Normalizacion

automatico
y exacto

Reconstruccion
(a)
36 / 40

Salazar
(b)
(c)
Introduction
Conclusions
References
Conclusions
General conclusions
1
A fringe analysis scheme to phase demodulation for fast and automatic

applications was presented.
The simplicity of the developed algorithms, makes possible the implementation of

the fringe analysis toolbox in dedicated hardware to address real-time
applications.
By the flexibility of the proposed scheme, it may be implemented in many other

measurement areas such as machine vision and adaptive optics.
Particular conclusions
37 / 40
Like the Fourier fringe analysis method, the proposed fringe-pattern and the
generalized phase-shifting algorithms may be extended to more than two spatial
dimensions.
For the proposed generalized phase-shifting algorithm, if the optical setup is static
and the object is dynamic, the computed phase shift will correspond to the object.
It may be useful to extract the phase object without a phase shift. An application of
this may be optical coherent tomography.

Salazar
Introduction
Conclusions
References
References I
R. Juarez-Salazar, C. Robledo-Sanchez, C. Meneses-Fabian, F. Guerrero-Sanchez, and L. A. Aguilar,
Generalized phase-shifting interferometry by parameter estimation with the least squares method, Optics
and Lasers in Engineering 51, 626632 (2013).
Robledo-Sanchez, C., Juarez-Salazar, R., Meneses-Fabian, C., Guerrero-Sanchez,

F., Aguilar, L. M. A.,
Rodriguez-Zurita, G., and Ixba-Santos, V. Phase-shifting interferometry based on the lateral displacement of
the light source Opt. Express, 21(14):1722817233 (2013).
Arevalo
suppression using a parameter estimation approach Optical Engineering, 52(7):074109074109 (2013).
Gannavarpu Rajshekhar and Pramod Rastogi Fringe analysis: Premise and perspectives Optics and Lasers
in Engineering, 50(8):iiix (2012).
J. H. Bruning, D. R. Herriott, J. E. Gallagher, D. P. Rosenfeld, A. D. White, and D. J. Brangaccio, Digital
wavefront measuring interferometer for testing optical surfaces and lenses,.Appl. Opt. 13, 2693-2703 (1974).
K. Creath, Phase measurement interferometry techniques,n Progress in optics,vol. 26, E. Wolf, ed.
(Elsevier Science Publishers, 1988), pp. 349-393.
L. L. Deck, Suppressing phase errors from vibration in phase-shifting interferometry,.Appl. Opt. 48,
3948-3960 (2009).
38 / 40

Salazar
Introduction
Conclusions
References
References II
C. J. Morgan, Least-squares estimation in phase-measurement interferometry,.Opt. Lett. 7, 368-370 (1982).
J. E. Greivenkamp, Generalized data reduction for heterodyne interferometry,.Optical Engineering 23,
350-352 (1984).
G. Lai and T. Yatagai, Generalized phase-shifting interferometry,J. Opt. Soc. Am. A 8, 822-827 (1991).
L. Z. Cai, Q. Liu, and X. L. Yang, Generalized phase-shifting interferometry with arbitrary unknown phase
steps for difraction objects,.Opt. Lett. 29, 183-185 (2004).
X. Xu, L. Cai, H. Yuan, Q. Zhang, G. Lu, and C. Wang, Phase shift selection for two-step generalized
phase-shifting interferometry,.Appl. Opt. 50, H171-H176 (2011).
A. Patil and P. Rastogi, .Approaches in generalized phase shifting interferometry,.Optics and Lasers in
Engineering 43, 475-490 (2005).
C. T. Farrell and M. A. Player, Phase-step insensitive algorithms for phase-shifting
interferometry,Measurement Science and Technology 5, 648 (1994).
Pramod K. Rastogi. Digital speckle pattern interferometry and related techniques. John Wiley and Sons, LTD,
2001.
D. Malacara, ed., Optical shop testing (John Wiley & Sons, Inc., 2007), 3rd ed.
39 / 40

Salazar
Introduction
Conclusions
References
Thank you very much for your attention
Any question?
40 / 40

Salazar

Computer Vision For Robotic Navigation and Augmented Reality Applications

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Computer Vision For Robotic Navigation and Augmented Reality Applications

Uploaded by

Copyright:

Available Formats

Introduction

Computer vision by phase encoding methods