2012 Fifth International Symposium on Computational Intelligence and Design
A 3D plane recognition algorithm based on image matching
Jun JIANG, Xu HU, Hua ZHANG, Yu-feng XIAO
Robotics Laboratory, School of Information Engineering
Southwest University of Science and Technology
Mian yang, China
E-mail: jiangjun1289@foxmail.com
Abstract—The three-dimensional(3D) plane-recognition
technology is the basic research of 3D object matching, 3D scene
modeling and scene segmentation. Traditional algorithms rely on
specific devices and particular environment or the priori
calibration parameters. The algorithm fails completely when lose
those information. For the shortcomings of the traditional
algorithms, the 3D multi-plane detection algorithm based on
image matching has been found. It only requires two pictures
from different perspective for a project to identify objects'
different planes. The algorithm's operation is simple, adaptable
and flexible. The algorithm is a new algorithm for 3D plane
recognition, and the effectiveness of the algorithm has been
verified by experiments.
Keywords-Three-dimensional plane recognition; Image
matching; Three-dimensional target recognition
I. INTRODUCTION
Image processing technology acts as an important role in
current era whose main theme is intelligence. 3D multi-planar
recognition technology, an important branch of Image
recognition, is the basis of some frontier subjects, such as
stereo matching, 3D scene modeling, scene segmentation, etc.
As a basic research, there are a lot of good algorithms to solve
the problem. The mainstream algorithm includes multi-planar
fitting technology based on point clouds data got by 3D laser,
multi-planar recognition technology based on multi-camera
vision, multi-planar recognition technology based on camera
calibration, etc. The above methods already have quite mature
solutions, and their precision is more accurate than some other
methods. But they are based on specific equipment, specific
environment or specific calibration parameter. When the
images have already been obtained or it is difficult to obtain the
calibration parameters, the above methods would fail. In this
paper, a multi-planar recognition algorithm based on image
matching is proposed. The algorithm can distinguish the
different plane of a 3D target only requiring two images of the
target from different perspective. It is simple to operate, and is
of strong adaptability. Furthermore, it is flexible to process.
The algorithm is a new algorithm in multi-planar recognition
field which can be applied to stereo matching, scene
segmentation, etc. The framework of the algorithm is shown in
Figure 1.
978-0-7695-4811-1/12 $26.00 © 2012 IEEE 551
DOI 10.1109/ISCID.2012.143
Obtaining two images
from different view
Searching matching
points through image
matching
Extracting changing
characteristics of
matching points
Classifying matching
points
Distinguishing different
plane
Figure 1. The framework of Algorithm
II. IMAGE MATCHING
As the basic research of image processing technology,
Image matching technology has sprung up a batch of very
classically algorithms in the efforts of many researchers. In
1988, Harris proposed a corner detector called harris[1], which
was invariant to rotation and scale. Mikolajczyk presented
Hessian Laplace[2] detector in 2002 which improved the
matching rate of images. Another detector named MSER[3],
proposed by J. Matas and O. Chum, made the image matching
technology more robust. From 1999 to 2004, David G. Lowe
proposed SIFT descriptor[4]. It gave a huge boost for image
matching field, and opened a new era in image matching. On
this basis, a series of improved algorithms based on SIFT were
raised. The PCA-SIFT[5] descriptor proposed by Y. Ke and R.
Sukthankar in 2004 aimed at reducing the computing time of
SIFT descriptor by using dimensionality reduction technique;
In 2006, Farag proposed CSIFT descriptor who introduced
SIFT descriptor to color space[6]. SURF[7] proposed by Bay in
2006 which was known as the enhanced version of SIFT
descriptor. Its greatest advantage comparing with SIFT was the
high computing speed. What’s more, the number of feature
points is almost the same as SIFT. Then, Jean-Michel Morel
and Guoshen Yu proposed ASIFT descriptor[8] in 2009. The
descriptor built a full affine space. The number of extracted
feature points is considerably more than other algorithms. The
proposed algorithm in this paper is based on ASIFT descriptor.
Firstly, a full affine space is built. Then, matching pairs are
obtained based on SIFT descriptor. Finally, different plane is
recognized according to the properties of the matching pairs.

A. Camera Model

n
l
ψ
λ Figure 3. ASIFT Affine space
θ
M
φ
u0 ASIFT descriptor simulated the angle θ and φ in camera
imaging model.

Figure 2. The camera imaging model C. SIFT Algorithm

Figure 2 shows a camera imaging model of the objects in
space. Supposing M is a fixed plane which is perpendicular to
physical object’s plane u0 . l is the optical axis of camera
imaging. The main factors affecting camera imaging quality
include: The angles φ and θ are the camera optical axis
longitude and latitude; the angle ψ parameterizes the camera
spin. The camera focal is λ; the math expressions is shown in
formula 1.
u  S1G1 Au0 (1)
Where u is the image of camera imaging, S1 is the
standard sampling operator on a regular grid with mesh l, G1 is
optical blur of camera lens, A is the affine translation matrix,
u0 is original image. The expression of affine translation is
From 1999 to 2004, SIFT was carried out by David G.
Lowe. It is a matching algorithm which is invariant to scale,
rotation and translation, and it has played a powerful role in
promoting the field of image matching [4]. The main idea of
SIFT is to find a stable point in the image that will be marked
as the key point, and the scale and direction were given to key
points. The left image in Figure 4 is a Gaussian pyramid, the
parameter σ reflects the scale size λ, and the right picture is the
describer of feature points, the gradient of key points reflects
the image’s main direction ψ. And then in the neighborhood of
the key points, the feature point description was constituted by
128-dimensional feature vectors that ware extracted with
specific rules. Finally, matching points will be found by
measuring the Euclidean distance among feature points [4].
Octave 5
Ă

8σ Octave 4
carried out as below.
Ă

4σ Octave 3 x
a b 
Ă

A  H  R1 ( )Tt R2 ( ) 2σ

c d 
Octave 2 y

(2)
Ă

 cos  sin  t 0  cos   sin   3σoct 3σoct

   
3σoct

sin cos 
0 1 
sin  cos  
3σoct

σ x
Octave 1
Where λ is camera focal length, ψ parameterizes the camera
spin, φ and θ is the camera optical axis longitude and latitude,
Ă

1 Figure 4. SIFT keypoints

and t  .
cos
The transformation from u  to v ' is shown in formula 4.
B. ASIFT Affine Space u  and v ' are, respectively, the transformation from u and v
using ASIFT algorithm, where f is translation vector. SIFT
ASIFT is a novelty algorithm based on full affine space descriptor simulates the focal λ in camera imaging model and
which was proposed by JEAN-MICHEL and GUOSHEN YU the rotation angle ψ along the optical axis l.
in 2009. The algorithm has strong ability of anti-affine
transformation, and can covers almost all the affine v '  [ H  ' R1 ( ')]1 H  R1 ( )u   f
transformation space [8]. ASIFT builds a matching framework (4)
 H  R1 ( ' )u   f
in fact. Every image is transformed into affine space by means
'
of formula 3. In order to have ASIFT invariant to any affine
transform, one needs to sample the t and φ at fixed intervals.
The built affine space is shown in Figure 3. III. STEREO PLANE DETECTION
From the above analysis, the process of building affine
 t 0  cos   sin  
space framework simulates the θ and φ in camera imaging
u '  Tt R2 ( )u    u (3)

0 1 
sin  cos   model, and the matching process of SIFT simulates the focal λ
and rotation angle ψ. Therefore, in the matching process using

552
ASIFT, one would acquire the values of θˈφˈλˈψ. The  '
|A|=|B| and  ; where  ,  is the maximum eigenvalues
matching process from u to v can be summarized in the  '
following steps. of matrix A and B, respectively.  ' ,  ' is their minimum
1. ASIFT affine space is built as below. eigenvalues, respectively. It is worth noting that    .
 u '  Tt R2 ( )u Experiments showed that most matching points generated
 (5)
v '  Tt ' R2 ( ')v
by ASIFT algorithm met the above conditions. However, there
is still some points are not. They didn’t affect identification
2. The SIFT matching process from u ' to v ' is shown in
effect, because the number of these points is small. According
formula 6, f is translation vector.
 t
to the above experiments, it can be known that N  0.
v '  [ H  ' R1 ( ')]1 H  R1 ( )u   f 't'
 H  R1 ( ' )u   f (6) If we assume that  is the maximum eigenvalue of matrix N
' and  is the minimum eigenvalues, then   r  . A
3. The translation from u to v can be got according to relationship can be got by the equation below.
formula 5 and formula 6, which is shown in formula 7.
Tr  N        r  1  r  2  1
2 2 2

v  H  [Tt ' R2 ( ')]1 R1 ( ' )Tt R2 ( )u  F  (9)

'
Det  N   r r
1 Where Tr  N   Dxx  Dyy ˈ Det  N   Dxx Dyy  Dxy Dxy , if
  t ' 0  cos  '  sin  '  
   
 ' 
0 1 
sin  ' cos  '   Tr  N 
2

r 1 , then ! r  2 is close to linear, e.i. the ratio

 cos( ' )  sin( ' )  Det  N 


sin( ' ) cos( ' )  of eigenvalues of matrix N can be obtained through computing
Tr  N 
2
 t 0  cos   sin  
  u  F (7) S .

0 1 
sin  cos   Det  N 
  cos  '/ t ' sin  ' 
  First of all, it is usually to build a two-dimensional vector V:
 '
 sin  '/ t ' cos  '  (|N|, S) for each feature point. Then, the similarity between
 cos( ' )  sin( ' )  each vector V is calculated based on Euclidean distance. Using
 K-means[9] classification method, the feature matching point of

sin( ' ) cos( ' )  each plane obtain can be obtained, i.e. the classification of
 t cos  t sin   different plane is realized. Table ĉ shows the values of vector
 u  F V of some feature points (Only listing 10 points) belong to

sin  cos   different plane in figure 4. In plane 1, the expectation and
F is the translation vector after transforming. A transformation variance of |N| is 1.393 and 0.002, respectively. The
is carried out as below: expectation of S is 26.879, its variance is 0.819. In plane 2, the
  cos  '/ t ' sin  '  expectation of |N| is 0.634, and 0.007 is its variance. 17.873
N  and 1.006 is, respectively, the expectation and variance of S.
 '
 sin  '/ t ' cos  '  It can be seen from the table that the difference of numerical
 cos( ' )  sin( ' )  features of different planes is rather obvious. It can be used to
 (8) classify accurately. Figure 5 shows two renderings after plane

sin( ' ) cos( ' )  segmentation. Fig a shows the matching effect. Fig b is the
 t cos  t sin   plane segmentation effect. Figure 6 shows the rendering after
 segmenting the three planes of an object (Note: the same

sin  cos  
colored points are in the same planes).
If matching points of u or v are in the same plane, then the
transformation matrix N should be similar. According to the
property, different planes in 3D space can be identified. TABLE I. ELEMENT VALUES IN VECTOR V

The similarity of two matrixes is judged by the property of Plane 1 Plane 2

Number
their eigenvalues. But it is very complicated to computing |N| S |N| S
eigenvalues of a matrix. Hence, in order to reduce the amount 1 1.418 25.486 0.561 18.256
of calculation, one could simplify the judging method of the
similarity of second order matrix A and B introducing the 2 1.414 27.684 0.512 18.347
following theorem. 3 1.401 26.544 0.710 19.346
4 1.389 26.346 0.672 17.345
Proof˖ The sufficient and necessary conditions is that the
second-order matrix A and B (|A|>0, |B|>0) being similar are 5 1.390 26.454 0.703 16.346

553
 6 1.393 28.562 0.690 18.365 algorithm, points in different stereo planes can be distinguished.
The relocation for objective can be realized by computing
7 1.423 27.569 0.720 19.455
perspective transformation matrix of matched feature points.
8 1.435 25.684 0.513 17.466 The proposed algorithm, as a new algorithm of 3D multi-planar
9 1.391 26.896 0.544 17.346 recognition, subverts the recognizing concept of traditional
algorithms. It is of strong research significance.
10 1.279 27.565 0.715 16.458
Ă Ă Ă Ă Ă
ACKNOWLEDGMENT
Thanks my mentor has been my care and help. The
following is the funded projects on this topic. The first is
“Science and technology planning project in Sichuan
(2012GZ0023)”.Another is “Graduate student innovation fund
in Southwest University of Science and Technology
(10ycjj26)”.

Fig a. ASIFT matching Fig b. Effect of plane segmentation
Figure 5. The result of plane segmentation
Figure 6. The result of planes segmentation
IV. CONCLUSION
The theoretical analysis, experiments and system tests show
that the 3-dimensional plane detection algorithm based on
image matching can obtain good results. It doesn’t have to
meet the extreme demands for traditional algorithm in
equipment, priori knowledge and the scene. Using this
REFERENCES
[1] C. Harris and M. Stephens, A combined corner and edge detector, in
Proceedings of the Fourth Alvey Vision Conference, Vol. 15, 1988, pp.
147–151.
[2] K. Mikolajczyk and C. Schmid, An affine invariant interest point
detector, in Proceedings of the Seventh European Conference on
Computer Vision (ECCV), Springer-Verlag, London, 2002, pp.128–142.
[3] J. MATAS, O. CHUM, M. URNAN, AND T.PAJDLA, Robust wide-
baseline stereo from maximally stable extremal regions, Image Vision
Comput. 22 (2004), pp. 761–767. Amsterdam, TheNetherlands2.
Elsevier, 1990.
[4] D. Lowe, Distinctive image features from scale-invariant key points, Int.
J. Comput. Vis., 60 (2004), pp.91–110. Dublin Ireland, 2000, 751-767.
[5] Yan Ke and Rahul Sukthankar, PCA-SIFT:a more distinctive
representation for local image descriptors[A].In Proceedings of IEEE
International Conference on ComputerVision and Pattern Recognition
[C],Washington,DC,USA, 2004,2:504~513.
[6] Alaa E㸬Abdel̾Hakim㸪Aly A㸬 Farag㸬 CSIFT㸸A SIFT descriptor
with color invariant characteristics [C] 㸬 IEEE Computer Society
Conference on Computer Vision and Pattern Recognition㸪2006㸪2:
1978㹼1983.
[7] Bay H. SURF: Speeded Up Robust Features[J]. Computer Vision and
Image Understandin g, San Diego: Academic Press Inc Elsevier
Science ,2008, 110 (3): 346-359.
[8] Jean-Michel Morel and Guoshen Yu. ASIFT: A New Framework for
Fully Affine Invariant Image Comparison[J]. SIAM J. IMAGI
NGSCIENCES Vol. 2, No. 2, pp. 438–469.
[9] S. P. Lloyd, Least Squares Quantization in PCM,IEEE Transactions on
Information Theory 28(1982),129-137.

554