Who invented Machine Vision. . .???

Machine vision was invented by Jerome Lemelson, an American

Inventor who has more than 500 patents on his name.

Machine Vision in operation can be
described by a four-step flow:
Application Types:
o Locate
o Measure
o Inspect
o identify

Locate
In locating applications, the purpose of the vision system is to find
the object and report its position and orientation.

In robot bin picking applications the camera finds a reference
coordinate on the object, for example center of gravity or a
corner, and then sends the information to a robot which picks up
the object.



Measure
In measurement applications the purpose of the vision
system is to measure physical dimensions of the object.

Examples of physical dimensions are distance, diameter,
curvature, area, height and volume.
In the example to the right, a camera measures multiple
diameters of a bottle-neck.

Inspect
In inspection applications the purpose of the vision system is
to validate certain features, for example presence or
absence of a correct label on a bottle, screws in an assembly,
chocolates in a box, or defects.
Identify:
In an identification application, the vision system reads various
codes and alphanumeric characters (text and numbers).

For example a Camera reads the best before date on a food
package.

Branch Types
Automotive & Electronics
Food
Logistics
Manufacturing & Packaging
Robotics
Pharmaceutical
Machine Vision
VCR
Industrial robots
Fax Machine
Cordless telephone
Videocassette Recorder
Camcorders
Magnetic Tape drives Used in Sony Walkman etc.
Few of them Includes:
Automatic extraction of information from digital images.

Industrial Vision" or Vision Systems.

Industrial manufacturing and quality control processes.

Inspection process.

Robot control or type verification.

Computer science, optics, mechanical engineering.

Industrial automation.
Machine Vision Basics

Industrial image processing.

Required because of :
high-speed,
high-magnification,
24-hour operation,
repeatability of measurements.

The way in which the system will "see" and the optimum
conditions which have to be created for it to perform its task
in a repeatable manner.

Computer vision v/s Machine vision

Machine Vision Flow Chart
The most common use for deploying machine vision
is to perform the following tasks:

- Position recognition

- Identification (by codes, characters etc.)

- Shape and dimension checks

- Completeness checks

- Image and object comparison

- Surface inspection

Components of Machine
Vision System
Digital cameras

A synchronizing sensor for part detection

A computer program

Input/output hardware
Machine Vision Flow
- Automotive

- Semi-conductor & Electronics

- Medical Devices and Pharmaceutical

- Printing and Packaging

- General Mechanical Engineering

- Food Processing

- Solar Production
Machine Vision Uses
Machine Vision Software Available in Market :

Vision reader plus : Used in Barcode Reading ,Reading a
Particular text .

Measure plus : Useful to get geometry measurement values of
parts like length, radius, angles etc. with very high accuracy.
Used in mainly Automotive and Packaging Industries

MVP Match-it : It is a very low cost machine vision solution for
all the industries where simple pattern matching type
application works. It is useful to math Features of parts like
assembly presence/absence, object matching etc.

Image
Acquisition
Image
Filtering &
Enhancement
Image
Restoration
Color Image
processing
Wavelets
Compression
Morphological
processing
Segmentation
Representation
& Description
Object
Recognition
Knowledge
base
Problem
Domain
Basic definition of an IMAGE :

A two dimensional function, f(x,y), where x & y are plane
Coordinates, & the amplitudes f at any pair of coordinates (x,y)
is called intensity or gray level of the image at that point.

A digital image is composed of a finite number of elements,
each of which has a particular location & value.
These elements are called picture elements, image elements
pels and pixels.

For displaying image converted it into analog signal, which
is the part of computer vision.
Image Formation Model

Intensity proportional to energy radiated by physical source.
0 < , <

, = i , r ,

(illumination) (reflectance)

0 < i , <
0 < r , < 1

(total absorption) (total reflectance)




Representing Digital Image

2-D array , containing M rows & N columns.
x = 0,1,2. . . . . M-1
y = 0,1,2 . . . . . N-1





Matrix of real numbers.
If M N = 600 600 means 36,000 elements (pixels).
(
(
(

=
) 1 , 1 ( ... ).... 1 , 1 ( ) 0 , 1 (
) 1 , 1 ( ... ).... 1 , 1 ( ) 0 , 0 (
) 1 , 0 ( ... ).... 1 , 0 ( ) 0 , 0 (
) , (
N M f M f M f
N f f f
N f f f
y x f

Introduction to Image
Filtering

Filtering accepting or rejecting certain frequency components.

Two types : Linear Spatial Filter
Filters in Frequency Domain.

Spatial Filtering :
Filtering creates a new pixel with coordinates equal to the
coordinates of the center of the neighborhood.

A filtered image is generated as center of filter visit each pixel in
the input image.
Smoothing Linear Filters :

For , Blurring
Noise reduction

To blur an image for the purpose of smaller object blends with
the back ground and larger image become bloblike and easy to
detect.

Output : the average of the pixels(intensity level) contained in
the neighborhood, called Averaging Filters. (Lowpass)

Reduce sharp transition.

3 3 smoothing filter Mask,

Equal coefficients,
Averaging :
R =
1
9

9
=1

(box filter)

Different Coefficients,
Weighted Average :
Weighing the center point highest
and then reducing the value of coefficients
as a function of increasing distance.

1 1 1
1 1 1
1 1 1
1
9

1 2 1
2 4 2
1 2 1
1
16

100 100 100 100 100
100 200 205 203 100
100 195 200 200 100
100 200 205 195 100
100 100 100 100 100
100 100 100 100 100
100 144 167 145 100
100 167 200 168 100
100 144 166 144 100
100 100 100 100 100
100 + 100 + 100 + 205 + 200 + 195 + 100 + 100 +200 = 1300
Now, 1300 / 9
Weighted average reduce blurring in the smoothing process.
Neighborhood
Median filter:

Based on ordering the pixels.

Taking the median value instead of the average or weighted
average of pixels in the window.
Median: sort all the pixels in an increasing order,
take the middle one.
Excellent noise reduction.
Less blurring.
Effective in presence of impulse noise (salt-and-pepper)
Median
Filter
100 100 100 100 100
100 200 205 203 100
100 195 200 200 100
100 200 205 195 100
100 100 100 100 100
100 100 100 100 100
100 100 200 100 100
100 200 200 200 100
100 100 195 100 100
100 100 100 100 100
In increasing order :
( 100 , 100 , 100 , 100 , 100 , 195 , 200 , 200 , 205 )
Median = 100
Salt & pepper noise 3 3 averaging mask 3 3 median filter
Sharpening Spatial Filters :

To highlight transition in intensity.

Accomplished by spatial differentiation.

Strength of the response of a derivative is proportional to the
degree of intensity discontinuity.

Enhance edges and other discontinuities.

Enhanced image = original image + scaled version of
the line structures and
edges in the image

Based on first and second order derivatives.

Second derivative for sharpening The Laplacian

Let ,

2
2
2
2
2
y
f
x
f
f
c
c
+
c
c
= V
) , ( 2 ) , 1 ( ) , 1 (
2
2
y x f y x f y x f
x
f
+ + =
c
c
) , ( 2 ) 1 , ( ) 1 , (
2
2
y x f y x f y x f
y
f
+ + =
c
c
) ( 4 )] 1 , ( ) 1 , ( ) , 1 ( ) , 1 ( [
2
x f y x f y x f y x f y x f f + + + + + = V
0 1 0
1 - 4 1
0 1 0
) , ( y x f
Considering diagonal terms also,
) ( 8 )] 1 , 1 ( ) 1 , 1 ( ) 1 , 1 ( ) 1 , 1 (
) 1 , ( ) 1 , ( ) , 1 ( ) , 1 ( [
2
x f y x f y x f y x f y x f
y x f y x f y x f y x f f
+ + + + + + + +
+ + + + + = V
1 1 1
1 - 8 1
1 1 1
)] , ( [ ) , ( ) , (
2
y x f c y x f y x g V + =
So, Laplacian for image sharpening,
) , ( y x f
- i/p image
) , ( y x g
- sharpened image
Useful MATLAB Commands
for Image Processing
imread() : To import an image from any supported graphics
image file format.
imwrite() : To export image data from the MATLAB workspace
to a graphics file.
imshow() : Displays Image in their respective format.
Imtool() : Opens a new image tool in empty state.
Imfilter(A,H) : Filters the multidimensional array with the
multidimensional filter H.
Im2bw() : Convert image to binary image, based on
threshold.

Greythrash(I) : Uses Otsus method, which chooses the threshold
to minimize the intraclass variance of the black and
white pixels

% Commands to Introduce noise
Imnoise(I,type) : Add noise to image

'gaussian : Gaussian white noise with constant mean and
variance
'localvar : Zero-mean Gaussian white noise with an intensity-
dependent variance
'poisson : Poisson noise
'salt & pepper : On and off pixels
'speckle : Multiplicative noise
MATLAB code for Image Filtering

Introduction to Image
Segmentation

Purpose: To partition an image into meaningful
regions with respect to a particular application.

Based on measurements taken from the image
and might be grey level, colour, texture, depth or
motion.
Applications of image segmentation include

Identifying objects in a scene for object-based
measurements such as size and shape.

Identifying objects in a moving scene for object-based
video compression (MPEG4).

Identifying objects which are at different distances from
a sensor using depth measurements from a laser range
finder enabling path planning for a mobile robots.

Example 1 : Segmentation based on greyscale









Very simple model of greyscale leads to inaccuracies
in object labelling.
Example 2 : Segmentation based on texture










Enables object surfaces with varying patterns of grey
to be segmented.
Example 3 : Segmentation based on motion







The main difficulty of motion segmentation is that an
intermediate step is required to estimate an optical flow
field.

The segmentation must be based on this estimate and
not, in general, the true flow.


Example 3 : Segmentation based on depth




v



This example shows a range image, obtained with a
laser range finder.

A segmentation based on the range is useful in
guiding mobile robots.
Original image Range image
Segmented
image
Two very simple image segmentation techniques that are
based on the grey level histogram of an image:

Thresholding
Ostus METHOD
MEAN TECHNIQUE
P-TILE TECHNIQUE
HISTOGRAM DEPENDENT TECHNIQUE (HDT)
EDGE MAXIMIZATION TECHNIQUE (EMT)
VISUAL TECHNIQUE

Clustering
K-means Clustering
Hierarchical Clustering

Segmentation Techniques
Thresholding

To extract object from background, select threshold T

Segmented image,
g(x,y) = 1 if f(x,y) > T (object point)
= 0 if f(x,y) <= T (background point)

o Global thresholding (T const.)
o Variable thresholding (T changes)

T depends on property of neighborhood of (x,y).

Ostus Method :

Maximizes the between class variance.

Let, {0,1,2. . . . L-1} distinct intensity levels of size M N
Histogram components,

- no. of pixels with intensity i.

So,

1
=0
= 1

Divide into 2 classes,
class

class

1
(k) =

=0

2
(k) =

1
=+1

Mean intensity,

1
(k) =
1

1
(k)

=0

2
(k) =
1

2
(k)

1
=+1


Global mean,

(k) =

1
=0


So,
1

1
+
2

2
=

Global variance,

2
= (

)
2

1
=0




between class variance,

2
=
1
(
1

)
2
+
2
(
2

)
2

2
- separability between classes

2
> 0 (zero when all intensity level in image are same)


A method by which large sets of data is grouped into clusters of
smaller sets of similar data.

Example : Clustering of balls of same colour. There are a total of 9
balls which are of three different
colours. We are interested in clustering of balls of the three
different colours into three different groups.


The balls of same colour are clustered into a group as shown below



Clustering
Clustering Algorithms

A clustering algorithm attempts to find natural groups of
components (or data) based on some similarity.

The clustering algorithm finds the centroid of a group of data
sets.

To determine cluster membership, most algorithms evaluate the
distance between a point and the cluster centroids.





Initially, the number of clusters must be known, or chosen, to be
say K.

Choose a set of K instances as centres of the clusters. Often
chosen such that the points are mutually farthest apart.

Next, the algorithm considers each instance and assigns it to the
cluster which is closest.

The cluster centroids are recalculated either after each instance
assignment, or after the whole cycle of re-assignments.

This process is iterated until no further minimum distance
obtained.
K-means Clustering

Using cluster centroid to represent cluster
Assigning data elements to the closest cluster (centre).
Goal: Minimise the sum of the within cluster variances
Variations of K-Means
Initialisation (select the number of clusters, initial
partitions)
Updating of center
Hill-climbing (trying to move an object to another cluster).

Other K-Means Algorithms features
The K-Means Clustering Method

Example :



















0
1
2
3
4
5
6
7
8
9
10
0 1 2 3 4 5 6 7 8 9 10
0
1
2
3
4
5
6
7
8
9
10
0 1 2 3 4 5 6 7 8 9 10
0
1
2
3
4
5
6
7
8
9
10
0 1 2 3 4 5 6 7 8 9 10
0
1
2
3
4
5
6
7
8
9
10
0 1 2 3 4 5 6 7 8 9 10
Comments on the K-Means Method

Strength
Relatively efficient: O(tkn), where n is # instances, c is # clusters, and t
is # iterations. Normally, k, t << n.

Often terminates at a local optimum. The global optimum may be
found using techniques such as: simulated annealing or genetic
algorithms

Weakness
Applicable only when mean is defined; what about categorical data?

Need to specify c, the number of clusters, in advance

Unable to handle noisy data and outliers

Not suitable to discover clusters with non-convex shapes
The K-Means Algorithm

1. Choose a value for K, the total number of clusters.

2. Randomly choose K points as cluster centers.

3. Assign the remaining instances to their closest
cluster center.

4. Calculate a new cluster center for each cluster.

5. Repeat steps 3-5 until the cluster centers
do not change.

MATLAB code for
Image Segmentation

Pattern Recognition

PATTERN RECOGNITION Pattern + Recognition
PATTERN : A set of objects or phenomena or concepts where
the elements of the set are similar to one
another in certain ways/aspects.


Example : Humans, Radar Signals, insects, Animals, sonar
signals, Fossil records, Micro organisms signals,
clouds etc.
RECOGNITION : The knowledge or feeling that the present object has
been met before (the process of knowing again).

A pattern is an arrangement of descriptor.
Descriptor differentiate one class of object from another.

Pattern vector,
x =

2
.
.
.
.

A pattern class is a family of pattern share some common
properties. {
1

2

3
. . . .

}
Recognition based on Decision-Theoretic Methods :

Let, x = (
1
,
2
. . . .

pattern class,
1

2

3
. . . .

Decision function,
1
(x) ,
2
(x). . .

(x)

If x belongs to class

(x) >

(x) j = 1,2. . . W

Decision boundary

given by,

(x) =

(x)

(x)

(x) = 0

(x) > 0 class

(x) < 0 class

Matching :

Minimum distance classifier approach.
Compute Euclidean distance between unknown and each
of the prototype vectors.

Minimum distance classifier:

The prototype of each pattern class to be the mean
vector,

=
1

j = 1,2. . . W
Using Euclidean distance,

= ||

|| j = 1,2. . . W
Using the property of Euclidean norm,

1
2

j = 1,2. . . W

Boundary between classes

&

(x)

(x) =

(x)

(x) =

)
1
2
(

)
= 0

The surface given by above equation is perpendicular
bisector of line segment joining

&

.
Example,

Two pattern classes : A & B denoted by

1
&
2
,
have sample mean vector

1
= (4.3,1.3)

&
2
= (1.5,0.3)

.

The decision function,

1
=

1
-
1
2

1

= 4.3
1
+ 1.3
2
- 10.1

2

A
B

2
=

2
-
1
2

2

= 1.5
1
+ 0.3
2
- 1.17
Boundary equation,

12
(x) =
1
(x)
2
(x)
= 2.8
1
+ 1.0
2
- 8.9
= 0

12
(x) > 0 class
1

12
(x) < 0 class
2

2

A
B
2.8
1
+ 1.0
2
- 8.9 = 0
DEFINITION.1.
It is a study of ideas and algorithms that
provide computers with a perceptual capability to put
abstract objects, or patterns into categories in a simple and
reliable way.


DEFINITION.2.
It is an ambitious endeavor of mechanization of
the most fundamental function of cognition.

Definition of Pattern
Recognition

METHODOLOGY OF PATTERN
RECOGNITIONS
1. We observe patterns
2. We study the relationships between the various patterns.
3. We study the relationships between patterns and ourselves
and thus arrive at situations.
4. We study the changes in situations and come to know about
the events.
5. We study events and thus understand the law behind the
events.
6. Using the law, we can predict future events.

Pattern Recognition System

Post
Processing
Classification
Feature
Extraction
Grouping Sensing input
output
Pattern Recognition covers a wide spectrum of
disciplines such as,

1. Cybernetics
2. Computer Science
3. System Science
4. Communication Sciences
5. Electronics
6. Mathematics
7. Logic
8. Psychology
9. Physiology
10.Philosophy
1. Medical diagnosis
2. Life form analysis
3. Sonar detection
4. Radar detection
5. Image processing
6. Process control
7. Information Management systems
8. Aerial photo interpretation.
9. Weather prediction
10. Sensing of life on remote planets.
11. Behavior analysis
12. Character recognition
13. Speech and Speaker recognition
etc.

Application of Pattern
Recognition

An essential part of the broader field of Artificial Intelligence.

An important aspect of applying computers to solve problems in
science and engineering.

A unified frame work to study a variety of techniques
Motivation

1.SPATIAL PATTERNS- These patterns are located in space.
* images of ground covers in remote sensing
* images of medical diagnosis.

2.TEMPORAL PATTERN- These are distributed in time.
* Radar signal, speech recognition, sonar signal etc.

3.ABSTRACT PATTERNS-Here the patterns are distributed neither
in space nor time.
* Medical diagnosis based on medical history and other
medical tests.
* Classification of people based on language they speak.

Types of Pattern Recognition

?
Any Question
Digital Image Processing (3
rd
edition)
by Rafael C. Gonzalez & Richard E. Woods

Pattern Classification (2nd Edition)
by Richard O. Duda ,Peter.E.Hart, David. G Stork, Wiley
Publication.

MATLAB

http://eeweb.poly.edu/~yao
Image Filtering: Noise Removal, Sharpening.


References