TECHNICAL UNIVERSITY OF CRETE

DEPARTMENT OF ELECTRONIC AND COMPUTER

ENGINEERING

MACHINE VISION
Euripides G.M. Petrakis
Michalis Zervakis
http://www.intelligence.tuc/~petrakis
http://courses.ece.tuc.gr
Chania 2010

E.G.M. Petrakis Machine Vision (Introduction) 1

 Machine Vision

• The goal of Machine Vision is to create a

model of the real world from images
– A machine vision system recovers useful
information about a scene from its two
dimensional projections
– The world is three dimensional
– Two dimensional digitized images

E.G.M. Petrakis Machine Vision (Introduction) 2

 Machine Vision (2)

• Knowledge about the objects (regions) in a

scene and projection geometry is required.
• The information which is recovered differs
depending on the application
– Satellite, medical images etc.
• Processing takes place in stages:
– Enhancement, segmentation, image analysis
and matching (pattern recognition).

E.G.M. Petrakis Machine Vision (Introduction) 3

Illumination

Image Machine
Acquisition Vision System

2D Image
Scene
Digital Image Description

Feedback

The goal of a machine vision system is to compute a

meaningful description of the scene (e.g., object)
 Machine Vision Stages
Image Acquisition • Analog to digital
(by cameras, scanners etc) conversion
• Remove noise/patterns,
Image Processing improve contrast
Image Enhancement
Image Restoration • Find regions (objects) in
the image
Image Segmentation • Take measurements of
objects/relationships
Image Analysis • Match the above
(Binary Image Processing) description with similar
description of known
Model Matching objects (models)
Pattern Recognition
E.G.M. Petrakis Machine Vision (Introduction) 5
 Image Processing
Image Processing

Input Image Output Image

• Image transformation
– image enhancement (filtering, edge detection, surface detection,
computation of depth).
– Image restoration (remove point/pattern degradation: there exist a
mathematical expression of the type of degradation like e.g. Added
multiplicative noise, sin/cos pattern degradation etc).

E.G.M. Petrakis Machine Vision (Introduction) 6

 Image Segmentation
Image Segmentation

Input Image Regions/Objects

• Classify pixels into groups (regions/objects of interest)

sharing common characteristics.
– Intensity/Color, texture, motion etc.
• Two types of techniques:
– Region segmentation: find the pixels of a region.
– Edge segmentation: find the pixels of its outline contour.
E.G.M. Petrakis Machine Vision (Introduction) 7
 Image Analysis
Image Analysis

Input Image
Segmented Image Measurements
(regions, objects)

• Take useful measurements from pixels, regions, spatial

relationships, motion etc.
– Grey scale / color intensity values;
– Size, distance;
– Velocity;

E.G.M. Petrakis Machine Vision (Introduction) 8

 Pattern Recognition
Model Matching
Pattern Recognition

Image/regions

•Measurements, or Class identifier
•Structural description

• Classify an image (region) into one of a number of known

classes
– Statistical pattern recognition (the measurements form vectors
which are classified into classes);
– Structural pattern recognition (decompose the image into primitive
structures).
E.G.M. Petrakis Machine Vision (Introduction) 9
 Digital Image Representation

• Image: 2D array of gray level or color values

– Pixel: array element;
– Pixel value: arithmetic value of gray level or color
intensity.
• Gray level image: f = f(x,y)
- 3D image f=f(x,y,z)
• Color image (multi-spectral)
f = {Rred(x,y), Ggreen(x,y), Bblue(x,y)}
E.G.M. Petrakis Machine Vision (Introduction) 10
 What a computer “sees” is very different from what
a human sees. A computer sees pixels (arithmetic values)
while a human sees shapes, structures etc.

E.G.M. Petrakis Machine Vision (Introduction) 11

 Relationships to other fields

• Image Processing (IP)

• Pattern Recognition (PR)
• Computer Graphics (CG)
• Artificial Intelligence (AI)
• Neural Networks (NN)
• Psychophysics

E.G.M. Petrakis Machine Vision (Introduction) 12

 Image Processing (IP)

• IP transforms images to images

– Image filtering, compression, restoration
• IP is applied at the early stages of machine
vision.
– IP is usually used to enhance particular
information and to suppress noise.

E.G.M. Petrakis Machine Vision (Introduction) 13

 Pattern Recognition (PR)

• PR classifies numerical and symbolic data.

– Statistical: classify feature vectors.
– Structural: represent the composition of an
object in terms of primitives and parse this
description.
• PR is usually used to classify objects but
object recognition in machine vision usually
requires many other techniques.
E.G.M. Petrakis Machine Vision (Introduction) 14
 Statistical Pattern Recognition

• Pattern: the description of an an object

– Feature vector
– (size, roundness, color, texture)
• Pattern class: set of patterns with similar
characteristics.
• Take measurements from a population of
patterns.
• Classification: Map each pattern to a class.
E.G.M. Petrakis Machine Vision (Introduction) 15
 Structure of PR Systems
input
Sensor

Processing

Measurements

Classification
class
E.G.M. Petrakis Machine Vision (Introduction) 16
 Example of Statistical PR
• Two classes:
I. W1 Basketball players
II. W2 jockeys
• Description: X = (X1, X2) = (height, weight)
X1 W1 .. ……
W2 + . … ..
.. .. ……
. ..
. . .. . D(X) = AX1 + BX2 + C = 0
- Decision function
X2
E.G.M. Petrakis Machine Vision (Introduction) 17
 Syntactic Pattern Recognition

• The structure is important

• Identify primitives
– E.g., Shape primitives
• Break down an image (shape) into a sequence of
such primitives.
• The way the primitives are related to each other to
form a shape is unique.
– Use a grammar/algorithm
– Parse the shape

E.G.M. Petrakis Machine Vision (Introduction) 18

•Primitives

•G1,L(G1) : submedian Grammar

•G2,L(G2) : telocentric Grammar
E.G.M. Petrakis Machine Vision (Introduction) 19
 •Each digit is represented by a waveform representing
black/white, white/black transitions (scan the image from
Left to right.

E.G.M. Petrakis Machine Vision (Introduction) 20

 Computer Graphics (CG)

• Machine vision is the analysis of images

while CG is the decomposition of images:
– CG generates images from geometric primitives
(lines, circles, surfaces).
– Machine vision is the inverse: estimate the
geometric primitives from an image.
• Visualization and virtual reality bring these
two fields closer.
E.G.M. Petrakis Machine Vision (Introduction) 21
 Artificial Intelligence (AI)

• Machine vision is considered to be sub-field of AI.

• AI studies the computational aspects of
intelligence.
• CV is used to analyze scenes and compute
symbolic representations from them.
• AI: perception, cognition, action
– Perception translates signals to symbols;
– Cognition manipulates symbols;
– Action translates symbols to signals that effect the
world.

E.G.M. Petrakis Machine Vision (Introduction) 22

 Psychophysics

• Psychophysics and cognitive science have

studied human vision for a long time.
• Many techniques in machine vision are
related to what is known about human
vision.

E.G.M. Petrakis Machine Vision (Introduction) 23

 Neural Networks (NN)

• NNs are being increasingly applied to solve

many machine vision problems.
• NN techniques are usually applied to solve
PR tasks.
– Image recognition/classification.
• They have also applied to segmentation and
other machine vision tasks.

E.G.M. Petrakis Machine Vision (Introduction) 24

 Machine Vision Applications

• Robotics
• Medicine
• Remote Sensing
• Cartography
• Meteorology
• Quality inspection
• Reconnaissance

E.G.M. Petrakis Machine Vision (Introduction) 25

 Robot Vision

• Machine vision can make a robot manipulator

much more versatile.
– Allow it to deal with variations in parts position and
orientation.

E.G.M. Petrakis Machine Vision (Introduction) 26

 Remote Sensing

• Take images from

high altitudes (from
aircrafts, satellites).
• Find ships in the aerial
image of the dock.
– Find if new ships have
arrived.
– What kind of ships?

E.G.M. Petrakis Machine Vision (Introduction) 27

 Remote Sensing (2)

• Analyze the image

– Generate a description
– Match this descriptions
with the descriptions of
empty docs
• There are four ships
– Marked by “+”

E.G.M. Petrakis Machine Vision (Introduction) 28

 Medical Applications

• Assist a physician to
reach a diagnosis.
• Construct 2D, 3D
anatomy models of the
human body.
– CG geometric models.
• Analyze the image to
extract useful features.

E.G.M. Petrakis Machine Vision (Introduction) 29

 Machine Vision Systems

• There is no universal machine vision system

– One system for each application
• Assumptions:
– Good lighting;
– Low noise;
– 2D images
• Passive - Active environment
– Changes in the environment call for different actions
(e.g., turn left, push the break etc).
E.G.M. Petrakis Machine Vision (Introduction) 30
 Vision by Man and Machine

• What is the mechanism of human vision?

– Can a machine do the same thing?
– There are many studies;
– Most are empirical.
• Humans and machines have different
– Software
– Hardware

E.G.M. Petrakis Machine Vision (Introduction) 31

 Human “Hardware”

• Photoreceptors take measurements of light signals.

– About 106 Photoreceptors.
• Retinal ganglion cells transmit electric and
chemical signals to the brain
– Complex 3D interconnections;
– What the neurons do? In what sequence?
– Algorithms?

• Heavy Parallelism.

E.G.M. Petrakis Machine Vision (Introduction) 32

 Machine Vision Hardware

• PCs, workstations etc.

• Signals: 2D image arrays gray level/color values.
• Modules: low level processing, shape from
texture, motion, contours etc.
• Simple interconnections.
• No parallelism.

E.G.M. Petrakis Machine Vision (Introduction) 33

 Course Outline

• Introduction to machine vision, applications,

Image formation, color, reflectance, depth,
stereopsis.
• Basic image processing techniques (filtering,
digitization, restoration), Fourier transform.
• Binary image processing and analysis, Distance
transform, morphological operators.

E.G.M. Petrakis Machine Vision (Introduction) 34

 Course Outline (2)

• Image segmentation (region segmentation, edge

segmentation).
• Edge detection, edge enhancement and
linking. Thresholding, region growing, region
merging/splitting.
• Relaxation labeling, Hough transform.
• Image analysis, shape analysis. Polygonal
approximation, splines, skeletons. Shape features,
multi-resolution representations.

E.G.M. Petrakis Machine Vision (Introduction) 35

 Course Outline (3)

• Image representation, image - shape recognition

and classification. Attributed relational graphs,
semantic nets.
• Image - shape matching (Fourier descriptors,
moments, matching in scale space).
• Texture representation and recognition, statistical
and structural methods.
• Motion, motion detection, optical flow.
• Video
E.G.M. Petrakis Machine Vision (Introduction) 36
 Bibliography

• “Machine Vision”, Ramesh Jain, Rangachar

Kasturi, Brian G. Schunck, Mc Graw-Hill, 1995
(highly recommended!).
• "Image Processing, Analysis and Machine
Vision", Milan Sonka, Vaclav Hlavac,
Roger Boyle, PWS Publishing, Second
Edition.
• "Machine Vision, Theory, Algorithms,
Practicalities'', E. R. Davies, Academic Press,
1997.
E.G.M. Petrakis Machine Vision (Introduction) 37
• "Practical Computer Vision Using C'', J.
R. Parker, John Wiley & Sons Inc., 1994.
• Selected articles from the literature.
• Lecture notes
(http://www.intelligence.tuc/~petrakis)
• Webcourses (http://courses.ece.tuc.gr)

E.G.M. Petrakis Machine Vision (Introduction) 38

 Grading Scheme

• Final Exam (F): 40%, min 5

• Assignments (Α): 40%
• Two assignments
– Obligatory

E.G.M. Petrakis Machine Vision (Introduction) 39