Image analysis for biology

Thanks to Hao Yuan Kueh, Eugenio Marco, and Sivaraj Sivaramakrishnan 1) Why Image Analysis?......................................................................................................................3 Image Analysis Strategies................................................................................................................4 Layout of this tutorial.......................................................................................................................5 2) Basics............................................................................................................................................... im!rite"f# $filename$)...........................................................................................................................% im!rite"f# $filename$# format)..............................................................................................................% Image ty&es# 'ata classes an' image classes....................................................................................% Basic Segmentation using (hreshol'ing........................................................................................1) g * im2+!"f#()..................................................................................................................................1) Image histograms ......................................................................................................................1) ,, im * imrea'"$rice.&ng$)-................................................................................................................11 ...........................................................................................................................................................11 ,, imhist"im)-...................................................................................................................................11 ( * graythresh"im).............................................................................................................................11 ,, im * imrea'"$rice.&ng$),, im * mat2gray"im)-.......................................................................................................................12 ,, le.el * graythresh"im)-.................................................................................................................12 ,, im+ * im2+!"im#le.el)-................................................................................................................12 /L# num0 * +!la+el"f).......................................................................................................................12 1 * region&ro&s"L# &ro&erties)..........................................................................................................12 2etamor&h Stac3 4iles ..............................................................................................................14 56578IS5S...................................................................................................................................15 3) 8ontrast a'9ustments......................................................................................................................1: g * c;log"1<'ou+le"f)) g * 1.="1<"m.="'ou+le"f) < e&s)).>5)..................................................................................................1% 4) S&atial 4iltering..............................................................................................................................2) Smoothing filters............................................................................................................................22 5'ge 'etection filters......................................................................................................................24 La&lacian filter...............................................................................................................................24 2e'ian filter...................................................................................................................................25 56578IS5S...................................................................................................................................25 5) 2or&hological image &rocessing...................................................................................................2: 1ilation...........................................................................................................................................2: se * strel"sha&e# &arameters).............................................................................................................2? 56578IS5.................................................................................................................................2? 5rosion............................................................................................................................................31 56578IS5.................................................................................................................................32 @&ening an' closing.......................................................................................................................32

56578IS5.................................................................................................................................32 A''itional useful image &rocessing tools......................................................................................33 4illing holes................................................................................................................................33 8learing +or'er o+9ects..............................................................................................................34 ) Image Segmentation.......................................................................................................................34 5'ge 'etection................................................................................................................................34 ,, h * fs&ecial"$so+el$),, Ay * imfilter"im# h),, AB * imfilter"im# h$)-.....................................................................................................................35 ,, A * "AB.>2 < Ay.>2).>"1=2).........................................................................................................35 g * e'ge"f#Cmetho'C# &arameters)......................................................................................................3 ,, im * imrea'"Dactin2.tifC)..............................................................................................................3 ,, e'geEso+el * e'ge"im# $so+el$),, e'geE&re!itt * e'ge"im# $&re!itt$)-...............................................................................................3 BW * e'ge"I#$canny$#(F75SF#SIA2A)..........................................................................................3 2or&hological Watershe's.............................................................................................................3: L * !atershe'"f)................................................................................................................................3: ,, im * imrea'"$circles.tif$)-..............................................................................................................3% ,, A * +!'ist"im).............................................................................................................................3% ,, B * +!'ist"Gim)- .........................................................................................................................3% ,, 8 * HB...........................................................................................................................................3% ,, L * !atershe'"8)-........................................................................................................................3? ,, im"L ** )) * )............................................................................................................................................................3? ,, im * imrea'"$steel.tif$)..................................................................................................................3? ...........................................................................................................................................................3? ,, L * !atershe'"imcom&lement"im))-............................................................................................3? g * imhmin"f# F)...............................................................................................................................3? ,, im2 * imcom&lement"im),, im3 * imhmin"im2#2)),, L * !atershe'"im3)-......................................................................................................................3? 56578IS5S...................................................................................................................................4) :) Analysis of motion in +iological images........................................................................................43 1esigning Ara&hical Iser Interfaces in 2A(LAB.......................................................................43 ,, gui'e.............................................................................................................................................43 D(agC * &lanesli'er D2inC * 1 D2aBC * 1) DJalueC * 1.........................................................................................................................................4 Kymogra&hs...................................................................................................................................4% ,, st3 * st3rea'"$fluBings&in'le.st3$)-...............................................................................................4% c * im&rofile"I#Bi#yi)..........................................................................................................................4% /y#B#&0 * im&iBel ...............................................................................................................................4% 1ifference Images# 2aBimum Intensity Lro9ections......................................................................4? Image 8rossHcorrelation.................................................................................................................51 ,, + * normBcorr2"im1#im2)-....................................................................................................52

,, figure- mesh"+)-.....................................................................................................................52 ,, /yc#Bc0 * siMe"im2),, yoff * y H yc,, Boff * B H Bc-.........................................................................................................................52 ,, cim2 * Meros" yc<yoff# Bc<Boff# 3),, cim2"yoffN"yc<yoffH1)# BoffN"Bc<BoffH1)# 1) * mat2gray"im1),, cim2"1Nyc# 1NBc# 2) * mat2gray"im2),, imsho!"cim2)-......................................................................................................................52 56578IS5S...............................................................................................................................52 Larticle (rac3ing............................................................................................................................53 %) 754575O85S...............................................................................................................................54

1).................................Why Image Analysis?

Biological images contain a !ealth of o+9ects an' &atterns# !hich may con.ey information a+out un'erlying mechanism in +iology. (a3e a loo3 at the follo!ing microsco&y imagesN

(he left microsco&y image sho!s a fiel' of .ie! of tissueHculture cells. @ne can as3N ho! many cells are there in this fiel' of .ie!? What is the a.erage siMe? Fo! much 1OA is in each of the cells? Fo! are the microtu+ule an' actin cytos3eletons organiMe' s&atially? 4or the mo.ie of the s&ec3le' s&in'le on the right# one can as3N What is the 'istri+ution of &olymer mass in the s&in'le? What is the fluB rate? 1oes it 'e&en' on the &osition along the s&in'le? Where is monomer getting incor&orate' an' lost? Image &rocessing an' analysis &ro.i'es a means to eBtract an' Puantify o+9ects an' &atterns in image 'ata an' o+tain ans!ers to meaningful +iological Puestions. It offers t!o a'.antages o.er tra'itional more manual metho's of analysisN 1) Fuman .ision# !hile highly sensiti.e# can +e easily +iase' +y &reHconcei.e' notions of o+9ects an' conce&ts- automate' image analysis &ro.i'es an un+iase' a&&roach to eBtracting information from image 'ata an' testing hy&otheses. 2) @nce an imageHanalysis routine is 'e.ise'# it can +e a&&lie' to a large num+er of microsco&y images# facilitating the collection of large amounts of 'ata for statistical analysis.

Image Analysis Strategies

Image analysis in.ol.es the con.ersion of features an' o+9ects in image 'ata into Puantitati.e information a+out these measure' features an' attri+utes. 2icrosco&y images in +iology are often com&leB# noisy# artifactHla'en an' consePuently rePuire multi&le image &rocessing ste&s for the eBtraction of meaningful Puantitati.e information. An outline of a general strategy for image analysis is &resente' +elo!N 1) (he starting &oint in image analysis ty&ically in.ol.es a 'igital image acPuire' using a 881 camera. 7a! microsco&y images o+taine' on 'igital 881 cameras are su+9ect to .arious im&erfections of the image acPuisition setu&# such as noise at lo! light le.els# une.en illumination# 'efecti.e &iBels# etcQ We often nee' to first &rocess the image to correct for such 'efects an' also to enhance the contrast to accentuate features of interest in the image for su+sePuent analysis. In section II# !e intro'uce .arious image transformation an' s&atial filtering techniPues that can +e use' for this &ur&ose. 2) Fa.ing correcte' artifacts an' enhance' contrast in the images# !e can a&&ly .arious com&utational techniPues to eBtract features an' &atterns from the images. In the follo!ing section# !e 'escri+e .arious tools of mor&hological image &rocessing an' image segmentation that can +e use' for this &ur&ose. 3) After +iological im&ortant features ha.e +een segmente' from images# !e can then 'eri.e Puantitati.e information from these features an' o+9ects. 2A(LAB &ro.i'es a set of tools that can +e use' to measure the &ro&erties of regions- the matriB re&resentation of images in 2A(LAB also allo!s for easy mani&ulation of 'ata an' calculation of Puantities from microsco&y images. Fere is an outline of the &rocessN
1) Correct image acquisition artifacts, by contrast adjustment and spatial filtering

2) Extract features using segmentation and morphological operations

3) Quantify image data, measure properties of image features

Layout of this tutorial

(he goal of this tutorial is to ta3e you through the aforementione' &rocess of eBtracting 'ata from ra! microsco&y images in 2A(LAB. In Section I# you !ill learn to loa' an' sa.e images an' con.ert +et!een 'ata ty&es in 2A(LAB. Rou !ill also go through a sim&le eBam&le of segmentation an' 'ata eBtraction from segmentation# using as an eBam&le an image of rice grains o.er a 'ar3 +ac3groun'. (he sim&le eBam&le of segmentation co.ere' in Section I !ill highlight some image artifacts an' im&erfections that !ill &re.ent accurate eBtraction of 'ata from ra! images. Sections IIHIJ co.er .arious techniPues that can correct for 'efects in either ra! images or segmente' images. Section II 'escri+es techniPues of contrast a'9ustment# !hich can +e use' to im&ro.e the contrast of features of interest to facilitate the segmentation &rocess. Section III 'iscusses the +asic &rinci&les an' some sim&le a&&lications of s&atial filtering. S&atial filters ser.e many 'ifferent &ur&oses in image anaylsis# from smoothing to shar&ening +oun'aries to 'etecting e'ges. Section IJ 'iscusses mor&hological image &rocessing# !hich is a &o!erful techniPue for correcting 'efects in segmente' images. 5ach section 'iscusses +oth the +asic math an' i'eas +ehin' these techniPues# an' also gi.es &ractical im&lementation of the tools in 2A(LAB. Section J 'escri+es t!o a''itional segmentation techniPues S 1) segmentation +ase' on e'ge 'etection# an' 2) segmentation +y the !atershe' algorithm. (hese techniPues are com&lementary to threshol'H+ase' segmentation an' may +ecome useful for 'ifferent ty&es of images. Section J then closes !ith a set of &ro+lems +ase' on +iological images that gi.e you the o&&ortunity to re.ie! all the techniPues you ha.e learnt an' also integrate them in a sim&le image &rocessing routine. (hey are not easy# +ut you are strongly encourage' to try them an' &erha&s +ang your hea's against the !all in the &rocessT (hey !ill &ro+a+ly gi.e you a goo' taste of !hat a realHlife image anaylsis &ro+lem is li3e. Aoo' luc3T

2)

Basics

In this section !e &resent the +asics of !or3ing !ith images in 2atla+. We !ill see ho! to rea'# 'is&lay# !rite an' con.ert images. We !ill also tal3 a+out the !ay images are re&resente' in 2atla+ an' ho! to con.ert +et!een the 'ifferent ty&es. (he 2atla+ comman' for rea'ing an image is imread('filename') Oote that !e su&&ress the out&ut !ith a semicolon# other!ise !e !ill get in the out&ut all the num+ers that ma3e the image. 'filename' is the name of a file in the current 'irectory or the name of a file inclu'ing the full &ath. (ry >> f = imread('chest-xray.tif'); We no! ha.e an array f !here the image is store' >> whos f Name f

Size 494x600

Bytes 296400

Class uint8 array

Grand total is 296400 elements using 296400 bytes f is an array of class uint8# an' siMe 4?4B )). (hat means 4?4 ro!s an' )) columns. We can see some of this information !ith the follo!ing comman's >> size(f) ans = 494 600

>> class(f) ans = uint8 We !ill tal3 later on a+out the 'ifferent image classes. Sometimes it is useful to 'etermine the num+er of ro!s an' columns in an image. We can achie.e this +y means of >> [M, N] = size(f);

(o 'is&lay the image !e use imshow >> imshow(f) Rou !ill get a !in'o! similar to this

Oote that in the figure tool+ar !e ha.e +uttons that allo! us to Moom &arts of the image. (he syntaB imshow(f, [low high])'is&lays all &iBels !ith .alues less than or ePual to low as +lac3# all &iBels !ith .alues greater or ePual to high as !hite. (ry >> imshow(f,[10 50]) 4inally# >> imshow(f,[]) sets the .aria+le low to the minimum .alue of array f an' high to its maBimum .alue. (his is .ery useful for 'is&laying images that ha.e a lo! 'ynamic range. (his occurs .ery frePuently !ith 1 H+it images from a microsco&e. We can also 'is&lay &ortions of an image +y s&ecifying the range >> imshow(f(200:260,150:220)) Another matla+ tool a.aila+le to 'is&lay images an' 'o sim&le image mani&ulations is imtool. (ry

>> imtool(f) In the figure !in'o! !e ha.e no! a.aila+le the follo!ing toolsN o.er.ie!# &iBel region# image information# a'9ust contrast an' Moom. (ry them. Images can +e !ritten to 'is3 using the function imwrite. Its format is imwrite(f, 'filename') !ith this syntaB# filename must inclu'e the file format eBtension. Alternati.ely imwrite(f, 'filename', format) sa.es f using format. 4or eBam&le >> imwrite(f, 'test', 'jpeg', 'quality', 25) In the hel& you can fin' more information a+out a.aila+le formats an' their o&tions.

Image types, data classes and image classes

(here are 'ifferent image ty&es an' image classes a.aila+le in 2A(LAB. (he t!o &rimary image ty&es you !ill +e !or3ing !ith are as follo!s Intensity images o uint1 /)# 55350 "881 cameras on microsco&es) o uint%/)# 2550 "4rom your stan'ar' 'igital camera) o 'ou+le /-1)3)%# 1)3)%0 Binary images "+lac3 an' !hite) o logical# ) or 1 7a! images ty&ically come in the form of an unsigne' integer "uint16 'enotes 1 H+it unsigne' integer# an' uint8 'enotes %H+it unsigne' integer). Fo!e.er floatingH&oint o&erations "mathematical o&erations that in.ol.e a 'ecimal &oint# such as log(a)) can only +e 'one !ith arrays of class double. Fence# to !or3 on a ra! image# first con.ert it from uint16 or uint8 to double using the double functionN >> f = imread('actin.tif'); >> g = double(f);

Oo! ty&e >> whos; to see the 'ifferent 'ata ty&es associate' !ith each .aria+le. Oote that !hile the 'ata ty&e changes# the actual num+ers after the con.ersion remain the same. 2any 2A(LAB image &rocessing o&erations o&erate un'er the assum&tion that the image is scale' to the range /)#10. 4or instance# !hen imshow 'is&lays an double image# it 'is&lays an intensity of ) as +lac3 an' 1 as !hite. Rou can automatically create a scale' double image using mat2grayN >> h = mat2gray(g); UuestionN Fo! !oul' you !rite a function to manually scale a double image? 8ertain image &rocessing comman's only !or3 !ith scale' 'ou+le images. 4inally# !e can con.ert an intensity image into a +inary image using the comman' im2bw(f, T), !here T is a threshol' in the range /)# 10. 2atla+ con.erts f to class double# an' then sets to ) the .alues +elo! T an' to 1 the .alues a+o.e T. (he result is of class logical. See the follo!ing eBam&le. We !ish to con.ert the follo!ing double image >> f = [1 2; 3 4] f = 1 2 3 4 to +inary such that .alues 1 an' 2 +ecome ) an' the other t!o .alues +ecome 1. 4irst !e con.ert it to the range /)# 10 >> g = mat2gray(f) g = 0 0.6667 0.3333 1.0000

We can con.ert the &re.ious image to a +inary one using a threshol'# say# of .alue ). N >> gb = im2bw(g, 0.6) gb = 0 1 0 1

Oote that !e can o+tain the same result using relational o&erators >> gb = f > 2 gb = 0 1 0 1

Binary images generate' +y threshol'ing often form the +asis for eBtracting mor&hological features in microsco&y images. In the neBt section# !e !ill eBtract some +asic Puantitati.e information a+out o+9ects in an image +y first using threshol'ing to generate a +inary image an' then using the regionprops comman' to eBtract Puantitati.e information from the +inary image. UuestionN Fo! might you re&resent a color image?

Basic Segmentation using Thresholding

2any +iological images com&rise of light o+9ects o.er a constant 'ar3 +ac3groun' "es&ecially those o+taine' using fluorescence microsco&y)# in such a !ay that o+9ect an' +ac3groun' &iBels ha.e gray le.els grou&e' into t!o 'ominant mo'es. @ne o+.ious !ay to eBtract the o+9ects from the +ac3groun' is to select a threshol' ( that se&arates these mo'esN g(x,y) * * 1 ) if f(x,y) , T other!ise

!here g(x,y) is the threshol'e' +inary image of f(x,y). We can im&lement the threshol'ing o&eration in 2A(LAB +y the follo!ing functionN g im2bw(f,T)

(he first argument f gi.es the in&ut image# an' the secon' argument T gi.es the threshol' .alue.

Image histograms
We nee' to choose a threshol' .alue ( that &ro&erly se&arates light o+9ects from the 'ar3 +ac3groun'. Image histograms &ro.i'e a means to .isualiMe the 'istri+ution of grayscale intensity .alues in the entire image. (hey are useful for estimating +ac3groun' .alues# 'etermining threshol's# an' for .isualiMing the effect of contrast a'9ustments on the image "neBt section). (he matla+ function to .isualiMe image histograms is imhist >> f = imread('chest-xray.tif'); >> imhist(f);

(he histogram has 25 +ins +y 'efault. (he follo!ing comman' ma3es 2) +ins >> imhist(f,20);

A goo' .alue for ( can +e o+taine' +y .isually ins&ecting the image histogram o+taine' using the imhist comman'N !! im imread('ri"e#png')$

!! imhist(im)$ Base' on the histogram# &ic3 a grayscale .alue manually that se&arates the light rice grains from the 'ar3 +ac3groun'. (hen threshol' the image an' 'is&lay the results. 2A(LAB &ro.i'es a function graythresh that automatically com&utes a threshol' .alueN T graythresh(im)

!here im is the in&ut image an' T is the resulting threshol'. graythresh calculates the threshol' .alue +y essentially maBimiMing the !eighte' 'istances +et!een the glo+al mean of the image histogram an' the means of the +ac3groun' an' foregroun' intensity &iBels.

EXAM !E In this eBam&le# !e threshol' the image of rice grains o&ene' a+o.eN !! im !! im imread('ri"e#png')$ mat2gray(im)$

8alculate the threshol' .alueN !! le%el graythresh(im)$

an' create a ne! +inary image using the o+taine' threshol' .alueN !! imb im2bw(im,le%el)$

Oote that the threshol'ing o&eration segments the rice grains Puite !ell. Fo!e.er# a &ro+lem in this image is that the rice grains near the +ottom of the image arenCt segmente' !ell S the +ac3groun' is une.en an' is lo! at the +ottom# lea'ing to incorrect segmentation. WeCll see a !ay to correct for this une.en +ac3groun' using image &rocessing later. Ising the +inary image# !e can then calculate region &ro&erties of o+9ects in the image# such as area# 'iameter# etcQ An o+9ect in a +inary image is a set of !hite &iBels "ones) that are connecte' to each other. We can enumerate all the o+9ects in the figure using the bwlabel comman'N [&, num] bwlabel(f)

!here L gi.es the la+ele' image# an' num gi.es the num+er of o+9ects. (o la+el the +inary image of the rice grains# ty&eN >> [L, N] = bwlabel(imb); Oo! loo3 at the la+ele' image L using imtool. What are the .alues of the o+9ects in the &iBels? A'9ust the contrast to see the range of intensity .alues in the image. @nce the image has +een la+ele'# use the regionprops comman' to o+tain Puantitati.e information a+out the o+9ectsN ' regionprops(&, properties)

(hereCs a lot of useful statistical information a+out o+9ects that can +e eBtracte' using regionprops. FereCs a listN
'Area' '%ounding%ox' 'Eu"er#umber' 'Extent' '$rientation' 'Perimeter'

'Centroid' 'Con(exArea' 'Con(ex*u""' 'Con(ex'mage' 'Eccentricity'

'Extrema' ')i""edArea' ')i""ed'mage' ''mage' 'Ma,orAxis!engt-'

& ixe"'dx!ist& ' ixe"!ist' '+o"idity' 'SubarrayIdx'

'Equi(.iameter' 'MinorAxis!engt-'

5Btract the area an' &erimeter of in'i.i'ual o+9ects in the la+ele' image as follo!sN >> D = regionprops(L, 'area', 'perimeter'); NOTE: (he information in D is store' in an o+9ect calle' a structure array. A structure array is a .aria+le in 2A(LAB that contains multi&le fiel's for storage of information. Rou can access the fiel' information in D as follo!sN >> D D = 151x1 struct array with fields: Area Perimeter Access an in'i.i'ual element in the structure array +y referring to its in'eB in &arenthesisN

>> D(1) ans = Area: 145 Perimeter: 58.2843

TIP:

sing structure arra!s in M"T#"$

Access an in'i.i'ual fiel' !ithin in the structure array +y referring to its name after a 'ot ".) . 4or instanceN >> D(1).Area

Rou can assign .alues to structure arrays using the assignment o&erator "*)N >> D(1).Area = 888; "n.+. !e 'o not con'one 'ata mani&ulation) An' 'efine ne! fiel's +y using a ne! fiel' name 'uring assignmentN >> D(1).test = 1 (o get an array of .alues from one fiel' in the structure '# ty&eN >> w = [D.Area] Rou can get the mean# stan'ar' 'e.iation an' the full histogram of the Areas from w as follo!sN >> meanarea = mean(w) >> stdarea = std(w) >> hist(w) Oo! loo3 at the histogram. 1i' the histogram loo3 the !ay you eB&ecte'? Fo! 'o you account for the features in the histogram? "Why are there so many .alues less than 5)? Why are there .alues aroun' 45)? Why is there a s&rea' of .alues +et!een 5) an' 2))?)

Metamorph Stack Files

@ne &o&ular format for microsco&y mo.ies &ro'uce' +y 2etamor&h are .stk files# !hich are similar to tiff files# +ut contain a''itional information li3e the eB&osure time# timestam&# stage &osition# etc. 4ranVois Oe'elec has create' a matla+ function calle' stkread eBtracts the information from the files "foun' in the materials 'irectory). (ry thisN >> S = stkread('yeastcdc42.stk') S = 1x31 struct array with fields: filename NewSubfiletype width height bits photo_type info x_resolution y_resolution resolution_unit software datetime

MM_private1 MM_stack MM_wavelength MM_private2 data S is a structure array !ith the num+er of elements ePual to the num+er of frames in the stac3. (he images are store' in the fiel' data. (o see the first frame !e can 'o >> imshow(S(1).data,[]) the image acPuisition times "in ms) are store' in the fiel' MM_stack. We can eBtract it for all frames using for index = 1:length(S) S(index).timestamp=S(index).MM_stack(4)-S(1).MM_stack(4); end

EXERCISES
1. Loa' the images pollen.tif an' bone-scan-GE.tif. 1is&lay them !ith imtool. Loo3 at the &iBel .alues in some area. 8on.ert them into double. 1is&lay an' loo3 at the .alues again. 2. Ise threshol'ing an' regionprops to count the nuclei in the follo!ing imagesN
nuclei1.tif nuclei2.tif

Fo! many nuclei are there in the &icture? What is the a.erage area of a nucleus? Which image !as easier? Why? Fo! can the segmentation +e im&ro.e'? WeCll see 'ifferent !ays shortly. (ry to !rite your &rograms in a generic fashion. 8omment an' may+e e.en inclu'e functions. We !ill eB&an' the &rogram you are !riting no! in future eBercises. Aoo' ha+its no! !ill ma3e changing the &rogram later easyT 3a. 5liminate W+a'X rice granules from the rice.png &icture. Ise region&ro&s to i'entify the rice you !ant to eliminate. Ise the fin' function to change eliminate these grains from the la+ele' image. "Rou !ill &ro+a+ly !ant to !rite a loo& that goes through each o+9ect in the la+ele' file) 8om&are the original image to your ne! image "imsho! an' hist). Fo! !ell 'i' you 'o? Uualitati.ely# ho! many real grain got eliminate'=ho! many +a' grains

remain? +. 7ela+el your resulting image from 3a so that your o+9ects are la+el 1 to total o+9ects. 8om&are the mean an' st' of the area of these o+9ect .ersus the original o+9ects. c. Oo! calculate the total intensity of each rice grain. Rou !ill ha.e to use a com+ination of the original rice image an' your ne! la+ele' image.

3)

Contrast adjustments

@ften# images ha.e a lo! 'ynamic range an' many of its features are 'ifficult to see. We !ill &resent 'ifferent intensity transformations that !ill im&ro.e the a&&earance of the images. Im&ro.ing the a&&earance of an image 'oes not merely ser.e an aesthetic role S often# it can hel& im&ro.e the &erformance of image segmentation algorithms an' feature recognition. 1uring contrast a'9ustment# the intensity .alue of each &iBel in the ra! image is transforme' using a transfer function to form a contrastHa'9uste' image. (he most common transfer function is the gamma contrast a'9ustmentN

Fere low(in an' low(high gi.e the lo! an' high grayscale intensity .alues for the contrast a'9ustment# an' gamma gi.es the eB&onent for the transfer function. (he +asic matla+ function for contrast a'9ustment is imadjust. It has the syntaBN g = imadjust(f,[low_in high_in],[low_out high_out],gamma) Without the o&tional arguments (g = imadjust(f))# the out&ut image has 1Y of the 'ata in f is saturate' at lo! an' high intensities. (ry the follo!ing comman's "negati.e image) >> f = imread('chest-xray.tif'); >> imshow(f) >> g1 = imadjust(f); >> imshow(g1) >> g2 = imadjust(f, [0 1], [1 0]); >> figure,imshow(g2)

Fo! 'oes changing the .alue of gamma affect the image? What ty&es of images are 'ifferent .alues of gamma goo' for? EXE/C'+E 8hec3 out the 1OA an' tu+ulin staining of these mammalian cancer cell linesN )2*+ Tubulin#tif )2*+ 'api#tif 4in' also the maBimum an' minimum of the images# an' try to a&&ly 'ifferent gamma a'9ustments to the images. What features of the images are accentuate' 'uring each transformation? (he negati.e of an image can +e o+taine' !ith >> g = imcomplement(f); (ry these other comman's >> >> >> >> g3 = imadjust(f, [0 0.2], [0 1]); figure,imshow(g3) g4 = imadjust(f, [ ], [ ], .2); figure,imshow(g4)

(!o other useful intensity transformations are the logarithmic an' contrastHstretching. (hey can +e im&lemente' in matla+ !ith the functionsN g g ",log(1-double(f)) 1#.(1-(m#.(double(f) - eps))#/0)

In the follo!ing figure !e can see the sha&e of the contrastHstretching transformation. (he .alue of the threshol' gi.en +y m an' the stee&ness of the threshol' is gi.en +y 5

In the limiting case of 0 !! 1# the image is threshol'e' at the intensity m. (ry the follo!ing comman's

>> f = imread('spectrum.tif'); >> imshow(f) >> g = im2uint8(mat2gray(log(1+double(f)))); >> figure, imshow(g) EXE/C'+E+ 1. 8reate a function that 'oes a contrast stretching transformation for 'ifferent .alues of m an' 0. A&&ly it to spe"trum#tif. Also try a logarithmic transformation. 2. Loa' the images pollen#tif an' bone1s"an120#tif. 1is&lay them !ith imtool. (ransform the images an' 'is&lay them again.

4)

S atial !iltering

A large .ariety of image &rocessing tas3s can +e accom&lishe' +y a techniPue calle' s&atial filtering. S&atial filtering in.ol.es a mas3# consists of an array of .alues "a-i) an' has a center "gray)N a b c d e f g h i (he mas3 is &lace' on the image of interestN 2ASK

an' translate' across all &ossi+le &iBel &ositions on the image. A ne! "filtere') image is &ro'uce' +y re&lacing the intensity .alue at the center +y a linear com+ination of the intensity .alues of the center &iBel an' all neigh+oring &iBels co.ere' +y the mas3. (he coefficients "aHi) in the mas3 array gi.e the !eights of each &iBel in the linear com+ination. Fere is an eBam&leN

4ilters can &erform many 'ifferent functions. (he function of the filter is essentially 'etermine' +y the .alue of the mas3 coefficients "a-i). In this tutorial# !e consi'er &rimarily t!o classes of filtersN 1) Smoothing filters# !hich re'uce noise in an image# an' 2) e'ge 'etection "or 'eri.ati.e) filters# !hich can 'etect o+9ect +or'ers. (he +asic comman' in matla+ isN g = imfilter(f, w) (he tool+oB su&&orts a num+er of &re'etermine' 2H1 linear s&atial filters. (he comman' to generate them is fs&ecialN w = fspecial(type, parameters)

(he +est !ay to get an i'ea for !hat a gi.en filter is 'oing is to loo3 at its sha&e.

Smoothing filters
(he follo!ing is a 3B3 a.erage filterN 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9 1/ 9

!hich can +e o+taine' using the follo!ing comman'N >> fspecial('average',3) (he a.erage filter re&laces a .alue at the center of the mas3 +y an a.erage of all grayscale intensity .alues in the mas3. (he a.erage filter ten's to smooth out shar& image features smaller than the siMe of the filter. It is im&ortant to &ic3 a filter siMe that smooths out the noise in your image# +ut at the same time &reser.es the features of interest. In the follo!ing eBam&le !e !ill a&&ly an a.erage filter of siMe 31 to a &attern an' see the results of using 'ifferent o&tions. >> f = imread('original_test_pattern.tif '); >> f2 = im2double(f); >> >> >> >> imshow(f2) w = ones(31); gd = imfilter(f2, w); figure,imshow(gd, [])

Oote the &resence of +lurring at the image +oun'aries. When the filter reaches a +oun'ary# it 'oes not ha.e enough &iBel information to filter near the image +oun'aries. By 'efault# 2A(LAB &a's the +oun'aries of the image !ith Meros an' runs the filter across this eB&an'e' image# gi.ing rise to the +lurring o+ser.e'. It is &ossi+le to s&ecify alternate &a''ing con'itions. 4or instance# try the o&tions replicate or symmetric in the filteringN >> gd2 = imfilter(f2, w, 'symmetric'); >> figure,imshow(gd2, []) A Aaussian filter is one that has &ea3 at the center of the mas3# an' has a Aaussian 'ecay a!ay from center. 8hec3 out# for instanceN fspecial(gaussian, 10, 5) (he Aaussian filter also &erforms a.eraging# +ut &erforms a !eighte' a.erage to gi.e more em&hasis on &iBels near the center of the mas3. (he Aaussian filter is more commonly use' for smoothing than the a.erage filter- it is ra'ially symmetric an' is hence less li3ely to intro'uce image artifacts such as shar& 'irectional 'iscontinuities. A5O57AL 7IL5 Z1N In general# the sum of all .alues in mas3 for a smoothing filter must ePual one. Why?

Edge detection filters

LetCs ta3e a loo3 at the Lre!itt filterN

>> f = fspecial('prewitt') 1 0 1 1 0 1 1 0 1

What 'oes it 'o? Where 'oes it gi.e maBimum contrast? (ry out the filter on the rice grain eBam&le of Section I. Where is the grayscale intensity maBimum? minimum? Mero? A5O57AL Z2N Aenerally s&ea3ing# a filter &ro.i'es maBimum contrast in regions of the image that resem+le itself. Oo! fli& the filter aroun' ?) 'egrees +y trans&ositionN >> f = f (ry a&&lying this filter to the rice grain eBam&le again. (he Lre!itt filter an' the So+el filter +elong to a class of filters calle' DfirstH'eri.ati.eC filters# !hich essentially calculate the slo&e of grayscale intensities in an image in a gi.en 'irection. (hey gi.e a maBimum .alue "or minimum .alue) at regions !ith 'iscontinuities in grayscale .alues an' are frePuently use' in e'ge 'etection. In section J# !e !ill com+ine use e'ge 'etection as a means to recogniMe close' +oun'aries of o+9ects in an image# an' use that information to segment the image. A5O57AL 7IL5 Z3N In general# the sum of all .alues in mas3 for a smoothing filter must ePual Mero. Why?

Laplacian filter
>> f = fspecial('laplacian',0); (he La&lacian filter loo3s li3e thisN 1 1 1 8 1 1

What features 'oes this filter &ic3 u&? (he sum of all .alues in this mas3 is Mero. What 'oes this mean? 7un this filter on moon.tif. Where 'o the regions of maBimum contrast = minimum contrast occur? (he La&lacian filter mathematically calculates the local cur.ature of grayscale intensities in an image an' ser.es as a metho' for &ic3ing u& e'ges that is in'e&en'ent of e'ge orientation. Ise the imtool comman' to eBamine the e'ges of the filtere' image of moon.tif. Oote that !hen you a&&roach a the e'ge of a +right o+9ect from the outsi'e# the &iBel intensity .alues first increase# then 'ecrease. (his change reflects the change in cur.ature eB&ecte' along a 'iscontinuity +et!een a +right o+9ect an' a 'ar3 +ac3groun'. Oo! form a ne! image +y su+tracting the la&lacian filtere' image from the original image of moon.tif. Fo! 'oes the resultant image 'iffer from the original? (his com+ination of filtering o&erations is a commonly use' techniPue an' is "counterintuiti.ely) referre' to as an [unshar& mas3C. Why 'oes it &ro'uce this effect? FIO(N (hin3 a+out !hat ha&&ens along an e'ge !hen you 'o the image su+traction.

Median filter
All the &re.ious filters are calle' linear# +ecause the final &iBel .alue is the result of a linear com+ination of &iBel .alues from the original image. Oonlinear filters# on the contrary# assign nonlinear com+inations of the &iBel .alues to the filtere' image. 2A(LAB has functions to im&lement nonlinear filters# +ut !e !onCt stu'y them in 'etail. We consi'er# an im&ortant nonlinear filter# the me'ian filter# !hich has its o!n im&lementation in matla+. 2atla+ also has a function to generate noise in an image. We !ill see in the follo!ing eBam&le ho! to corru&t an image "!ith imnoise) an' ho! to try to restore it !ith a me'ian filter. g = medfilt2(f, [m n]) (ry the follo!ing >> >> >> >> >> >> f = imread('ckt-board-orig.tif'); imshow(f) fn = imnoise(f, 'salt & pepper', 0.2); figure, imshow(fn) gm = medfilt2(fn); figure, imshow(gm)

EXERCISES

1. (ry a''ing 'ifferent ty&es of noise to ckt-board-orig.tif. Loo3 in the hel& for the 'ifferent o&tions in imnoise. (hen choose the a&&ro&riate filter to remo.e the noise. 2. 7e&eat for pollen#tif.

")

#or hological image

rocessing

2athematical mor&hology is a &o!erful tool that can +e use' to eBtract features an' com&onents from an image. It is often use' to &reH&rocess or &ostH&rocess images to facilitate analysis. In mor&hology# a small sha&e "structuring element) is translate' across the image 'uring the course of &rocessing. 8ertain mathematical logic o&erations are &erforme' on the image using the structuring element to generate the &rocesse' image. In this section# !e first intro'uce 'ilation an' erosion# t!o fun'amental o&erations in mathematical mor&hology. We then 'escri+e more com&leB mor&hological o&erations o+taine' +y com+ining erosion an' 'ilation. We 'escri+e a han'ful of mor&hological tools that are es&ecially useful in image analysis.

ilation
1ilation is an o&eration that gro!s o+9ects in a +inary image. (he thic3ening is controlle' +y a small structuring element. In the follo!ing figure you can see the structuring element on the right an' the result after a&&lying 'ilation on a rectangle.

(ry the follo!ing eBam&le >> A = imread('broken-text.tif');

>> B = [0 1 0; 1 1 1; 0 1 0]; >> A2 = imdilate(A, B); >> imshow(A), figure, imshow(A2) 1. Aenerate 'ifferent structuring elements an' a&&ly them to broken-text.tif. (ry also a&&lying se.eral 'ilations consecuti.ely. (he strel function can +e use' to +uil' structuring elements in a .ariety of sha&es. (hey are store' in a s&ecial format that ma3es use of 'ecom&ositions of the elements into sim&ler structuring elements. Its syntaB isN se strel(shape, parameters)

See the ta+le for &ossi+le structuring elements. >> se = strel('diamond', 5)

EXERCISE
1. Loa' 'broken-text.tif'. 1ilate the image !ith 'ifferent structuring elements. An' see the 'ifferent results. 2. 1o the same !ith noisy1fingerprint#tif

Erosion
5rosion shrin3s or thins o+9ects in a +inary image. After erosion the only &iBels that sur.i.e are those !here the structuring element fits entirely in the foregroun'.

In the follo!ing eBam&le one can see ho! 'ifferent erosions affect the features of an image >> A = imread('wirebond-mask.tif'); >> imshow(A) >> se = strel('disk',10);

>> A2 = imerode(A,se); >> figure,imshow(A2) (ry 'is3s of ra'ius 5 an' 2).

EXERCISE
1. Loa' noisy1fingerprint#tif an' broken-text.tif. A&&ly 'ifferent erosions.

!pening and closing

8om+inations of mor&hological o&erations can +e .ery useful an' hel& remo.e many artifacts &resent in images. (his !ill +ecome .ery useful after segmenting an image. (he first o&eration !e !ill see is o&ening# !hich is an erosion follo!e' +y 'ilation. @&ening smoothes o+9ect contours# +rea3s thin connections an' remo.es thin &rotrusions. After o&ening# all o+9ects smaller than the structuring element !ill 'isa&&ear. 8losing is a 'ilation follo!e' +y erosion. 8losing smoothes o+9ect contours# 9oins narro! +rea3s# fills long thin gulfs an' fills holes smaller than the structuring element. (he follo!ing eBam&le !ill sho! the utility of o&ening an' closing. >> >> >> >> >> >> >> >> >> f = imread('shapes.tif'); imshow(f) se = strel('square',20); fo = imopen(f, se); figure, imshow(fo) fc = imclose(f, se); figure, imshow(fc) foc = imclose(fo, se); figure, imshow(foc)

EXERCISE
1. 7eo&en rice.&ng Ise imo&en to eliminate all the rice grains. 8on.ert all num+ers so that they are +et!een ) an' 1 Ise mesh to .ie! image "see 'etails of DmeshC comman' in hel& section) Ise this ne! array to mo'ify the original rice.&ng. Llug this +ac3 into your co'e from eBercise 3

"the &re.ious rice grain eBercise). Is it easier or har'er to correctly &ic3 all the rice grains.? Fo! 'oes the Puantitation com&are? 2. Loa' an' a&&ly o&ening an' closing to noisy1fingerprint#tif an' brokentext.tif.

Additional useful image processing tools

A .ery useful mor&hological transformation to su+tract the +ac3groun' from an image is the so calle' to&hat. (o&hat is the su+traction of an o&ene' image from the original. @ne can 'o o&ening in gray images# remo.ing all features smaller than the structuring element. In the follo!ing eBam&le !e !ill sho! its usefulness >> >> >> >> >> >> >> >> >> I = imread('rice.png'); imshow(I) background = imopen(I,strel('disk',15)); figure, imshow(background) figure surf(double(background(1:8:end,1:8:end))),zlim([0 255]); set(gca,'ydir','reverse'); I2 = imsubtract(I,background); figure, imshow(I2)

(he same result can +e o+taine' using >> I2 = imtophat(I, strel('disk',15)); >> figure, imshow(I2) What ha&&ens !hen you change the siMe of the structuring element? What criteria shoul' you use to choose the a&&ro&riate siMe?

Filling holes
4rePuently# after some mor&hological o&eration !e nee' to fill the holes in a +inary image. 4or eBam&le# !e 'etect the +oun'ary of a cell an' !ant to o+tain an o+9ect !hich is fille' an' co.ers the cell. In this eBam&le !e !ill see its effect in teBt.

g = imfill(f, 'holes') >> >> >> >> f = imread('book-text.tif'); imshow(f) f2 = imfill(f, 'holes'); figure, imshow(f2)

Clearing border objects

After segmenting an image there are usually o+9ects touching the +or'er of the image. Since !e can not o+tain com&lete information a+out them it is usually the +est to remo.e them. imclearborder is the matla+ function to remo.e o+9ects touching the +or'er. g = imclearborder(f) >> >> >> >> f = imread('book-text.tif'); imshow(f) f2 = imclearborder(f); figure, imshow(f2)

$)

Image Segmentation

Segmentation refers to the &rocess in !hich an image is su+'i.i'e' into constituent regions or o+9ects. (hese o+9ects can +e further &rocesse' or analyMe' for the eBtraction of Puantitati.e information. Biological image 'ata is usually messy an' noisy# an' as a result 'ifficult to segment &ro&erly. 2ulti&le image &rocessing ste&s are often rePuire' in the &rocess of segmentation. We often com+ine segmentation !ith .arious mor&hological &rocessing an' filtering techniPues 'escri+e' a+o.e to achie.e accurate an' ro+ust segmentation of an image. We loo3e' at a +asic segmentation techniPue earlier S threshol'ing. In this section# !e loo3 at other com&lementary segmentation techniPues that may +e useful in 'ifferent situations.

Edge detection
@ne !ay to fin' +oun'aries of o+9ects is to 'etect 'iscontinuities in intensity .alues at the e'ge of a region. (hese 'iscontinuities can +e foun' +y calculating the first an'=or secon' or'er 'eri.ati.es of an image. (he first 'eri.ati.e of choice in image &rocessing is the gra'ient# 'efine' as the .ectorN grad f = [Gx Gy] !here Gx = df/dx an' Gy = df/dy are the &artial 'eri.ati.es in the horiMontal an' .ertical 'irections of the image. (he magnitu'e of this .ector is

|grad f| = (Gx2 + Gy2)1/2 (he gra'ient .ector &oints in the 'irection of stee&est ascent. (he angle of stee&est ascent is gi.en +yN a(x,y) = ta -1(Gy/Gx) We can estimate the 'eri.ati.es Gx an' Gy 'igitally +y linearly filtering the image !ith the follo!ing 3 +y 3 3ernelsN

(he Lre!itt an' So+el o&erators are among the most use' in &ractice for com&uting 'igital gra'ients. (he Lre!itt mas3s are sim&ler to im&lement than the So+el mas3s# +ut the latter ha.e slightly su&erior noiseHsu&&ression characteristics. (o gain an intuition for !hat a gra'ient image loo3s li3e# try o&ening some grayscale images into im an' creating a gra'ient image as follo!sN !! h !! 2y !! 23 fspe"ial('sobel')$ imfilter(im, h)$ imfilter(im, h')$

We can also .isualiMe the magnitu'e of the gra'ientN !! 2 (23#/2 - 2y#/2)#/(1.2)

Oote that the magnitu'e of the gra'ient is high at an e'ge# !here the intensity of a &iBel changes ra&i'ly +et!een neigh+oring &iBels. A threshol'ing o&eration is then a&&lie' to the image 2 to segment the +oun'ary &iBels. @+9ects can then +e i'entifie' +y filling in the holes of 'etecte' close' contours using the imfill function "see &re.ious section) 2A(LAB &ro.i'es a general e'ge 'etection comman' that is sensiti.e to horiMontal or .ertical e'ges or +oth. (he general syntaB for this function is g edge(f,4method4, parameters)

Where f is the in&ut image# method gi.es to the e'ge 'etection algorithm use'# an' parameters gi.es the &arameters that accom&any the algorithm. g is +inary image !here e'ges ha.e a .alue of 1. EXAM !E LetCs a&&ly e'ge 'etection to the segmentation of actin filamentsN !! im imread(5a"tin2#tif4)

4irst try to use threshol'ing "in the first section) to segment the actin filaments. LetCs run the t!o e'ge 'etectors on the microtu+ules image for com&arisonN !! edge(sobel edge(im, 'sobel')$ !! edge(prewitt edge(im, 'prewitt')$ (he most &o!erful e'ge 'etector is the 8anny e'ge 'etector &ro.i'e' +y the function edge. (he 8anny e'ge 'etector incor&orates information a+out the 'irection of the image gra'ient in 'etermining !hether a gi.en &iBel is an e'ge &iBel. It also has t!o threshol's# a !ea3 threshol' (1 an' a strong threshol' (2 , (1. LiBels a+o.e the !ea3 threshol' count as an e'ge only if there are &iBels a+o.e the strong threshol' a'9acent to them. (he 8anny e'ge 'etector gi.es goo' e'ge 'etection for +iological images# !hich are ty&ically noisy an' ha.e less !ellH'efine' e'ges. 67 edge(8,'"anny',T9:0+9,+82;<)

Where T9:0+ is a t!o element .ector in !hich the first element is the lo! threshol'# an' the secon' element is the high threshol'. +82;< is the stan'ar' 'e.iation of a Aaussian filter that is a&&lie' to the image &rior to e'ge 'etection. (ry a&&lying the canny e'ge 'etector the microtu+ule image# an' &lay aroun' 'ifferent .alues of the threshol' an' smoothing. 5'ge 'etection can +e use' in con9unction !ith hole filling "imfill) to recogniMe an' segment o+9ects !ith !ell 'efine' +oun'aries.

Morphological "atersheds
In the &re.ious sections# !e 'iscusse' segmentation +ase' on 1) 'etection of 'iscontinuities in an image# an' 2) threshol'ing. 2or&hological !atershe's &ro.i'e a com&lementary a&&roach to the segmentation of o+9ects. It is es&ecially useful for segmenting o+9ects that are touching one another. (o un'erstan' the !atershe' transform S !e .ie! a grayscale image as a to&ological surface# !here the .alues of f(3,y) corres&on' to heightsN

8onsi'er the to&ogra&hic surface on the right. Water !oul' collect in one of the t!o catchment +asins. Water falling on the !atershe' ri'ge line se&arating the t!o +asins !oul' +e ePually li3ely to collect into either of the t!o catchment +asins. Watershe' algorithms then fin' the catchment +asins an' the ri'ge lines in an image. (he algorithm !or3s as follo!sN Su&&ose a hole is &unche' at each regional local minimum an' the entire to&ogra&hy is floo'e' from +elo! +y letting the !ater rise through the holes at a uniform rate. LiBels +elo! the !ater le.el at a gi.en time are mar3e' as floo'e'. When !e raise the !ater le.el incrementally# the floo'e' regions !ill gro! in siMe. 5.entually# the !ater !ill rise to a le.el !here t!o floo'e' regions from se&arate catchment +asins !ill merge. When this occurs# the algorithm constructs a oneH&iBel thic3 'am that se&arates the t!o regions. (he floo'ing continues until the entire image is segmente' into se&arate catchment +asins 'i.i'e' +y !atershe' ri'ge lines. (he !atershe' algorithm is im&lemente' in the 2A(LAB image &rocessing tool+oB asN & watershed(f)

!here f is the in&ut image an' & is a la+ele' matriB image ha.ing &ositi.e integer .alues at 'ifferent regions an' ) at the !atershe' ri'ge lines. (he 3ey +ehin' using the !atershe' transform for segmentation is thisN Change your image into another image whose catchment basins are the objects you want to identify. In the follo!ing eBam&les# !e consi'er 'ifferent !ays to &reH&rocess images to ma3e them amena+le to !atershe' segmentation. EXAM !E 0 1aters-ed segmentation by t-e distance transform

8onsi'er the tas3 of se&arating t!o touching o+9ects circles in this +inary image. !! im imread('"ir"les#tif')$

Fo! can !e mo'ify this image so its catchment +asins are t!o circular o+9ects? (o 'o this !e$ll use another ne! tool in the Image Lrocessing (ool+oBN bwdist# !hich com&utes the di!ta ce tra !f"r#. (he 'istance transform of a +inary image is the 'istance from e.ery &iBel to the nearest nonMeroH.alue' &iBel# as this eBam&le sho!s.

$ !#a%% bi ary i#age (%eft) a d it! di!ta ce tra !f"r# (right)& 8alculate the image 'istance transform of imageN !! < bwdist(im)

(his image is not .ery useful# +ecause there is only one catchment +asin s&anning the entire image. Instea'# try com&uting the 'istance transform of the image$s com&lementN !! 6 bwdist(=im)$

(his image is closer# +ut !e nee' to negate the 'istance transform to turn the t!o +right areas into catchment +asins. !! > 16 sin for each o+9ect# so !e call the !atershe' function.

Oo! there is one catchment +a

!! &
L

watershed(>)$

is calle' a %abe% #atrix# an' it contains &ositi.e integers corres&on'ing to the locations of each catchment +asin. We can use the MeroH.alue' elements of L# !hich are locate' along the !atershe' lines# to se&arate the o+9ects in the original image. !! im(& ?) ?$

EXAM !E 0 o(ersegmentation 8onsi'er the follo!ing microsco&y image of steel grainsN !! im imread('steel#tif')

try running the !atershe' algorithm on the com&lement of this imageN !! & watershed(im"omplement(im))$

As you can see# there are too many !atershe' ri'ge lines that 'o not corres&on' to o+9ects in !hich !e are intereste'. @.ersegmentation is common &ro+lem !hen using !atershe's. @ne !ay to counter the &ro+lem of o.ersegmentation is to remo.e the minima that are too shallo! using the follo!ing comman'N g imhmin(f, 9)

!hich ta3es an in&ut image f an' su&&resses the minima !hose 'e&th is less than 9. (ry thisN !! im2 !! im@
!! &

watershed(im@)$

im"omplement(im)$ imhmin(im2,2?)$

(he result is much im&ro.e'.

EXERCISES
1. %uantum dots. 1e.ise a segmentation strategy to segment the follo!ing image of Puantum 'ots "Puantum'ots.tif)N

5Btract the follo!ing &arameters from the imageN 1) num+er of 'ots# 2) histogram of areas an' 3) histogram of integrate' intensities 2. &I' images o( ce))s*

4in' the +oun'ary of this cell using any of your fa.ourite image segmentation techniPues. "'icEcell.tif) 3. "ctin (i)aments 1e.ise an image &rocessing strategy to o+tain the 'istri+ution of filaments from this image# an' su+sePuently calculate 1) the mean filament length# 2) the .ariance in filament length#

an' 3) the num+er of filaments. OoteN image segmentation may +e com&licate' +y the lo! light le.els in the image. "actin.tif)

4.

+i,ed ce))s. Fere are some cells that are fiBe' an' staine' for actin "re')# tu+ulin "green) 1OA "+lue)# an' a histone mar3er "not sho!n). 1e.ise an image &rocessing strategy to segment the cells. Rou may o&erate on any of the color channels in the image "or a multi&le of them). (his &ro+lem is es&ecially tric3y +ecause many of the cells are touching. "4colorEcells.st3)

5. "dditiona) E,ercises. 1e.elo& a criteria to 'istinguish +et!een the 1OA an' tu+ulin mor&hologies of these cells treate' !ith 'ifferent 'rugsN U2OS !"# $a%ol.tif U2OS !"# &'()*.tif U2OS !"#.tif U2OS $ubulin $a%ol.tif U2OS $ubulin &'()*.tif (. 2easuring transferrin en'ocytosis

(he images in the AtransferrinA 'irectory are in matche' &airs. (he first image sho!s 1ALI staining an' the secon' image sho!s staining of transferrin that has +een en'ocytose'. Write a &rogram to segment the cells "most li3ely ta3ing a'.antage of the nuclei staining) an' Puantify ho! much transferring is +eing en'ocytose'. Rour Puantification can +e 9ust the sum of the total amount of transferring en'ocytose' or ta3e into account the num+er=Pualities of in'i.i'ual .esicles "more a'.ance'). :. 4in'ing cell +or'ers in a sheetN (he images in the AsheetA 'irectory are in matche' &airs of 1ALI an' mem+rane staining. Write a &rogram to i'entify each cell an' its +or'ers.

%)

Analysis o& motion in 'iological images

(he &re.ious sections focuse' on eBtracting information from in'i.i'ual images using a .ariety of filtering# mor&hological &rocessing an' segmentation techniPues. In this section# !e turn our attention to the analysis of 'ynamics an' motion in a mo.ie# !hich com&rises a sePuence of images ta3en at successi.e time inter.als. 4rom the mo.ies# !e can eBtract information a+out 'ynamical &rocesses in +iology# from the 3inetics of +iochemical reactions to the mo.ements of &roteins an' cellular structures. We +egin +y 'escri+ing !ays of .isualiMing motion in a sePuence of images. A common a&&roach to .isualiMing 'ynamics in.ol.es con.erting the 'imension of time in a series of images into a s&atial 'imension. Kymogra&hs are essentially t!oH'imensional &lots !ith time running along one of the 'imensions. We neBt 'iscuss !ays of eBtracting Puantitati.e information +y &erforming mathematical mani&ulations +et!een images in a stac3. 8rossHcorrelation an' &ro9ection o&erations fall into this category. 4inally !e 'iscuss .arious techniPues for trac3ing &articles an' their mo.ements o.er time.

esigning #raphical $ser Interfaces in MATLAB

Before learning ho! to analyMe motion in microsco&y mo.ies# !e !ill first learn ho! to create gra&hical user interfaces "AIIs) in 2A(LAB# !hich !ill ma3e it easier to interact !ith your functions an' change an' test &arameters. It !ill also ma3e it easier for other &eo&le to use the image &rocessing routines that you ha.e !ritten. We here go through the &rocess of a creating a AII for .ie!ing 2etamor&h stac3s. Rou can easily customiMe an' eBten' this AII to incor&orate the image &rocessing routines that you 'e.elo&e'. (he +asic 2A(LAB interface for creating AIIs in 2A(LAB is the AII 'e.elo&erCs en.ironment# !hich is run +y the comman' guideN !! guide 8reate a ne! Blan3 AII "1efault) +y &ressing @K. Rou !ill see something li3e thisN

Edit the 2Hfile for your AII 'ommands for a''ing stuff to the +ac3'ro&.

/ N the AII

$ackdro. for a''ing +uttons# aBes# etcQ

Fere are the 3ey com&onentsN 1. (he - I $ackdro. is the +ac3groun' on !hich you a'' AII com&onents such as +uttons# teBt# aBes# teBt +oBes an' all sorts of other +ells an' !histles. 2. (he 'ommands allo! you to a'' .arious +ells an' !histles to the Bac3'ro&. 3. @nce you ha.e 'esigne' the +asic layout for your AII# you can attach .arious functions to your AII com&onents +y &ressing the +utton to Edit your 2Hfile. 4. @nce e.erything is com&lete# run the AII +y &ressing the /un +uttonT Before a''ing any AII com&onents onto the +ac3'ro&# letCs first ta3e a loo3 at the 2Hfile +ehin' the AII. Lress the +utton D2Hfile e'itorC to e'it your 2Hfile. When &rom&te'# sa.e the AII file as DmyguiC. When the file o&ens# scroll 'o!n toN
function mygui_OpeningFcn(hObject, eventdata, handles, varargin)

(his is the function that eBecutes +efore the AII is ma'e is .isi+le. Any initialiMation co'e is a''e' +elo! here. 4or the meantime# letCs a'' a .aria+le that acts as a flag as to !hether a stac3 is loa'e'N
handles.loaded = 0

Fan'les is a structure that acts a glo+al .aria+le that can +e seen +y all functions insi'e the AII 2H file. (o sa.e the changes to handles ma'e +y a function# !e run the follo!ing lineN
guidata(hObject, handles)

So the .alue handles.loaded = 0 can no! +e seen +y all com&onents insi'e the AII. When you a'' com&onents to your AII# ne! functions !ill a&&ear in your 2Hfile. LetCs a'' a +utton for loa'ing a 2etamor&h stac3 image into 2A(LAB. A'' a Push $utton onto the +ac3'ro&. 1ou+le clic3 on it to e'it its &ro&erties. Set the D(agC of the Lush Button to loadstac!. A uniPue tag i'entifies each com&onent in the AII# an' the tag !ill a&&ear in the names of AII com&onent functions in the 2Hfile. Also# set the string to DLoa' Stac3C an' set its tag as loadstac!. 8hec3 the 2A(LAB hel& for eB&lanations of the &ro&erties of AII com&onents. Ao to the e'itor for the 2Hfile. Rou shoul' see the follo!ing function a&&earN
function loadstac!_"allbac!(hObject, eventdata, handles)

As you can see# the name of the function is 'etermine' +y its tag. (he call+ac3 function eBecutes !hene.er the +utton is &resse'. A'' the follo!ing lines of co'eN
# "all up a $%& to load to choose the 'etamorph stac! "urrent(ir=p)d *filename, path+ = uigetfile(,-.st!,, ,"hoose stac!.,) #load the stac!, if it e/ists if(filename) *st!,stac!length+=st!read(*path filename+) handles.st! = st! # entire stac! structure )ritten by metamorph handles.loaded = 0 else cd("urrent(ir) return end figure(0) imsho)(st!(0).data,*+) guidata(hObject, handles)

(he co'e calls u& uigetfile to retrie.e the filename of the 2etamor&h stac3 file to +e rea'# an' rea's the stac3 using the comman' stBread. (he stac3 is then a''e' to the handles structure# !hich is then sa.e' using the comman' guidata(hObject, handles). We also set the flag handles.loaded = 0 to sho! that a file has +een loa'e'. Oo! letCs a'' a sli'er to the +ac3'ro& that can +e use' to cycle through 'ifferent &lanes in the stac3. A'' a s)ider. (he sli'er has arro! +uttons an' a +oB that can +e mo.e'# an' is set to

assume the .alue of the current image &lane. Oo! 'ou+le clic3 on the sli'er to mo'ify its &ro&erties. SetN 5Tag4 5;in4 5;a34 5Calue4 planeslider 1 1? 1

(he a+o.e comman's initialiMe the minimum# maBimum an' current .alues of the sli'er. (he current .alue gi.es the current image &lane that is to +e 'is&laye' in the figure !in'o!. When !e loa' a 2etamor&h stac3# !e !ill set the maBimum .alue of the sli'er to ePual the num+er of image &lanes in the stac3. (he follo!ing function a&&ears in the 2HfileN
function planeslider_"allbac!(hObject, eventdata, handles)

(he function eBecutes !hene.er the sli'er is touche' in the AII. (he lo!er .alue of the sli'er is 1 an' the u&&er .alue of the sli'er is O# !hich is the num+er of &lanes in the stac3. We can set these .alues after loa'ing the stac3 +y a''ing this to loadstac!_"allbac!N
set(handles.planeslider,,'in,,0,,'a/,,stac!length)

Oote that the han'les structure also hol's information regar'ing the status of .arious AII com&onents# !hich can +e get or set in the manner sho!n a+o.e. 4or instance# the current &osition of the sli'er can +e o+taine' as follo!sN
get(handles.planeslider, ,1alue,)

(o get a com&rehensi.e list of AII com&onent &ro&erties that can +e get or set# refer to the 2A(LAB hel&. We no! a'' the follo!ing co'e to planeslider_callbac!# to +asically u&'ate the image in the figure !in'o! e.ery time the sli'er is mo.e'N
if (handles.loaded == 0) # get the value of the current plane slideval = get(handles.planeslider, ,1alue,) plane = round(slideval) # retrieve image from stac! and find minimum and ma/imum for contrast # adjustment im = handles.st!(plane).data imin = min(min(im)) ima/ = ma/(ma/(im)) # display image (optimi2ed to prevent flic!ering) figure(0) imagesc(,cdata,, handles.st!(plane).data, ,erasemode,,,none,, *imin ima/+) colormap(,gray,)

children = get(gca,,children,) delete(children(end)) /label(*,plane , num3str(plane)+) end

(his co'e retrie.es the current .alue of the sli'er# retrie.es the image from the stac3 an' 'is&lays the image in figure(0). So +y &ressing the arro!s in the sli'er# !e can scroll through all the &lanes in the stac3. (here are many features that can +e a''e' to the a+o.e AII. 4or instance# !e can a'' teBt +oBes to 'is&lay the &lane num+er or the timestam& of the current frame of the mo.ie insi'e the AII. We can further eBten' this +asic AII +y a''ing image &rocessing +uttons an' functions for &rocessing images an' stac3s an' to sa.e the results of the image analysis. We can e.en a'' e'it teBt +oBes to allo! us to .ary the segmentation &arameters easily. 5fficient AII 'esign can enhance &ro'ucti.ity an' allo! for other &eo&le to use your routines more easily.

%ymographs
@ne !ay to .isualiMe 'ynamical +eha.ior is to &lot time along a s&atial aBis. (ime traces are oneH 'imensional gra&hs are one 'imensional &lots !here time t is &lace' on a single aBis an' a 'ynamical trac3e' '(t) is trac3e' o.er time. Kymogra&hs are a t!oH'imensional analog of time traces# !here time t occu&ies one aBis# s&ace along a oneH'imensional contour x occu&ies another aBis# an' the 'ynamical .aria+le '(x,t) is .isualiMe' as an image. Kymogra&hs &ro.i'e a fast an' con.enient !ay to .isualiMe motion an' 'ynamics in microsco&y images. Loa' the follo!ing image of a s&in'le la+ele' !ith s&ec3le amounts of tu+ulin using the AII you 9ust create'. !! stB stBread('flu3ingspindle#stB')$

Llay the mo.ie a fe! times. Oote that the tu+ulin s&ec3les fluB to!ar's the +oth s&in'le &oles. @ne !ay to measure the rate of fluB is to create a 3ymogra&h along a straight line that runs from one s&in'le &ole to the other. We start off +y creating a scan of intensities along a straight line connecting the s&in'le &oles. @ne !ay to 'o this in 2A(LAB is to use to use the comman' improfile# !hich returns the &iBelH .alue crossHsections along line segmentsN " improfile(8,3i,yi)

!here 8+ t,e input image+ and 3i+ yi# !hich are ePualHlength .ectors s&ecifying the s&atial coor'inates of the en'&oints of the line segments. (o fin' the B an' y coor'inates for the en'&oints# !e can use the comman' [y,3,p] impi3el

!hich allo!s the user to interact !ith the current figure !ith the mouse to o+tain coor'inate .alues. 5ach mouseHclic3 a''s an coor'inate .alue to 3 an' y an' a right mouse clic3 terminates the interacti.e &art of the comman'. We ta3e the intensity &rofiles along the line segment for all images in the &lane# an' then concatenate all the intensity &rofiles to form a t!o 'imensional imageN
st! = handles.st!
imsho)(st!(3).data,*+) # display image *y,/,p+ = impi/el # use impi/el to interactively find endpoints of the spindle # build !ymograph !ymo = *+ # loop through all image plans, find the linescan, and concatenate to form # the !ymograph for i = 0.length(st!) im = im3double(st!(i).data) c = improfile(im, /, y) !ymo = *!ymo c+ # concatenation end # transpose !ymograph !ymo = !ymo,

Oote that the resultant image has many 'iagonal lines. (hese lines corres&on' to mo.ing tu+ulin s&ec3les# an' the .elocity of motion is gi.en +y the slo&e of these lines. Oote that image analysis can +e &erforme' on the 3ymogra&hs for the eBtraction of Puantitati.e information# 9ust li3e any other regular image. EXE/C'+E+ 1. A'' the 3ymogra&h as an a''itional tool in the AII you create' a+o.e. Lro.i'e a means to sa.e the 3ymogra&h into a file. 2. Instea' of sim&ly ta3ing an intensity &rofile along a line# it is common to ta3e an a.erage "or maBimum) of intensities of the image in the 'irection orthogonal to the line segment !hen ma3ing a 3ymogra&h. Fo! !oul' you im&lement this? 3. Im&lement a 3ymogra&h that ta3es intensity .alues along the circumference of a circle of a gi.en ra'ius an' center. 8an you thin3 of any +iological situations !hen this might +e useful?

ifference Images, Ma&imum Intensity 'ro(ections

It is often &ossi+le to .isualiMing 'ynamics in image stac3s +y &erforming mathematical mani&ulations in.ol.ing neigh+oring images in a stac3# or e.en +y &erforming mani&ulations on the

entire stac3. In this section# !e 'iscuss t!o &ossi+le mani&ulations- 'ifference images an' maBimum intensity &ro9ections. &i((erence Images A 'ifference image of successi.e frames in a mo.ie is gi.en +yN 19 "B#y) * I9<1"B#y) S I9"B#y) !here 1 is the 'ifference image# an' 9 is a gi.en &lane in the stac3. 1ifference images highlight features of the image that change ra&i'ly o.er time# much in the same !ay that a s&atial gra'ient enhances contrast aroun' an e'ge. (he change is usually +rought a+out +y mo.ement of an o+9ect in the image# or +y a 3inetic &rocess that has not reache' stea'yHstate "li3e &hoto+leaching). EXE/C'+E 1. Photo0)eaching o( '&'123. Loa' yeast"d"D2#stB. (a3e 'ifference images +et!een successi.e frames. What 'o the &ea3s an' troughs in intensity corres&on' to? Ma,imum Intensit! Projections A maBimum intensity &ro9ection of an entire image stac3 is gi.en +y 2"B#y) * maB9"I9"B#y)) (he maBimum on the right han' si'e is the maBimum in intensity .alue at a gi.en &iBel .alue o.er all stac3s in an image. 2aBimum &ro9ections colla&se the entire 'ynamics of the stac3 onto a single &lane# an' is es&ecially useful for .isualiMing entire tra9ectories of mo.ing &articles on a single &lane. EXE/C'+E 1. 4isua)i5ing )isteria trajectories. (i!teria is an intracellular &athogen that mo.es insi'e the cell +y &olymeriMing actin at its rear surface. Loa' listeria"ells#stB an' use a maBimum intensity &ro9ection to .isualiMe the entire tra9ectory of the mo.ing +ug. 8an you a&&ly image &rocessing tools to trace an' segment the tra9ectory?

Image Cross)correlation
A common !ay to 'etect motion of o+9ects o.er successi.e frames in a mo.ie is to &erform a correlation of successi.e images !ith one another. When there is concerte' motion +et!een t!o image frames# the crossHcorrelation of the t!o images !ill eBhi+it a &ea3 at a nonHMero location# an' the offset of the &ea3 from Mero !ill then gi.e the 'istance of mo.ement o.er the &erio' of time. 8rossHcorrelation is most effecti.e !hen there is concerte' motion o.er large regions of the microsco&y image. In this section# !e first go through an eBam&le of registering an image using crossHcorrelation. We then co.er some +iological eBam&les !here crossHcorrelation may +e useful in eBtracting 'ynamical information in the system of interest. 4irst# letCs loa' the image !! im1 !! im2 imread('"ells1#tif')$ imread('"ells2#tif')$

(he images are out of registerN

8reate a su&erim&ose' color image of the t!o images to simultaneously .isualiMe +oth images. (he comman' "at is use' to concatenate matrices in the thir' 'imension an' can +e use' to create 7AB color imagesN !! blanB Eeros(siEe(im1))$ !! "im "at(@, mat2gray(im1), mat2gray(im2), blanB)$ !! imshow("im,[])$ As you can see# the t!o images are translate' !ith res&ect to each other. (o 'etermine ho! much one image is translate' !ith res&ect to the other# you can &erform a crossHcorrelation on the t!o images. 2A(LAB has a comman' for &erforming a normaliMe' crossHcorrelationN

!! b

norm3"orr2(im1,im2)$

Oo! .isualiMe the crossHcorrelation function using the mesh comman'N !! figure$ mesh(b)$ (he out&ut of this crossHcorrelation function is a matriB b that is the sum of the siMe of the t!o in&ut images. b has a single maBimum# !hich is offset from the center of the matriB +y a small amount. (his offset corres&on's to the translation of im1 !ith res&ect to im2. (o 'etermine the location of this &ea3# !e use the find comman'# !hich returns the locations of nonMero in'ices in a gi.en matriB. We can cou&le this comman' to con'itional statement to fin' the maBimum &oint of this matriBN !! [y,3] find(b ma3(b(F)))$

Oote that# in matriB su+scri&t notation# ro!s "y) are a''resse' +efore columns "3). Oo! !e 'etermine the offset of these &ositions from the origin of the matriBN !! [y",3"] siEe(im2)$ !! yoff y 1 y"$ !! 3off 3 1 3"$ Oo! letCs create a large color image to .isualiMe the su&er&osition of the t!o framesN !! !! !! !! "im2 Eeros( y"-yoff, 3"-3off, @)$ "im2(yoffF(y"-yoff11), 3offF(3"-3off11), 1) "im2(1Fy", 1F3", 2) mat2gray(im2)$ imshow("im2)$ mat2gray(im1)$

EXERCISES
1. Lerform crossHcorrelation +et!een successi.e frames of the s&in'le image flu3ingspindle#stB. 8rossHcorrelate the first image "2) !ith images that are further a&art in time. What 'oes the crossHcorrelation &lot loo3 li3e? What 'oes this tell us a+out the +iology? 2. A common !ay of eBtracting flo!s in flui' 'ynamics is to &erform crossHcorrelation on su+regions of a mo.ie to o+tain a flo! .ector fiel'. Write a &rogram that calculates flo! .ector fiel's +y &erforming crossHcorrelation in a localiMe' region near the .ector to +e calculate'. (reat the siMe the neigh+orhoo' as a free &arameter an' test the &rogram on flu3ingspindle#stB

'article Trac*ing
In &articleHtrac3ing# !e attem&t to eBtract the tra9ectories of mo.ing &ointHli3e o+9ects from a stac3 of images. (here are t!o main ste&sN 1) Segmentation* We first nee' to i'entify the &articles in all of the images. (he segmentation techniPues taught in the &re.ious sections can +e use' to segment these &articles an' eBtract their locations in the microsco&y images. 2) Motion corres.ondence. Ai.en the locations of segmente' &articles in successi.e microsco&y images# !e nee' to 3no! !hich &articles in one frame corres&on' to !hich &articles in a frame ta3en a small timeHinter.al later. We essentially nee' to match &articles at gi.en time t !ith &articles at a time t+dt later# an' 'o this for all image framesN

t t2dt
It is often the case in image acPuisition that the time inter.al +et!een successi.e frames is so small that these mo.ing &articles mo.e only small 'istances 'uring this time inter.al. In such a case# it is often a'ePuate to match &articles !ith the closest ones near+y. In such a case# !e fin' a matching that minimiMes a cost function 'etermine' +y the 'istance tra.ele' +y &articles. A cost function that !or3s !ell for 'iffusi.e motion is gi.en +y the sum of sPuares of the 'istancesN

8 * 'i2
Fere the summation is o.er all matche' &articles an' the cost function is minimiMe' o.er all &ossi+le matching &airs. In &ractice it is com&utationally too eB&ensi.e to go through all &ossi+le matching &airs to fin' the minimum. @ne ty&ically im&oses a threshol' 'istance +eyon' !hich no &air of &articles in successi.e frames can +e matche'. A &article trac3er in 2A(LAB that sol.es the motion corres&on'ence &ro+lem for the a+o.e cost function can +e foun' atN

httpF..www#deas#har%ard#edu.proGe"ts.weitElab.matlab.tutorial#html

(here are other &ro+lems an' 'ifficulties that may arise !hen sol.ing the motion corres&on'ence &ro+lem. 4or instance# &articles an' a&&ear or 'isa&&ear from the fiel' of .ie! 'uring the course of the mo.ie. 2oreo.er# &articles may +ecome tem&orarily occlu'e' from the fiel' of .ie!. It is im&ortant to mo'ify the motion corres&on'ence algorithms to a''ress such &otential &ro+lems. Ai.en a set of &articles in a stac3 of segmente' images an' a corres&on'ence +et!een &articles in successi.e images# it is &ossi+le to o+tain a set of tra9ectories that 'escri+e the motion of these in'i.i'ual &articles. EXE/C'+E+ 1. Kinetochores. In a s&in'le# microtu+ules attach to chromosomes through structures calle' 3inetochores. In the stac3 Bin#stB# 3inetochores are la+ele' !ith a fluorescent mar3er. Write a &article trac3ing algorithm to trac3 the mo.ement of in'i.i'ual 3inetochores. Is there anything you can say a+out their 'ynamics? "ckno6)edgements (han3s to 2ary Ba'on# @nn Bran'man# Aaron Aroen# 1an Oee'leman# \a'e Shi an' Luc3 @hi for images.

()

)*!*)*+C*S

We relie' on the follo!ing +oo3s to &ut together &arts of the tutorialN AonMaleM 7.8.# Woo's 7.5. "2))1) 1igital Image Lrocessing. 2n' e'ition# I&&er Sa''le 7i.er# Lrentice Fall AonMaleM 7.8.# Woo's 7.5.# 5''ins S.L. "2))4) 1igital Image Lrocessing using 2A(LAB# I&&er Sa''le 7i.er# Lrentice Fall 8hec3 out the follo!ing !e+site for lots of image &rocessing eBam&lesN htt&N==!!!.image&rocessing&lace.com= Some of the eBam&les !ere &ulle' of fromN htt&N==!!!.image&rocessing&lace.com=1ILI2='i&umEtutorials=tutorials.htm Iseful &article trac3ing referencesN Jallotton L.# Lonti A.# WatermanHStorer# 8.2.# Salmon 5.1 an' 1anuser A. "2))3) 7eco.ery# JisualiMation# an' Analysis of Actin an' (u+ulin Lolymer 4lo! in Li.e 8ellsN A

4luorescent S&ec3le 2icrosco&y Stu'y. Bio&hys \. %5N12%?H13) Jeenman# 8. \.# 2. \. (. 7ein'ers# an' 5. Bac3er. 2))1. 7esol.ing motion corres&on'ence for 'ensely mo.ing &oints. )*** Tra !& +atter $ a%y!i! a d ,achi e ) te%%ige ce& 23N54S :2.