Professional Documents
Culture Documents
FASTFRAMEHYBRIDOPTOELECTRONICNEURAL
OBJECTRECOGNITIONSYSTEM
TAHERDAUD,TIENHSINCHAO,ANDANILKUMARTHAKOOR
CenterforSpaceMicroelectronicsTechnology
JetPropulsionLaboratory,CaliforniaInstituteofTechnology
4800OakGroveDrive,Pasadena,CA911098099
ABSTRACT
The Hybrid Optoelectronic Neural Object
RecognitionSystem(HONORS)developmentatthe
JetPropulsionLaboratory(JPL)promiseshighspeed
(>1000 frames/s) processing of large frame size
(1000x1000pixel)images.Itconsistsoftwomajor
building blocks: (1) an advanced grayscale optical
correlator(OC);and(2)amassivelyparallel,VLSI
basedneural3Dprocessor(N3DP). TheOC,with
itsinherentparallelprocessingandshiftinvariance,
is used for target of interest detection and
segmentation. The N3DP, with its high speed
(1012 /s)convolutionoperationsandneurallearning
capability is used for target classification and
identification. This paper presents the system
architectureandprocessingalgorithms. Inaddition,
the results from simulations and experiments,
including the detection, classification, and tracking
oftanksandaircraftaresummarized.
KEYWORDS:
Object recognition, optoelectronic, MACH filter,
neuralnetworks,3Dprocessor.
INTRODUCTION
Automatic target recognition (ATR) applications
requirehighspeedprocessing,recognitionofobjects
from clutteredbackground,andintelligent decision
making. Acompact, highspeed ATR system isa
movingtarget.Enhancementinprocessorspeedsgo
handinhandwithbetterandlargerimagersrequiring
higher data processing rates and autonomy in
processing and decision making. Various civil,
military,andspaceapplicationswouldseeordersof
magnitudeadvancement intheircapabilitieswitha
systemperformingATRat1000framespersecond
especially with large format (1000x1000pixel)
imagers.
JPLisdevelopingahybridoptoelectronic
neural object recognition system (HONORS) for
SYSTEMARCHITECTURE
ThesystemblockdiagramofHONORSisshownin
Figure1. TheentireATRfunctionisimplemented
infiveconsecutivesteps[1]:
(1) Sensor data formats are fed through a frame
buffer(FB)device.
(2) After format conversion, FB feeds OC for
preprocessing for target of interest (TOI)
detection and segmentation based on target
shape,size,texture,andgrayscaledistribution.
Training target images are computed with a
distortion invariant correlation filter algorithm
anddownloadedintotheopticalcorrelator.
Use W ord 6 .0c or later to
vi ew Ma ci ntosh pi cture.
(4) TOIimagesarefedtothecubeconsistingof:an
eigenvectorbased feature extraction 3D
stacked electronic processor and an analog
neuralnetworkclassificationchipforfinaltarget
classificationandidentification.
(5) TheoutputofN3DPisaviableinputfortarget
tracking, navigation & guidance, sensor
retasking,andmissionreplanning.
A highspeed camcordersized grayscale
optical correlator [24] consisting of a grayscale
inputSLMreplacesthebinarySLMusedinprevious
stateoftheart optical correlator systems. A
ferroelectric liquid crystal (FLC) SLM capable of
encoding realvalued data is also used for the
correlationfilterimplementation.
A system architecture for this innovative
OC and a photograph of the palmcorder size
grayscaleOCisshowninFigure2.A25mwdiode
laser,emittingat690nm,iscollimatedandusedas
the light source. A 640x480pixel liquid crystal
display(LCD) with a2.4mm pitch isused asthe
inputSLM. TheFLCSLMisusedasafilterand
operatesinatransmissionmodewhiletheLCDSLM
operates inareflection mode. Theinput imageis
Fouriertransformedanddirectedtoaddressthefilter
SLMviaapolarizingcubicbeam splitter. Ahalf
waveplateisinsertedbetweenthebeamsplitterand
thefilterSLMtosteerthepolarizationorientationof
the throughput light beam to ensure bipolar
amplitudeoutputfromtheFLCSLM. Thereadout
beam,reflectedbackfromthefilterSLMisinverse
Fouriertransformedatthe
OCFILTERALGORITHM
OPTICALMACHFILTERDEMONS
TRATIONOFDISTORTION
INVARIANTTARGETDETECTION
In a laboratory experiment for distortion invariant
targetdetectionusingopticallyimplementedMACH
filter,asequenceof15IRimagesofatankmoving
downwardadesertterrainisusedastheinput. In
preparing the MACH filter, 5 of the inputs were
selectedfortrainingandtherestusedfortesting.A
MACHfilteriscomputedandthendownloadedinto
thefilter SLM. InFigure3,resultsoftheoptical
correlatoroutput areprovided. All15inputswere
successfully detected with a single MACH filter
trainedforrecognizingthetankimages.Fourofthe
images are shown in Fig. 3 (a), and their
corresponding correlation peaks and 3D plots are
shownin(b) and(c)respectively. Notice that the
correlation peaks remain sharp and uniform across
alltheinputimagesincludingthosewithscaleand
perspectivevariations. Thescaleratiobetweenthe
topandbottomimagesisabout2:1.Thisexperiment
validates that an optical correlator is an effective
targetdetectionpreprocessor.
(a)
(b)
(c)
FIGURE3.TARGETDETECTIONRESULTSOF
ANOPTICALMACHCORRELATORFILTER.
NEURAL3DPROCESSOR(N3DP)
N3DPisaground/airborneimageprocessing/target
recognition processor being developed as an
enhancedversionofa3dimensionalartificialneural
network (3DANN) processor architecture. Its
successiscoupledwiththeinnovativeeigenvector
based algorithm development anddemonstrationin
simulation.
ARCHITECTURE: A block diagram of the front
partoftheN3DPconsistingofaCLICandanNPM
cube (without the subsequent multilayer neural
network) is shown in Figure 4. It also shows a
photographofthehardware.ThefunctionofN3DP
is to have a 64x64pixel image window as an
electronicinputtobeconvolvedsimultaneouslywith
64selectedandstoredeigenvectortemplates(each
of 64x64pixel size) and provide the 64 electronic
outputs(oneinnerproductpertemplate)ataspeedof
about4MHz,reducingtheinputbandwidthof4096
tojust64outputs.
CLICconsistsofasetof8bitshiftregisters
and a 64x64 array of 8bit static random access
memory (SRAM) cells attached to respective
multiplyingdigitaltoanalogconverters(MDAC)[8].
The inputsare thusconverted byMDACsto4096
analog voltage signals, and fed inparallel intothe
NPMcubeevery250nanoseconds.
FIGURE4.THE3DCONVOLVERNETWORKCONSISTSOFANEURALPROCESSINGMODULE(NPM
CUBE) WITH 3D STACKED 64 CHIPS, EACH WITH A 64X64 SYNAPSE ARRAY BASED ON 8BIT
MULTIPLYING DIGITALTOANALOGTECHNOLOGY ANDINCORPORATINGASPECIALPURPOSE
IMAGEWRITEDEVICETERMEDCOLUMNLOADINGINPUTCHIP(CLIC). CLICISBUMPBONDED
TOTHENPMCUBE.THECUBEHASBEENREALIZED(PHOTOGRAPH)ASA10gm,3cm 3PACKAGE,
WITHPOWERCONSUMPTIONOF~2.5W.
Helicopter Plane
Missile
OC
OUTPUTS
CLIC
3DANN-M Templates
(Eigenvectors of
target objects)
FIGURE5.GENERALTARGETRECOGNITIONISACHIEVEDUSINGEIGENVECTORPROJECTIONSIN
CONJUNCTIONWITHANEURALNETWORKCLASSIFIERTRAINEDONSELECTEDDATASETS.
definethepartitionandprojectedontoeigenvectors
suitableforthatparticulardistributionofdata.
Informationabouttheobject(class,identity,or
orientation)isprocessedinacoarsetofinemanner.
Forinstance,afterdetectinganobjectinaframe,a
roughestimateofimageorientation/scaleismade,a
resultthatcanthenbeusedtolimitthevariationthat
needs to be considered during subsequent object
classificationstep.Insimulation,objectrecognition
ratesof96%inclutteredbackgroundareobtained
CONCLUSIONS
HONORS,apowerfulATRsystemdescribedherein
combinesanadvancedopticalcorrelatoranda3D
integrated neural network based processor in a
compactpackagetoperformobjectrecognitionwith
unprecedented speed. Algorithms have been
identified and simulated for both the optical
correlator and N3DP. Demonstrations for realtime
detection,classification,andprecisiontracking,with
groundbased and airborne experiments on live
targetsareplanned. Itcanbeprojectedthatsucha
highperformancesystemwillfindvariedusesboth
intheNASAarenaandforcommercialandmilitary
applications.
ACKNOWLEDGMENTS
Theresearchdescribedinthispaperwascarriedout
bytheJetPropulsionLaboratory,CaliforniaInstitute
of Technology, and was jointly sponsored by the
Ballistic Missile Defense Organization and the
National Aeronautics and Space Administration.
TheauthorswishtoexpresstheirsincerethankstoS.
Suddarth,S.Udomkesmalee,T.Duong,C.Padgett,
andG.Reyesforhelpfultechnicaldiscussions.
REFERENCES
1. TH.Chao,T.Daud,andA.Thakoor,Honors:
Advanced Miniature Processing Hardware for
ATR Applications, Proceedings of the 4th
InternationalNEURAPConf.,Marseille,France,
March1113,1998,pp.451456.
2. TH. Chao, E. Hegblom, B. Lau, and W. J.
Miceli,"OptoelectronicallyImplementedNeural
NetworkwithaWaveletPreprocessor,"Proc.of
SPIEvol.2026,1993,SanDiego,CA.,pp.472
482.
3. TH. Chao and A. Yacoubian, Realtime
updatable optical wavelet processor, Proc. of
SPIE,vol.2490,1995,pp.118124,.
4. TH.Chao,A.Yacoubian,andB.Lau,Portable
opticalwavelet processor, Proc.ofSPIE,vol.
2752,1996,pp.254259,.
5. B.V.K. Vijaya Kumar, Tutorial survey of
compositefilterdesignsforopticalcorrelators,
Appl. Opt., vol. 31, no. 23, pp. 47734801,
1992.
6. A.Mahalanobis,B.V.K.VijayaKumar,S.Song,
S.R.F.SimsandJ.F.Epperson,Unconstrained
correlationfilters,Appl.Opt,vol.33,pp.3751
3759,1994.
7. A. Mahalanobis, Correlation filters for object
tracking, target reacquisition, and smart
aimpointselection, Proc.ofSPIE,vol.3073,
1997,pp.2532.
8. T.Duong,T.Thomas,T.Daud,A.Thakoor,and
B. Lee, 64x64 Analog input array for 3
dimensional neural network processor,
Proceedingsofthe3rdInternationalConference
on Neural Networks and Their Applications,
Marseilles,France,March1997.
9. W.S. McCulloch and W. Pitts, A logical
calculus of the ideas immanent in nervous
activity, Bulletin of Mathematical Biophysics,
vol.5,pp.115133,1943
10. T.Duong,S.Kemeny,M.Tran,T.Daud,andA.
Thakoor, Low power analog neurosynapse
chips for a 3D sugarcube neuroprocessor,
Proceedings of the IEEE International
ConferenceonNeuralNetworks,vol.III,1994,
Orlando,FL,pp.19071911.
11. M. Turk and A. Pentland, Eigenfaces for
recognition, J.ofCognitiveNeuroscience,vol.
3,pp.7186,1991.
12. C. Padgett, G. Cottrell, and R. Adolphs,
Categorical perception in facial emotion
classification,Proceedingsofthe18thAnnual
Conference of the Cognitive Science Society,
Hilldale,1996,pp.201207,.
13. C.Padgett,M.Zhu,andS.Suddarth,Detection
and object identification using VIGILANTE
processing system, Proc. SPIE, vol. 3077,
1997.