INTEGRATIONOFAUGMENTEDREALITYINTOTHECADMODELINGAND

ENGINEERINGDRAWINGTRAININGOFDESIGNERS
GojkoVladi1,NedaMili1,Stefanurevi1,RastkoMiloevi1,MladenStani2
1
UniversityofNoviSad,FacultyofTechnicalSciences,DepartmentofGraphicEngineeringand
Design,NoviSad,Serbia
2
UniversityofBanjaLuka,FacultyofTechnology,DepartmentofGraphicEngineering,BanjaLuka,
BosniaandHerzegovina
Abstract:Thegraphicdesigners,andproductdesignersespecially,inthemodernjobmarketareexpectedto
have some engineering abilities and skills. Those include comprehension of technical documentation and at
least rudimentary skills to produce understandable technical drawings. Usual process of training for
engineering drawing in the first steps includes training in using a specific set of descriptive geometry
procedures. Those procedures allow an imaginary object to be represented on a twodimensional surface in
such a way that it may modeled in 3d space, real or virtual, in true scale and shape and viewed from any
position in space. This enables the designers to turn research ideas into technical plans for prototypes and
production using computeraided design (CAD). Recently augmented reality has become able to provide
integration variety of multimedia formats in order to enrich education experience. Augmented reality is not
newtechnology,butonlywithrecentdevelopmentofmobiletechnologiesitbecomeabletobeusedtoitsfull
potentialineducation.Thispaperaimstopresentpossibilitiesforintegrationofaugmentedrealitytechnology
into the cad modeling and engineering drawing training of designers. This can lead to far better
comprehensionandeasierlearningprocess.
Keywords:Augmentedreality,education,cadmodeling,engineeringdrawing

1.INTRODUCTION
The graphic designers, and product designers especially, in the modern job market are expected to have
someengineeringabilitiesandskills.Thoseincludecomprehensionoftechnicaldocumentationandatleast
rudimentary skills to produce understandable technical drawings. Traditional education and training for
engineering drawing in the first steps includes training in using a specific set of descriptive geometry
procedures,thishaswellprovenbenefits.Thoseproceduresallowanimaginaryobjecttoberepresentedon
atwodimensionalsurfaceinsucha way that it maymodeled in 3d space,real orvirtual,in truescaleand
shapeandviewedfromanypositioninspace.Thisenablesthedesignerstoturnresearchideasintotechnical
plans for prototypes and production using computeraided design (CAD). Students can explore a 3D
visualizationoftheteachingmaterial,thusenablingthemtounderstandmoreeffectivelythroughinteractivity
with multimedia content (Liarokapis et al, 2004). Educational process must be simple in order to provide
users with clear and comprehensible information and to increase the level of understanding the subject.
Apart from these issues, the digitization of the teaching material must be done carefully so that all
informationwillbeaccuratelyandclearlypresentedtotheusers.Thisdigitizationorcontentpreparationis
usuallyanofflineprocessandconsistsofmanydifferentoperations,dependingonthetargetapplication.
Recentlyaugmentedrealityhasbecomeabletoprovideintegrationvarietyofmultimediaformatsinorderto
enricheducationexperienceandensureeasyandefficientinteractionbetweenthestudentsandtheteaching
material. So far numerous educational systems based on augmented reality have been developed for
differentfields(Liarokapisetal,2002).Thispaperaimstopresentpossibilitiesforintegrationofaugmented
reality technology into the cad modeling and engineering drawing training of designers. Enabling them to
exploreteachingmaterialthroughaugmentationandconcentrateandstudypresentedprocedures,whichin
turnwillleadtofarbetterskillandknowledgeadoption.

1.1.Augmentedrealitytechnology
AugmentedrealitytechnologyasaconcepthasitsrootsinTVtechnologywherelayerofinformation,textor
images,islayeredoverthevideofootage.Thecurrentformofaugmentedreality originatesfrom1990s,it
waspatentedatthebeginningof2000sbytheXeroxCorporation(Harringtonetal,2002).Everydayusageof
this technology to its full potentialin interactiveapplications is still considered a novelty.Advancementsof
the hardware performance in mobile devices made possible capturing the real world images in sufficient
quality,processingthem,simultaneouslyprocessingthevirtualcontentandcombiningthemintoaugmented
reality displayed on the devices screen. Augmented reality basically overlays computergenerated virtual
elements onto the realworld images, thus enhancing their informative or entertainment value. Most
important characteristic of this concept is that the overlay content is contextsensitive, which means that
information displayed is triggered by the real world object in real time. This technology is widely available
through different education contents, games, etc. mostly intended for the mobile devices. (Azuma, 1997;
Azumaetal,2001).
There are different approaches to the augmented reality system design. Design rout first and foremost
dependsontheintendeduseofthefinalapplication,consideringfieldofuse,intendedusergroup,devices
andenvironment.Allofthesefactorsinfluencethedecisiononthetechnologiestobeutilized.Accordingto
tracking technique, display technology and interaction techniques, augmented reality technology can be
classified:

1. Trackingtechniques
a. Sensorbasedtechniques
b. Visionbasedtechniques
c. Hybridtechniques
2. Displaytechnology
a. Seethroughdisplays
b. Projectionbaseddisplays
c. Handhelddisplays
3. Interactiontechniques
a. TangibleARinterface
b. CollaborativeARinterface

Foreducation fieldVisionbasedtracking technique is especiallyinteresting havingin mind that augmented

realityisusually usedasan upgradeto printedlearningmaterials.Thesetechniquescan befurtherdivided
into Marker tracking technique where the corresponding image descriptors are provided beforehand and
stored into the database and Markerless (Nonmarker) tracking technique where application recognizes
imagesthatwerenotprovidedtotheapplicationbeforehand.
Most of the applications in the use today are using markerbased technique, as Markerless technique is
muchmoredifficulttoimplementbecausetherecognitionalgorithmshouldindependentlyidentifypatterns,
colorsorsomeother"features"thatmayexistincameraframes(Tangetal,2003).
Examplesforthesetwotechniquesareshowninfigure1.Figure1arepresentsmarkerbasedtrackingwhere
thevirtualobjectisplaceduponarecognizedimage(infiniti.co.uk).Muchmoredifficulttoachieveismarker
lesstrackingshowninfigure1basthesoftwareneedstorecognizeshapeofthehumanhandandtheposition
of the fingers in order to place virtual content and enable its interactivity with the hand movement
(ilab.cs.ucsb.edu).
Sensorbased tracking offers utilization of some specific technologies such as optical, magnetic, inertial,
acoustic, ultrasonic sensors. Hybrid tracking techniques combine the best characteristics of previously
mentionedtechniques(Yangetal,2008).

a)

b)
Figure1.ARtrackinga)Markertracking,b)Markerlesstracking

1.2.

Augmentedrealityhardware

Driving force behind explosion of augmented reality applications and move from the laboratories to the
everyday use is advancement of hardware, mainly of mobile devices. Minimal device requirements image
capturingability,processingmachineanddisplay.
Simplest setups consist of personal computer equipped with web camera and monitor, figure 2a
(sketchupdate.blogspot.rs). In order to ensure mobility and full usability of the technology for education,
modernmobiledevicesuchasphoneortabletmustbeequippedwithbackfacingcameraandappropriate
application installed, figure 2b (architectmagazine.com). Usability can be expanded even further with
wearabletechnologysuchasGoogleglassorsimilarproducts.Suchwearabletechnologyfreesusershands
andenableshimtoperformmanualtaskswhileheisgivenadditionalinformation orinstructions,figure2c
(33rdsquare.com).

a)

b)

c)

Figure2.ARhardwarea)Personalcomputer,b)Phone,c)Wearablehardware

Futuredevelopmentandintegrationofwearableelectronicsintoeverydaylifewillprovidenewhorizonsfor
the implementation of the augmented reality in different human activities and pushing humancomputer
interactionfurther.
UsingothertechnologiessuchasEyetrackingwhichenablesthecomputertotrackmomentarilyuserspoint
ofgaze,orpointofattention,canfurtherintensifyemersionbyprovidingvirtualcontentonlyintheareasof
users interest. Furthermore by incorporating an electromyogram, electroencephalogram, and electro
oculogram headgear eye movements, facial muscle movements, and brain waves can be translated into
computerinput,thusenablingothermeansofcontrollingthesystem.
2.2.

Augmentationprocess

Theaugmentationisproducedafteraseriesoftransformations,asshownintheprocessforthecreationof
AR using markerbased tracking, figure 3. First, the real video image is captured and transformed.
Transformedimageisprocessedinordertodeterminepositionofthemarkers(containinganimagepattern
thatiscomparedtopatternsstoredinadatabase).Next,thealgorithmdeterminespatternorientationasthe
baseforthecoordinatesframeandcalculatestherealpositionofthedigitalcamerainrelationtothephysical
marker.Afterthat,thevirtualobjectsareplacedoverthemarkers,andthefinalimageisrenderedandsent
tothedisplay(Soaresetal,2012).

Figure3.TheprocessforthecreationofARusingmarkerbasedtracking

Dependingontheapplication,virtualobjectsplacedoverrealimagecanbestillimages,videocontent,2Dor
3D graphics. Virtual objects can be also interactive, offering further functionality to the user, such as
additionalinformationonthecontentcapturedintherealworld.

2.METHODS
ThispaperaimstopresenttheprocessofcreatinganAugmentedRealityapplicationforCADmodelingand
engineeringdrawingtrainingofdesigners.Softwaretoolsneededforproductionofsuchtrainingapplication
based on augmented reality technology can be divided in to the software for teaching material production
andaugmentedrealityapplicationproduction.
3D modeling software, video and raster graphics editing software and screen capture software are usually
used to produce teaching material. There are plenty of high quality, 3D modeling tools. Majority of them,
whetherintendedforCADorpolygonalmodelingcanserveasagoodtoolforproductionofthe3Dmodelsor
trainingvideos.Asidefromprofessionaltoolstherearefreesoftwaresthatposesallthefeaturesneededfor
tasksrequiredinthisprocess.AutoDesk3DSmax2016wasusedforproductionofthevideomaterialsforthis
application.
Sameasinthecaseof3dmodelingtoolsthereisvideverityofsoftwaresuitableforafterproduction.Video
materialrenderedandexportedfromthe3Dmodelingsoftwarealmostalwaysneedstobemanipulatedin
after production. Manipulating video quality, video montage or even adding additional elements are usual,
especially in production of educational or training videos. Tool contained in Adobe Creative suite 6 Master
collectionwereusedforthispurpose.
Imageandvideoscreencaptureiswellestablishedtechniqueforproductionoftrainingmaterialforsoftware
users.TechsmithCamtasiausedinthiscaseisoneofthemoreversatilesolutions,offeringafterproduction
andpossibilityofexporttomanypopularformats.
Software needed for augmented reality application production is usually array of different softwares
dependingontheplatformusedforproductionandtheplatformonwhichfinalapplicationwillbeusedon.
CombinationofUnitity3D,Vuforia,MicrosoftVisualstudioandAndroidstudiowasusedtoproducethefinal
application.

2.RESULTS
Applicationthatwasproducedasacasestudyforintegrationofaugmentedrealityintothecadmodelingand
engineering drawing training of designers offers three instructional components: interactive 3D model
augmentation and two video augmentations. This application intended for Android platform and can be
downloadedfromhttp://www.grid.uns.ac.rs/symposium/download/cadedu.apk.Whenapplicationisinstalled
and run, interactive 3D model triggered by the image shown in figure 4 offers possibility to the user to
manipulatethemodelandtogetmuchbetterunderstandingoftheobjectshape.

Figure4.Interactive3Dmodeltrigger

Video augmentation triggered by the image shown in figure 5 explain placement and creation of the
orthographicprojections.VideoisproducedusingAutoDesk3DSmaxandcompositedusingAdobeCreative
suite6Mastercollection.

Figure5.Videoaugmentationtrigger,placementandcreationoftheorthographicprojections

The image shown in figure 6 triggers video material explaining usage of certain tools for creation of CAD
modelusingCATIAv5software.ThisvideowascapturedandeditedusingTechsmithCamtasia.

Figure6.Videoaugmentationtrigger,creationofCADmodelusingCATIAv5software

5.CONCLUSIONS
This paper presented the possibility for integration of augmented reality into the cad modeling and
engineeringdrawingtrainingofdesigners.Thebasicpathoftheaugmentedrealityapplicationproductionis
explained.Alltheresultsshownareeffortofthesmallgroupofauthorsandwithevolvementoftheexpertsin
eachofthefieldsmentionedresultscanonlybeimproved.
The future plans include creation of more educational and training material using augmented reality
technology, integration with existing teaching materials and optimization of old material to fit better with
futuretrends.Applicationofthistechnologyineducationpracticeisanimperative.Onlytherealwordusage
cananswerthequestionsaboutadvantagesanddisadvantagesofthetechnology.Enhancingtherealismof
theintegrationbetweenrealandvirtual,usingaugmentedshadows,takingintoaccounttransparencies,etc.
isalsooneofthefuturegoals.

6.ACKNOWLEDGMENTS
The research is supported by Ministry of education, science and technology development of Republic of
Serbia,projectnumber:35027Developmentofsoftwaremodelforscientificandproductionimprovementin
graphicindustry

7.REFERENCES
[1] Azuma,R.,ASurveyofAugmentedReality.PresenceTeleoperatorsandVirtualEnvironments6(4):355
385,1997.
[2] AzumaR,BaillotY,BehringerR,FeinerS,JulierS,MacIntyre,B.,RecentAdvancesinAugmentedReality.
IEEEComputerGraphicsandApplications21(6):3447,(2001).
[3]Liarokapis,F.,Mourkoussis,N.,White,M.,Darcy,J.,Sifniotis,M.,Petridis,P.,Basu,A.,Lister,P.F.,Web3D
andAugmentedRealitytosupportEngineeringEducation,WorldTransactionsonEngineeringandTechnology
Education,Vol.3,No.1,(2004).
[4]Liarokapis,F.,Petridis,etal,MultimediaAugmentedRealityInterfaceforELearning(MARIE),World
TransactionsonEngineeringandTechnologyEducation,UICEE,Vol.1,No.2,173176,(2002).
[6] Soares,A,Andrade,A.,Lamounier,E.,Cardoso,a.,VirtualandAugmentedReality:ANewApproachtoAid
UsersofMyoelectricProstheses,DOI:10.5772/50600,(2012).
[5]StevenJ.Harrington,JamesR.Vallino,AugmentedrealitydisplaymethodandsystemUS6408257B1,
(2002).
[7] A.Tang,C.Owen,F.Biocca,andW.Mou,Comparativeeffectivenessofaugmentedrealityinobject
assembly,inProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystems.ACM,pp.73
80,(2003).
[8] Yang,p.,wu,w.,Moniri,m.,Chibelushi,c.c.,ASensorbasedSLAMAlgorithmforCameraTrackingin
VirtualStudio.InternationalJournalofAutomationandComputing,05,152162,(2008).
[9] https://www.infiniti.co.uk/cars/newcars/q30/q30augmentedreality.html(lastrequest:20160809).
[10] http://ilab.cs.ucsb.edu/projects/taehee/HandyAR/HandyAR.html(lastrequest:20160809).
[11] http://sketchupdate.blogspot.rs/2011/01/augmentedrealityforsketchup.html?m=1(lastrequest:2016
0809).
[12]http://www.architectmagazine.com/technology/products/threeaugmentedandvirtualrealityappsfor
designandconstruction_o(lastrequest:20160809).
[13] http://www.33rdsquare.com/2016/03/willaugmentedrealityheadsetsreplace.html(lastrequest:2016
0809).