The Reality of Augmented Reality

Mark Billinghurst mark.billinghurst@hitlabnz.org The HIT Lab NZ, University of Canterbury December 2011

Falling in Love

1989

Virtual Reality Was COOL!

Joining the HIT Lab in Seattle

Only $250K for 1500 polygons/sec!

My VR Predictions
I knew everyone would use VR when:
HMDs were cheap (<$300) Computers generate millions of polys/sec Tracking was inexpensive Good 3D input devices

Cheap HMDs

Gartner Hype Cycle

1990-95

1995-2000

 April 2007 Computer World

VR Voted 7th on list of 21 biggest technology flops
- MS Bob Winner

Back to Reality
1999 Fred Brooks Whats Real about Virtual Reality In 1994 VR barely works In 1999 VR is now really real

3 stages of application maturity:

Demonstration Pilot Production There are successful VR application domains

VR Business Today

> $3-5 Billion VR business (+ > $150 Billion Graphics Industry)

Visualization, simulation, gaming, CAD/CAE, multimedia, graphics arts

> $3-5 Billion

Lessons learned
Dont believe the hype Many factors determine technology acceptance
- Design for users

Need to move from Demo to Production

- Profitable niche markets

Follow the money

Whats Real About Augmented Reality?

Key Questions
Where is AR technology today? What are the key opportunities? What are research obstacles? Where will the technology be in 5-10 years?

AR History
1960s 80s: Early Experiments 1980s 90s: Basic Research
Tracking, displays

1995 2005: Tools/Applications

Interaction, usability, theory

2005 - : Commercial Applications

Games, Medical, Industry

2007 - AR Reaches Mainstream

MIT Technology Review March 2007 list of the 10 most exciting technologies Economist Dec 6th 2007 Reality, only better

Google Searches for AR

2009 Key crossover point More people interested in AR than VR

Gartners top 10 disruptive technologies 2008-2012:

Multicore and hybrid processors Virtualisation and fabric computing Social networks and social software Cloud computing and cloud/Web platforms Web mashups User Interface Ubiquitous computing Contextual computing Augmented reality Semantics

AR Technology Today

1977 Star Wars

Key Features
Classic Augmented Reality (Azuma 97)
Combines Real and Virtual Images
- Display technology

Interactive in Real-Time
- Real time graphics

Content Registered in 3D
- Viewpoint tracking

Other Features
Shared 3D viewing
- Individual views

HMD, HMD, HMD...

Tracking, Tracking, Tracking

Feature Points Contours

Surfaces

1999 - HIT Lab US Shared Space

AR Conferencing

Moves conferencing from the desktop to the workspace

Virtual Viewpoint Generation

3D Live System

2008 - CNN

The Red Planet (2000)

Outdoor AR Flexible Lens Surface

Bimanual interaction Digital paper analogy

AR FlexiLens

Real handles/controllers with flexible AR lens

2008: Location Aware Phones

Motorola Droid

Nokia Navigator

HIT Lab NZ Outdoor AR Platform

Cross platform
Android, iPhone

3D onsite visualization
Intuitive user interface

Positions content in space

Camera, GPS, compass

Client/Server software architecture Targeting museum guide/outdoor site applications

Earthquake Reconstruction

See past, present and future building designs Earthquake survivor stories shown on map view Collect user comments Android platform

IronMan2

Natural Hand Interaction (2011)

Using bare hands to interact with AR content

MS Kinect depth sensing Real time hand tracking Physics based simulation model

AR Today
Key Technologies Available
- Robust tracking (Computer Vision, GPS/sensors) - Display (Handheld HMDs) - Input Devices (Kinect, etc) - Developer tools (Qualcomm, Metaio, ARTW)

Commercial Business Growing

- Gaming, GPS/Mobile, Online Advertisement
>$5 Billion USD by 2016 (MarketsandMarkets) >$1.5 Billion USD in Mobile AR by 2014 (Juniper Research)

AR Business Today
Marketing
Web-based, mobile

Mobile AR
Geo-located information and service Driving demand for high end phones

Gaming
Mobile, Physical input (Kinect, PS Move)

Upcoming areas
Manufacturing, Medical, Military

Research Directions

Important Research Directions

Fundamental Technologies
Unobtrusive displays Ubiquitous tracking

Improving the user experience

Interaction, Information management Social networking + AR

Standards (Content, interaction) Integrating with related technologies

AR User Experience

Future Displays

Always on, unobtrusive

Contact Lens Display

Babak Parviz
University Washington

MEMS components
Transparent elements Micro-sensors

Challenges
Miniaturization Assembly Eye-safe

Contact Lens Prototype

Information Presentation

Public and private annotations Aid recognition, extended memory

Wikitude www.mobilizy.com
Blah
Blah Blah Blah Blah

Blah Blah
Blah Blah

Blah

Blah

Blah Blah

Blah Blah Blah Blah

Blah

Blah Blah

Blah

Blah

Information Filtering

Information Filtering
Information Filtering (Julier et al. 00)

Remove clutter by goal- and distance based filtering Users task is route finding: Sniper and relevant buildings are
displayed; objects, which are determined to be unnecessary, removed

Metaverse
Neal Stephensons SnowCrash The Metaverse is the convergence of:
1) virtually enhanced physical reality 2) physically persistent virtual space

Metaverse Roadmap
http://metaverseroadmap.org/

Metaverse Dimensions
Augmentation technologies that layer information onto our perception of the physical environment. Simulation refers to technologies that model reality Intimate technologies are focused inwardly, on the identity and actions of the individual or object; External technologies are focused outwardly, towards the world at large;

Mirror Worlds
Mirror worlds are informationally-enhanced virtual models or reflections of the physical world.
Google Earth, MS Street View, Google Maps

LifeLogging
Technologies record and report the intimate states and life histories of objects and users
Nokia LifeBlog, Nike+

Ubiquitous AR (GIST, Korea)

How does your AR device work with other devices? How is content delivered?

ubiHome @ GIST
Media services Light service MR window ubiTrack

Where/When

Tag-it
ubiHome

ubiKey
Who/What/ When/How

What/When/How

PDA

Couch Sensor

Door Sensor

Who/What/When/How

When/How

When/How

CAMAR GIST
(Context-Aware Mobile Augmented Reality)

Conclusions

Conclusions
AR is becoming a real industry Key areas of everyday use include
Location Based, Gaming Web-based, Mobile AR

Important research for the future

Tracking, Interaction, Displays, User experience

More Information

Mark Billinghurst
mark.billinghurst@hitlabnz.org

Website
http://www.hitlabnz.org/