GIS with Augmented Reality

1. Augmented reality :
A new visualisation method for GIS
PhD Researcher Sung Hyun Jang
Supervised by Mike Batty and Andy Hudson-Smith

2. Outline
• Introduction • Initial results
• Background – AR map using ARTag
– Geovisualisation Trend – UCL buildings in Layar
g y
– Ubiquitous computing • Conclusion
• Literature review • Future Work
– Geovisualisation
– About AR
• Definition
• History
– AR applications
• Methodology
– Data extract
– Scenario
– Matching
– Implementation and Evaluation 249

3. Introduction
“ If a picture is worth a thousand words, then an interface is
worth a thousand pictures” (Shneiderman 2001)
(Shneiderman,
“If an interface is worth a thousand pictures, then
a geovisualisation can save more than a
thousand lives”
349

4. Introduction
449

5. Introduction
Soho
+ Cholera death
Water pump
549

6. Background
Geovisualisation Trend
Data
Life
Cycle
C l
GIS Geovisualisation GIS
Platform Trend Uses
GIS
Data
649

7. Background
GIS data life cycle
1st Gen. : linear and finite
Data
Acquisition
A i iti
Data
Processing
P i
Geo-
Visualisation
749

8. Background
GIS data life cycle
2nd Gen. : Cloud Source
Data
Acquisition
A i iti
Data
Processing
P i
Geo-
Visualisation
Openstreetmap
849

9. Background
GIS data life cycle
3rd Gen. : Enhancing reality
g y
Data
Acquisition
q
Data Realistic
Processing view
Geo
Geo-
Visualisation
Openstreetmap
Real Accessible
time
ti
949

10. Background
Geovisualisation Trend – GIS Platform
Enterprise Pervasive
Desktop GIS Web GIS
GIS GIS
Embedded
ArcInfo ArcView &
Micro
GIS Specialist Enterprise WWW
1049

11. Background
Geovisualisation Trend – GIS data
2D 3D 3D + Realistic
3D
SHP DEM /DTM in real-time
Coverage 3D Model In real place
etc. etc. Enhancing reality
1149

12. Background
Geovisualisation Trend – GIS Uses
User Demand
Public Individual Context
Awareness
Policy
Decision Everyday Intelligent Cognition,
Cognition
Making technology Everyday technology,
Management
g and Calm technology
UIS StreetMap
(Urban Information System)
LIS Navigation
g
(Land Information System)
1249

13. Background
Short summary of new Geovisualisation trend
User demand
Pervasive
3D+Realistic Context
GIS Awareness
3D Intelligent cognition,
I t lli t iti
Embedded
E b dd d
in real-time Everyday technology,
Ubiquitous
&
In real place and Calm technology Computing
Micro
Enhancing reality
g y
1349

14. Background
Visualization for Ubiquitous Environment.
“Specialized elements of hardware and
software, connected by wires, radio waves
, y ,
and infrared, will be so ubiquitous that no
one will notice their presence”
p
(The Computer for the 21th Century, Mark Weiser, 1988)
1449

15. Background
Ubiquitous Space
The 1st Space Physical Space (Real space)
The 2nd Space Cyber Space (Virtual space)
Physical Space + Cyber Space = Linked Operating Space
y p y p p g p
Th 3rd S
The Space
(Ubiquitous Space)
Th 1st S
The Space Th 2nd S
The Space Th 3rd S
The Space
Space
(Real Space) (Virtual Space) (Ubiquitous Space)
Tech. Agriculture / Info.(IT, Computer) Space (Ubiquitous)
Revolution Industrial Revolution Revolution Revolution
Time 19th Century 20th Century 21st Century
• Ubiquitous IT revolution and the 3rd space, 2002, DH Kim 1549

16. Background
Ubiquitous Space
Internet
Location Based
Service
Invisible Embodied
I i ibl E b di d
Micro / Calm technology
Embedded
Intelligent
Devices
1649

17. Background
Ubiquitous Interface
Workers Everybody
1749

18. Background
Geovisualisation in Ubiquitous Interface
• Geovisualisation beyond the desktop GIS, web GIS
(Fixed location, screen, k b d and mouse)
(Fi d l ti keyboard d )
• Requirements:
q
Handheld, Portable, Wearable Devices
Very different scales or styles of output
Implicit input – location time, context
location, time
1849

19. Background
Geo-objects in Ubiquitous Environment
1949

20. Background
Visualise Geo-objects in Ubiquitous Environment
RFID
Building
information RFIDs
RFID Street Tree
RFID managing RFIDs
RFID RFID
RFID
Car information
RFIDs
Traffic Lights
Managing RFIDs
Pipe managing
RFIDs
Communication
Cables
Water Pipes
Gas Pipes
2049

21. Research Question
• Can geovisualisation support the ubiquitous
environment?
• What is the best way to visualise intelligent geo-
objects in the ubiquitous environment?
j q
2149

22. Overall research aim
IT Visualisation
Ubiquitous computing Geovisualisation
GIS in Geovisualisation
Ubiquitous With AR
Computing
p g
How can we visualise geo objects in ubiquitous environments?
geo-objects
– Dynamic visualisation: real-time, instant
– Intuitive interface: Calm technology, awareness
gy,
– with Augmented Reality
2249

23. Research Objectives
GIS in Geovisualisation
G i li ti
Ubiquitous Computing With AR
• Select GIS data can be used with AR
• P id d
Provide dynamic & real-time GIS data
i l ti d t
visualization using AR
• D
Develop GIS b
l GIS-based augmented reality service
d t d lit i
2349

24. Literature review
• Geovisualisation
– Definition of visualisation, geovisualisation
visualisation
– Challenges for geovisualisation
• Augmented reality
– Definition of AR
– History of AR
– Applications
2449

25. Literature review
What is geovisualisation?
• Visualisation
• The formation of mental visual images, the act or process of
interpreting in visual terms (Classic definition)
• In 1987, McCormick proposed visualisation taxonomy
• Exploring data and information g p
p g graphically, as a means of g
y gaining
g
GIS
understanding and insight into the data with
All activities
Software &and Wiseman, 1992: Brodlie et al., 1993)
(Earnshaw GIS data
Hardware
• Geo + visualisation
– The first use of the term visualisation in cartographic literature
Interface
– Ten principles common to cartography and geography
(Philbrick, 1953)
– A set of tools and techniques supporting geospatial data analysis
through the use of interactive visualisation (K k and M
(Kraak d Maceachren, 1999)
h
2549

26. Literature review
Challenges for geovisualisation
• Paper maps were designed to be both database and
presentation media. (MacEachren and Kraak, 2000)
• 80% of all di it l d t generated t d i l d geospatial
f ll digital data t d today include ti l
referencing (e.g., geographic coordinates, address, postal
codes, etc.)
codes etc ) (MacEachren and Kraak 2000)
Kraak,
• Geovisualisation has the potential to provide ‘windows’
into the complexity of the phenomena and processes
involved, through innovative scene construction, virtual
environments and collaboration (MacEachren and Kraak, 2000)
• Two current keywords in geovisualisation: Interaction,
Dynamic (MacEachren, 1994)
2649

27. Literature review
Why Augmented Reality
2749

28. Literature review
Why Augmented Reality
• Intuitive: : A more intuitive interface to access complex real
world
• Real – time: Real world should be a back ground
• Dynamic: Dynamic features
• Instant: Quick response
• Link real and virtual world: Enhancing the real world
2849

29. Literature review
Definition of AR
• A combination of a real scene viewed by a user and
a virtual scene generated by a computer that
augments the scene with additional information.
(Azuma 1997; Azuma, Baillot, Behringer, Feiner, Julier & MacIntyre 2001)
2949

30. Literature review
Reality – Virtuality (RV) Continuum
(Milgram & Colquhoun 1999)
3049

31. Literature review
A Brief History of AR
• 1960’s :
• Ivan E Sutherland
E.
• The ultimate display
The ultimate display would, of course, be a room
within which the computer can control the existence
of matter. A chair displayed in such a room would be
good enough to sit in.
…….
With appropriate programming, such a display could
literally be the Wonderland into which Alice walked
walked.
3149

32. Literature review
A Brief History of AR
• Early 1990’s : Boeing coined the term “AR” in its
Wire harness assembly application
(Anthony E. Majoros , Paul R. Jackson)
• Early 1990’s : UNC ultrasound project
y j
• 1994: Motion Stabilized display
3249

33. Literature review
A Brief History of AR
• 1994: Fiducial tracking in video see through AR
see-through
• 1996: UNC hybrid magnetic-vision tracker
• 2000: Augmented sports broadcasts
• 2009: Mobile AR ( o e 3GS, Android etc )
009 ob e (iPhone d o d etc.)
3349

34. Literature review
Components of augmented reality system
Component Type Description
Outdoor GPS
Positioning
Indoor 2D Tag, RFID
Digital Compass Direction of your sight (Horizontal)
User
Perspective
Accelerometers Angle of your sight (Vertical)
Optical see through Optical combiners in front of the user s eyes
user’s
Combining a closed-view HMD with one or
Display Video see through
two cameras
Combining virtual objects and real objects on
Monitor base
a monitor
3449

35. Literature review
Display materials
3549

36. Literature review
Tag based augmented reality
• ARToolKit: Washington Univ. HIT lab
• ARTag: Columbia Univ
Univ.
• Library for tag-based AR applications
• Open Source, Multi-Platform
• Solves two significant p
g problems in AR
• Tracking
• Registration
• Overlays 3D virtual objects on real markers
• Uses single tracking marker
3649

37. Literature review
ARToolKit
3749

38. Literature review
Tag-based Flash AR
3849

39. Literature review
Mobile augmented reality
• 2009 was the turning point for mobile augmented
reality
• Combined Location Based Service (LBS)
• C
Capabilities of mobile d i
biliti f bil devices hhave i
improved
d
– Built in GPS, Digital Compass, Accelerometers
• Wireless communication widely used
– Improving positioning
– WiFi, 3G
3949

40. Literature review
Nearest Tube station
4049

41. Initial Result
GIS ARMap
• Library : ARTag (2D tag-based)
– 2D rectangles with distinctive p
g patterns
• 3D data format
– 3D objects loaded from WRL (VRML) OBJ (Wavefront
(VRML), (Wavefront,
Maya), ASE (3D-Studio export)
4149

42. Initial Result
GIS ARMap
4249

43. Initial Result
UCL in Layar
• Platform : Layar
• Data format: JSON
• Information: UCL buildings
– Location, Image, Website, Phone Number
• Architecture
4349

44. Initial Result
UCL in Layar
4449

45. Conclusion and Future work
Conclusion
• Augmented reality in GIS
– Promising visualisation method for GIS data
– AR supports dynamic (time, location, user interface)
geovisualisation
– Need to improve positioning/ rendering technology
• GIS in Augmented reality
– Combine Location Based Service
–NNew fi ld with various challenges
field ith i h ll
– Need to select suitable GIS data for augmented reality
4549

46. Future work
Application Scenario
• AR Urban Information System
4649

47. Future work
Future work
• New user interface in static situation
– FLARTookit 2.0
20
• Image matching
– Visual Tag
– A l 3D t 2D T
Apply to Transformation
f ti
• Use tags as Ground Control Points (GCPs)
• Matching GIS data with real world
• Indoor positioning
– RFID
4749

48. Future work
Future work
3D 2D Tag
GIS With
Data Geo ref
3D t 2D
to
Transformation
2D Screen
4849

49. Future work
3D to 2D Transformation
4949

50. Future work
Application Scenario
• London Olympic 2012 with GIS AR (Olympic 2012 team)
Location Based
Future Sight Service
What’s going on
Service 5049

51. Future work
Future work
• Data acquisition
– POI data related to the Olympics
• Stadiums, tourist Information desks, ticket offices, etc.
– 3D model of the Olympic buildings
• Application Development
– 3D data on the Layar platform
– Using DBMS with Layar
• D
Develop more scenarios relating t th L d
l i l ti to the London
Olympics
5149

52. Questions and
comments?
Thank
Th k you.
5249