This presentation is a part of a talk I was invited to give on the topic of Augmented Reality and Virtual Worlds. This talk, organized by IEEE, aimed at introducing the technology to students and discuss the scope and research associated with it. Qualcomm's Vuforia platform is used as a prototype.

1. Augmented Reality
Possibilities and Challenges
Parth Darji
Senior AR Engineer
Kagiso Interactive Media

2. Augmented Reality
Possibilities and Challenges
Introduction to Augmented reality
Types of Augmented Reality
Abilities and Scope
Current Research
Demo applications
Introduction to Virtual Reality
Applications of Virtual Reality
Qualcomm Vuforia™ AR SDK
Hands-on experience of developing an AR application
Choosing AR and VR as career path
Conclusion
Questions and suggestions
Annexure: Further Reading
Annexure: Useful links

3. Introduction to Augmented
Reality
Quoting Wikipedia, “Augmented reality (AR) is a live direct or indirect view
of a physical, real-world environment whose elements are augmented (or
supplemented) by computer-generated sensory input such as sound, video,
graphics or GPS data.”
Google says “Augmented reality (AR) a technology that superimposes a
computer-generated image on a user's view of the real world, thus providing
a composite view.
But I think Jay Wright, business-development director at technology
company Qualcomm Inc. describes it best when he says – ‘It’s the real world
– only better’.

4. GPS and Compass based:
• This one uses the on-board GPS and compass to determine a user’s current
location and augment the camera screen – intelligently.
• Usually for mobile phones and other hand-held devices.
Advantages:
• It can give the user an intelligent data about his/her surroundings by augmenting
it with location markers, navigation, etc.
Disadvantages:
• The current limitations of GPS technology makes
• tracking inaccurate.
Examples?
Future scope?
Types of Augmented
Reality

5. Marker-based
Types of Augmented
Reality
• Probably the most common AR type in the market currently.
• It uses a camera and a visual marker to determine the center, orientation and range of its
spherical coordinate system.
Advantages
• Easily affordable, lots of SDKs in the market.
Disadvantages:
• Comparatively weak
• Few useful applications,
• Usually ends up being just a ‘wow’ factor.
New SDKs have entered the which allow
using real-world 3D objects as markers.

6. Marker-less
• Currently the best technology for tracking.
• Augments real world environments without the
need of specially placed markers.
• Allows more complex applications of AR.
Examples?
Types of Augmented
Reality

7. Types of Augmented
Reality
Wearable techs:
• This one blurs the lines between augmented reality and
virtual reality, in that wearable devices not only augment the
environment around the user, but also try to immerse the
user into a partly computer-generated environment.
• The recent buzz around Google Glass has been a pointer to
what this technology is going to be in the near future.
• There is plenty of research going on on wearable techs and
their applications in Augmented Reality and Virtual worlds.
• Other Examples: Oculus VR, Meta, DAQRI Smart Helmet, etc.

8. Types of Augmented
Reality
Special mention – Vuforia Smart Terrain™
• A special type of AR which combines both marker-based and
markerless.
• Here, one marker is used to calibrate the real world and the
augmented world coordinate systems, and then anything else in
the camera view can be used to reconstruct and augment the
physical environment into its graphical counterpart, to create new
kinds of gaming and visualization applications.

9. Smart Terrain™ in action
Video Link:
https://www.youtube.com/watch?v=9ZcB3-
35VFE
• More examples?

10. Abilities and Scope of AR
So, after looking at the basic theory around AR,
what could we possibly achieve using this
technology?
• Entertainment
• Ease of access to information using graphically put content
• Road safety (iOnRoad)
• Games
• The “wow” factor
• Others?

11. Scope:
Abilities and Scope of AR
• AR is a relatively new technology in practice and there’s a lot of research
going on in this area. Although as a concept, AR emerged as early as
1990 and a first head mounted display was built as early as 1996, AR has
been available for open development very recently. With the current
crop of wearable technology and cutting edge computer vision, AR is
developing exponentially each passing day. I personally believe that
within the next decade, it would be a technology we would use in our
day-to-day lives.
Quick fact: AR and VR market is expected to grow at a CAGR
of 15.18% from 2013 to 2018 and reach $1.06 Billion in 2018.

12. A look at current research
Majority of current research revolves around wearable techs. A lot of companies are working on
Computer Vision, transparent displays, low powered processors, depth sensors, etc. to make the AR
experience as immersive and interactive as possible. A lot of these projects are crowd-funded and
others are funded by tech giants like Google and Facebook.
Israel is doing a tremendous job in terms of Computer Vision R&D and development of nano-sized
depth sensors which we’ve started seeing in our smart phones (HTC One m8).
The next crop of smart phones would commonly have these depth sensors which would allow us to
gather more information of the real world around us and develop applications which would use this
data and do meaningful processing out of it.
A major role in AR application is played by how the application interacts with the users. So a lot of
research is also going on in the field of Human Computer Interaction. Presenting such a complex
technology to laymen is a tremendous task and researchers are working to make it all the more
accessible.

13. A look at current research
Head mounted displays are becoming more and more popular, thanks to the advent of Google
glass. The days are not far when we would be able to play our favorite RPG games where we
would be the heroes of the game actually being inside it.
Smart watches are being a common thing these days and almost every big smart phone manufacturer is trying
its hands on pushing a new smart watch into the market every now and then. With processors becoming
smaller and more battery intensive, these watches would be able to do wonders. Can you think of a smart
watch which has a camera on it and as you move your hand, it shows you the Facebook profile of the person
you’re looking at? Fancy much, eh?
Think of your smart phone as a dashboard mounted GPS device. You can see the distance from the car ahead of you along with
the speed at which it is moving so you can control your speed accordingly. You enter a destination and it colors the entire road
green on your mobile phone so you no longer have to worry about missing a right-turn along the way. You’re busy talking to the
person next to you and your phone suddenly shows you a red flag on the road telling you that you’re going off track?
Okay, how about a tablet for each student at school where he/she can scan the textbook and all the content
comes up as an animated 3d model or a movie? How about that for learning? The DNA molecule that you had
problems cramming would be popping out of your textbook and rotating in 3d and you can zoom in/out on
particular areas which you find difficult to understand? Would that help you remember it better?
All this has been possible or will be possible only because of the rigorous research being put into
the area of Augmented Reality and Virtual Worlds. The opportunities are endless and so are the
possibilities. The only bottleneck to this technology would be your imagination and your
creativity.

14. A look at current research
A lot of research is being put into using this technology for military purposes. AR could provide a
lot of information to the troops about their surroundings.
Gaming: AR can drastically change the course of gaming. Current research is based on involving
real world objects into a game as well as placing computer generated content in the real world
which the user can interact with. Example: Zombie killer.
Medical: A lot of research is going on in the field of medical applications of AR. One such example
would be superimposing MRI scan data directly on to a patient’s head so as to pinpont the
location of a tumor. Similarly, AR can also help achieve greater insight into some of the most
unreachable parts of the human body.
Education: Researchers are trying to make lab experiments more immersive and
educational. AR can help students learn more about the experiments they are
participating in.

15. GPS and Compass based:
• Wikitude
• Yelp monocle
• iOnRoad
• Google Ingress
Marker based:
• Layar
• Blippar
Markerless:
• Kinect, Xbox games
Demo applications

16. Introduction to Virtual
Reality!
“Virtual Reality (VR), sometimes referred to as immersive multimedia, is
a computer-simulated environment that can simulate physical presence in places
in the real world or imagined worlds.” – Wikipedia
Virtual reality can recreate sensory experiences, which include virtual
taste, sight, smell, sound, touch, etc., but here we will primarily be looking at
vision based virtual reality.
Basically, virtual reality is a technology where a user is tried to be placed in an
environment other than the real world. For example, when you put on a head
mounted display, you see a computer generated graphic content around you in a
360 degree view. You still are in the real world, but your senses perceive it as a
virtual world.

17. Introduction to Virtual
Reality!
Lets look at a video which exactly emphasizes the value of this technology.
Video Link:
https://www.youtube.com/watch?v=dO4k2
Rvs94I

18. Introduction to Virtual
Reality!
Let’s look at another video – a demo video for Oculus VR and we would
be able to better understand the concept, possibilities and scope of VR.
Video Link:
https://www.youtube.com/watch?v=kznm
_qMR0-4

19. Virtual Reality has been very popular with the Hollywood film industry. A lot of
movies are made using green screen chroma-keying techniques and inserting virtual
reality content in the real world.
Education and training
• Training and education can be done using VR because it can prepare you for dangerous
situations and put the worker in risky real life situation without the risk of them being
hurt.
• Examples: driving simulator, flight simulator, etc.
Museums
• Visitors can have a virtual tour to history by making them the characters of a 3d
environment which depicts the details of history as it happened.
Others?
Other Applications of
Virtual Reality

20. Qualcomm Vuforia™
Augmented Reality SDK
Now, lets delve into some practical stuff where we would pick an
SDK and try to develop a working AR application from it. For
simplicity, we will use the Android platform, but you can convert
the application to iOS with relative ease.
For this, we shall choose the Qualcomm Vuforia™ SDK because of
the following reasons
• Free to use (almost)
• Vuforia™ delivers best-in-class computer-vision, advanced detection, tight optical
tracking, and cloud-based image recognition.
• Flawless Unity3d integration.
• Multiple platform support – Android and iOS.
• Variety of markers you can use for tracking.
• A community of over 1,00,000 developers so help is just a forum post away.

21. Qualcomm Vuforia™
Augmented Reality SDK
• Choosing Unity3d over native development?
• Advantages
– Cross-platform development.
– Support of powerful .NET libraries
– Easy integration of 3D objects
• Disadvantages
– Unity3d is an expensive software
– Increases application size
– Cannot use native library support

22. Choosing AR as a career
• Looking at all the prospects with this technology and the impressive
things it can do, I think anyone with a passion of doing research in
an evolving field would be greatly interested here.
• You get to work on something as interesting as space travel or
virtual teleportation – More on this later.
• You get to take yourself and the world to whole different level of
interaction with the technology around us.
• There is so much going on in the crowd funding domain that almost
anything that is vaguely interesting would find a decent amount of
money to work on the idea. All the tech giants are investing millions
of dollars in this area of research.
• Tremendous research opportunities: There are universities that
have dedicated AR and VR labs – MIT, GaTech, UCSD, UCB, Stanford,
Cornell, etc.
• Do you see a future here?
path

23. Conclusion
AR systems will instantly recognize what someone is looking at, and retrieve and
display the data related to that view.
There are hundreds of potential applications for such a technology, gaming and
entertainment being the most obvious ones.
Any system that gives people instant information, requiring no research on their
part, is bound to be a valuable to anyone in any field.
As AR evolves, the fine line between real and computer generated data would
become more and more blurry and our senses of sight, smell, hearing would be
enhance.

24. So what do you think? Like it? Don’t like
it? Indifferent?
Questions?
Suggestions?
Questions and
Suggestions

25. Annexure: Further
Reading
We looked at one of the many AR platforms available in the market. You can
go ahead and look at other SDKs that specifically cater to the need of your
applications.
Let me list down a few of them here:
• Metaio
• OpenCV
• Total Immersion
• Wikitude
• PointCloud
You can also go ahead and check out gesture based SDKs like Kinect, Intel
Perceptual Computing, xBox, etc. Details about these SDKs is out of the
scope of this talk, but you can always get back to me if you’re interested and
want more info

26. Annexure: Useful Links
• Really cool AR apps
• Promising AR devices
• Augmented World Expo - 10 best AR SDKs for
developers
• Download links:
– Unity3d download
– Vuforia Download - Registration required
• Tutorials:
– Vuforia - Getting Started
– Vuforia - Unity3d integration