SlideShare a Scribd company logo
1 of 19
Download to read offline
www.itee.uq.edu.au/research/co-innovation
Co-Innovation Research ITEE
Language in social technologies:
what happens when AI meets VR?
Janet Wiles
Co-Innovation Group
ARC Centre of Excellence for the Dynamics of Language &
School of Information Technology & Electrical Engineering
University of Queensland
SIGGRAPH Brisbane, 17 November 2019
Overview
1. Robots learning language
Studies in grounding
2. Language in AI: the promise of VR/AR
Common ground: learning new words, words in context;
shared understanding; directing attention
3. VR and the brain
Conversation: checking back, interactions, speed
3
A family of child-friendly robots for
research into the effects of social
robots on learning from tablets.
Design: ongoing, iterative, immersive
UQ Social robots
A bio-inspired robot for research at the
intersection of neurorobotics,
neuroscience and embodied cognition.
Design: D Ball; Programming: S Heath
OPIEiRat
Lingodroids: Could a robot
understand language?
Scott Heath
Ruth Schulz and
the Lingodroids
team
CATgrounded meaning
Common ground and
the triangle of meaning
Representamen
Object Interpretant
Can a child teach the
robot words for colour?
Kristyn Hensby
Scott Heath
OPAL Team
OPIE
Language
games
purple
purple
purple
purple
INTENSITY
HUE
blue
Colour space
blue
name-this-colour?
blue
what’s-the-difference?
Scott Heath
lighter
gifugrounded meaning
Common ground and
the triangle of meaning
Representamen
Object Interpretant
yozozuku
Which word refers to space
and which to time?
Scott Heath
spacetime
Heath et al (2016)
Lingodroids:
Cross-situational
learning, IEEE
Trans CDS
An individual robot’s semiotic network
(based on Pierce’s semiotic triangle)
The Lingodroid sign system
Internal representation
Symbol
Referent
social
grounding
symbol grounding
private
grounding
Schultz et al (2012)
Adaptive Behavior
Representamen
Object Interpretant
Robot sensors and “Deixis”
4/5
Insights from the Lingodroid studies
• The Lingodroids are embodied and enactive
• Words are not grounded in the direct perceptions of the
robots, but rather, in the integrated navigation system: the
nodes of the map, and the relational spaces of distance
and direction
Each word has a meaning as part of a wider network (extended)
• Common ground depends on shared culture
(understanding of the activity the robots are engaged in –
which game they are playing: where-are-we vs how-far)
Culture assigns meaning to activities which then becomes
encoded when the robots name common ground
Activities that involve common ground are intrinsically
embedded in culture
In theory there is no
difference between theory
and practice.
In practice there is.
Saying in robotics; attributed variously to baseball
player Yogi Berra, computer scientist Jan L. A. van
de Snepscheut and physicist Albert Einstein.
iRat
Biological systems are
clock-free, grid-free, scale-free & symbol-free
Brain regions synchronize over a range of timescales.
Fast local coupling enables rapid communication,
slow episodes require sustained coupling.
Buzsaki&Mizuseki,NatureReviewsNeuro,2014
1 second
Time
(msecs) Function
< .1 location from stereo sound;
1 neuron spike width
10 communication in neural regions
100 social moments;
1000 cognitive moments
Timescales of fluid conversation
(human-to-human)
Speaker 1
200
time (milliseconds)
700
• The gap between speakers is typically 200 msec
• The time to process an utterance is 500-700 msec
• Social micro abilities provide a scaffold for these rapid exchanges
Speaker 2
How the brain represents space
16Drieu and Zugaro, Hippocampal sequences… Front. Cell. Neurosci., 2019
Placecells
How a brain
constructs
space-time
The representation of space
in the brain
Grieves & Jeffery, (2017)
Behav Proc. 135 Fig. 8
17
“VR breaks the laws of physics,” Mayank Mehta, UCLA
• “The rats in the virtual maze found
hidden rewards as skillfully as rats in a
real maze. … Despite this, the neural
activity in their hippocampus was highly
abnormal in virtual reality. More than half
of all neurons in the hippocampus shut
down in virtual reality and the remaining
neurons fired in a disordered fashion
• “If the vision says you are moving in
space, but the sounds and smells tell
otherwise, as is the case in virtual reality,
the neural activity becomes very strange.
• “VR … removes the consistent
relationship between different stimuli in
the world that all the animals have used
for millions of years.”
www.wired.com/2013/05/virtual-reality-spatial-sense/
Space, Time & Imagination: How Virtual Reality Changes the Brain | Mayank Mehta | TEDxUCLA
ITEE/Co-Innovation Group: Janet Wiles, Scott Heath, Jonathon Taufatofua, Nik Rybak, Josh Arnold,
Josh Riddell, Gautier Durantin, Ola Olsson, Carlos Ramirez, Andrew Back; Scott Frazier, Beth Cave,
Dylan, Alumni: Kris Rogers, Michael Smith, Rollsy Ponmattam Madassery, Lachlan Fitzpatrick,
Autumn Qiu Jing Chew, Nick Buckeridge, Hugh Raynor, Wilson Kong; ITEE/Robotics: Pauline Pounds;
ITEE/Interaction Design: Pete Worthy, Jason Weigel, Steve Viller, Ben Matthews, Marie Boden
School of Psychology Early Cognitive Development Centre: Kristyn Hensby, Virginia Slaughter, Mark
Nielsen, Tamara van der Zant
School of Communication and Arts/Discursis: Dan Angus, Yvonne Yu
Bond U+ITEE: Helen Chenery, Jacki Liddle, Brooke-Mai Whelan
Melbourne U: Jill Wigglesworth; Ngukurr language centre: Greg Dickson, Jackie van den Bos,
Angelina Joshua, Tamara Joshua, Grant Thompson, Jordan
UCSD Machine Perception Lab: Deborah Forster, Javier Movellan, Marian Bartlett,
Mohsen Malmir; HARLIE: David Ireland (CSIRO), Christina Atay, + Harlie team
computer scientists
mechatronics
electrical & software
engineers
roboticists
designers
psychologists
Speech pathologists
occupational
therapist
linguists
CoEDL Scholars
MPhil, PhD
THE SOCIAL TECHNOLOGIES TEAMS: LINGODROIDS, OPAL, HARLIE, DISCURSIS
j.wiles@uq.edu.au

More Related Content

What's hot

Improving Artificial Intelligence by Studying the Brain
Improving Artificial Intelligence by Studying the BrainImproving Artificial Intelligence by Studying the Brain
Improving Artificial Intelligence by Studying the BrainTahoe Silicon Mountain
 
2010 - Projeto Abelhas de Blackawton
2010 - Projeto Abelhas de Blackawton 2010 - Projeto Abelhas de Blackawton
2010 - Projeto Abelhas de Blackawton Ezequiel Redin
 
Remote control brain technology
Remote control brain technologyRemote control brain technology
Remote control brain technologyPankaj Rattan
 

What's hot (7)

superintelligence
superintelligencesuperintelligence
superintelligence
 
Improving Artificial Intelligence by Studying the Brain
Improving Artificial Intelligence by Studying the BrainImproving Artificial Intelligence by Studying the Brain
Improving Artificial Intelligence by Studying the Brain
 
2010 - Projeto Abelhas de Blackawton
2010 - Projeto Abelhas de Blackawton 2010 - Projeto Abelhas de Blackawton
2010 - Projeto Abelhas de Blackawton
 
200905 - Sociable machines
200905 - Sociable machines200905 - Sociable machines
200905 - Sociable machines
 
Remote control brain technology
Remote control brain technologyRemote control brain technology
Remote control brain technology
 
Foundation of A.I
Foundation of A.IFoundation of A.I
Foundation of A.I
 
Research takefuji lab_20140609 Taketoshi Daikoku
Research takefuji lab_20140609 Taketoshi DaikokuResearch takefuji lab_20140609 Taketoshi Daikoku
Research takefuji lab_20140609 Taketoshi Daikoku
 

Similar to Siggraph Asia 2019 VR & AI workshop-wiles-handout

From artificially intelligent systems towards real thinking tools and human s...
From artificially intelligent systems towards real thinking tools and human s...From artificially intelligent systems towards real thinking tools and human s...
From artificially intelligent systems towards real thinking tools and human s...Jorn Bettin
 
Marianne Wolf
Marianne WolfMarianne Wolf
Marianne WolfIRI
 
The importance of collective intelligence, and empathy in digital environments
The importance of collective intelligence, and empathy in digital environmentsThe importance of collective intelligence, and empathy in digital environments
The importance of collective intelligence, and empathy in digital environmentsKatri Saarikivi
 
Sensing And Manipulation Through Touch
Sensing And Manipulation Through TouchSensing And Manipulation Through Touch
Sensing And Manipulation Through TouchAmanda Hengel
 
Phys of Life Reviews 2014_11_83-84
Phys of Life Reviews 2014_11_83-84Phys of Life Reviews 2014_11_83-84
Phys of Life Reviews 2014_11_83-84Subrata Ghosh
 
Digital Art and Philosophy #5
Digital Art and Philosophy #5Digital Art and Philosophy #5
Digital Art and Philosophy #5Melanie Swan
 
Oshri,Khandwala,Chopra199
Oshri,Khandwala,Chopra199Oshri,Khandwala,Chopra199
Oshri,Khandwala,Chopra199Barak Oshri
 
SR-1011_S&T_Map_2005-2055
SR-1011_S&T_Map_2005-2055SR-1011_S&T_Map_2005-2055
SR-1011_S&T_Map_2005-2055Jake Wachman
 
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...Numenta
 
Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?
Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?
Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?Seismonaut
 
NeuroWeb Roadmap: Results of Foresight & Call for Action
NeuroWeb Roadmap: Results of Foresight & Call for ActionNeuroWeb Roadmap: Results of Foresight & Call for Action
NeuroWeb Roadmap: Results of Foresight & Call for ActionPavel Luksha
 
*Handout10 tranimal cardiff
*Handout10 tranimal cardiff*Handout10 tranimal cardiff
*Handout10 tranimal cardiffKatie King
 
AI Fables, Facts and Futures: Threat, Promise or Saviour
AI Fables, Facts and Futures: Threat, Promise or SaviourAI Fables, Facts and Futures: Threat, Promise or Saviour
AI Fables, Facts and Futures: Threat, Promise or SaviourUniversity of Hertfordshire
 

Similar to Siggraph Asia 2019 VR & AI workshop-wiles-handout (20)

The importance of collective intelligence, and empathy in digital environment...
The importance of collective intelligence, and empathy in digital environment...The importance of collective intelligence, and empathy in digital environment...
The importance of collective intelligence, and empathy in digital environment...
 
From artificially intelligent systems towards real thinking tools and human s...
From artificially intelligent systems towards real thinking tools and human s...From artificially intelligent systems towards real thinking tools and human s...
From artificially intelligent systems towards real thinking tools and human s...
 
Marianne Wolf
Marianne WolfMarianne Wolf
Marianne Wolf
 
The importance of collective intelligence, and empathy in digital environments
The importance of collective intelligence, and empathy in digital environmentsThe importance of collective intelligence, and empathy in digital environments
The importance of collective intelligence, and empathy in digital environments
 
Transhumans: Technology Powered Superhumans
Transhumans: Technology Powered SuperhumansTranshumans: Technology Powered Superhumans
Transhumans: Technology Powered Superhumans
 
Knowledge and Self
Knowledge and SelfKnowledge and Self
Knowledge and Self
 
Sensing And Manipulation Through Touch
Sensing And Manipulation Through TouchSensing And Manipulation Through Touch
Sensing And Manipulation Through Touch
 
Phys of Life Reviews 2014_11_83-84
Phys of Life Reviews 2014_11_83-84Phys of Life Reviews 2014_11_83-84
Phys of Life Reviews 2014_11_83-84
 
Digital Art and Philosophy #5
Digital Art and Philosophy #5Digital Art and Philosophy #5
Digital Art and Philosophy #5
 
Oshri,Khandwala,Chopra199
Oshri,Khandwala,Chopra199Oshri,Khandwala,Chopra199
Oshri,Khandwala,Chopra199
 
SR-1011_S&T_Map_2005-2055
SR-1011_S&T_Map_2005-2055SR-1011_S&T_Map_2005-2055
SR-1011_S&T_Map_2005-2055
 
The ever existing gaps in indian research –
The ever existing gaps in indian research –The ever existing gaps in indian research –
The ever existing gaps in indian research –
 
Nano wylie
Nano wylieNano wylie
Nano wylie
 
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
Brains@Bay Meetup: Open-ended Skill Acquisition in Humans and Machines: An Ev...
 
Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?
Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?
Micah Allen: Zombies or Cyborgs: Is Facebook eating your brain?
 
NeuroWeb Roadmap: Results of Foresight & Call for Action
NeuroWeb Roadmap: Results of Foresight & Call for ActionNeuroWeb Roadmap: Results of Foresight & Call for Action
NeuroWeb Roadmap: Results of Foresight & Call for Action
 
*Handout10 tranimal cardiff
*Handout10 tranimal cardiff*Handout10 tranimal cardiff
*Handout10 tranimal cardiff
 
Alien Minds
Alien MindsAlien Minds
Alien Minds
 
AI Fables, Facts and Futures: Threat, Promise or Saviour
AI Fables, Facts and Futures: Threat, Promise or SaviourAI Fables, Facts and Futures: Threat, Promise or Saviour
AI Fables, Facts and Futures: Threat, Promise or Saviour
 
Ai its nature and future
Ai its nature and futureAi its nature and future
Ai its nature and future
 

More from Joaquim Jorge

Matera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdfMatera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdfJoaquim Jorge
 
Presentation at SMI 2023
Presentation at SMI 2023Presentation at SMI 2023
Presentation at SMI 2023Joaquim Jorge
 
How to Craft and Deliver Winning Presentations
How to Craft and Deliver Winning PresentationsHow to Craft and Deliver Winning Presentations
How to Craft and Deliver Winning PresentationsJoaquim Jorge
 
Virtual and Augmented Reality in Health Care & Rehabilitation
Virtual and Augmented Reality in Health Care & RehabilitationVirtual and Augmented Reality in Health Care & Rehabilitation
Virtual and Augmented Reality in Health Care & RehabilitationJoaquim Jorge
 
Siggraph asia 2019 Course slides
Siggraph asia 2019 Course slidesSiggraph asia 2019 Course slides
Siggraph asia 2019 Course slidesJoaquim Jorge
 
Siggraph asia 2019 panel pesentation ian hall
Siggraph asia 2019 panel pesentation ian hallSiggraph asia 2019 panel pesentation ian hall
Siggraph asia 2019 panel pesentation ian hallJoaquim Jorge
 
Morie frontiers nov2019
Morie frontiers nov2019Morie frontiers nov2019
Morie frontiers nov2019Joaquim Jorge
 
Embodiment cognition
Embodiment cognitionEmbodiment cognition
Embodiment cognitionJoaquim Jorge
 
Empathic Extended Reality
Empathic Extended RealityEmpathic Extended Reality
Empathic Extended RealityJoaquim Jorge
 
Debevec vrai-s-asia2019
Debevec vrai-s-asia2019Debevec vrai-s-asia2019
Debevec vrai-s-asia2019Joaquim Jorge
 
Glassner Puppies and Dragons vr+ai workshop
Glassner Puppies and Dragons vr+ai workshop Glassner Puppies and Dragons vr+ai workshop
Glassner Puppies and Dragons vr+ai workshop Joaquim Jorge
 
Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?Joaquim Jorge
 
Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?Joaquim Jorge
 
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...Joaquim Jorge
 
top 10 ways to get your paper rejected at Computers and Graphics Journal
top 10 ways to get your paper rejected at Computers and Graphics Journaltop 10 ways to get your paper rejected at Computers and Graphics Journal
top 10 ways to get your paper rejected at Computers and Graphics JournalJoaquim Jorge
 
Virtual Reality for Health Applications - siggraph asia 2018
Virtual Reality for Health Applications - siggraph asia 2018Virtual Reality for Health Applications - siggraph asia 2018
Virtual Reality for Health Applications - siggraph asia 2018Joaquim Jorge
 
ACM Europe October 2016
ACM Europe October 2016ACM Europe October 2016
ACM Europe October 2016Joaquim Jorge
 
Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...
Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...
Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...Joaquim Jorge
 
Game Over? New Approaches to Teaching Engineering Courses
Game Over? New Approaches to Teaching Engineering CoursesGame Over? New Approaches to Teaching Engineering Courses
Game Over? New Approaches to Teaching Engineering CoursesJoaquim Jorge
 

More from Joaquim Jorge (20)

Matera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdfMatera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdf
 
Presentation at SMI 2023
Presentation at SMI 2023Presentation at SMI 2023
Presentation at SMI 2023
 
How to Craft and Deliver Winning Presentations
How to Craft and Deliver Winning PresentationsHow to Craft and Deliver Winning Presentations
How to Craft and Deliver Winning Presentations
 
Virtual and Augmented Reality in Health Care & Rehabilitation
Virtual and Augmented Reality in Health Care & RehabilitationVirtual and Augmented Reality in Health Care & Rehabilitation
Virtual and Augmented Reality in Health Care & Rehabilitation
 
Siggraph asia 2019 Course slides
Siggraph asia 2019 Course slidesSiggraph asia 2019 Course slides
Siggraph asia 2019 Course slides
 
Siggraph asia 2019 panel pesentation ian hall
Siggraph asia 2019 panel pesentation ian hallSiggraph asia 2019 panel pesentation ian hall
Siggraph asia 2019 panel pesentation ian hall
 
Morie frontiers nov2019
Morie frontiers nov2019Morie frontiers nov2019
Morie frontiers nov2019
 
Embodiment cognition
Embodiment cognitionEmbodiment cognition
Embodiment cognition
 
Empathic Extended Reality
Empathic Extended RealityEmpathic Extended Reality
Empathic Extended Reality
 
Debevec vrai-s-asia2019
Debevec vrai-s-asia2019Debevec vrai-s-asia2019
Debevec vrai-s-asia2019
 
Glassner Puppies and Dragons vr+ai workshop
Glassner Puppies and Dragons vr+ai workshop Glassner Puppies and Dragons vr+ai workshop
Glassner Puppies and Dragons vr+ai workshop
 
Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?
 
Empathic Computing
Empathic ComputingEmpathic Computing
Empathic Computing
 
Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?Augmented Reality and Artificial Intelligence: Friends with Benefits?
Augmented Reality and Artificial Intelligence: Friends with Benefits?
 
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
 
top 10 ways to get your paper rejected at Computers and Graphics Journal
top 10 ways to get your paper rejected at Computers and Graphics Journaltop 10 ways to get your paper rejected at Computers and Graphics Journal
top 10 ways to get your paper rejected at Computers and Graphics Journal
 
Virtual Reality for Health Applications - siggraph asia 2018
Virtual Reality for Health Applications - siggraph asia 2018Virtual Reality for Health Applications - siggraph asia 2018
Virtual Reality for Health Applications - siggraph asia 2018
 
ACM Europe October 2016
ACM Europe October 2016ACM Europe October 2016
ACM Europe October 2016
 
Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...
Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...
Touching More than 3 Dimensions Research Into Novel Interfaces – three dimen...
 
Game Over? New Approaches to Teaching Engineering Courses
Game Over? New Approaches to Teaching Engineering CoursesGame Over? New Approaches to Teaching Engineering Courses
Game Over? New Approaches to Teaching Engineering Courses
 

Recently uploaded

Multicomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdfMulticomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdfGiovanaGhasary1
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabusViolet Violet
 
Pioneering Process Safety Management: The Future Outlook
Pioneering Process Safety Management: The Future OutlookPioneering Process Safety Management: The Future Outlook
Pioneering Process Safety Management: The Future Outlooksoginsider
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid BodyAhmadHajasad2
 
ingles nivel 3 ucv 2024 - modulo 3 _ppt2
ingles nivel 3 ucv 2024 - modulo 3 _ppt2ingles nivel 3 ucv 2024 - modulo 3 _ppt2
ingles nivel 3 ucv 2024 - modulo 3 _ppt2nhuayllav
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptxSaiGouthamSunkara
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxKISHAN KUMAR
 
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxVertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxLMW Machine Tool Division
 
How to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfHow to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfRedhwan Qasem Shaddad
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxwendy cai
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Amil baba
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingMarian Marinov
 
OS Services, System call, Virtual Machine
OS Services, System call, Virtual MachineOS Services, System call, Virtual Machine
OS Services, System call, Virtual MachineDivya S
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Apollo Techno Industries Pvt Ltd
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxNaveenVerma126
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Sean Meyn
 

Recently uploaded (20)

Multicomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdfMulticomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdf
 
Lecture 2 .pptx
Lecture 2                            .pptxLecture 2                            .pptx
Lecture 2 .pptx
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabus
 
Pioneering Process Safety Management: The Future Outlook
Pioneering Process Safety Management: The Future OutlookPioneering Process Safety Management: The Future Outlook
Pioneering Process Safety Management: The Future Outlook
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
 
ingles nivel 3 ucv 2024 - modulo 3 _ppt2
ingles nivel 3 ucv 2024 - modulo 3 _ppt2ingles nivel 3 ucv 2024 - modulo 3 _ppt2
ingles nivel 3 ucv 2024 - modulo 3 _ppt2
 
Présentation IIRB 2024 Chloe Dufrane.pdf
Présentation IIRB 2024 Chloe Dufrane.pdfPrésentation IIRB 2024 Chloe Dufrane.pdf
Présentation IIRB 2024 Chloe Dufrane.pdf
 
Lecture 2 .pdf
Lecture 2                           .pdfLecture 2                           .pdf
Lecture 2 .pdf
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptx
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptx
 
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxVertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
 
How to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdfHow to Write a Good Scientific Paper.pdf
How to Write a Good Scientific Paper.pdf
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptx
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
 
Litature Review: Research Paper work for Engineering
Litature Review: Research Paper work for EngineeringLitature Review: Research Paper work for Engineering
Litature Review: Research Paper work for Engineering
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & Logging
 
OS Services, System call, Virtual Machine
OS Services, System call, Virtual MachineOS Services, System call, Virtual Machine
OS Services, System call, Virtual Machine
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
 

Siggraph Asia 2019 VR & AI workshop-wiles-handout

  • 1. www.itee.uq.edu.au/research/co-innovation Co-Innovation Research ITEE Language in social technologies: what happens when AI meets VR? Janet Wiles Co-Innovation Group ARC Centre of Excellence for the Dynamics of Language & School of Information Technology & Electrical Engineering University of Queensland SIGGRAPH Brisbane, 17 November 2019
  • 2. Overview 1. Robots learning language Studies in grounding 2. Language in AI: the promise of VR/AR Common ground: learning new words, words in context; shared understanding; directing attention 3. VR and the brain Conversation: checking back, interactions, speed
  • 3. 3 A family of child-friendly robots for research into the effects of social robots on learning from tablets. Design: ongoing, iterative, immersive UQ Social robots A bio-inspired robot for research at the intersection of neurorobotics, neuroscience and embodied cognition. Design: D Ball; Programming: S Heath OPIEiRat
  • 4. Lingodroids: Could a robot understand language? Scott Heath Ruth Schulz and the Lingodroids team
  • 5. CATgrounded meaning Common ground and the triangle of meaning Representamen Object Interpretant
  • 6. Can a child teach the robot words for colour? Kristyn Hensby Scott Heath OPAL Team OPIE Language games
  • 8. gifugrounded meaning Common ground and the triangle of meaning Representamen Object Interpretant
  • 9. yozozuku Which word refers to space and which to time? Scott Heath spacetime Heath et al (2016) Lingodroids: Cross-situational learning, IEEE Trans CDS
  • 10. An individual robot’s semiotic network (based on Pierce’s semiotic triangle) The Lingodroid sign system Internal representation Symbol Referent social grounding symbol grounding private grounding Schultz et al (2012) Adaptive Behavior Representamen Object Interpretant
  • 11. Robot sensors and “Deixis” 4/5
  • 12. Insights from the Lingodroid studies • The Lingodroids are embodied and enactive • Words are not grounded in the direct perceptions of the robots, but rather, in the integrated navigation system: the nodes of the map, and the relational spaces of distance and direction Each word has a meaning as part of a wider network (extended) • Common ground depends on shared culture (understanding of the activity the robots are engaged in – which game they are playing: where-are-we vs how-far) Culture assigns meaning to activities which then becomes encoded when the robots name common ground Activities that involve common ground are intrinsically embedded in culture
  • 13. In theory there is no difference between theory and practice. In practice there is. Saying in robotics; attributed variously to baseball player Yogi Berra, computer scientist Jan L. A. van de Snepscheut and physicist Albert Einstein. iRat
  • 14. Biological systems are clock-free, grid-free, scale-free & symbol-free Brain regions synchronize over a range of timescales. Fast local coupling enables rapid communication, slow episodes require sustained coupling. Buzsaki&Mizuseki,NatureReviewsNeuro,2014 1 second Time (msecs) Function < .1 location from stereo sound; 1 neuron spike width 10 communication in neural regions 100 social moments; 1000 cognitive moments
  • 15. Timescales of fluid conversation (human-to-human) Speaker 1 200 time (milliseconds) 700 • The gap between speakers is typically 200 msec • The time to process an utterance is 500-700 msec • Social micro abilities provide a scaffold for these rapid exchanges Speaker 2
  • 16. How the brain represents space 16Drieu and Zugaro, Hippocampal sequences… Front. Cell. Neurosci., 2019 Placecells
  • 17. How a brain constructs space-time The representation of space in the brain Grieves & Jeffery, (2017) Behav Proc. 135 Fig. 8 17
  • 18. “VR breaks the laws of physics,” Mayank Mehta, UCLA • “The rats in the virtual maze found hidden rewards as skillfully as rats in a real maze. … Despite this, the neural activity in their hippocampus was highly abnormal in virtual reality. More than half of all neurons in the hippocampus shut down in virtual reality and the remaining neurons fired in a disordered fashion • “If the vision says you are moving in space, but the sounds and smells tell otherwise, as is the case in virtual reality, the neural activity becomes very strange. • “VR … removes the consistent relationship between different stimuli in the world that all the animals have used for millions of years.” www.wired.com/2013/05/virtual-reality-spatial-sense/ Space, Time & Imagination: How Virtual Reality Changes the Brain | Mayank Mehta | TEDxUCLA
  • 19. ITEE/Co-Innovation Group: Janet Wiles, Scott Heath, Jonathon Taufatofua, Nik Rybak, Josh Arnold, Josh Riddell, Gautier Durantin, Ola Olsson, Carlos Ramirez, Andrew Back; Scott Frazier, Beth Cave, Dylan, Alumni: Kris Rogers, Michael Smith, Rollsy Ponmattam Madassery, Lachlan Fitzpatrick, Autumn Qiu Jing Chew, Nick Buckeridge, Hugh Raynor, Wilson Kong; ITEE/Robotics: Pauline Pounds; ITEE/Interaction Design: Pete Worthy, Jason Weigel, Steve Viller, Ben Matthews, Marie Boden School of Psychology Early Cognitive Development Centre: Kristyn Hensby, Virginia Slaughter, Mark Nielsen, Tamara van der Zant School of Communication and Arts/Discursis: Dan Angus, Yvonne Yu Bond U+ITEE: Helen Chenery, Jacki Liddle, Brooke-Mai Whelan Melbourne U: Jill Wigglesworth; Ngukurr language centre: Greg Dickson, Jackie van den Bos, Angelina Joshua, Tamara Joshua, Grant Thompson, Jordan UCSD Machine Perception Lab: Deborah Forster, Javier Movellan, Marian Bartlett, Mohsen Malmir; HARLIE: David Ireland (CSIRO), Christina Atay, + Harlie team computer scientists mechatronics electrical & software engineers roboticists designers psychologists Speech pathologists occupational therapist linguists CoEDL Scholars MPhil, PhD THE SOCIAL TECHNOLOGIES TEAMS: LINGODROIDS, OPAL, HARLIE, DISCURSIS j.wiles@uq.edu.au