My keynote at the 7th EAI International Conference on Interactive Digital Media (https://icidm.eai-conferences.org/2021/). It discusses the potential of games for serious purposes and the use of Artificial Intelligence (AI) for games. It discusses how AI can be used in games applied to learning of social skills and how AI can improve the social qualities of artificial agents in games. It concludes with a discussion of the use of AI for game production, in particular, to generate content and to automate playtesting.

1. How AI can help games fulfil
their purpose
Rui Prada
INESC-ID and Instituto Superior Técnico
Universidade de Lisboa, Portugal
EAI ICIDM 2021 - 7th EAI International Conference on Interactive Digital Media
July 30 – August 1, 2021 - Johor Bahru, Malaysia (Online)

2. Games with purpose
Artificial Intelligence
in Games
1
2

3. Games are means for
Entertainment
Communication
Participation
Experience

4. Games with Purpose (Serious
Games)
Good means to transmit information and
show how things work
(Multimedia)
Capture attention and maintain interest

5. Games for Education
Promote practice, exploration and error
(Interactivity)
Personal emotional experience

6. Examples of
Games with Purpose

7. Play with Seismic Phenomena
Treme-treme
https://treme-treme.pt/
https://youtu.be/e1Mq8Egy9ko
• For kids (and all)
• Learn how to prepare
• Learn how to protect
• Learn the dangers (during
and after)
• Promote discussion

8. Play with Math and Problem Solving*
Festarola
*Ricardo Rodrigues, Paula Costa Ferreira, Rui Prada, Paula Paulino, Ana Margarida Veiga Simão, “Developing Children's Regulation of Learning in Problem-Solving
With a Serious Game” in IEEE Computer Graphics and Applications, vol. 40, no. 5, pp. 26-40, Oct. 2020. IEEE.
• For kids
• To develop problem solving
skills
• Promote self-regulation of
learning and self-efficacy
• Engage students in the subject
of mathematics

9. Play with Organic Chemistry
ChemTouch
https://youtu.be/fIC9GTJFQcs
• VR game for university
students
• Learn how to build molecules
• Learn atoms’ connections
• Stereochemistry (3D
Chemistry)
• Understand Structure and
symmetry

10. Games for People with Intellectual Disability
• Quality of Life Training Game
• Cognitive functions: calculus and problem solving, memory, spatial
orientation, language, etc.
• Quality of Life dimensions: interpersonal relationships, social
inclusion, self-determination, physical and emotional well-being,
etc.
15 million people with intellectual
disability in the European Union
https://id-gaming-project.eu

11. Artificial Intelligence (AI)
in Games

12. AI in Games
AI to build the social dimension of the game
Play against the player
Populate the social world, play social roles in games
AI to support the development of the game
To generate content for games
To automate testing
1
2

13. AI to build the social dimension
in Games

14. Social roles that AI agents play in games*
*Diogo Rato; Rui Prada: “A taxonomy of social roles for agents in games” (to appear) in proceedings of ICEC’21 - the 20th International Conference on
Entertainment Computing, pp. 1-12, Coimbra, Portugal, October 2021. IFIP.

15. Social roles that AI agents play in games*
*Diogo Rato; Rui Prada: “A taxonomy of social roles for agents in games” (to appear) in proceedings of ICEC’21 - the 20th International Conference on
Entertainment Computing, pp. 1-12, Coimbra, Portugal, October 2021. IFIP.

16. Social roles that AI agents have in gameplay*
*Diogo Rato; Rui Prada: “A taxonomy of social roles for agents in games” (to appear) in proceedings of ICEC’21 - the 20th International Conference on
Entertainment Computing, pp. 1-12, Coimbra, Portugal, October 2021. IFIP.

17. Social Roles in AI agents
•Agents need to understand social reality
• Culture and group dynamics
• Social relations
• Social emotions
*Frank Dignum, Rui Prada, Gert Jan Hofstede “From autistic to social agents” in Proceedings of the 2014 international conference on Autonomous Agents and
Multi-Agent Systems, pp. 1161-1164. 2014. IFAAMAS
*Samuel Mascarenhas, Nick Degens, Ana Paiva, Rui Prada, Gert Jan Hofstede, Adrie Beulens, Ruth Aylett: “Modeling culture in intelligent virtual agents: From
theory to implementation” in Autonomous Agents and Multi-Agent Systems. pp. 1-32, 2015. Springer.
*Gonçalo Pereira, Rui Prada, Pedro A. Santos: “Integrating social power into the decision-making of cognitive agents” in Artificial Intelligence. vol. 241, pp. 1-44,
December 2016. Elsevier.

18. Socially Situated Cognition*
• Social meaning of objects
• E.g. An apple can be food, a gift, a toy, a weapon, …
• Social categorization and social identity
• The agents and their social groups in a given context
• Social affordances
• What you can do with the agents and objects in the context
• Socially affordable
• What is acceptable
*Diogo Rato, Samuel Mascarenhas, Rui Prada “Towards Social Identity in Socio-Cognitive Agents” arXiv preprint arXiv:2001.07142 (2020).

19. Social AI agents in Minecraft
• Context
• Time, location, agents
• Social practices
• Activated by context
• Social roles
• Locations have social properties
• Expected activity
• Ownership
• Agents have categories/identities
• Define relevant social practices
• E.g. Lumberjack
Background/Observer

20. https://fatima-toolkit.eu
*Samuel Mascarenhas, Manuel Guimarães, Rui Prada, João Dias, Pedro A. Santos, Kam Star, Ben Hirsh, Ellis Spice, and Rob Kommeren “A Virtual Agent Toolkit
for Serious Games Developers” in proceedings of 2018 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1-7. IEEE, August 2018.

21. Space Modules Inc. Sports Team Manager
Social AI agents with FAtiMA Toolkit
Challenger/Contender Subordinate/Master

22. *Ana Salta, Rui Prada, Francisco S. Melo “A Game AI Competition to foster Collaborative AI research and development” in IEEE
Transactions on Games, pp. 1-14, 2020. IEEE. DOI: 10.1109/TG.2020.3024160
Teammate
Cooperative play in Geometry Friends*
http://gaips.inesc-id.pt/geometryfriends/

23. Social robots and games

24. *André Tiago Pereira, Rui Prada, Ana Paiva: “Improving social presence in human-agent interaction” in proceedings of CHI’2014 - 32nd annual ACM
conference on Human Factors in Computing Systems, pp. 1449-1458, Toronto, Canada. April 2014. ACM.
Opponent
Playing boardgames*
• Contextual gaze
• Emotional reactions to
game events
• Interpersonal relations
• Increased the social
presence of the artificial
player
Emys the Risk player

25. Playing Team Games*
• Displaying emotions as individual or group
• Determine the cognitive unit for the emotional appraisal
• Positive effects for group identification, trust and likability
*Filipa Correia, Samuel Mascarenhas, Rui Prada, Francisco S. Melo, Ana Paiva: "Group-based emotions in teams of humans and robots" in proceedings of
HRI'18 - International Conference on Human-Robot Interaction, pp. 261-269, Chicago, IL, USA, March 2018. ACM/IEEE.
Teammate
Opponent

26. Social Robots as Team Leaders*
• Leadership types
• Transactional (TA): focus on task
• Transformational (TF): focus on people
• 108 people (Portuguese companies) 36
teams of 3
• Productivity: higher for TA
• Engagement: higher for TF
• Role Ambiguity: no sig. difference
• Trust: no sig. difference
*Sara L. Lopes, José Bernardo Rocha, Aristides I. Ferreira, Rui Prada “Social robots as leaders: leadership styles in human-robot teams” (to appear) in
proceedings of RO-MAN 2021 – The 30th International Conference on Robot and Human Interactive Communication, Canada 2021. IEEE.
Subordinate/Master

27. AI to support the development of
games

28. Procedural Content Generation for Games
• Automate the creation of content for games
• Reduce production costs
• Adapt content to players
• Computational creativity
• Co-creation
• Humans and agents together
Levels of Geometry Friends

29. Procedural Content Generation for Cooperative
Games
• Generate levels for the Geometry Friends game
• Use of Genetic Algorithms

30. Procedural Content Generation for Cooperative
Games
• Population
• Chromosome: [RectangleSpawn, CircleSpawn, PlatformArray]
• 8 platforms: [Activation, Position, Dimentions]
• Ex:[120,200,300,240,1,340,120,400,30,0,230,30,200,50,...]
• Evolution process
• Selection: Elitism with selection of 30%
• Crossover: 2 parents breed one child
• Gets random parts from each parent (e.g. platforms)
• Mutation: change the activation of a platform or its characteristics

31. Procedural Content Generation for Cooperative
Games
• Fitness: character reachability
• Circle only
• Rectangle only
• With collaboration
• Both
• Intersection with user input
• Min value of intersection
User Input
User Input
User Input

32. Procedural Content Generation for Cooperative
Games
• Sample results: population 50 and 2000 generations
input area
reachability
level

33. Automated Testing
• Software testing with autonomous (testing) agents
• Agents for User eXperience testing
• Social and emotional agents
• Predict UX by running the agents in the environment
• Predict emotion from interaction traces
*Pedro M. Fernandes, Manuel Lopes Rui Prada “Agents for Automated User Experience Testing” in proceedings of the AIST’2021 – the 1st International Workshop on Artificial
Intelligence in Software Testing, International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp.. 247-253, IEEE. 2021
*Rui Prada, ISWB Prasetya, Fitsum Kifetew, Frank Dignum, Tanja EJ Vos, Jason Lander, Jean-yves Donnart, Alexandre Kazmierowski, Joseph Davidson, Pedro M Fernandes “Agent-
based Testing of Extended Reality Systems” in proceedings of ICST’- 13th International Conference on Software Testing, Validation and Verification (ICST), pp. 414-417, IEEE. 2020.
https://iv4xr-project.eu

34. Predict Emotion from Interaction Traces
• Machine Learning approach
• Users play a game
• Self-report annotation: PAD model
https://iv4xr-project.eu

35. Predict Emotion from Interaction Traces
Arousal
Accuracy: 69.22 %
Pleasure
Accuracy: 64.52 %
Dominance
Accuracy: 62.58 %
https://iv4xr-project.eu
Preliminary results
• 88 participants, 3 maps
• 264 traces
ML approach: Random Forests
• Predict 3 classes: increase, decrease, stable

36. Conclusions
Games can be used for a purpose: educate,
inform, motivate and persuade
AI has an important role in game development
AI to create social agents in games
AI to create content
AI to automate testing

37. Thank you!
rui.prada@tecnico.ulisboa.pt
http://gaips.inesc-id.pt/rprada
http://labjogos.tecnico.ulisboa.pt
https://iv4xr-project.eu
https://id-gaming-project.eu EAI ICIDM 2021 - 7th EAI International Conference on Interactive Digital Media
July 30 – August 1, 2021 - Johor Bahru, Malaysia (Online)