How reinforcement learning affects human behavior?
Reinforcement learning (RL) is a machine learning (ML) technique that trains the policy of the agent, who “must discover which actions yield the most reward by trying them (Sutton et al., 2018)” in a given situation. During trial-and-error iterations the agent evaluates the given feedback after each interaction, where the feedback can be a reward or a punishment, and then uses this experience “to solve new tasks efficiently (Tessler et al., 2016)”.
While in RL decisions are driven by rewards, in games it could be provided as a high score or a new level. The agent’s policies improve under the pressure of a game situation, such as learning how to react when attacked or how to achieve a specific goal.
RL models can be classified based on the number of agents in an environment or players taught by the algorithm: single-agent RL deals with one agent in a situation, whereas in a multi-agent environment, several agents act simultaneously, which affects the learning process of other agents, these agents must act independently but must learn to interact with other agents. This is an immensely difficult problem because with co-adapting agents the environment is constantly changing.
In video games, agents have typically been trained to win against humans, like the famous chess agent AlphaZero, which was trained by self-learning, that is, by playing against itself (McGrath et al., 2021). The purpose of games is to develop one’s physical and mental abilities, and a competitive situation would certainly improve strategic thinking, dexterity, and other abilities (Togelius et al., 2018).
Project Paidia (Project Paidia: a Microsoft Research & Ninja Theory Collaboration, 2022) team of Microsoft Research aims to provide collaborative agent algorithms that support rather than hinder the player during gameplay, and these models are available to developers to improve their own games. Read more about Project Paidia in this article. This technology is used in various existing games, including Microsoft’s Halo Infinite (Halo – Official site (en), 2022), players choose a companion to help in a particular situation, with the goal of improving gameplay and people’s success.
While Halo’s agents enhance the game experience for experienced, they can degrade it for less skilled players. Algorithms of both opponents and companions have a serious impact on the gameplay, causing various disorders if the game is too difficult, anti-social behavior, and even addiction if players consistently achieve high scores (Dimita et al., 2021).
Advances in RL allow developers to provide a more personalized gaming experience for their players, but increasing capabilities also create higher expectations, which can lead to game performance degradation – either because the game is too easy or too hard, or because companies don’t know how to deal with increasing demands, negative feedback, and the loss of employees due to deterioration in mental or physical health.
Serious Gaming (SG) forces players to interact with real-world environments, and such games aim to teach citizens how to use resources efficiently while examining human behavior from social, ethical, and organizational perspectives (Carvalho et al., 2015).
One approach is suggested in (Predescu et al., 2021), a location-based game in which players or citizens report issues about the water infrastructure such as leaks, pollution, or service outages by sharing data from their mobile devices’ integrated sensors, including pinpointing their exact location. After the record is evaluated and validated, the details of the problem appear on the world map, and the reporter receives a reward: the more accurate and realistic the data, the greater the reward for the player. This motivates citizens to actively participate and interact, resulting in accurate experimental results for water use and infrastructure issues, even though low-grade sensors may degrade the quality of the reported information.
Integrating pervasive computing and reinforcement learning results in serious privacy, security, and data quality challenges. As the vast amount of data collected by sensors and human behavior are evaluated along with the reported information to reward players, the game must be reliable and transparent. Blockchain technology improves the execution of transactions, as well as the secure, immutable storage, and validation of the information provided (Predescu et al., 2021). Thanks to smart contracts, stored records are available to companies in real-time, monitoring and analyzing this data makes water distribution more efficient (Blockchain and the water industry – Smart Contracts – EMA, 2018). While citizens may choose to report information anonymously, location information must be provided. The private blockchain is applied to solve data privacy issues (Yang et al., 2019).
We have discussed how enhancing the personalized gaming experience affects the performance of players, improving their physical and mental abilities. However, Smart Games have simpler graphics and aim to successfully solve or improve environmental problems by teaching citizens how to use resources efficiently and the importance of cooperation in solving serious issues related to resource management.
Blockchain and the water industry – Smart Contracts – EMA (2018) Available at: https://www.ema-inc.com/news-insights/2018/10/blockchain-and-the-water-industry-smart-contracts/ (Accessed: 14 November 2022)
Carvalho, M.B. et al. (2015) ‘An activity theory-based model for serious games analysis and conceptual design’, Computers and Education, 87, pp. 166–181. Available at: https://doi.org/10.1016/j.compedu.2015.03.023.
Dimita, G. et al. (2021) ‘Wellbeing, mental health, and video games: A shifting narrative from player to industry perspective’, Interactive Entertainment Law Review. Edward Elgar Publishing Ltd., pp. 85–86. Available at: https://doi.org/10.4337/ielr.2021.02.00.
Halo – Official site (en) (2022) Available at: https://www.halowaypoint.com/ (Accessed: 31 October 2022)
Huynh-The, T. et al. (2022) ‘Artificial Intelligence for the Metaverse: A Survey’. Available at: http://arxiv.org/abs/2202.10336.
Jameel, F. et al. (2020) ‘Reinforcement learning in blockchain-enabled IIoT networks: A survey of recent advances and open challenges’, Sustainability (Switzerland). MDPI. Available at: https://doi.org/10.3390/su12125161.
Julian Togelius; Jesper Juul; Geoffrey Long; William Uricchio; Mia Consalvo, “1 In the Beginning of AI, There Were Games,” in Playing Smart: On Games, Intelligence, and Artificial Intelligence, MIT Press, 2018, pp.1-10.
McGrath, T. et al. (2021) Acquisition of Chess Knowledge in AlphaZero. Available at: http://arxiv.org/abs/2111.09259.
Project Paidia: a Microsoft Research & Ninja Theory Collaboration (2022) Available at: https://www.microsoft.com/en-us/research/project/project-paidia/ (Accessed: 31 October 2022)
Richard S. Sutton and Andrew G. Barto (2018) Reinforcement Learning, Second Edition : An Introduction. Cambridge, Massachusetts: Bradford Books (Adaptive Computation and Machine Learning).
Predescu, A. et al. (2021) ‘A serious gaming approach for crowdsensing in urban water infrastructure with blockchain support’, Applied Sciences (Switzerland), 11(4), pp. 1–32. Available at: https://doi.org/10.3390/app11041449.
Tessler, C. et al. (2016) A Deep Hierarchical Approach to Lifelong Learning in Minecraft. Available at: http://arxiv.org/abs/1604.07255.
Yang, M. et al. (2019) ‘A blockchain-based location privacy-preserving crowdsensing system’, Future Generation Computer Systems, 94, pp. 408–418. Available at: https://doi.org/10.1016/j.future.2018.11.046.