The Psychology behind using Music in Video Games

There has been a relative boom in gameplay research that has brought about a new understanding of the multi dimensional relationship between music and play, allowing players to feel more immersed in a video game. The study of video game music is referred to as ludomusicology. This term takes into consideration the anthropological, psychological, cultural, societal, economic and musicological aspects of the music used in video games³. Furthermore, it examines multiple facets of video game music such as its history, audio, interactivity, technology, and composition of the music⁸. Others also consider the ways in which music and play interact and how music can be understood as a mode of gameplay¹⁷. The psychological implications of ludomusicology are of keen interest as several studies have emerged that comment on the immersive experience, better player performance, and physiological impacts of using music in video games.

Gameplay models represent key elements that structure the gameplay experience as it relates to all modes of immersion factors. They illustrate the complex internal organization of a game and focus on the consciousness structured by the interplay⁴. Examples of gameplay models include the SCI model and the ALI model. The SCI model is a heuristic model that includes three components: sensory, challenge-based, and imaginative immersion. Sensory immersion is when the player becomes entirely focused on the game world and its stimuli. This can sometimes be accomplished with large screens close to the player’s face and powerful sounds. The goal being to overpower the sensory information coming from the real world. Challenge-based immersion is achieved when there is a satisfying balance of challenges and abilities whereas imaginative immersion is when the player becomes absorbed with the stories and the world of the game⁴. Another representation of gameplay is the ALI (affect, literacy, interaction) model. The ALI model seeks to explain the meaning and identification of video game music. The three components are musical affect, media literacy, and interaction. This model emphasizes the importance of the emotional connections that video game players create through music. Affect is related to memory, emotion, and identification and is often driven by media literacy. Players who have previous knowledge of certain tropes and musical conventions are able to assimilate at a subconscious level through other audiovisual media²⁵.

Player Experience

As supported by the ALI and SCI models, music plays a key role in the feeling of being immersed in a game. The player begins to feel as though they are “there” in the world when the game creates spatial presence. Spatial presence involves using images, movement, sounds, etc. in the game to help the player, either consciously or subconsciously, form a mental model and decide whether they feel as though they are in the imagined world²⁶. Immersion is the feeling of being completely enthralled by the other reality that it takes over all our attention, and entire perceptual system. The creation of this illusion is often closely connected to the entertainment value of the game. Feelings of immersion are often reported alongside a lack of awareness to time¹¹. The addition of music has been shown to increase spatial presence, or the players’ sense of being there^{5, 14}. Background music is suggested to make the video game environment more authentic and convincing, allowing the player to form a richer mental model of the game world²⁶.

The music in video games can be used to facilitate certain experiences for the players. For instance, by creating an association between musical sounds and actions or situations, auditory cues can be used to improve not only the experience but also the performance of the player²⁵. Sound features can provide basic information to the player within the game, such as if certain points have been achieved through an action, a mission was completed, or if the player has enemies waiting up ahead. A high pitch sound, usually that of a small bell, is commonly associated with a reward whereas an ascending or descending glissando signifies an increase or loss of energy of the character. This type of relationship assumes the player recognizes the integration of the elements of the sound to the mechanics of the video game. Accordingly, the player can decipher the meaning and make in-game decisions⁶.

Background music complements the gameplay by creating a new attractive world for the user to escape to and can reinforce the story within the game. Game soundtrack music may not only trigger individual associations, but can also activate memories of related media experiences, such as movies (characters, scenes, actions, etc.) that might have an effect on the player’s imagination and their overall game enjoyment. When video game players can recognize the musical traits and composition used in the game, they become more actively involved in the dynamics of the world⁶. During the development of game music, composers will create sounds and music that fit into the overall narrative setting, mood, and visual aesthetics of the world. Furthermore, the soundtrack will often be congruent to what is happening in a game situation¹⁴. For the player, if the instruments and sounds are consistent with the game’s world, they are likely to feel that the world is more realistic and legitimate. Music can be used as a storytelling tool by accentuating the emotions and themes of the story, especially those that might recur throughout the game. Furthermore, music can reinforce the emotional impact of story events to the player, such as battlescenes. The development and evolution of a musical theme over the course of the story reinforces the connection and emotion of the changes to the character or place. In addition, it can help the player represent these elements in their minds which can allow them to better remember them later. Furthermore, music can be used as a guiding measure through the game. It can help the player remember different elements of the story that are associated with a melody and musical pattern and, thus, strengthen their emotional connection to those elements¹⁸. In this way, the player will feel more immersed in the story and the world of the game.

Music can also reinforce immersion by targeting the emotional affect of the player¹⁴. Studies have shown that the elements in music can trigger or amplify specific emotions in players^{10,12, 21}. For instance, if a game contains music fragments of rapidly changing frequencies, non standard harmonies, or those that imitate animal cries or screams, during the appearance of a boss enemy, the game is likely to either activate or intensify the emotion of fear²¹. Overall, the effects of soundtrack music in video games has been linked to greater positive emotional response and enjoyment during gameplay due to the extent that players value more intense emotions^{14, 22}. This effect also applies equally for positive emotions elicited from video game music, such as the feeling of relief and pride the player experiences when listening to joyful music accompanying the defeat of the boss enemy¹³. However, according to one study, players were more immersed when they enjoyed the music. Thus, a badly designed soundtrack may hinder a player’s ability to feel immersed in the game²⁰. Music soundtrack, when developed appropriately, reinforces the player’s emotional connection to the story, characters, and the game world.

Player Performance

Not only does music heighten the immersive experience of the player but it can also improve their performance in the game as well. According to one study, music present in video games increase the player’s high score, as male gamers were able to score almost twice as many points while playing the first-person shooter game DOOM with the sound on compared to those playing with no sound²³. In other studies, results were more complex. While playing a driving game, people playing with ‘highly arousing’ music drove the fastest but made the greatest number of mistakes, such as hitting barriers or knocking over road cones². However, in role-playing games, such as Legend of Zelda, players performed worse when playing with both music and sound effects off. As more game audio was progressively added, performance improved. According to the same study, participants performed best when playing with background music on a boombox that was unrelated to the game²⁴. Similarly, another study has shown that the presence of self-selected music by the player increased not only their enjoyment of the game, but also their performance. Players were more efficient, perceived lowest distraction, and experienced a reduction in tension- anxiety compared to when researchers selected high arousing music². A reason why player performance improves with music could be due to greater proactive cognitive control when music is playing. Background music might add cues to the game, such as linking sound patterns to enemy attacks that will allow time for the player to make an appropriate response or quickly modify their behavior⁷. In turn, they will begin to memorize and recognize certain musical cues in the game, which can help them improve their performance in the future.

Physiological Impacts

Lastly, the physiological impacts of music in games is also of great research interest. Studies have shown that there is an increase in cortisol levels when music accompanies gameplay showing that the auditory input contributes significantly to the stress response found during video game playing¹⁰. Studies have shown that there are effects of sound on physiological activity. For example, classical music has been shown to elicit relaxing effects on the listener, as well as cause a decrease in blood pressure and heart rate¹⁵. On the other hand, when listening to rock/ techno music, listeners experienced increased self-ratings of anxiety, aggressiveness, tension, and discomfort¹⁶. Furthermore, the participants were found to have significant increases in heart rate, systolic blood pressure, and cortisol levels when listening to heavy metal music, rock, or techno music ^{1, 9, 19}. “First person shooter” games generally use techno and rock music and, therefore, the effects found in this study are representative of those that would be obtained with those games¹⁰.

Music in video games, whether it be sound tracks originating from music compositions created specifically for the game or music derived from the adaptive reuse of orchestral compositions that originated from the production of a tv show or movie, or simply sound effects meant to accentuate an action, have shown to enhance the experience. Original music created for games has found new revenue streams as stand alone content repurposed for advertising, ringtones, or other film/tv content, both long form and short form. Enormous numbers of user generated content found on tik tok or youtube readily utilize music from games. Because of the impact on the user experience and the potential for revenue streams beyond the purchase of the game, gaming companies are investing in the creative process to produce original music for their games. This trend has opened new opportunities within the creative community. The empirical evidence has influenced this investment. As the science progresses in the field of ludomusicology, investment will surely follow. Coupled with technology advances, there will be an evolution to sounds and music found in the newest games. Within most major game producers, there is a large number of composers, voice actors, foley artists, and sound engineers that make up their incredibly large sound department. New technologies centered on AR and Generative AI will challenge these music artists to create more immersive experiences and soundscapes within the game experience. The future of ludomusicology will be fascinating, no doubt.

Bio: Emily E. Garcia is a senior at Tulane University, with a double major in Neuroscience and Psychology and a minor in Strategy, Leadership, and Analytics. Upon graduation, Emily plans to pursue her PhD. in Cognitive Neuroscience with a focus on Neurocinematics and Neuromusicology. This whitepaper is a product of her internship at RSG media Summer 2023 as part of a consulting engagement with one of their clients.

References

1. Burns, J. L., Labbé, E., Arke, B., Capeless, K., Cooksey, B., Steadman, A., & Gonzales, C. (2002). The effects of different types of music on perceived and physiological measures of stress. Journal of music therapy, 39(2), 101-116.
2. Cassidy, G., & MacDonald, R. (2009). The effects of music choice on task performance: A study of the impact of self-selected and experimenter-selected music on driving game performance and experience. Musicae Scientiae, 13(2), 357–386. https://doi.org/10.1177/102986490901300207
3. Cleeve, S. (2017). REVIEW | Ludomusicology: Approaches to video game music.  IASPM Journal, 7(1), 119-122. DOI: 10.5429/2079-3871(2017)v7i1.11en
4. Ermi, L., & Mäyrä, F. (2005). Fundamental Components of the Gameplay Experience: Analysing Immersion. DiGRA Conference.
5. Evans, R. (2018). An Investigation into the Effect of Music on Immersion in Video Games.
6. Fernández-Cortés, J., Cook, K. (October 2021). Ludomusicology: Normalizing the study of video game music.Journal of Sound and Music in Games; 2 (4): 13–35. doi: https://doi.org/10.1525/jsmg.2021.2.4.13
7. Fu X. (2017). The influence of strategy video game and its background music on cognitive control. Ment Health Addict Res 2. DOI: 10.15761/MHAR.1000125
8. GAME. The Italian Journal of Game Studies. (2016). An interview with Mark Sweeney—The ludomusicology research group. In H. C. Rietveld & M. B. Carbone (Eds.), GAME. Issue 6, 2017—Hear the Music, Play the Game. Music and Game Design: Interplays and Perspectives. Available https://www.gamejournal.it/an-interview-with-mark-sweeney-the-ludomusicology-research-group/
9. Gerra, G., Zaimovic, A., Franchini, D., Palladino, M., Giucastro, G., Reali, N., Maestri, D., Caccavari, R., Delsignore, R., & Brambilla, F. (1998). Neuroendocrine responses of healthy volunteers to ‘techno-music’: relationships with personality traits and emotional state. International journal of psychophysiology : official journal of the International Organization of Psychophysiology, 28(1), 99–111. https://doi.org/10.1016/s0167-8760(97)00071-8
10. Hébert, S., Béland, R., Dionne-Fournelle, O., Crête, M., & Lupien, S. J. (2005). Physiological stress response to video-game playing: the contribution of built-in music. Life sciences, 76(20), 2371–2380. https://doi.org/10.1016/j.lfs.2004.11.011
11. Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., & Walton, A. (2008). Measuring and defining the experience of immersion in games. Int. J. Human-Computer Studies, 66, 641- 661.
12. Juslin, P. N., Liljestrom, S., Laukka, P., Vastfjall, D., & Lundqvist, L.-O. (2011). Emotional reactions to music in a nationally representative sample of Swedish adults: Prevalence and causal influences. Musicae Scientiae, 15(2), 174–207. doi:10.1177/1029864911401169
13. Klimmt, C., Blake, C., Hefner, D., Vorderer, P., & Roth, C. (2009). Player performance, satisfaction, and video game enjoyment. In S. Natkin & J. Dupire (Eds.), Lecture notes in computer science: Vol. 5709. entertaiment computing: Proceedings of the 8th International Conference on Entertainment Computing (ICEC 2009) (pp. 1–12). Berlin, Germany: Springer.
14. Klimmt, C., Possler, D., May, N., Auge, H., Wanjek, L., & Wolf, A.-L. (2019). Effects of soundtrack music on the video game experience. Media Psychology, 22(5), 689–713. https://doi.org/10.1080/15213269.2018.1507827
15. Knight, W. E., & Rickard PhD, N. S. (2001). Relaxing music prevents stress-induced increases in subjective anxiety, systolic blood pressure, and heart rate in healthy males and females. Journal of music therapy, 38(4), 254–272. https://doi.org/10.1093/jmt/38.4.254
16. McCraty, R., Barrios-Choplin, B., Atkinson, M., & Tomasino, D. (1998). The effects of different types of music on mood, tension, and mental clarity. Alternative therapies in health and medicine, 4(1), 75-84.
17. Moseley, R. (2013) “Playing Games with Music (and Vice Versa): Ludomusicological Perspectives on Guitar Hero and Rock Band.” In Taking It to the Bridge: Music as Performance, edited by N. Cook and R. Pettengill, pp. 279–318. Ann Arbor: University of Michigan Press.
18. Pozderac-Chenevey, S. (2019). The Narrative Function of Pre-existing Music in Video Games [Doctoral dissertation, University of Cincinnati]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1562060451119602
19. Salamon, E., Bernstein, S. R., Kim, S. A., Kim, M., & Stefano, G. B. (2003). The effects of auditory perception and musical preference on anxiety in naive human subjects. Medical science monitor : international medical journal of experimental and clinical research, 9(9), CR396–CR399.
20. Sanders, T., & Cairns, P.A. (2010). Time perception, immersion and music in videogames. British Computer Society Conference on Human-Computer Interaction.
21. Scherer, K. R., & Coutinho, E. (2013). How music creates emotion. In T. Cochrane, F.Bernardino, & K. R. Scherer (Eds.), The emotional power of music (pp. 121–145). Oxford University Press. doi:10.1093/acprof:oso/9780199654888.003.0010
22. Schramm, H., & Wirth, W. (2010). Exploring the paradox of sad-film enjoyment: The role of multiple appraisals and meta-appraisals. Poetics, 38(3), 319–335. doi:10.1016/j.poetic.2010.03.002
23. Tafalla, R. J. (2007). Gender differences in cardiovascular reactivity and game performance related to sensory modality in violent video game play. Journal of Applied Social Psychology, 37, 2008-2023.
24. Tan, S.-L., Baxa, J., & Spackman, M. P. (2010). Effects of Built-in Audio versus Unrelated Background Music on Performance In an Adventure Role-Playing Game. International Journal of Gaming and Computer-Mediated Simulations, 2(3). https://link.gale.com/apps/doc/A430319095/AONE?u=anon~3477b067&sid=googleScholar&xid=fe6021fe
25. Van Elferen, I. (2016). Analysing game musical immersion: The ALI model. In Kamp, Michiel; Summers, Tim; Sweeney, Mark (eds.). Ludomusicology: Approaches to Video Game Music. (pp. 34-39). Sheffield: Equinox.
26. Wirth, W., Hartmann, T., Böcking, S., Vorderer, P., Klimmt, C., Schramm, H., … Jäncke, P.(2007). A process model of the formation of spatial Presence experiences. Media Psychology, 9(3), 493–525. doi:10.1080/15213260701283079