The sound of music: from increased personalization to therapeutic values
Ileana Caceres and Souvick Ghosh
Introduction. Music providers like Spotify leverage music recommendation systems to connect users with relevant music. Based on content-based and collaborative-filtering statistical methods, these machine learning algorithms quantify user-song probabilities and present the highest-ranked songs. However, most music providers do not fully address their users’ music seeking and retrieval needs. Likewise, the fields of Recommender Systems, Music Recommendation Systems (MRS) and Music Information Retrieval (MIR) remain disconnected from real-world use cases of music seeking.
Method. In this conceptual paper, we review the literature of the Recommender Systems, MRS, MIR and Music Therapy (MT) academic fields. We discuss trends towards greater user control and personalization in the MRS and MIR fields and the connections between MT and positive health outcomes such as reductions in stress, anxiety and heart rate.
Analysis. We argue that greater control and visibility into the characteristics of songs and recommended items can generate positive downstream benefits. We recommend features that empower users to better seek, find, store, retrieve and learn from their musical catalogs.
Results. We suggest design enhancements that recognize music’s wider psychological and physiological benefits and create opportunities to build domain knowledge.
Conclusions. Unlocking music’s myriad benefits through the enhancements proposed would catalyze positive outcomes for business stakeholders, users and society.
DOI: https://doi.org/10.47989/irisic2201
Introduction
As more people live online and more digital content becomes available, users encounter more automated recommendations from content recommender systems. Recommendation or recommender systems present digital consumers with a curated sliver of suggestions based on machine learning-based models' computed prediction of consumers' preferences. These systems leverage personal user data, machine learning and statistics to predict which items are most relevant to that particular user. In the music domain, businesses like Spotify, Apple Music, YouTube and other commercial music platforms deploy complex recommendation systems to help users discover relevant music, retain paying customers subscribed to their monetized services and maintain high engagement with the platform (Millecamp, et al., 2018). Users rely on these commercial music platforms to provide an ecosystem and feature set that enables them to manage their personal music library and discover new content based on their individual preferences.
In this conceptual paper, we argue that commercial music platforms must implement features that allow for stronger user control and customization in order to address their users' music information needs in the use cases discussed, as well as to differentiate themselves amongst competitors. First, we review relevant scholarly articles and conference proceedings from the information and computer science fields of Recommender Systems, Music Information Retrieval (MIR) and Music Recommender Systems (MRS). Next, we discuss recent findings in Music Therapy (MT), a multidisciplinary field of the health sciences. Following this literature review, we connect the MT use cases with MRS and MIR trends to suggest features that empower users to configure their music recommendations and control the interface and the information retrieval system of their music collections. Finally, we argue that these product enhancements would improve user satisfaction and provide them with tools to increase their domain expertise.
From the business perspective, adopting these features would increase user adoption, improve customer retention, minimize churn and provide incentives to maintain loyalty to the commercial music platform. Along with these business reasons, users would gain an improved music listening experience from an MRS perspective (i.e., increase in usability, perceived value, user satisfaction, user expertise and domain knowledge) and from an MT perspective (i.e., decrease in stress, improved personal and public health outcomes).
The original contribution of this conceptual paper is to synthesize the latest research from the fields of MRS and MIR -- which often focus on Computer Science -- with the MT field, which focuses on the Health Sciences. This synthesis provides a compelling rationale for why commercial music streaming platforms should empower users with the interactive tools necessary to better control and direct their music information-seeking journey and meet their discovery, retrieval and organization needs. Furthermore, we propose design features that commercial music platforms should consider to meet the users' real-world needs and enhance the system's human-computer interaction (HCI) capabilities. We argue that music streaming platforms, ubiquitous in many people's everyday lives, have the potential to improve the health and overall lives of their users through increased user control of their collection's metadata (descriptive features of the content and consumption metrics) and increased flexibility and interactivity in the system's recommendations.
The rest of the paper is organized as follows. First, we present our review of literature from Recommender Systems, Music Recommender Systems (MRS), Music Information Retrieval (MIR) . Then, we summarize MRS and MIR trends, and present a comprehensive analysis of Music Therapy (MT). Finally, we synthesize our findings from previous sections and propose design suggestions that would better satisfy the music information needs of the users.
Literature Review
In the following subsections, we review literature on Recommender Systems, Music Recommender Systems (MRS) and Music Information Retrieval (MIR).
Recommender Systems
Recommender or recommendation systems (otherwise known as RecSys or RS) emerged in the 1990s (Park, et al., 2012). Recommender systems employ machine learning algorithms and data mining methods to predict users' preferences in a growing range of domains. As the application of recommender systems in multiple businesses and industries has proliferated, so has academic research on the topic (Bunnell, et al., 2020). Recommender systems use cases include recommendations for movies and tv shows (Gomez-Uribe and Hunt, 2015), online videos (Covington, et al., 2016), points of interest like restaurants (Binacalana, et al., 2013), stocks (Bunnell, et al., 2020), health outcomes (Kamyshev, et al., 2020) and people themselves, such as with LinkedIn or online dating (Bunnell, et al., 2020). While recommender system is related to the field of Information Retrieval (IR), it requires the personalized retrieval of items that are ranked in order of preference for that specific user (Bunnell, et al., 2020).
Two overarching focus areas within the field of recommender systems are: 1) the system components of computational algorithms; and 2) the user components of the recommender's design. Indeed, Bunnell, et al. (2020) chose user and system as the top-level classes of their ontology of issues within the field of recommender systems due to the dichotomous nature that these two topic areas are often approached. In this paper, we will be focusing more on the User issues of recommender systems in the music domain.
Evaluation is a persistent challenge in the recommender system field. The relevance of a recommendation is generally measured through offline computations rather than user studies, which may not reflect actual user perceptions in the real world (Knijnenburg, et al., 2012). Recommender system researchers often employ evaluation metrics that focus on minimizing prediction error and come from the field of statistics, such as mean absolute error (MAE) and root-mean-square deviation (RMSE). Metrics from Information Retrieval such as precision and accuracy are also utilized, along with the newer, recommender system-focused beyond-accuracy metrics such as diversity, novelty and serendipity (Schedl, et al., 2018). Aside from these metrics, Jannach and Bauer (2020) highlight the individual consumer, business and societal benefits that recommender systems have the potential to catalyze.
Music Recommendation Systems (MRS)
Music was one of the earliest use cases of recommender systems (Jannach, et al., 2018). Music recommender systems commonly employ the following machine learning techniques: collaborative-filtering (CF) techniques, content-based (CB) techniques, or a hybrid of both techniques (Schedl, et al., 2014).
The MRS field faces domain-specific challenges such as: 1) millions of recommendable items with incomplete metadata and of relatively short duration; 2) the elicitation of users' musical preferences is largely informed by implicit signals which a MRS can misinterpret; 3) users' musical preferences may change over time; 4) recommendations depend on the users' purpose in music listening and their temporal and spatial context; 5) the steady addition of new songs creates item churn; 6) the emotional component of music is difficult to compute; and 7) social influences can impact music preferences (Jannach, et al., 2018; Lee, et al., 2010; Schedl, et al., 2018).
Although the MRS field suffers from the same over-reliance on algorithmic measures and lack of user-centricity that characterizes the general recommender system field, recent literature in the MRS field has emphasized the importance of holistic, user-centric design over computational accuracy (Jannach and Bauer 2020; Lee, et al., 2010; Weigl and Guastavino, 2011).
Music Information Retrieval (MIR)
The field of Musical Information Retrieval grew rapidly in the 1990s and 2000s due to a confluence of technical advancements such as the rise in computing power and audio compression techniques, mobile music players and the emergence of streaming services like Pandora and Spotify. These changes expanded users' access to a wider range of music that could be accessed in almost any environment and at any time (Schedl, et al., 2014).
In the past decade, the proliferation of cloud-based music storage and services has also expanded the potential for regular music listeners to collect a vast collection of songs. The move to cloud-based music services allows users to maintain a digital music collection removed from any physical files. Often this may come with relinquishing ownership of the music in favor of subscribing to the music provider's colossal catalog of music (Datta, et al., 2017; Lee and Price, 2016). These changes in users' music storage capacity and ownership have significant implications in MIR, where researchers are concerned with how users seek, utilize, store, retrieve and share music. Figure 1 shows the interaction and application of information retrieval (IR), recommender systems and medicine with the music, which leads to the specialized domains of music information retrieval (MIR), music recommender systems (MRS) and music therapy (MT) respectively.
Trends In MRS and MIR
In this section, we review current trends in the fields of MRS and MIR that point towards the critical importance of personalization and suggest how music content providers can adopt recommendations and insights from the research literature to better meet the needs of their multiple stakeholders (i.e., users, artists, advertisers).
Fundamental Uniqueness of User
A commonly accepted challenge of MIR and MRS is the subjectivity inherent in music recommendation. Music preferences can be socially influenced, purpose-driven and context-dependent (Jannach, et al., 2018; Schedl, et al., 2018). An individual's background, domain knowledge and mood can affect their perception of a song, so their reaction may be different from that of another user (Schedl, et al., 2013). Moreover, the specific memories or experiences which users associate with a particular song can uniquely bias them towards or against that song. When Cunningham, et al. (2005) asked survey respondents (students) why they hated a nominated song, 5\% of them cited unpleasant personal memories – negative or traumatic memories – associated with a song as the reason. Whether through content-based, collaborative-filtering, or hybrid methods, recommender system developers would be hard-pressed to code an MRS that would predict such use cases.
Researchers have also documented the differences in music information seeking behaviour between musical and non-musical users (Jin, et al., 2020; Jin, et al., 2018; Laplante, 2010). Users with musical knowledge are better at wording search terms and effectively interacting with a novel music recommender to gain perceived value. Such users are better equipped to articulate what type of music they are seeking, which may lead to improved search results and greater perceived effectiveness of the recommendation system (Jin, et al., 2018). Researchers have found that when given more user control over the recommendation process, musically sophisticated users will interact more with the MRS and be able to identify relevant songs that the system or they themselves would have overlooked (Millecamp, et al., 2018). On the other hand, the lack of musical knowledge reduces the ability of the users to define an effective search term that reflects their music information need. Therefore, a commercial music recommender system that provides users with opportunities for musical learning would improve users' ability to interact with and extract value from their system.
Users often connect certain music to specific times in their lives, so it is important to include time-based information in commercial music systems. Users’ tastes and preferences change over time and they may prefer music that reminds them of a specific time in their life (Bogdanov, et al., 2013). The system should track these changes, enabling users to request music they liked from specific time periods. Apple Music does have a useful feature called Smart Playlists that enables users to automatically create a playlist by defining Boolean rules of what type of songs to include or exclude. The feature allows defining a time-period when a song was added to the library. For example, a user can set the Smart Playlist's rules to include only songs added to the Apple Music library within February to July of 2009, with a Play Count higher than 15 and whose Genre is not Classical. More such features are needed that empower users to understand their own musical history, explore their past and delve into nostalgia. Sedikides, et al. (2021) argue that music-induced nostalgia has positive psychological benefits such as generating stronger feelings of 'sequentially social connectedness' (Sedikides, et al., 2021), self-esteem and a sense of meaning in life. Commercial music providers could create features and tools that enable users to tap into their own personal songs that induce nostalgia.
Psychology and Personality of the User
Despite the complexities of music preferences inherent in a single user, researchers have found evidence that particular music preferences correlate with personality types and associated characteristics like cognitive ability and extraversion (Gardikiotis and Baltzis, 2011; Rentfrow and Gosling, 2003). Rentfrow and Gosling (2003) identified four dimensions of musical preference: 1) reflective and complex (e.g., blues, jazz, classical and folk); 2) intense and rebellious (e.g., rock, alternative and heavy metal); 3) upbeat and conventional (e.g., country, soundtrack, religious and pop); and 4) energetic and rhythmic (e.g., rap/hip-hop, soul and funk and electronica/dance music). The authors found a strong correlation between users' musical preferences and their personality types and cognitive abilities. Users who preferred Upbeat and Conventional music scored lower in verbal IQ scores and in the Openness to New Experiences personality trait. Similarly, Gardikiotis and Baltzis (2011) found that participants who scored high on Openness to Change in personality questionnaires were more likely to report a preference for complex, non-conventional music. Ferwerda, et al. (2019) found that certain personality types prefer certain music searching techniques. For example, users whose personality tests scores were high in Openness to New Experience preferred to browse music by mood, while those who scored high in Neuroticism preferred to browse music by activity or genre.
MIR and MRS providers may leverage the user personality types to inform their MRS models. However, they should not forget the importance of individuality, as many users may enjoy music from multiple or all dimensions of music. Also, these preferences can change over time and the user's lifetime, thereby providing more evidence for the inclusion of time-based data.
Moving Towards Better User Experience
Over the past decade, Recommender system research has overly focused on offline studies that intend to achieve an ever-higher accuracy score with an algorithmic prediction model and has failed to sufficiently study real users and real-world scenarios (Bogdanov, et al., 2013; Jannach and Bauer, 2020; Knijnenburg, et al., 2012; Weigl and Guastavino, 2011). Non-algorithmic factors must be considered in order to achieve a holistic understanding of the user (Knijnenburg, et al., 2012). Jannach and Bauer (2020) emphasize the need to pivot research towards studying real world users rather than focusing solely on algorithmic prediction accuracy. Studying the subjective components of a user's experience more closely enables deeper understanding of the overall user experience (UX). The recommender system field has realized in recent years that the front-end UI/UX aspects of a MIR system are a critical part of holistic user-centric evaluation, along with the system's algorithmic features (Lee and Price, 2016; Weigl and Guastavino, 2011).
Changes in the digital music ecosystem also call for a deeper look at the UI/UX of commercial music platforms. Datta, et al., (2017) discuss how the adoption of streaming has led to changes in user behaviour. Changes observed include increases in the quantity and diversity of the music consumed which resulted in higher levels of discovery and play count of new music (Datta, et al., 2017). The adoption of a music streaming business model enables users to rent access to a vast catalog of music titles for a subscription fee. This digital ownership model, where additional music titles come with a fee, increased music consumed, and requires new MIR features and tools for the users to better manage their collection and recommendations.
Most commercial music platforms lack sufficient control mechanisms for users to provide feedback over system-generated recommendations, aside from a standard like/dislike button (Jin, et al., 2020). To create a more personalized UX that enables stronger user control, Narducci, et al. (2020) propose the adoption of Conversational recommender systems (CoRSs), which allow users to interact with an recommender system and shape its recommendations by providing answers in a series of dialog boxes, similar to a chatbot. Jin, et al. (2020) echo the need for personalized user control in describing how lower levels of user control lead to lower levels of trust in the recommender system. However, they also warn that cognitive overload can occur if too many interactions and decisions are required of the user. The interaction preferences and cognitive load thresholds vary depending on personal characteristics such as the users' levels of experience or musical sophistication, personality traits, demographic characteristics and cognitive skills.
To account for user diversity, Jin, et al. (2020) recommend providing UI widgets that enable varying levels of user control: low, medium and high. The low level allows users to remove and sort recommendations, the middle level allows users to modify their user profile to tweak potential sources of inaccurate recommendations, and a high level enabling users to adjust the item weights associated with the recommendation algorithm. By providing users with more control over their recommendations, their user profile, and the parameters of the recommendation algorithm itself, commercial music providers can offer more personalized and targeted recommendations that will distinguish them from the competitors.
Music Therapy (MT)
In this section, we introduce the field of Music Therapy (MT) to highlight how music recommendation and retrieval systems can and do impact individuals and societies. First, we review the history and literature of Music Therapy in study and practice. Then, we discuss the successful application areas of MT. The multiple use cases of Music Therapy provide evidentiary support for empowering music seekers with better tools for discovery, retrieval and learning. We suggest that insights from this growing field could inspire research directions in the academic fields of MIR and MRS. In the business realm, commercial music platforms could adopt design changes that consider MT use cases to improve their products and services.
History of MT
Music Therapy (MT), as a modern research and academic field, arose in the post-World War II period, in part due to the development of government-funded music programs (Beyers, 2016). From the 1950s to the 1970s, MT has developed into a formal interdisciplinary field in the US, leading to the development of formal degree programs, the publication of academic journals on MT and an examination-based certification to be a Registered Music Therapist (RMT). The use of Music Therapy expanded in the 1980s and onward as more patient populations gained interest in MT services, such as patients with brain injuries, chronic pain, trauma, AIDS and dementia, with more applications every year. Today, music therapy is used for pain management, occupational therapy (for language or movement), education and psychotherapy (Beyers, 2016).
Psychological Benefits of Music
Music reduces aggression in the society, enables social cohesion, communicates social messages, reduces anxiety about human mortality, helps pass the time, assists in achieving transcendence and can lead to a pleasurable experience for both the listener and the performer (Schäfer, et al., 2013). Cross-cultural evidence suggests that humans may have evolved to become musical because of the power of music to promote social bonding, particularly among large and complex population (Savage, et al., 2020). Tian, et al. (2021) found that a person's appearance often reflects their musical tastes, which others will interpret and leverage to identify people with similar musical tastes who could be potential candidates for social bonding.
Three overarching motivations have been identified behind the purposes of listening to music: 1) self-awareness; 2) social connection; and 3) regulating mood and arousal (Schäfer, et al., 2013). Self-awareness focuses on the cognitive and emotional functions for music-listening, such as self-related thoughts, emotions and sentiments, absorption, escapism, coping, solace and meaning. Social relatedness refers to how music can connect to a particular affiliation. Through music, people can learn about their environment, connect with others and express their identity (Schäfer, et al., 2013). Lastly, arousal and mood regulation focus on the emotional and physiological- (or arousal-) related function of music. Music can be used to passively regulate moods to maintain the desired level of physiological arousal. For example, music is often utilized as a background noise where the listeners need not pay devoted attention to the music. If emotional regulation is the goal, rather than mood regulation, listeners may need to employ a more attentive and active listening style (Schäfer, et al., 2013). To illustrate how music can generate measurable changes to the human brain, we review the literature on music's effects on stress, anxiety and depression.
Use Cases for MT - Treating Stress, Anxiety and Depression
In this section, we look at successful and potential application of Music Therapy (MT) to real-world scenarios.
Through neuroimaging and brain lesion studies, researchers can observe how music evokes emotions that affect all the behavioural and emotional regulatory structures of the brain (Koelsch, 2010). Emotional processing is associated with both the limbic and paralimbic structures (de Almeida, et al., 2020; Koelsch, 2010), and these neural structures are critical to the processing of emotions that ultimately determine the survival of the individual and species as a whole.
Cognitive science researchers have studied the physiological responses our human bodies have to music. Blood and Zatorre (2001) identified how changes in regional cerebral blood flow to the amygdala are detected when a person experiences music-induced chills, which are extreme emotional experiences usually accompanied by goosebumps or shivers (Koelsch, 2010). Music-induced chills are shown to reduce activity in the parts of the brain that are connected to anxiety (Schäfer, et al., 2013). Mori (2022) found that musically induced 'peak emotions' such as chills or crying can generate useful psycho-physiological effects for the listener such as calming or arousal. Music's potential to regulate emotional activity suggests that music playing or listening be employed to reduce symptoms of depression (Koelsch, 2010).
To study the effects of music therapy on stress, Pelletier (2004) conducted a meta-analytic review of 22 quantitative studies of music and music-assisted relaxation therapy covering myriad use cases, such as decreasing occupational stress, decreasing student anxiety, a cognitive behavioural intervention for anxiety and pain, during surgery and medical procedures, in preparation for and during labor, and for patients with terminal illness diagnoses. MT was shown to significantly decrease arousal associated with stress and anxiety among all groups, but certain groups were most likely to be affected by Music Therapy, such as adolescents, females and musicians (Pelletier, 2004). Knight and Rickard (2001) also suggest that females may have a stronger physiological reaction to music than males. The authors unequivocally state that music treatment can help decrease anxiety, heart rate, and blood pressure as well as help people avoid the physiological effects of stress.
While the benefits of music therapy are promising, it remains out of reach for many who cannot afford or have access to mental health services of any kind. Mental health access can be particularly limited for racial and ethnic minorities. This disparity in access to mental health services can lead to increases in suicide, preventable hospitalization and decreased productivity, thereby affecting society as a whole (Lee, et al., 2014). While commercial music streaming applications cannot replace the services of a licensed therapist, they can provide features or services that strengthen their users' ability to extract mental health benefits from using their platform.
As the COVID-19 pandemic has pushed mental health to the forefront of societal needs and conversations, the potential impact of music should not be diminished as anything less than a lifegiving tool. MRS platforms should build features and tools that empower music therapists and lay users to better organize, retrieve and discover relevant music items according to the kaleidoscopic needs of the user. The examples presented in this section highlight how MIR/MRS systems could leverage music therapy to promote personal health and overall wellness among the users.
Synthesis and Recommendations
In the previous sections, we presented arguments on why research in MIR and MRS must refocus from achieving small increases to algorithmic predictive scores to catalyzing real-world benefits for the users. Then, we reviewed use cases and potential application areas of MT in several health domains, from reducing stress to managing cerebral artery strokes. In this final section, we suggest key design recommendations for commercial music platforms to consider music's wider psychological and physiological benefits and build systems for social good. We argue that companies providing MIR and MRS products and services should re-imagine themselves as enablers of self-music therapy. These companies possess the power to drastically improve people's lives far beyond entertainment needs. As part of this re-imagination, these providers should consider adopting design, usability, search and recommendation features that catalyze the diverse benefits that music has been scientifically shown to generate. We propose simple changes to the UI and metadata that should generate myriad benefits for the users.
Expanding View of Content Metadata
Music information needs are sometimes too complex to be predicted by even the most robust machine learning models. Expanding the visibility of metadata that describes valuable information about the music item – such as its tempo, key and valence – should enable users to better select music based on their personal music information needs. Allowing users to select these additional descriptive metadata attributes could also increase the users' self-awareness, reveal the characteristics of music they enjoy, catalyze further discovery on the platform and increase domain expertise. Next, allowing users to view musical metadata such as piece's key, score, instruments used, or beats-per-minute (BPM) could permit music students, musicians, DJs and music aficionados to discover music based on these descriptive features. Dougan (2012) found that a majority of undergraduate and graduate music students employ commercial music platforms such as YouTube and iTunes to search for musical scores. Commercial music platforms already track and store many of these fields, but they are not available through the platform's GUI. For example, Spotify tracks the key and BPM of their songs, but one must access a site like TuneBat (https://www.tunebat.com) to pull these stats from Spotify's API or access the Spotify API directly through computer programs. This limitation in Spotify's UI diminishes the user's capacity for new music discovery and knowledge-building by creating roadblocks for users to maneuver around to access this important metadata.
Commercial music providers could also leverage the sample identification feature of the site WhoSampled (https://www.whosampled.com/). This site allows users to query a specific song, for example, Electric Relaxation by a Tribe Called Quest, and then returns the specific songs that are sampled in Electric Relaxation and also what songs sampled Electric Relaxation afterwards (https://www.whosampled.com/A-Tribe-Called-Quest/Electric-Relaxation/). This feature would further the user's discovery of songs, increase user satisfaction and extend the user's time on the platform. Including these musical fields can help the listener engage in active music therapy and reap the psychological and physiological benefits of participating in music production. Most people do not have a therapist, and enabling these features could engender passive music therapy for users. We recommend adopting features from the guitar-tab site and app, Ultimate Guitar (https://www.ultimate-guitar.com), which provide guitar tabs for over a million songs while offering a wealth of interactive, customizable features that truly improve the user experience while learning to play the guitar.
The act of learning to play or read music can improve one's cognitive and communication skills (Pelletier, 2004). Furthermore, including play-along features in the commercial streaming platform itself -- without the user having to navigate to another site like Ultimate Guitar or Tunebat -- can enable users to learn more about the music domain, which as the literature shows, can lead to more effective search and discovery behaviour and an improved perception of the value of the MRS. Allowing users to view metadata information about the key and vocal ranges covered and the lyrics in a song would enable singing along to the music. Weigl and Guastavino (2011) review user studies and show the potential value of including lyrics in the metadata. If an alto singer is practicing how to sing songs in the key of C major, they may be interested in contralto songs that are just in the C major key. Current or prospective players and singers of music would benefit from visibility into these fields since specific musical keys are more difficult to play or sing. An expanded view of the music catalog's comprehensive metadata would support users in the search and retrieval process as well as provide insight into one's own musical preferences and abilities.
Access to Personal Consumption Metrics
Commercial music platforms should not stop at increasing access to the descriptive attributes of songs themselves. Instead, they should allow users to easily view metadata reflecting how they interacted with the song, with automated fields such as Play Count and Date Added. The inclusion of these personal metadata fields can reveal to users which songs they listen to the most and during which time periods. Users can then learn about their music preferences and consumption trends and revisit previous time periods of their lives through music. Apple Music has always offered these two metadata fields (Play Count and Date Added), but Spotify and Amazon Music still do not. Forcing users to wait for end-of-year playlists created by the music platform does not empower users to learn about their music interests in real-time.
The broad appeal of metadata-based analytics is evidenced through the popularity of Spotify Wrapped, their end of year release of personal usage stats that users' share on social media. Furthermore, when considering the successful application of music in reminiscence therapy for patients with dementia (Pacchetti, et al., 2000; Särkämö, et al., 2008; Tsoi, et al., 2018, Woods, et al.; 2005), the individual's time-specific musical preferences are crucial to identify and make visible. Commercial music platforms should be wary of not including these crucial fields, as the lack of this metadata could become the primary reason for a user to leave the platform.
Conclusion
Considered to be one of the most successful applications of machine learning and artificial intelligence, the field of recommender systems and its applications continue to grow. As Jannach and Bauer (2020) emphasize, these recommenders have multiple stakeholders beyond the business of providing the MRS product or service. These stakeholders include individual users, the suppliers of the content and society. Commercial streaming platforms can be dynamic forces of positive societal and individual change if they develop features that empower the users to seek, find, store and learn about their preferred music in a manner that allows them to reap the greatest holistic benefit. The adoption of these proposed design features is in the interest of business stakeholders, who are the decision-makers behind the popular commercial streaming platforms. Business stakeholders could create unique incentives for users to choose their music platform over competitors by adopting these user-centric features.
Aside from potentially expanding the user base, other desired business outcomes include increased customer loyalty and reliable monthly recurring revenue from subscriptions. Unlike Netflix or Disney+, most of the content on commercial music platforms is not exclusive to that platform. For example, one can listen to any particular song on Spotify, YouTube, Apple Music and several other platforms. Therefore, music platforms should invest in retrieval and recommendation features that focus on user control, customizability and deep personalization. This direction should allow the platforms to provide unique value to users, differentiate themselves from competitors and encourage customer loyalty. No two users are identical, and the purposes of music listening are complex. If music platforms develop features that empower users to better interact with and customize the recommendations, and organize and retrieve their music collection, they will stand out in a crowded marketplace.
Beyond the immediate benefits to the commercial music platforms, the design features suggested could generate positive outcomes for individual music seekers and the society in general. We discussed the psychological, developmental and physiological benefits of music listening and music education at length. Although some countries like Sweden prioritize music education (Georgii-Hemming and Westvall, 2010), yet many children around the world do not have access to music education (Goble, 2010). More users could enjoy the full benefits of music listening, singing and playing if provided with the opportunity to self-learn music knowledge. Similar to the described limitation of music knowledge, many users do not have access to therapists (Lee, et al., 2014), particularly music therapists. Therefore, commercial music platforms can provide compelling features that help their MRS generate the benefits of music therapy, thereby positively affecting the health of their users. These motivations need not be entirely altruistic, as users are likely to appreciate these features and respond positively to the business. We would conclude the paper with a quote by Schedl, et al. (2014) that encapsulates our perspective on how commercial music platforms should re-imagine themselves:
Musical companions: music has a great potential in different aspects of human life such as cognitive development, education, therapy and well-being; we foresee MIR systems as personalized musical companions along our lives, systems connecting music to personal memories or experiences, and systems helping to regulate emotions, for instance in stressful or even depressing human situations. MIR will hence help improving humankind's overall well-being.
About the authors
Ileana Caceres is a Senior Learning Consultant at Pure Storage, California. She is a recent graduate Master’s in Library and Information Science (MLIS) from the School of Information, San José State University, USA. She researches Music Recommender Systems, Music Information Retrieval and Human-Computer Interaction. She can be contacted at icaceres@purestorage.com.
Dr. Souvick Ghoshis an Assistant Professor and the Program Coordinator (for the Bachelors of Science in Information Science and Data Analytics Program) at the School of Information, San José State University, One Washington Square, San José, California. He received his Bachelor’s and Master’s in Computer Science from Jadavpur University, India and his Ph.D. from the Rutgers University, New Jersey, USA. His research is focused on Conversational and Interactive Information Retrieval, Human-Computer Interaction, and Machine Learning. He can be contacted at souvick.ghosh@sjsu.edu.
References
- Beyers, K.L.H. (2016). A history of the music therapy profession: Diverse concepts and practices. Barcelona Publishers.
- Biancalana, C., Gasparetti, F., Micarelli, A., & Sansonetti, G. (2013). An approach to social recommendation for context-aware mobile services. ACM Transactions on Intelligent Systems and Technology, 4, 279-314. https://doi.org/10.1145/2414425.2414435
- Blood, A. J., & Zatorre, R. J. (2001). Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. Proceedings of the National Academy of Sciences of the United States of America, 9898(20), 11818–11823. https://doi.org/10.1073/pnas.191355898
- Bogdanov, D., Haro, M., Fuhrmann, F., Xambó, A., Gómez, E., & Herrera, P. (2013). Semantic audio content-based music recommendation and visualization based on user preference examples. Information Processing and Management, 49(1), 13-33. https://doi.org/10.1016/j.ipm.2012.06.004
- Bunnell, L., Osei-Bryson, K. M., & Yoon, V. Y. (2020). Recsys issues ontology: A knowledge classification of issues for recommender systems researchers. Information Systems Frontiers, 22(6), 1377–1418. https://doi.org/10.1007/s10796-019-09935-9
- Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for youtube recommendations.RecSys 2016 - Proceedings of the 10th ACM Conference on Recommender Systems,191–198. https://doi.org/10.1145/2959100.2959190
- Cunningham, S. J., Downie, J. S., & Bainbridge, D. (2005). The pain, the pain: Modelling music information behavior and the songs we hate. ISMIR 2005 - 6th International Conference on Music Information Retrieval.
- Datta, H., Knox, G., & Bronnenberg, B. J. (2017). Changing their tune: How consumers’ adoption of online streaming affects music consumption and discovery. Marketing Science, 37(1). https://doi.org/10.1287/mksc.2017.1051
- de Almeida, J. S., Lordier, L., Zollinger, B., Kunz, N., Bastiani, M., Gui, L., Adam-Darque, A., Borradori-Tolsa, C., Lazeyras, F., & Hüppi, P. S. (2020). Music enhances structural maturation of emotional processing neural pathways in very preterm infants. NeuroImage, 207, paper 116391. https://doi.org/10.1016/J.NEUROIMAGE.2019.116391
- Dougan, K. (2012). Information seeking behaviors of music students. Reference Services Review, 40(4), 558–573. https://doi.org/10.1108/00907321211277369
- Ferwerda, B., Yang, E., Schedl, M., & Tkalcic, M. (2019). Personality and taxonomy preferences, and the influence of category choice on the user experience for music streaming services. Multimedia Tools and Applications, 78(14), 20157–20190. https://doi.org/10.1007/s11042-019-7336-7
- Gardikiotis, A., & Baltzis, A. (2011). ‘Rock music for myself and justice to the world!’: Musical identity, values, and music preferences. Psychology of Music, 2. https://doi.org/10.1177/0305735610386836
- Georgii-Hemming, E., & Westvall, M. (2010). Music education - a personal matter? examining the current discourses of music education in sweden. British Journal of Music Education, 27(1), 21–33. https://doi.org/10.1017/S0265051709990179
- Goble, J. S. (2010). What’s so important about music education? Routledge. https://doi.org/10.4324/9780203853221
- Gomez-Uribe, C., & Hunt, N. (2015). The netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems, 6(4). https://doi.org/10.1145/2843948
- Jannach, D., & Bauer, C. (2020). Escaping the mcnamara fallacy: Toward more impactful recommender systems research. AI Magazine, 41(4), 79–95. https://doi.org/10.1609/aimag.v41i4.5312
- Jannach, D., Kamehkhosh, I., & Bonnin, G. (2018). Music recommendations. algorithms, practical challenges and applications. In S. Berkovsky, I. Cantador & D. Tikk (Eds.), Collaborative recommendations: Algorithms, practical challenges and applications (pp. 487–525). https://doi.org/https://doi.org/10.1142/9789813275355_0015
- Jin, Y., Tintarev, N., Htun, N. N., & Verber, K. (2020). Effects of personal characteristics in control-oriented user interfaces for music recommender systems. User Modeling & User-Adapted Interaction, 30(2), 199–249. https://doi.org/10.1007/s11257-019-09247-2
- Jin, Y., Tintarev, N., & Verbert, K. (2018). Effects of personal characteristics on music recommender systems with different levels of controllability. RecSys 2018 - 12th ACM Conference on Recommender Systems, 13–21. https://doi.org/10.1145/3240323.3240358
- Kamyshev, K. V., Kureichik, V. M., & Borodyanskiy, I. M. (2020). Review of the recommender systems application in cardiology. Cardiometry, (16), 97–105. https://doi.org/10.12710/cardiometry.2020.16.97105
- Knight, W. E., & Rickard, N. S. (2001). Relaxing music prevents stress-induced increases in subjective anxiety, systolic blood pressure, and heart rate in healthy males and females. Journal of Music Therapy, 38(4), 254–272. https://doi.org/10.1093/jmt/38.4.254
- Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012). Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22(4-5), 441–504. https://doi.org/10.1007/s11257-011-9118-4
- Koelsch, S. (2010). Towards a neural basis of music-evoked emotions. Trends in Cognitive Sciences, 14(3), 131–137. https://doi.org/10.1016/j.tics.2010.01.002
- Laplante, A. (2010). Users’ relevance critera in music retrieval in everyday life. 11th International Society for Music Information Retrieval Conference.
- Lee, J. H., Cho, H., & Kim, Y. S. (2016). Users’ music information needs and behaviors: Design implications for music information retrieval systems. Journal of the Association for Information Science and Technology, 67(6), 1301–1330. https://doi.org/10.1002/asi.23471
- Lee, J. H., Cho, H., & Kim, Y.-S. (2010). Analysis of user needs and information features in natural language queries seeking music information. Journal of the American Society for Information Science and Technology, 61(5), 1025–1045. https://doi.org/10.1002/asi.21302
- Lee, J. H., & Price, R. (2016). User experience with commercial music services: An empirical exploration. Journal of the Association for Information Science & Technology, 67(4), 800–811. https://doi.org/10.1002/asi.23433
- Lee, S. Y., Xue, Q. L., Spira, A. P., & Lee, H. B. (2014). Racial and ethnic differences in depressive subtypes and access to mental health care in the united states. Journal of Affective Disorders, 155(1), 130–137. https://doi.org/10.1016/J.JAD.2013.10.037
- Millecamp, M., Htun, N. N., Jin, Y., & Verbert, K. (2018). Controlling spotify recommendations: Effects of personal characteristics on music recommender user interfaces. Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, 101–109. https://doi.org/10.1145/3209219.3209223
- Mori, K. (2022). Decoding peak emotional responses to music from computational acoustic and lyrical features. Cognition, 222, paper 105010. https://doi.org/10.1016/j.cognition.2021.105010
- Narducci, F., Basile, P., de Gemmis, M., Lops, P., & Semeraro, G. (2020). An investigation on the user interaction modes of conversational recommender systems for the music domain. User Modeling & User-Adapted Interaction, 30(2), 251–284. https://doi.org/10.1007/s11257-019-09250-7
- Pacchetti, C., Mancini, F., Aglieri, R., Fundaró, C., Martignoni, E., & Nappi, G. (2000). Active music therapy in parkinson’s disease: An integrative method for motor and emotional rehabilitation. Psychosomatic Medicine, 62(3), 386–393. https://doi.org/10.1097/00006842-200005000-00012
- Park, D. H., Kim, H. K., Choi, I. Y., & Kim, J. K. (2012). A literature review and classification of recommender systems research. Expert Systems with Applications, 39(11), 10059–10072. https://doi.org/10.1016/j.eswa.2012.02.038
- Pelletier, C. L. (2004). The effect of music on decreasing arousal due to stress: A meta-analysis. Journal of Music Therapy, 41(3), 192–214. https://doi.org/10.1093/jmt/41.3.192
- Rentfrow, P. J., & Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality correlates of music preferences. Journal of Personality and Social Psychology, 84(6), 1236–1256. https://doi.org/10.1037/0022-3514.84.6.1236
- Särkämö, T., Tervaniemi, M., Laitinen, S., Forsblom, A., Soinila, S., Mikkonen, M., Autti, T., Silvennoinen, H. M., Erkkilä, J., Laine, M., Peretz, I., & Hietanen, M. (2008). Music listening enhances cognitive recovery and mood after middle cerebral artery stroke. Brain, 131(3), 866–876. https://doi.org/10.1093/brain/awn013
- Savage, P. E., Loui, P., Tarr, B., Schachner, A., Glowacki, L., Mithen, S., & Fitch, W. T. (2020). Music as a coevolved system for social bonding. Behavioral and Brain Sciences, 44. https://doi.org/10.1017/S0140525X20000333
- Schäfer, T., Sedlmeier, P., Städtler, C., & Huron, D. (2013). The psychological functions of music listening. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00511
- Schedl, M., Flexer, A., & Urbano, J. (2013). The neglected user in music information retrieval research. Journal of Intelligent Information Systems, 41(3), 523–539. https://doi.org/10.1007/s10844-013-0247-6
- Schedl, M., Gómez, E., & Urbano, J. (2014). Music information retrieval: Recent developments and applications. Foundations and Trends in Information Retrieval, 8(2-3), 127–261. https://doi.org/10.1561/1500000042
- Schedl, M., Zamani, H., Chen, C. W., Deldjoo, Y., & Elahi, M. (2018). Current challenges and visions in music recommender systems research. International Journal of Multimedia Information Retrieval, 7(2), 95–116. https://doi.org/10.1007/s13735-018-0154-2
- Sedikides, C., Leunissen, J., & Wildschut, T. (2021). The psychological benefits of music-evoked nostalgia. Psychology of Music, paper 03057356211064641. https://doi.org/10.1177/03057356211064641
- Tian, L., Alaei, R., & Rule, N. O. (2021). Appearance reveals music preferences. Personality and Social Psychology Bulletin, 01461672211048291. https://doi.org/10.1177/01461672211048291
- Tsoi, K. K., Chan, J. Y., Ng, Y. M., Lee, M. M., Kwok, T. C., & Wong, S. Y. (2018). Receptive music therapy is more effective than interactive music therapy to relieve behavioral and psychological symptoms of dementia: A systematic review and meta-analysis. Journal of the American Medical Directors Association, 19(7), 568–576. https://doi.org/10.1016/j.jamda.2017.12.009
- Weigl, D., & Guastavino, C. (2011). User studies in the music information retrieval literature. Proceedings of the 12th International Society for Music Information Retrieval Conference.
- Woods, B., Spector, A. E., Jones, C. A., Orrell, M., & Davies, S. P. (2005). Reminiscence therapy for dementia. Cochrane Database of Systematic Reviews, (2). https://doi.org/10.1002/14651858.CD001120.pub2