By Tristan Vonet, Junior Researcher
At the beginning of the Autumn semester 2022, during a class called ‘Navigating Complexity’, I was introduced to ETHOS lab and their Junior Researcher program (JRP). If you are reading this blogpost, you most likely already know what the JRP is, so I will spare you the details about what it entails (and you can read about the lab on this website). At that time, I had spent four years away from academia; the first two years working in eCommerce as an integration specialist, and the last two years starting to work my way into the music industry.
As anyone who has been working for some time in the music industry will tell you, the landscape of it is very complex. Following the journey from the release of a single/album/music all the way from the artist/band to its listeners, can take us through very different scenarios. The listener’s touchpoints are multiple, ranging from concerts, streaming platforms, movies, radio, printed and digital media, just to name a few. Likewise, a single musical piece has to go through the hands of several mediators – lawyers, managers, A&Rs, publishers, distributors – before being made available to the public through any of the aforementioned touchpoints. Mapping this complex interplay between the different actors/stakeholders is an interesting and not so trivial exercise, however, this is not the aim of this blogpost.
Within this constellation of actors and mediators, streaming platforms has always had a particular interest to me, such as Spotify, Apple Music or Deezer. This interest comes from the fact that within these organizations happens an interplay between society, culture and technology which has defined the way we have come to interact with music. The first, and most obvious interplay, has been the evolution of the format of music. Nowadays, music is mostly accessed as a digital audio format, either in .mp3, .wav or .flac file formats, however, a couple of decades ago, the only formats available were analog formats such as vinyl and cassettes. Digital audio formats have brought a much higher level of accessibility, connectivity, and portability to listeners (Gomes, 2016).
Once again, this particular interplay of culture and technology is not the object of study of this blogpost, however, it has led to the emergence and development of new technologies, such as recommender systems, which will be the focus of this this blogpost.
About Recommender Systems
Recommender Systems emerged in the early 2000’s as a way of providing personalized recommendations to users. Behind the scenes, ensembles of machine learning algorithms process many different types of data, from multiple sources, in order to make predictions about what content should be recommended to either a specific user or a group of users. In the case of streaming platforms, the output of these recommender systems – the predictions – feed into different elements of a streaming platform’s web or mobile application, such as curated playlist, personalized home screens and more, depending on the streaming platform one might be studying.
There are a number of inherent limitations and risks associated with recommender systems, which, when used to curate cultural content such as is the case with music, can have certain societal implications, impacting both the lives of listeners and musicians alike. One such risk, explained in the research of Chaney, Stewart and Engelhardt (2018), is algorithmic confounding – a feedback loop that occurs when a recommendation algorithm learns from its own recommendations – which can result in more homogeneous user activity and decrease the value that minority groups experience. In the world of music, these effects therefore tend to overrepresent popular music and misrepresent smaller niche genres in the algorithmically curated content that is outputted by the recommender systems. This phenomenon has been referred to as popularity bias (Hesmondhalgh et al., 2023). Other biases have also been observed in recommender systems, most notably gender biases. In a study by Shakespeare et al., it has been noted within a binary gender classification, “recommender systems can propagate a pre-existing bias” (2020, p.8), and cause harm to certain social groups.
These risks do not only stem from the technical aspects of recommender systems. It would not be difficult for any given music streaming platform to set up services for artists that would allow them to get recommended more, at a certain cost. In fact, Spotify launched in 2020 a service called Discovery Mode, in which artists relinquish a small part of their royalties in exchange for better algorithmic recommendations on their platform (Spotify, 2020). These sorts of initiatives, seeking to monetize recommender systems, do not come without their fair share of controversy, understandably so.
How to study them
It therefore comes as no surprise then that people have put extensive effort into finding ways to study these systems, in order to mitigate the negative effects that might be incurred by their use, or at least hold accountable those that develop them. One of these individuals is Nick Seaver, an assistant professor of anthropology at the Tufts University. In his book “Computing Taste: Algorithms and the Makers of Music Recommendation” (2022), he provides a presentation of the reality that the people involved in the development of these systems are living in when it comes to thinking about an individual’s taste in music. Already from the introduction of his book, the reader is warned not to succumb to the traditional view of algorithms as black-boxes that can’t be studied unless you have access to them. Seaver (2022), referring to observations made by feminist epistemologist Sandra Harding in 1986, tells us that this black-box concept “give[s] a poor account of the interactive and dynamic processes through which knowledge is made”. When it comes to recommender systems, their impact on cultural, economic, and societal dynamics are not only to be understood through the unattainable technical aspects of the technology – the source code for instance – but also through the eyes of their creators and the users and how they make meaning of them.
As such, many methods can be employed to study recommender systems, that stem from a broad range of academic disciplines. Seaver has spent over a decade talking to the makers of recommender systems, using the methods of ethnography, to create knowledge about these systems. He offers us a rather somber conclusion to this decade-long research: “Corporations are complex and ambivalent: they do not speak with one voice, and one employee’s point of view may be straightforwardly contradicted by another’s” (2022).
Ethnography is just one of the methods that can be employed to observe and analyze these relatively closed systems. In my case, I was limited by the given timeframe for this project and therefore sought to use some of the methods that we have been taught on the Digital Innovation & Management master’s program at the IT University. One such method was the walkthrough method, however, before we get to that, I would like to introduce you to the company I have been studying this past year.
Case study: Spotify
With recommender systems as the focus of my research project, I found it natural to look into what Spotify was doing. Since their launch in 2008, they have had a focus on technology, to the extent of developing their own peer-to-peer protocols for transferring data more efficiently (Astor, 2021), and have had, since 2020, dedicated engineering and R&D forums where articles and posts explain their tech stack – many of which are about their recommender systems. What’s more, Spotify has been a contributor to the ACM Conference Series on Recommender Systems since 2016, which is an international conference that involves some of the largest tech corporations, in which the latest advancements in recommender systems are discussed among experts (RecSys Community, 2016).
How recommender systems work at Spotify
On the technical side, the algorithms used by Spotify to recommend songs seek to do two things: 1) create a representation of a track, and 2) create a representation of each individual user (Pastukhov, 2022).
To generate track representations, two filtering methods are employed. The first one is called content-based filtering, which, as the name implies, analyzes the content itself. This analysis encompasses both a track’s metadata (track title, artist name, version of the track, etc.), the track’s raw audio signal, the lyrics (if present) and online music blogs and news outlets (Pastukhov, 2022). The second filtering method is called collaborative filtering. This method of filtering, in the context of Spotify, looks at whether song A appears in the same user playlist as song B, and in how many user playlists it does so (ibid). This provides an idea of the similarities – genre, moods, cultural references – between songs based on their inclusion within playlists.
With these two filtering methods, tracks are represented rather accurately, however they still need to be recommended to the right users. As such, Spotify also generates, for each user, a taste profile, so that the recommender systems will know which track should be recommended to which user. When we use Spotify, most if not all of our actions get logged. This often amounts to enormous amount of information, some of which is more obvious (playlist adds, tracks played, artist follows), while others are not so obvious (track playthrough, repeated songs). These types of activity logs are called active feedback and passive feedback, respectively (ibid), and create the taste profile of each individual user.
How these recommender systems work is a rather interesting topic, however, as has been noted by some researchers, algorithmic dynamics (Lazer et. al., 2014) – the fact that changes are made every day to the algorithm and how it works – makes the study of “how does it work” highly transient. Let’s say you had access to the source code on a given day. Through painstaking work, you might find out that certain parameters, for instance a track’s raw audio signal, might be weighted more than other parameters within a particular recommender system. This would undoubtedly inform you of the considerations that Spotify has put into practice. However, coming back a month later, you would find out that the weight given to certain parameters would have changed, and as such, your analysis would not be representative of the new reality.
The walkthrough method
With a technical analysis out of the question, I therefore, as mentioned earlier, turned to another method of critically assessing the use of recommender systems within Spotify: the walkthrough method. This method employs ethnographic methods and mobilizes concepts from Science and Technology Studies to conduct a critical analysis of an application (Light et. al., 2016). My thinking has been that even though I cannot access the back-end of Spotify’s services, I can still peer into their application. As Light et. al. mention: “Another way of understanding the influence of non-human actors [i.e. recommender systems] is through consideration of a technology’s materiality and the affordances it extends” (2016, p.886).
This method advocated by Light et al. is two-fold. The first part of the method consists of establishing the environment of expected use of an application. In order to do so, one needs to look at three facets of an application: the vision, which tells us what the user “is supposed to do and, by extension, implies how it can be used and by whom” (p. 889). The second facet to be researched is the company’s operating model, which “involves its business strategy and revenue sources, which indicate underlying political and economic interests” (p.890). The third facet is the governancepractices employed by the company, and “involves how the app provider seeks to manage and regulate user activity to sustain their operating model and fulfill their vision” (p. 890). These three facets of an application’s environment of expected use will then allow us to understand “how an app’s designers, developers, publishers and owners expect user to receive and integrate [the application] into their technology usage practice” (p. 890).
The second part of the walkthrough method consists of gathering data by “walking through” the app as a user would do. This is called the technical walkthrough. This allows the researchers to engage “directly with an app’s interface to examine its technological mechanisms and embedded cultural references to understand how it guides users and shapes their experiences” (p.882).
It is also necessary to touch upon the theoretical principles behind this methodology, as they are a necessary aspect to understand how researchers should interact with the elements of an application. The walkthrough method is grounded in Bruno Latour’s Actor-Network Theory, which sets a scene in which both human and non-human actors – most often of the technical kind – mutually shape each other’s existence in this world (Latour, 2005). Latour further defines the actors as either being intermediaries or mediators (Latour, 2005), where meditators are agents of change within the system in which they act, while intermediaries have a passive effect on the same system.
In the context of apps, Light et. al. point out that “user interfaces and functions are […] understood as non-human actors that can be mediators” (2018, p.886), and I will go further to say that recommender systems, which fall under the category of ‘functions’, are mediators with a high potential of transformativity. As technical systems that sort, recommend, and arrange content on an application’s interface, which in the context of Spotify, have monetary and career implications for many artists, the way these systems are engineered and integrated into an app’s user interface matters heavily.
Preliminary conclusion
Had this project of mine been an actual research project or exam paper, this would be the place where I present my methodology and findings. Unfortunately, partly due to poor time management on my end, as well as the scope of my project, I have not been able to synthetize my findings into a coherent form. Nonetheless, I have through this project gathered a solid amount of knowledge about the recommender systems, both within and outside music streaming platforms. This knowledge is not something that will be lost to me, and I have plans to give myself the opportunity to take this research project further, as I believe that it is both an interesting and important subject to study. And for you, the reader, I hope that this blogpost has sparked some interest into recommender systems and the critical research needed to be done in order to make sure that the makers of recommender systems do not reinforce existing biases in the world through misuse of their technology.
Bibliography
Chaney, A.J.B., Stewart, B.M. and Engelhardt, B.E. (2018). How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. Proceedings of the 12th ACM Conference on Recommender Systems – RecSys ’18. [online] doi:https://doi.org/10.1145/3240323.3240370.
Gomes, R. 2016. Audio Quality X Accessibility How Digital Technology Changed the Way We Listen and Consume Popular Music. Revista Vortex, 4(2).
Hesmondhalgh, David, et al. The Impact of Algorithmically Driven Recommendation Systems on Music Consumption and Production – a Literature Review. 9 Feb. 2023, www.gov.uk/government/publications/research-into-the-impact-of-streaming-services-algorithms-on-music-consumption/the-impact-of-algorithmically-driven-recommendation-systems-on-music-consumption-and-production-a-literature-review. Accessed 1 June 2023.
Latour, B. (2005). Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford University Press.
Lazer, D., Kennedy, R., King, G. and Vespignani, A. (2014). The Parable of Google Flu: Traps in Big Data Analysis. Science, [online] 343(6176), pp.1203–1205. doi:https://doi.org/10.1126/science.1248506.
Light et. al., (2016). The walkthrough method: An approach to the study of apps. New Media & Society, 20(3), pp.881–900. doi:10.1177/1461444816675438.
Pastukhov, D. (2022). Inside Spotify’s Recommender System: A Complete Guide to Spotify Recommendation Algorithms. [online] www.music-tomorrow.com. Available at: https://www.music-tomorrow.com/blog/how-spotify-recommendation-system-works-a-complete-guide-2022.
RecSys Community. “RecSys – ACM Recommender Systems.” Recsys.acm.org, 15 Sept. 2016, recsys.acm.org/recsys16/. Accessed 6 June 2023.
Shakespeare, D., Porcaro, L., Gómez, E. and Carlos Fernandez-del Castillo (2020). Exploring Artist Gender Bias in Music Recommendation.
Spotify (2020). Amplifying Artist Input in Your Personalized Recommendations. [online] Spotify. Available at: https://newsroom.spotify.com/2020-11-02/amplifying-artist-input-in-your-personalized-recommendations/ [Accessed 1 Jun. 2023].