Instead, it makes use of a user simulator, a model that estimates how a user would respond to a list of tracks picked by the agent. In model-based RL, the agent is not trained online against real users. To avoid letting an untrained agent interact with real users (with the potential of hurting user satisfaction in the exploration process), we make use of a model-based RL approach. To take into account the constraints and the sequential nature of music listening, we use a reinforcement learning approach.
For this reason, automatic playlist generation is a relevant and practical problem for music streaming platforms to create the best personalized experience for each user. Having a catalog of millions of tracks, it is not straightforward which tracks are best suited for the user that requested the playlist, as each user experiences music differently. Here, instead, we assume we have a user-generated response for multiple items in the slate, making slate recommendation systems not directly applicable.Īs an example, consider the case when users decide on a type of content they are interested in (eg, “indie pop”). Crucially, our use case is different from standard slate recommendation tasks, where usually the target is to select at maximum one item in the sequence.
#Autodesk cfd how to#
We frame the problem as an automatic music playlist generation: given a (large) set of tracks, we want to learn how to create one optimal playlist to recommend to the user in order to maximize satisfaction metrics. RL for Automatic Music Playlist Generation Finally, we demonstrate that performance assessments produced from our simulator are strongly correlated with observed online metric results. We show how these agents lead to better user-satisfaction metrics compared to baseline methods during online A/B tests.
#Autodesk cfd Offline#
We analyze and evaluate agents offline via simulations that use environment models trained on both public and proprietary streaming datasets. Using this simulator we develop and train a modified deep Q-Network, which we call the Action-Head DQN (AH-DQN), in a manner that addresses the challenges imposed by the large state and action space of our RL formulation. In particular, we developed a RL framework for set sequencing that optimizes for user satisfaction metrics via the use of a simulated playlist-generation environment. In this work, we apply RL to solve an automatic music playlist generation problem. Reinforcement learning (RL) is an established tool for sequential decision making.
0 kommentar(er)
