Designing with advanced artificial intelligence (2024-2025)

This study focuses on a core design question: in social gathering settings such as board game evenings, how can an AI-driven, context-aware music selection system be utilised to minimise social friction over music whilst striking a balance between automated recommendations and user autonomy? The research background highlights that music plays a vital role in social settings by setting the mood, enhancing immersion and fostering social connections; however, the highly subjective nature of musical preferences often leads to social friction—the question of ‘what song to choose’ frequently becomes a point of contention at gatherings. At the same time, whilst algorithmic recommendations have become deeply ingrained in everyday music listening behaviour, they may limit users’ autonomy in musical exploration due to the ‘echo chamber effect’. To address this, this study designed and developed a voice-interactive music playback system named ‘A/Muse’. The system utilises the Google Web Speech API for real-time speech recognition, capturing natural snippets of conversation between players during board games; it then employs the Gemini 1.5 Flash large language model to analyse the dialogue for sentiment and context, generating playlist titles that match the current atmosphere (such as ‘tense and intense competitive atmosphere’); finally, it uses the Spotify Web API to search for and play matching existing playlists. The system also features a high-fidelity mobile interface designed in Figma, allowing users to input contextual information such as the game type, whilst employing the Wizard-of-Oz method to have researchers simulate the entire “speech recognition → sentiment analysis → music recommendation” process in the background. Preliminary results from user testing conducted with five participants (playing Exploding Kittens), combined with the UEQ questionnaire and semi-structured interviews, indicate that: The system received high ratings across dimensions such as ‘Enjoyment’ (average score 6.2/7), ‘Comprehensibility’ (average score 6.2/7) and ‘Creativity’ (average score 2.2/7, on a reverse scale); A/Muse effectively enhanced the game’s immersion and atmospheric experience. The core contribution of this design lies in: integrating multimodal AI technologies (speech recognition + LLM sentiment analysis + API music retrieval) for the first time into a ‘shared autonomy’ music system tailored for social scenarios; validating the feasibility of context-aware automatic song selection in reducing the burden of social music decision-making; and revealing differences in user demands regarding ‘controllable randomness’ and ‘algorithmic transparency’—with users holding diametrically opposed attitudes towards the unpredictability of music (some view it as a pleasant surprise, whilst others regard it as a poor experience), providing empirical reference and design insights for the interaction design of future intelligent music systems on ‘how to strike a balance between automation and user control’.