Crafting Artificial Musicians: Building Solaris as a Co-operative, Perceiving and Creative AI
摘要
In this chapter, we discuss an approach for crafting an artificial musician. At the centre of this is the understanding that music is an embodied process involving joint action and mutual perception. In contrast to many music-AI projects, we have designed and built a system from the bottom-up by focusing on musical togetherness and embedding the mutual perception of liveness and responsivity rather than symbolic representations of music such as midi-files, notation and audio. We view this from the perspective of creative artistic researchers, and therefore from an insider perspective. By developing an artificial musician that is co-creative and capable of ‘perceptual experience’ (Kirk, Zombies and us: understanding consciousness, Bloomsbury, London, 2017) we believe this approach is appropriate for a wide range of musical systems. The practice-based project at the centre of this paper is Solaris a jazz quartet in which three musicians are AI, trained with a dataset collected over two years, and only one musician (co-author Vear) is human. This ensemble has been performing since 2019 in various guises, mostly with compositions in the spirit of jazz and free improvisation. In the first section of this chapter, we outline the core perspectives that define our understanding of music-making, behavioural and embodied AI, AI as media, the nature of togetherness, and the importance of mutual perception of liveness and perceptual experience (Kirk, Zombies and us: understanding consciousness, Bloomsbury, London, 2017) from our musicking partners. These have been developed and tested through a process of practice-based methods, with the co-authors operating as practitioner-researchers in this process. From this, we present a list of guiding principles which we used to craft Solaris. Following this we discuss how we crafted Solaris through (a) a new dataset that captured certain physiological parameters from a series of professional jazz pianists; (b) a new approach to an AI-Factory built on models trained from the dataset, and (c) a digital score that represented the inner workings and experience of Solaris’ AI. We tested Solaris in several scenarios. The results obtained from the interaction of human musicians with Solaris are surprising, as they suggest that human musicians see Solaris as a competent jazz improviser who acts and responds in real time to the musical space. In the discussion section, we dig deeper into the implications of Solaris’ perceptual experience and playfully present a conceptualisation of Solaris as a sentient Creative-AI system. We propose a framework based on Charles Sanders Peirce’s concept of Synechism to understand sentience in the context of Creative-AI and from this insider perspective. The idea that AI should be understood as a form of mind, following the precepts of Peirce’s synechism, opens up new avenues for collaboration between humans and apparatus, breaking the Cartesian duality (spirit-matter) that has plagued us since Descartes (Hull, J Soc Comput 4(3), 193–204, 2023). What we experience when interacting with artificial media systems like Solaris is not just a cold, programmatic interaction with inert matter, there is the presence of a collaborative and creative spirit (Poltronieri and Vear, ISEA—Awaiting publication, 2024) that has perceptual experiences (Kirk, Zombies and us: understanding consciousness, Bloomsbury, London, 2017).