+ WHO?
Ari Benjamin Meyers & Halsey Burgund
Collaborations:
CERTH
Delphos
MIT Open Documentary Lab (ODL)
Juliet Aliban
+ WHAT?
Inclusive social soundscapes for diverse public spaces ??
+ HOW?
??
While living in divisive times, the simple act of listening and interacting with each other has become fraught with a constant barrage of political and social baggage. With Invisible Choir, the goal is to circumvent some of these challenges by allowing strangers to create an evolving collective musical work by singing together, without words. Invisible Choir combines urban sound design, sound ecology, and technology to create a novel type of interactive social soundscape: an invisible landscape of music generated by an ever-evolving open-sourced choir. This transforms the anonymous, often isolating experience of urban environments into spaces of collective creativity. Beginning with musical “seeds” composed for specific locations, a choral composition is nurtured through the active, asynchronous participation of visitors who become members of the Invisible Choir through their vocal contributions, recorded and “planted” live on site.
This is made possible through a specially designed web-based application that facilitates both collective singing and the listening of “choirs.” By harnessing the power of new technologies—including artificial intelligence and advanced audio AR—the project introduces an innovative and urgent form for urban public spaces that encourages collective engagement centered around communal singing, music creation, and the establishment of new networks of citizen-singers. Ultimately, Invisible Choir reimagines what a public artwork can be and how it may allow people to connect with each other and their (sonic) environment in these polarized times.
audio augmented reality, co-creation, collective singing, geo-located audio, contributory, evolving composition, algorithmic composition, urban sound design, city soundscape, voices
Tools and Methodology: Design Methods & Frameworks
1. Audio Mixing Algorithm Experiments – A central question in the development of the algorithm was how to make the listening experience of Invisible Choir engaging for participants while having only marginal control over individual audio clips, which are short and looping. The algorithm was conceived as a dynamic puzzle with unknown inputs that must be assembled in real time, optimizing for the elusive metric of “sounding good.” At a high level, the prototype algorithm performs the following:
– Determines which singing voice clips are “available” based on the listener’s physical location.
– Selects a subset of available loops to play, synced and looped.
– Applies further preprocessing to a percentage of playing loops, including fractional playback and speed/direction adjustments.
– At each loop point, evaluates which loops play and how they play based on settings and probabilities.
– Continuously monitors the listener’s position and fades out audio from loops no longer in range.
– Includes numerous configuration options (e.g., min/max number of loops, update probability, loop fractions available) to craft the aesthetic output best suited for the artwork, refined through extensive experimentation.
2. AI Music Generation Experiments – Various commercial AI audio tools were researched and tested, including ElevenLabs voice cloning, Udio, Suno, and Replicate, to explore bespoke audio models. While some insights were gained, none of these services were ultimately suitable for integration into Invisible Choir or Roundware due to API limitations and challenges in preserving the human characteristics of input voices. Music generation services produced high-quality audio in diverse styles, but replicating the style and timbre of individual human voices proved difficult. Voice cloning services excelled at reproducing timbre and style for speech, but using them for musical purposes was largely off-label, limiting their effectiveness.
3. Extensive UX Iterations – A UI/UX designer collaborated on the visual and functional components of the web app. Multiple iterations were carried out both pre-build and alongside development to refine the interface, ensuring usability and participant engagement.
Tools and Methodology: Immersive Experiments
Several audio augmented reality experiments were conducted at various sites during development. The web app was designed to allow testing via a map interface rather than requiring physical presence on-site, which proved critical for many technical improvements. However, fully assessing the system’s effectiveness required on-site testing. Test sites were set up in Berlin and in Maine, Massachusetts, and Connecticut (USA), while a test site in Mumbai, India, was created by the development team.
Tools and Methodology: Interactive Modules & Web Tools
1. Roundware
– Roundware server – A cloud-based server built in Python using Django, maintaining a database of audio and related data and implementing a RESTful API for access.
– Web framework – A TypeScript framework module that implements the Roundware server API and in-browser audio mixing. It can be included in any web project via npm import.
– Web app – A React-based front-end app that implements the UI/UX and incorporates the Roundware framework.
2. Polygon Generator – A test web app was developed to experiment with and establish a critical visual element of the project, ensuring it is both functional and visually descriptive. The app allowed rapid testing of different polygon shapes, connectors, and animations within the map view.
3. Invisible Choir Web App – While serving as the primary interface for participants during public exhibitions (not merely a tool), the web app underwent multiple iterations to continuously refine front-end and back-end aspects. The goal was to make the app as intuitive and performant as possible within the constraints of web-based technologies. In this sense, it also served as a tool for rapid design and functionality iteration.
Collaborations
CERTH – Collaboration with AI developers at CERTH aimed to explore various ways AI could potentially be harnessed to assist in the automated composition process. One set of experiments analyzed a longer “base track” as well as shorter example participant recordings and optimized for harmonic placement of the latter within the former. Another experiment developed a model to pitch shift participant recordings to the most “effective” nearby pitch without losing the timbre of the voice.
Delphos – Consultation was held with the CEO of Delphos, an AI music generation company, to determine what compositional tools their music models could offer the project. Delphos models produce “soundworlds” trained on specific musical data and can “compose” musical pieces which are stylistically derived from these soundworlds.
MIT Open Documentary Lab (ODL) – Invisible Choir was presented to the ODL as an opportunity to showcase a new innovative form of “documentary” that Invisible Choir represents, as well as to gather feedback from a cohort of artist/academic/technical fellows.
Juliet Aliban – A visual designer was contracted to develop the UI and UX for the web app and define the overall design aesthetic for the project. It was critical that the design reinforced the conceptual and artistic goals of the project while providing a welcoming and intuitive experience for participants. Since the project relies heavily on participant contributions (singing), multiple iterations were carried out to determine a design that would maximize engagement. As the project involves an uncommon app experience—asking participants to sing along—considerable effort was made to ensure this felt natural and easy.
Impact
Experimentation and iteration have been critical to the project’s development. The experiments and collaborations described above were all necessary to reach the point of the initial exhibition—some because they proved successful and useful, and others because they revealed what would not work and should be avoided.
Developing the “looping” recording system for participant contributions provides a primary example. Numerous approaches were designed and tested before building the version that initially seemed most effective. Weaknesses were identified through testing; some were fixed, while other approaches were discarded in favor of new ideas. The first recording system was nearly impossible to use to create a good recording, but after four or five iterations, the team developed a system that feels much more intuitive and yields higher-quality contributions.
Scenarios / Tests / Exhibitions:
The project has followed a continuous and iterative process— design 🡪 develop 🡪 test 🡪 redesign—throughout its entire development. Much of this work was carried out by a small team of two artists and several developers, setting up test instances in Maine, Massachusetts, and Connecticut (USA), Mumbai (India), and Berlin (Germany). For each version of the design, sample voice melodies were recorded and “installed” on top of a physical landscape in proximity using the Roundware web admin system. The team then used the web app while walking through this installation to observe how the algorithm behaved, assess the UI/UX, and explore different ways to distribute the seed voices to achieve the desired aesthetic effect.
All of the core functional development accomplished for Invisible Choir, which expanded the Roundware platform, will be integrated back into the open-source Roundware codebase. This ensures that anyone interested in creating geo-located audio projects can use these tools in the future. Roundware can be easily incorporated into other web-based projects via the Roundware web framework, and the team looks forward to seeing how other artists, educators, and creators deploy and further develop the platform.
Lessons learned:
Several key lessons were learned throughout the experimentation, including:
– Technical limitations become fully apparent only when pushed to their limits during experimentation. Numerous constraints in mobile browsers related to audio recording and processing were discovered during experimentation.
– Creating an intuitive experience is more challenging than anticipated. The “sing-along” recording interface went through multiple design iterations, as initial ideas did not perform in reality as expected during the design phase.
– Balancing artistic vision with technical feasibility is a constant challenge. Technology can enable remarkable aesthetic experiences, but the team’s imagination often surpassed reality, requiring adjustments and rebalancing.
– AI currently functions best as a creative collaborator, not as an independent creative force. However, it is becoming increasingly capable at a rapid pace.
Value of the Approach
The project aims to transform the way:
– public spaces are perceived as areas of collective creation
– musical expression can bring people together
– embedding music into a landscape can transform the in-situ experience
The Pilot Use Case “Musical Experience Design” demonstrates the feasibility and potential of these transformations. The technical aspects of the project have been approached with an open-source ethos. Technological creativity—as with much of human creativity—builds upon previous work, and the team firmly believes that openly sharing their work, while drawing inspiration from existing projects, yields better results for both themselves and others. Invisible Choir implements a major expansion of the open-source Roundware platform, which will be released for others to build upon. Throughout development, the team prioritized creating a technical solution for Invisible Choir, while incorporating as much flexibility and configurability as possible. This approach supports both aesthetic experimentation and broader applicability for various artistic and educational use cases.
Adaptability / interoperability:
The project was built as a web-based, highly standards-compliant solution rather than a native mobile app (iOS/Android) because the web is inherently based on open protocols and designed for interoperability. The core functionality can therefore be integrated into other web-based projects with relative ease. Using common programming languages (JavaScript, TypeScript, Python) further expands the potential for development across the wider developer community. While significant restrictions exist when building for mobile browsers compared to native apps—making development more challenging—the team felt, and continues to feel, that these trade-offs are worthwhile for achieving participant convenience and broad interoperability.