In the TROMPA Choir singers use case, we are using multi-lingual singing voice synthesis to support individual choir singers rehearsal. During the first two years of the project, we have identified the users requirements, developed a working prototype and improved the synthesis models. Now, we are almost ready to start sharing our tools with choir singers! But, how did we get here?

A user-centered Pilot

In December 2018, we launched an open call for choirs in the Barcelona area to participate in a workshop to retrieve information about the requirements for the Choir Singers Pilot, the TROMPA application to support individual practice. In January 2019, 15 participants (including singers and conductors) from 5 different choirs took part in the workshop and provided feedback on a mock-up. This information was used to refine the mock-ups and iterate on the requirements for the pilot application.

From this point, Voctro Labs started working on successive iterations of the pilot, which is a web application compatible with different devices. Currently, we are working on visualisation modes for amateur singers and in the integration with other TROMPA components.  Development plans are to have a fully-functional prototype by next month (March 2020). This prototype will be tested in a workshop with potential users to validate its functionality and identify possible minor changes to be made to improve usability aspects. 

Exploiting existing resources

The use of singing synthesis technology for this use case offers a major advantage compared to the use of studio recordings: scalability. By using synthesis, we can generate singing performances from any piece available in digital symbolic formats. These scores exist in public repositories such as IMSLP or CPDL. We have built tools to import data from these existing repositories into the TROMPA Contributor Environment, such that pilots can access these data. Right now, we are working on the tools to generate synthetic performances from these scores that will be accessible through the pilot for choir singers.

Open datasets 

In TROMPA we also researched on technologies to analyze choral singing recordings in terms of intonation, to be exploited in the pilot. We are in the process of creating three open datasets for research on this topic: Choral Singing Dataset, released last year, ESMUC  ChoralSet (worked internally towards its publication and in collaboration with associated partner Escola Superior de Música de Catalunya), and Dagstuhl ChoirSet (submitted to an academic journal this month). All of them consist of recordings of choir singers singing simultaneously but recorded with individual microphones. The recordings are accompanied by information about the pitch of each singer, among other features, which is beneficial for the analysis of intonation of singers individually and as a group.

Multi-lingual singing synthesis

Singing synthesis is a complex problem. We are working on progressing beyond the state of the art for the specific case of choir singing, as well as in providing multilingual support for some of the most commonly found languages in the repertoire. The singing synthesis algorithm we use in TROMPA is based on the system called Neural Parametric Singing Synthesis (NPSS). As with any machine learning-based system, we need data to train the models. In this sense, we carried out recordings of a multi-lingual repertoire with the Francesc Valls choir. Currently, we are using these recordings to train the models that will be used for the pilot. At the same time, we are integrating a new model for F0 generation and experimenting with voice cloning to generate the multiple voices of the synthetic choir.

Stay in tune

So, since the pilot is almost ready… We are making plans to start running workshops with interested users during this Spring. Workshops will be held in the Barcelona area, but we plan to add streaming to allow for remote participation from all over Europe. 

Do you sing in a choir? Would you like to give a try to the TROMPA Choir Singers Pilot? Stay in tune and check the TROMPA news section; we will start running workshops very soon!