Home Tech/AIALS took away this musician’s voice. AI enabled him to sing once more.

ALS took away this musician’s voice. AI enabled him to sing once more.

by admin
0 comments
ALS took away this musician's voice. AI enabled him to sing once more.

As Patrick Darling’s song starts to play, tears fill the audience. It’s an emotional tune penned for his great-grandfather, whom he never had the opportunity to meet. However, the performance is poignant for another reason: It marks Darling’s return to the stage with his bandmates after he lost his ability to sing two years earlier.

The 32-year-old artist was diagnosed with amyotrophic lateral sclerosis (ALS) at the age of 29. Similar to various types of motor neuron disease (MND), it impacts the nerves that control the body’s muscles. Individuals suffering from ALS ultimately lose control over their muscles, including those required for movement, speech, and breathing.

Darling’s last live performance occurred over two years ago. By that time, he had already lost the ability to stand and play his instruments and was finding it hard to sing or talk. Recently, however, he managed to recreate his lost voice using an AI tool trained on clips from old audio recordings. Another AI tool has allowed him to use this “voice clone” to write new songs. Darling is making music once more.

“Sadly, I have lost the ability to sing and play my instruments,” Darling stated on stage at the event in London on Wednesday, using his voice clone. “Despite this, I spend most of my time now focused on composing and producing my music. It feels more crucial than ever for me.”

Losing a voice

Darling claims he has been a musician and composer since around age 14. “I taught myself to play bass guitar, acoustic guitar, piano, melodica, mandolin, and tenor banjo,” he shared at the event. “But my greatest passion was always singing.”

He met bandmate Nick Cocking over a decade ago while still in university, according to Cocking. Shortly after, Darling became part of Cocking’s Irish folk group, the Ceili House Band, with their first performance together taking place in April 2014. Darling joined as a singer and guitarist, significantly enhancing the band’s musicianship, as stated by Cocking.

The four bandmates pose with their instruments.
Patrick Darling (second from left) with his former bandmates, including Nick Cocking (far right).
COURTESY OF NICK COCKING

However, a few years back, Cocking and his fellow band members began observing changes in Darling. He started to act clumsily, recalls Cocking. He remembers a night when the band had to trek through rainy Cardiff: “He kept slipping and falling, tripping over paving stones and similar obstacles.”

At that moment, Cocking didn’t think much of it, but Darling’s symptoms progressively worsened. The disease initially impacted his legs, and by August 2023, he began needing to sit during performances. Soon after, he started losing the use of his hands. “Ultimately, he couldn’t play the guitar or banjo anymore,” says Cocking.

By April 2024, Darling struggled to talk and breathe simultaneously, according to Cocking. For that performance, the band lifted Darling onto the stage. “He called me the following day and said he couldn’t do it anymore,” Cocking shares, his voice trembling. “By June 2024, it was over.” That marked the last occasion the band performed together.

Re-creating a voice

Darling was connected with a speech therapist who suggested “banking” his voice. Those losing their ability to speak can choose to record their speech and utilize those recordings to produce sounds that can then be activated by typed text, whether manually or with an eye-controlled device.

Some users have found these tools to sound mechanical. However, Darling encountered a different challenge. “By that stage, my voice had already changed,” he indicated during the event. “It felt like we were preserving the wrong voice.”

Subsequently, another speech therapist exposed him to alternative technology. Richard Cave, a speech and language therapist and researcher at University College London, is also a consultant for ElevenLabs, an AI firm that produces audio, speech, video, and music technology. One of their tools can create “voice clones”—realistic imitations of actual voices generated from mere minutes or even seconds of a person’s recorded voice.

Last year, ElevenLabs unveiled an impact program pledging to provide complimentary licenses for these tools to individuals who have lost their voices due to ALS or other ailments, including head and neck cancer or stroke.

The tool is already assisting some users. “We’re not truly enhancing how quickly they can communicate or alleviating all the physical challenges individuals with MND face, such as issues with eating and breathing,” remarks Gabi Leibowitz, a speech therapist leading the program. “Yet what we are offering is a means … to create again, to flourish.” Users can extend their time in their jobs and “continue engaging in activities that make them feel human,” she adds.

Cave collaborated with Darling to utilize the tool to recreate his speaking voice from older audio samples.

“The first instance I heard the voice, I thought it was remarkable,” Darling remarked at the event, using the voice clone. “It sounded just like I used to, and you honestly couldn’t tell the difference,” he claimed. “I won’t disclose the first word I had my new voice utter, but I can tell you it commenced with ‘f’ and concluded with ‘k.’”

Patrick and bandmates with their instruments prior to his MND diagnosis

COURTESY OF PATRICK DARLING

Re-creating his singing voice proved to be more challenging. The tool typically needs roughly 10 minutes of clear audio to produce a clone. “I lacked high-quality recordings of myself singing,” Darling explained. “We had to rely on audio from videos taken on people’s phones, recorded in noisy pubs, and a few recordings of me singing in my kitchen.” Nevertheless, those snippets sufficed to craft a “synthetic version of [Darling’s] singing voice,” according to Cave.

In the recordings, Darling’s voice sounded somewhat raspy and “was slightly off” on some notes, notes Cave. The voice clone retains those same characteristics. It doesn’t sound flawless, acknowledges Cave—it sounds human.

“The ElevenLabs voice we’ve developed is fantastic,” Darling expressed at the event. “It certainly resembles me—[it] just feels like a different rendition of myself.”

ElevenLabs has also created an AI music generator named Eleven Music. This tool allows users to compose tracks by using text prompts to select musical styles. Several prominent artists have partnered with the company to license AI clones of their voices, including actor Michael Caine, whose voice clone is utilized for the narration of an upcoming ElevenLabs documentary. Last month, the company released an album containing 11 tracks created with the tool. “The Liza Minnelli track is genuinely a hit,” remarks Cave.

Eleven Music can generate a song in a minute, but Darling and Cave invested about six weeks refining Darling’s song. Any user can use text prompts to “create music and introduce lyrics in any style [they prefer],” Cave explains. Darling enjoys Irish folk, but Cave has also collaborated with someone in Colombia who is developing Colombian folk music. (The ElevenLabs tool currently supports 74 languages.)

Back on stage

Last month, Cocking received a call from Cave, who shared with him Darling’s finished track. “Upon hearing the first couple of words he sang, I had to turn it off,” he recounts. “I was just overwhelmed, in tears. It took me several attempts to listen to the entire track.”

Darling and Cave began planning to perform the track live at the ElevenLabs summit in London on Wednesday, February 11. Consequently, Cocking and bandmate Hari Ma arranged additional parts to be played on the mandolin and fiddle. They had a couple of weeks to rehearse before joining Darling on stage, two years after their last joint performance.

“I wheeled him onto the stage, and neither of us could believe it was actually happening,” Cave remarks. “He was ecstatic.” The song played as Darling remained on stage, with Cocking and Ma performing their instruments live.

Cocking and Cave mention that Darling intends to continue using the tools to create music. Cocking expresses his hope to perform with Darling again but acknowledges that due to the nature of ALS, making long-term plans is challenging.

“It’s incredibly bittersweet,” says Cocking. “Yet being on stage and seeing Patrick there filled me with immense joy. I know Patrick was really pleased as well. We’ve been discussing it … He felt deeply proud.”

ELEVENLABS/AMPLIFY

You may also like

Leave a Comment