How this Canadian producer is using AI to allow artists to sing in different languages

When Canadian producer JordanYoung started playing around with AI voice-cloning technology ultimately creating a fakeKanye West song that sounded, well, just like Kanye West it immediately prompted himto think about the ethical implications of the technology.

"As soon as I made it, I started to think ... if anybody can make music with anyone's voice, what does this mean for artists? What does this mean for music?" said Young, who has worked with Beyonc, BTS and Dua Lipa.

He and his company Hooky recently developed a technology that could make an artist sound like they were singing in a different language from their own.

That resonated with American pop singer Lauv, who at the time had just finished a Korean tour and wanted to find a meaningful way to thank his fans. Why not develop a new version of his song Love U Like That, transformed to make it sound like he wasperforming in Korean?

The process began with Young's company tapping Kevin Woo, a South Korean singer-songwriter who is fluent in English and Korean, to rewrite and sing the song in a new tongue. That interpretation has to capture the nuances of the Korean language, explained Young.

"It's not just, throw it into a translator website and whatever it spits out, that's your song. You have to be artistic about it," Youngsaid.

The recordings also had to match musically. For example,if Lauv sang in falsetto on a few bars, Woo did the same in his Korean-language demo.

Enter artificial intelligence.

The technology is trained on "every corner" of Lauv's voice, as Young puts it from the highest frequencies and tones to the lowest to build a comprehensive model of his vocals.

The Korean-language demo written and sung by Woo is then combined with the AI model of Lauv's voice, creating a Korean-language song that sounds like it's sung by Lauv.

It's been a year of reckoning for creative industries wrestling with artificial intelligence.

The music business in particular has seen an explosion of AI-generated songs made to sound like the world's most popular artistssuch as the viral trackHeart On My Sleeve, which featured computer-generated vocals that sounded likeDrake and The Weeknd.

Some artists are fighting the non-sanctioned use of their voices and likeness, while others likeGrimes, John Legend and Demi Lovato are embracing the techto varying degrees.

"At minimum, we should give artists control over how their voice is used," said Young, who is from Toronto, noting that's where the ideacame from to use AI to change languages.

"It was about creating a control mechanism for artists to be able to manage their likeness in a world where AI can totally disrupt that."

Other companies are working on technology that focuses on music and translation.

AudioShake, a San Franciscocompany that specializes in audio separation, isolates a song'svocaltrackfrom the music, then uses AI to transcribe and translate the song, building technology that maps the original English with the new language so that fans can follow the lyrics of the original song in their language. Pop singer Sia is one of their clients.

Meanwhile, South Korea's largest record label HYBE used in-house AI technology so that its artist Lee Hyun, known as MIDNATT, could record his song Masquerade in six different languages: English, Korean, Spanish, Japanese, Chinese and Vietnamese.

The singer recorded his song in each language, which was then combined with demosof native speakers reading out the lyrics. Hyun speaks Korean as well aslimited English and Chinese.

"It's a scary thing that people don't fully understand, and often AI is portrayed as this monolithic thing, and it's not," said Young.

"AI is actually many different technologies that can come together in many different ways."

Artists already trying to reach international fans

Even before AI came to the forefront this year, artists hadalready been trying to use different languages to target fans, according to Kristin Robinson with Billboard Magazine.

She points to Despacito, a 2017 song by Puerto Rican singer Luis Fonsi andPuerto Rican rapperDaddy Yankee, that saw its remixed version featuring Justin Bieberbecome a huge hit.

"Having Justin Bieberhop onto a Latin track is not something I would have ever anticipated. Having Beyonc try to sing in Spanish is not something I had on my bingo card for Beyonc to do."

But with AI now factored into this already successful trend, Robinson says people will likely have mixed feelings about "the authenticity" of the process.

It remains to be seen if this type of music will be well received.Robinson saysthat using the technology in a way that is both ethically and culturally appropriate is an important part of the conversation.

That was reflected in some of the online reaction to Lauv's Korean-language version ofLove U Like That.

Some fans saidthey were "obsessed"with the new version, while others questionedthe optics of a white artist using technology to insert his voice over a song sung by a Korean artist. Still others suggesteddifferent ways to have Lauv and Woo sing on the same track.

Getting 'nuance' oflanguage important: Toronto artist

Toronto jazz vocalist and songwriter Jacqueline Teh hopes to incorporate her family's language Hokkien, a Chinese dialect from the Fujian province into her future projects. As a multilingual artist, she can see the benefits and the downsides of using AI to create songs in different languages.

She saysthe technology seemed exciting at first, adding that it can be a "great springboard" for artists to access other languages.

But she sayscredit and agency are incredibly important to artists and that she wouldn't wantAIto come betweena potential relationship with a language that's new to her, or with a person who speaks a language different from her own.

LISTEN | AI in music generates existential questions for artists:

Front Burner29:01The Beatles and the future of AI music

Before his death, John Lennon recorded a demo of a new song, "Now and Then" on a cassette. His Beatles bandmates later tried to repurpose it for release, but abandoned the project in part because of the poor voice quality.This week, Paul McCartney revealed that, 43 years after Lennon's death, the song will drop thanks to AI technology.It's just the latest example of artificial intelligence's increasing presence in the music industry. Fake Drake songs, AI-generated Kanye covers and posthumous Biggie collabs have raised alarm about copyright, and existential questions about songwriting and creativity.Today, Saroja Coelho speaks with the host of Vulture's Switched on Pop podcast, Charlie Harding, about what the technology means for the music industry and art itself.For transcripts of this series, please visit: https://www.cbc.ca/radio/frontburner/transcripts

"I think with language there's so much nuance and there's so much slang and there's so much that can't really be translated just through technology," she said, which is why working with a native speaker makes the song more meaningful to her.

She saysas long as artists aren't using AI to bypass the hard work that goes intocreating meaningful relationships with fans and other artists, "there's so much that is accessible to us with technology."

Young, the producer,is working on a new translation project potentially with a Canadian artist. Hesays now that the genie is out of the bottle, the industry has to figure out how to steer the tech in the right direction.

"I'm never going to build something that removes the human being from the creative process," he said. "And there are lots of companies that are attempting to do that right now."