Game modder nikich340 recently released a The Witcher 3 PC mod called A Night to Remember. The mod takes off after events from the Blood and Wine expansion, with protagonist Geralt of Rivia once more taking up the hunt for Orianna.
Fans of the game were stoked to have a new, albeit unofficial addition to The Witcher 3 to play. Voice actors and some other observers, however, were less than thrilled. You see, A Night to Remember not only features new content, it features new voice lines. Specifically, the modder used AI trained on voice actor Doug Cockleâs speech to generate brand new voice lines for Geralt, the character he portrays.
âIf this is true, this is just heartbreaking,â tweeted Jay Britton, a voice actor with credits in Divinity: Original Sin 2 and Pathfinder. âYes, AI might be able to replace things but should it? We literally get to decide. Replacing actors with AI is not only a legal minefield but an utterly soulless choice.â
The Witcher 3 mod controversy came on the heels of video game developer Obsidian releasing a video about its work with Sonantic, which involves using AI voices as placeholders before adding voice actors into the game. The video explains that Obsidian uses Sonanticâs AI voices because the developer finds it helpful to listen back to its dialog to find out what works and what doesnât.
Naturally, these sorts of advances make video game voice actors nervous. âThereâs the kneejerk concern of âItâs going to take our jobs,â and I do believe, in some cases, that will happen,â Natalie Winter, a voice actor in games such as Assassinâs Creed: Valhalla, tells me via DM. With ever-improving technology and affordable internet, thereâs been a boom in voice acting that can be done from home, and, as Winter tells me, competition is pretty fierce. âIt is sad to think that if AI voices become good enough to be widely used,â she says, âthen those opportunities will decrease again.â
The pseudonymous modder behind the A Night to Remember project used software called CyberVoice to create new dialogue and voice lines for Geralt. CyberVoice is the product of the Russia-based Mind Simulation Lab, which is also behind CyberMind, which uses AI to form digital personalities for non-playable characters (NPCs) that gamers interact with. CyberMind provides the information used â for example, Geraltâs knowledge of the many monsters he encounters in The Witcher series â and CyberVoice gives that understanding a voice.
The CEO of Mind Simulation Lab, Leonid Derikyants, explains to me via email why such technology is needed in gaming. âWe create digital versions of [actorsâ] voices so that live NPCs can answer questions from players, outside of story missions, with the same voice,â he says. âAnd since they form their answers independently and remember new facts, itâs impossible to voice it all in advance. It would be strange if, in that case, they spoke with a different voice. That is why we need the advanced technology of voice parodying.â
Though Mind Simulation Lab has worked with voice actors, Geraltâs voice was created through the use of free audio tracks meshed with another voice. As Derikyants explains, Mind Simulation Lab carries out âsound engineering workâ that helps to manually change the voice so that itâs similar to the original. Then the company trains its speech synthesis on the audio. Derikyants says âparodying,â but this is more like parroting â the quality is that good.
This brings up a whole host of issues. Whatâs to stop anyone â be it a solo developer or a triple-A game studio â from using the voice of someone to express something, say, racist or homophobic without their consent?
âIf our actors are not comfortable with something, then itâs a no-go. We take misuse very seriously.â
Though Sonantic CEO Zeena Qureshi doesnât agree with the use of voice AI for A Night to Remember DLC, she says that, at least with her company, offensive speech shouldnât be a concern. Qureshi says that when Sonantic models the voices of their actors, the company makes sure that actors agree with the content their voice will be used for throughout the whole process.
Qureshi points to the companyâs âdisclosure system,â which lets Sonantic run content past actors before they accept anything. âIf our actors are not comfortable with something, then itâs a no-go,â she says. âWe take misuse very seriously.â
I reached out to a spokesperson for actorsâ union SAG-AFTRA about how voice AI could potentially affect an actor in regard to credit. If misuse did indeed occur and the actor left the project, would that seriously affect crediting rights? Their response, via email: This would ultimately be up to the actor, as currently âvideo game companies do not have a right to continue using the performerâs voice to create something new without their permission.
âIf the performer chooses to allow for this, we would negotiate for fair compensation,â the rep continues, adding that the union wants âto ensure there are protections in place that allow visibility into how the AI voice is used, for how long, that the data is protected, and that [the actors] are aligned with projects and companies that they choose.â
Mind Simulation Lab also says that it also pays special attention to the ethics of AI. According to Derikyants, the parody voice of Geralt is not available for public use. However, the modder of A Night to Remember, nikich340, tells me that theyâd asked for specific voice lines from the lab, which Mind Simulation Lab then provided â a turn of events that would seem to undercut Derikyantsâ argument.
âUnfortunately, now no one can prohibit synthesizing any voice.â
The Cockle parody audio, according to Mind Simulation Lab, will not be accessible for commercial purposes unless the actor joins the CyberVoice platform. If Cockle (who declined to comment for this piece) did take issue with the audio being used, however, Derikyants informed me that, in reality, thereâs nothing to be done, seeing as the audio is not his actual voice âbut simply similar.â Derikyants concedes that it is an issue: âUnfortunately, now no one can prohibit synthesizing any voice.â
Thomas Mitchells, a voice actor and voice director currently working on Baldurâs Gate 3, is wary of the new technology. âCompanies like Sonantic offer âprotection,â but with time people will be able to get their hands on this type of kit,â Mitchells says. Heâs right: There are a variety of voice-cloning systems being shared on GitHub at this very moment.
While Mitchells says heâs supportive of indie studios using voice AI for one-liners or call-outs (âGood job!â), thereâs something very different about using voice AI for characters in an immersive world. âNo AI is perfect,â he says. âYou donât get spontaneity, you donât get a personâs personal experience, you donât get that essence of humanity within the lines. You get a by-product of someone tweaking a bunch of knobs and dials in a plug-in to make delivery sound as plausible as possible.â
Mitchells cites the story of Sir Christopher Lee and his correction to director Peter Jackson on the set of Lord of the Rings about what sound a person makes when being stabbed in the back. In this situation, Leeâs experience during World War II brought something to the character that no AI could.
âIf your AI voice doesnât breathe, itâs never going to carry the emotional weight that a humanâs performance can.â
Winter, meanwhile, stresses the importance of the act of breathing in voice acting. âBreath is so key to expressing ourselves, especially through voice,â she says. âIf your AI voice doesnât breathe, itâs never going to carry the emotional weight that a humanâs performance can.â
Ultimately, for Mitchells and other voice actors, itâs the diminishment of their craft that feels unforgivable. âActors love to act,â he says. âThat's why they sacrifice so much to do it as a job. It is creatively fulfilling and when a character ends up with a fanbase behind it, itâs the most rewarding experience.
âNow, imagine becoming a character loved by many but you didn't do a single thing to contribute towards that role,â Mitchells adds. âZero creativity from the actor. Zero fulfillment. Zero art.â
UniverseBear on June 27th, 2021 at 15:59 UTC »
Welcome to how musicians felt with the advent of pre-recorded music.
Back in the 20s if your bar wanted music? Gotta hire musicians.
Your movie needs a soundtrack? Every movie theater had to pay for its own pit band.
Sc2SuperJack on June 27th, 2021 at 11:29 UTC »
I understand the complaints. But I've seen so many games cut the amount of dialogue just to be able to have voiced NPC's. Compare it to when dialogue was text based, part of me is looking forward to AI voices because we can once again expand on the dialogue. It will also help smaller studios that don't have the budget for voice actors to be able to have voice dialogue in their game.
JoeRig on June 27th, 2021 at 07:48 UTC »
I think this might go down the same path as music industry; where you get part of the revenue when your song is used.