Dead musicians are touring again, as holograms. It's tricky — technologically and legally

Dead musicians are touring again, as holograms. It's tricky — technologically and legally

In 2019, Amy Winehouse will tour the world. Sort of.

Key points: The technology used to bring dead singers back to life is getting better

It is a burgeoning industry, both competitive and litigious

Holograms of Tupac, Michael Jackson and others have performed

The much-loved British singer, who struggled for years with drug and alcohol addiction while producing hits like Rehab and Back To Black, died in 2011.

But a hologram version of Winehouse will tour internationally next year, her father announced recently, moving about the stage backed by a live band in a show that could last almost two hours.

It is not the first time a dead musician has been resurrected on-stage via the magic on technology. The late Roy Orbison played recently, Michael Jackson performed at the 2014 Billboard Music Awards and Tupac appeared at Coachella in 2012, more than a decade after his death.

The technology behind this post-human age of live entertainment is reaching a tipping point, with several companies clamouring — sometimes by way of the courts — to create a hologram performance that can be as engaging as a human one.

In the process, they hope to unlock vast amounts of money in the back catalogues of the 20th Century's biggest artists.

The technology is cutting edge, but based on old-fashioned theory

The current crop of productions are 2D video projections, rather than proper holograms, and they are pre-recorded, not live. That's according to Mike Seymour, a digital human researcher at the University of Sydney who has worked in visual effects in London and Hollywood.

While the imagery is photorealistic, using complex neural networks to build a reconstruction of a famous face, the method used to project that image onto a stage is not new.

"Hologram USA uses a high-tech, [high-definition] version of the 19th Century Pepper's Ghost technique," says billionaire Alki David, whose company Hologram USA owns the rights to create holograms of Billie Holiday, Jackie Wilson and others.

"It was the original 'smoke and mirrors' way to put a ghost on a stage."

Most productions start by filming a performance, usually by an impersonator, to nail down the dance moves and the general physicality of the celebrity, Mr Seymour says.

"So, you could obviously film someone dancing like Michael Jackson and then replace [the face] with a digital version of Michael Jackson's face, reconstructed from a tonne of images of Michael Jackson," Mr Seymour said.

That face you create will initially be inert. The next thing you need to do is "rig" it.

"That rigging phase is the second stage, and that you can think of it as producing a bunch of digital leavers or puppet strings that would allow you to pull an expression on the face — or for that matter the body — of a digital character that you had."

After that, you have got to "drive the puppet". This can be done by using cameras attached to an actor's head to capture their facial movements.

"So, if I've got that head rig on, if I say the word hello, it reads my lip movements, from the cameras that are on my head rig, and then it pulls the digital strings" to make the holographic singer say hello.

From there, the video will be rendered and the light will be fixed to make shadows fall where they should.

"Live performers on stage with a hologram can see reflections the audience can't, and adjust their movements to fit," Mr David said of his company's approach.

"But with our advancements, and powerful projectors, we're able to present holograms that are opaque, so that the problem of 'show through' is minimised."

Eyellusion, another US company with upcoming shows featuring a holographic Frank Zappa, uses a recording of the artist from a real live performance as the vocal track for the show.

As for interaction with the audience, "you can do banter in advance," the firm's CEO and founder, Jeff Pezutti, said.

"There is future technology where [live banter] can happen, and will happen, but for now it's a pre-produced show, so you've just got to be smart about where you put it so it feels seamless."

This is a competitive — and litigious — space

The Amy Winehouse tour is being put together by Base Hologram, which was also behind the Orbison show in Los Angeles in October.

Hologram tours are a potentially lucrative field for companies like Base, as well as its major competitors, and record labels.

This is because it allows cash to be squeezed out of heritage acts like Elvis or Holiday in an era when the value of recorded music has been eroded by the shift to streaming platforms.

In the commercial parlance of Base, which did not respond to a request for comment, it is "creating multiple revenue verticals to deliver value".

That has meant the technological race is tight, and the field sometimes brutal.

Hologram USA sued Pulse Evolution, created from the remnants of an earlier special effects firm partly founded by director James Cameron, for patent violation in 2014.

"The minute we started doing this there were wannabes and patent thieves trying to grab a piece of what The Hollywood Reporter called the next billion-dollar business," Mr David, whose family made its fortune bottling Coca-Cola, told the ABC.

As Mr Pezutti said: "Everyone realises this is the future. Everyone is trying to get out in front."

Last year, Hologram USA had a legal disagreement with the estate of Whitney Houston, with whom it was negotiating on a hologram performance. Mr David said the two sides were in talks and he hoped there would be an announcement in the New Year.

The technology is one hurdle, but the law is another

In order to put one of these performances on, a hologram tour promoter has to pay to use the entertainer's music.

In addition to that, in the US — where Orbison, Elvis and Holiday lived and died — there is also a thing called right to publicity, which gives someone an exclusive right to profit off their likeness.

But whether that right extends beyond death, via the person's family or estate, differs from state to state.

For example, in California, the right to publicity extends 50 years after death. In New York, that right ends at death.

Neither Australia nor the UK, were Winehouse lived and died, have specific protections for right to publicity, though both have laws against "passing off", or making it seem like someone endorses something they don't.

Once you've got all that down, will people actually buy tickets?

The original Tupac performance at Coachella created a lot of buzz. The Roy Orbison performance was positively reviewed earlier this year by the Los Angeles Times and received praise from the singer's children.

"People watched the first minute-and-a-half, and before you knew it, they had their devil horns up," Mr Pezutti said of the debut performance by a holographic Ronnie James Dio, the former Black Sabbath singer.

"When they sang back to the hologram, that's when I knew that we had literally captured something. You forget you are looking at a hologram. Now you are at a rock show."

For Mr Seymour, the success of a performance will come down to an individual audience member's willingness to buy in.

"If you are appalled by [the idea], because you think it's an atrocity to the original act, you are going to hate it," Mr Seymour said.

"And if you are a fan that just loves seeing that song being performed again, you are going to think it's the best thing ever."

Mr Seymour said the industry could have a "major impact" — if it can harness new technology, which he is seeing coming out of private labs and academia, to create interactive, real-time performers — ones that feel as genuine as they appear.

Topics: music, arts-and-entertainment, music-industry, computers-and-technology, united-states