How to Make Your Hologram Speak with Voice Cloning Technology

Realistic holograms in sci-fi movies have impressed audiences for decades. For a long time, holograms have been popular in hit films such as Star Wars and others, where holography was used as a form of direct communication across great distances. Despite the fact that 45 years have passed since the release of the film, holograms in films still look impressive and futuristic. Today however, along with the development of modern technologies, holography accompanied by audio is becoming a popular tool in many industries, including synthetic media.

What is a Hologram?

A hologram is a product of holography, a three-dimensional image created by a laser that reproduces an image of a three-dimensional object. Varying forms of holography present promise for the future of visual entertainment as it has been the most effective way to render 3D scenes to date.

Dennis Gabor invented the method for holography in 1947 and received the Nobel Prize in Physics in 1971 for his achievement. The method results in the appearance of an object which takes up a three-dimensional space. It can walk around, be viewed from all angles, and take up a depth that no other 3D display technology is able to match.

Modern projector models are capable of creating an array of 3D effects. The capabilities of holographic projectors are constantly expanding with the development of modern technologies, and the quality of images is improving. Over time, the technology has gotten smaller and more affordable.

Holograms in the Modern World

In the modern world, optical holography is used across a variety of industries. In medicine, holography creates 3D visualization of organs. At present, numerous projects use optical holography to render objects in 3D. In addition, scientists are actively developing holograms that can not only be viewed from all sides but can also be touched and appear as if they are completely three-dimensional.

The technology for creating holograms is based on the use of a polystyrene bead that is two millimeters wide. This bead is able to levitate through the use of multiple ultrasonic transducers that generate sound waves. Such ultrasonic transducers are capable of lifting a polystyrene ball at a speed of up to 30 km per hour. At this speed, the ball can outline a shape up to ten centimeters wide in less than a tenth of a second.

An unusual approach to creating this kind of illusion was chosen because the movement of a hyperactive ball does not initially betray its appearance as a hologram. But, thanks to this speed, scientists are able to create many holographic forms. Since the roller creates a completely three-dimensional shape, a person is able to view the hologram from all sides and from any angle, which, at the same time, does not compromise their detailed outlines or shapes.

Despite such an innovative approach when creating a new technology, scientists also had to consider the sounds the image should make. Possessing the ability to generate sound waves that are easily perceived by the human ear will allow you to communicate directly with the hologram.

By providing the opportunity to see and hear the hologram, a person will be able to touch it and receive tactile feedback when doing so. Despite scientists claiming that touching a hologram will not represent accurate textured sensations, a hologram will be able to let you know what it is. For example, you will be able to easily distinguish between a living hologram of a butterfly and a non-living one.

Today, some of the most popular use cases for holograms are:

• Telecommunications.
• Education
• Spatial navigation
• Marketing and direct sales
• Music shows
• The reanimation of iconic individuals

Learn more about these use cases here.

Examples of Audio Holograms

As you can see, audio plays a huge role in creating realistic holograms. AI voices have already been implemented across a number of industries to create different types of holograms, from “animating” digital avatars to reviving iconic stars and historical figures for shows and presentations. This integration of voice AI technology demonstrates the potential of AI generated content in holographic applications.

In 2017, a gigantic hologram of Eric Prydz's face was the closing act of his EPIC 5.0 show in London. In the evening, over 300 lasers formed a volumetric hologram of his head. Ever since its airing, the hologram of the DJ’s face was a constant companion in his shows.

In 2012, Tupac Shakur was “revived” by Digital Domain studio as a 3D hologram. Using an actor and body double, the studio created animations for a lifelike digital avatar of Tupac. In 2014, Tupac also appeared at Coachella in his digital human form. 

Roy Orbison passed away back in 1988, but this musician went on tour last winter in North America. Other artists who are no longer with us still “perform” live, these include Whitney Houston, Ronnie James Dio, and The Notorious B.I.G.

Holograms with voices and sounds are critical for educational projects such as creating a virtual history museum. In these museums, you can see a 3D model of a historical figure telling their story and educating people of different ages about historical events of their time.

Politicians are also interested in this technology - holography can become a convenient and safe way to run campaigns. And, what is especially crucial - holograms can speak to their audiences in any language. Today, studios can create a hologram of Joe Biden speaking to the people of Japan in fluent Japanese, leveraging the capabilities of voice AI.

How the Voice for Audio Holograms is Created

When creating the voice for an audio hologram, a voice cloning software first takes samples of the speech of the person whose hologram needs to speak. 

Usually, about an hour of their speech should be recorded, and 10-15 minutes of that recording is then used for the cloning process, leveraging the use of advanced voice cloning software. Even in cases when the client doesn’t have high-res sources available, Respeecher delivers the highest resolution audio across the board. Download this whitepaper about Respeecher's audio super-resolution algorithm to find out more. 

After this, the recordings are loaded into a neural network, which generates a voice that considers all the possible nuances of the speaker’s voice. The result is a AI-generated voice that is almost indistinguishable from the original. 

In order to make a hologram of a living star, politician, etc., speak different languages, a recording of their speech and a volunteer who will read the necessary text in the necessary language are required. After the recordings are made, the process is the same as described above.

Of course, creating 3D models, animating movements, and synthesizing authentic voices for holograms is no simple task. Respeecher can help creators reduce the costs and time associated with reproducing an authentic voice. We can not only reproduce your favorite musician’s voice but also help you create new content using that singer’s voice. Contact us to learn more about how synthetic media can enhance your projects!