The Trouble With 3D
It’s surely self-evident that headaches and nausea shouldn’t be in a trade-off with entertainment. Yet, with stereoscopic 3D – which is the technology underpinning the primary forms of 3D currently on the market, both in cinemas and at home – these problems are reported by a sizable minority. It has nothing to do with wearing glasses, either – it’s a fundamental issue with the way that stereoscopic 3D works, and it affects you at a physiological level, regardless of whether you’re staring through active shutter specs, polarised lenses, or looking at the polarised screen of the 3DS.
All of these display solutions use a process which projects two images, one for each eye, in order to create an illusion of depth. The problem arises because this forces your optical systems to contradict each other. When you look at an object in the real world, there are two processes at work: first, your eyes angle inwards to fix upon the object. Pull the object close enough, and you go cross-eyed. Look at something on the horizon, and your eyes are almost parallel. This is the primary way we judge depth, and the process that stereoscopic 3D hijacks to create its illusion. But there’s another process at work, too: inside each eye, your lenses try to bring the object into focus, stretching thin to accommodate distance vision and bulging to resolve objects up close.
“What happens with stereoscopic 3D is you get provided two 2D images, one for the left eye, one for the right,” says Hagen Stolle, CTO and business director of SeeReal, a Dresden-based company which has a long history in 3D display technology. “You have what’s called parallax in the images – a subtle difference in the angle of the object presented in the two images that mimics the different angles from which your eyes would view an actual object. When combined in your brain, this gives an impression that the virtual object is somewhere behind the screen. But these are still 2D images – and they are, in reality, being depicted on the screen and that’s where your eye is focused.”
SeeReal's holographic display recreates the lightwave distribution that enters your eye from an object in realtime, using off-the-shelf graphics cards
“So your eyes get cheated into converging on the virtual object [behind the display], but your focus isn’t cheated and stays glued to the display,” continues Bo Krøll, SeeReal’s chairman. “The greater the depth of the 3D, the greater disparity between the focus and convergence of your eyes. Half your brain says the object is here, the other half says no, no, no – it’s here!”
This conflict results, at the least, in the faint awareness of the image’s artificiality, and at worst in eye fatigue, headaches and nausea. Over the long term, the effects are unknown – advocates of stereoscopic 3D insist that we will simply adapt, but it is telling that Nintendo’s 3DS comes with a warning stating that is not suitable for young children whose eyes are still developing.
Not everyone suffers, of course, but even when someone is able to view stereoscopic 3D in perfect comfort, the technology’s capabilities are still limited by the convergence-focus contradiction. In order to reduce the eye fatigue inherent to viewing stereoscopic 3D, creators implement a depth budget. This means that the full depth of a 3D scene is compressed down to a small number of inches – at this limited range of depths, the conflict between where your eyes are focused and where they are angled is less pronounced (this is also why some viewers find 3DS screens less straining when the 3D effect slider is pulled down). The result is that stereoscopic 3D often produces an image that looks like layers of 2D cutouts – the fulsome 3D of each object having been squished to a plane. And the closer the display to your face, the smaller the depth budget.
“That budget totals, on average, four per cent of the distance between the viewer and the display,” says Krøll. “In practical terms, on average, that means [stereoscopic TVs] show little more than four inches of depth.”Nintendo’s hardware, by this calculation, produces comfortable 3D at around a centimetre of depth. And this is only the main problem with stereoscopic 3D. There are other secondary issues too – already much discussed by detractors of 3D cinema. Famed cinematographer Walter Murch has described an unpleasant strobing effect related to horizontal movement and edge perception, and decries the overall dimness of the display when using polarised glasses. The entertainment world has worked hard to evangelise 3D, but the result seems to have been a heavy push for a technology which doesn’t reproduce anything approaching the depth of reality, and is in some way incompatible with our optical systems.
Technologies exist which solve the major issues inherent to the current process, but it remains to be seen if the industry can afford to adopt them now that it has expended so many resources on promoting stereoscopic 3D. SeeReal’s sudden willingness to articulate the failings of stereoscopic 3D isn’t coincidental – it has something better. Holographic 3D technology produces a true 3D image, one which your eyes focus and converge upon in unison. It offers far greater depth, creating image-objects that appear to sit before your very nose, or disappear into the horizon.
“When you look at an object in reality, you’re not actually seeing the object but the light reflected from the surface of the object,” explains Stolle. “If I can recreate that same information – that same lightwave distribution that the real environment creates in your eyes – you could not tell the difference between the real object or the holographically reconstructed object. And since I know where the display is, and I know where you are – because we have a camera tracking your eyes – I can compute the lightwave distribution in your eye. I can then have a display which modulates the light in a particular way, called an interference pattern, so that when it hits your eye it has the same information as the natural 3D scene we wish to reconstruct. It’s physics.”
Though extremely clever, this has not been the main technological hurdle. “Holography has been around for 60 or 65 years, but has always been based on high-resolution emulsion films,” says Stolle. “It requires a very high-resolution medium – on film it’s easy, but on a display, to create a decent angle, I would need pixels of less than one micron in size! Which means I’d end up with more than 250,000 times HD resolution if I wanted to follow the classic approach to holography – which is why nobody has bothered! I’d have to compute every single one of those pixels at 60 frames a second. Nobody had a solution to do this practically. But if you look at the information this [imaginary super display] is creating, a lot of it is stuff you won’t see. The only stuff you’ll see is that which goes into your eyes – and that was our starting point. We only create the information an individual will see – which we can do because we know where the eyes are. So we can use a normal LCD to create a hologram, as long as we limit the information.”
The addition of eye-tracking hardware, not dissimilar to Kinect, is the vital component. SeeReal is confident that it will be able to track multiple pairs of eyes simultaneously, projecting to each a separate virtual 3D image. In order to do this, the TV multiplexes, displaying screens for each user alternately, but at such great speed that each viewer sees an unbroken moving image. Since the human eye can detect the individual frames of an animation if there are fewer than 60 per second, a high refresh rate is needed for multiple viewers.
SeeReal is able to limit its calculations to the light which enters your eyes, discarding unseen data.
“For two viewing windows you use 120hz,” says Stolle. “For three users you would need 360. The refresh rates [on consumer TVs] are improving every month now – technology-wise you can already serve four users.”
The early prototype for this technology currently does not, however. Since the phase modulator technology underpinning it can’t be bought from a shop yet, it had to be built from repurposed medical imaging equipment. The result is that the refresh rate is low, and the screen flickers while producing a full-colour image. On this early model, headtracking doesn’t allow for considerable movement, either. Nonetheless, the presentation of 3D is genuinely startling – sufficient for us to reassess our stance on 3D entirely. We’re able to toy with a 3D model of a spaceship, using a trackball to rotate it and move it closer and farther away – we can push it far into the distance beyond the screen, and drag it out of the display until it sits inches from our face. At all points it feels like a natural, physical object that we could reach out and hold – albeit a luminescent red one. Later, we’re shown a tank full of water, its length tapering away from us, which see-saws forward and back. The illusion of a physical object is extremely convincing.
We also watch some colour animations that extend into the screen and note that we can focus on elements within the image as we please. With holography, the entire object is presented as it is in reality, and the viewer chooses which bit to focus on – look at the foreground and the background blurs as it would when you look at a real vista, and vice versa. This necessitates some changes in the language used by cinematographers, who are accustomed to drawing the viewer’s eye with a shallow depth of field – ie, by blurring out all but one element of an image.
Even among traditional film experts, holographic 3D has advocates. Felix Forrest, a steadicam operator and director of photography, is one convert. “I am a big fan of cinematography and I have an absolute fear of and hatred for stereoscopic 3D because I think it destroys a lot of the artform,” he says. “That’s the position I started from. Then I sat down at the first [SeeReal] demonstration and thought: ‘I’ve got to change the way I shoot things’. I think we’ll end up going back to some of the more traditional techniques like you would see with film noir, where you use light and shadow to draw your attention to things. It’s a bit like lighting for theatre.”
So too will cinematographers grapple with a changing sense of scale: when viewers are aware of being in a perspective relationship with a scene, it can be difficult to escape the impression of viewing a tiny diorama. And, although SeeReal’s representatives say this is far less of an issue with holographic 3D than it is with stereoscopic 3D, quick cuts are also difficult to make, as the eye will be fatigued by rapid changes in focus. “It changes the pace at which you can tell a story,” explains Forrest. “Action films become quite difficult – you can’t do a lot of handheld camera work, so you lose a lot of the frenetic pace.”
Clearly, true 3D display forces fundamental changes to the way the creative industries work, but this is not necessarily a bad thing. We watch a few stereoscopic showreels, ostensibly to prove that SeeReal’s technology is backwards compatible, but the result is to underline holographic 3D’s superiority over its stereoscopic forebear – holography feels far less like a novel addition to the existing visual language and more like a new visual medium in itself. New areas of expression have opened up.
SeeReal's prototype produces a convincing 3D effect, but is flickery and dim. "We modified an amplitude display made for medical imaging to be a phase modulator," Stolle explains
“It’s much more participatory,” says Forrest. “Because you are free to change your focal point, it allows for greater means of piquing your interest and keeping you engaged. You feel immersed. It’s the difference between theatre and TV. And you can only get that with holographic 3D.”
It’s clear that the idea of viewer participation dovetails with the ambitions of an interactive medium, and it is conceivable that films will begin to take cues from videogames, too, presenting action with an enhanced spacial awareness. There is further potential: with the ability to project things into the space close to the viewer, the tracking hardware allows for a true Minority Report-style interface. And with the facial recognition already available in technologies such as Kinect, the multiplexing used to present different images to different viewers could equally be used to present entirely different films or games to people watching the same television. Splitscreen could become a thing of the past – multiple players would see only their own screen.
So, is this a case of ‘3D is dead, long live 3D’? Holographic 3D’s improvement over stereoscopy is palpable, and its potential to alter the way we present video media clearly profound. But it’s hard to say how quickly and inexpensively the adoption of holographic 3D will occur – such significant investment has already gone into stereoscopic 3D that it is difficult to see the heavyweights behind it being willing to change course suddenly. Nor is it clear how consumers will react to the prospect of a secondary technology upgrade so soon after the arrival of home 3D.
Although SeeReal has no great say in the price point of TVs using its technology, it isn’t in the business of designing toasters, after all; this will be an expensive piece of kit – at least at first. With stereoscopic TV prices now coming down, consumers could be caught in between the obsolete and the unaffordable. It’s hard to imagine holographic 3D, with its headtracking requirements, being able to scale for cinema viewing either. But though many such questions hang over the future of 3D, one thing has become very apparent on the basis of SeeReal’s demonstration: stereoscopic 3D could soon be a part of its past.