Hollywood has been quick to exploit facial motion-capture technology, with the first films making use of the technique due in cinemas this year, but it will be a little while longer before gamers see the fruits of these latest mo-cap specialists’ labors in interactive form.

“Our feature film clients have been using Contour to create CG faces that blend seamlessly into live action, and the next obvious step was to see if we could make it work in a videogame,” explains Mova’s founder and president, Steve Perlman, who developed QuickTime technology and WebTV.

Mova had a working demo of the new capture technology – a photorealistic talking head running on Epic Games’ Unreal Engine 3 – at GDC in February. With Contour, Mova took an entirely different approach to facial motion capture. The traditional way to do that has been to glue maybe a hundred markers to the actor’s face, paint dots on the face, or track gross motion of visible facial features. What you end up with is around 100 motion-tracking points on the face, rarely at ideal locations, to try to pick up all the subtleties of emotion on the human face. Critical motion that falls between these tracked points is lost, so it must be approximated and/or animated by hand, adding cost and limiting realism.

“With Contour, we use a special phosphorescent makeup that covers the entire surface of the skin, which allows Contour to capture about a million points, and then after the capture, the production team can retrospectively direct Contour to track the critical motion points of each particular face, whether that motion is visible, such as at the edge of the lips, or not,” explains Perlman.

“Then, the production team decides on what resolution is appropriate for the production medium, be it 2,500 points for a feature film close-up or 255 points for a game close-up (such as in Mova’s GDC08 demo), or 20 points for a face far in the background. Also, eyes, teeth and tongue are captured photographically. So, regardless of the medium or the shot, every nuance and every wrinkle of the performance is available, and the face can be rendered optimally.”

There are several significant advantages to using Contour for facial mo-cap, according to Perlman. For actors, the makeup is comfortable, they are not tied down, and their performance is captured exactly. For the production team, Contour consistently delivers photoreal results with a minimum amount of manual labor, is fast and inexpensive, and a single capture session can be used for movies, games, close-ups, long shots, etc. Also, Contour captures can be easily retargeted so that an actor’s performance can drive a different 3D face, be it a photoreal human face or a creature. And Contour captures more than faces: necks, shoulders, torsos, cloth, hands, etc, can all be captured and tracked in 3D precisely as the surface moves.

Anyone who’s played recent games knows of the problems that the Uncanny Valley – something that still plagues CGI features like Beowulf – can cause. Perlman explains that, within the game space, this problem occurs largely because developers start from very approximate motion capture and require hand animation to try to overcome the imperfections. Since Contour captures start out with photographic-resolution captures, at full resolution the faces are indistinguishable from live action, putting them beyond the Uncanny Valley, and then the production team selectively lowers the resolution as needed, given the limitations of the game platform.

“So while current methods struggle to approach photorealism at all, Contour allows you to start at photoreal, and then decide how much to dial it down, and with full creative control,” adds Perlman. “Contour allows games to leap over the Uncanny Valley and bring to life CG characters that people can finally empathies with. This will also push the convergence of film and games, allowing games to move toward a more narrative experience with more complex dialogues and character interactions.

While Contour-based games look really cool on current game hardware, they are running Contour at far below the resolution that is available from the capture. Also, once the facial shape and motion are perfect, other details of facial rendering, like skin surfaces or eye reflections, become far more noticeable if they are imperfect. We’ve been told that Contour capture data is really underscoring the limitations of current game hardware. Tests we’ve been doing with non-realtime renders simulating new systems really whet the appetite of developers.”

So look for a new wave of games and films to feature this cost-effective new technology, which motion-capture technicians and developers have already embraced with open arms.

Enjoyed the article? Why not subscribe to Edge?