NCSOFT has unveiled Project M, a proposal that blurs the line between video games and interactive films. The project uses Unreal Engine 5 as its foundation, but its true secret lies in the facial capture and digitization pipeline. Far from simple textures, the team has implemented full 3D facial scanning to capture every pore and micro-expression of the real actor, transferring that information directly to the MetaHuman Creator system.
Technical Pipeline: From Physical Scanning to Real-Time Animation 🎬
The workflow begins with volumetric capture of the actor's face using high-density photogrammetry. This base mesh is imported into MetaHuman Creator, where the necessary blendshapes for facial animation are generated. Unlike traditional methods that require manual sculpting, NCSOFT uses state-of-the-art motion capture tools that record the actor's live performance data. This data is mapped directly onto the metahumans in UE5, allowing expressions to be reproduced in real-time without losing the subtlety of micro-gestures. Optimization focuses on maintaining cinematic fidelity without sacrificing interactive fluidity, a technical challenge that few studios have managed to solve.
The New Standard for Digital Humanoids 🤖
Project M marks a before and after in the niche of digital humanoids. While cinematic projects like The Mandalorian use virtual sets with real actors, NCSOFT bets on a complete digital actor that must function in an interactive environment. The key difference is the elimination of the uncanny valley: by integrating real facial scanning with the flexibility of MetaHuman, the character not only looks real but behaves like a living human being. This approach surpasses titles like Hellblade 2 or The Matrix Awakens by focusing exclusively on facial expressiveness as the narrative engine.
Considering the facial hyper-realism of Project M with MetaHuman and UE5, how is the challenge of the uncanny valley addressed so that the audience does not question whether they are seeing a real human or a digital one?
(PS: Digital humanoids have the advantage that they never complain about the rigging.)