A light show ended in a cascade of failures when fifty drones collided mid-flight. Authorities and technicians suspect that the smoke generated by a pyrotechnic effect saturated the proximity sensors. To confirm this, a forensic reconstruction has been initiated, combining aerial photogrammetry and real-time simulation, seeking to replicate the occlusion error within a digital twin of the event.
Trajectory reconstruction with RealityCapture and Pix4D 🚁
The first step was to capture the show space using multiple aerial and ground shots. RealityCapture processed the images to generate a dense point cloud of the environment, including the exact position of each drone at the instant before impact. In parallel, Pix4D was used to calculate individual flight vectors from telemetry data, correcting drifts and aligning the routes with the 3D model. This process allowed identifying the areas with the highest density of smoke particles, where the occlusion sensors failed to detect the proximity of neighboring aircraft. The fusion of both programs produced a heat map of potential collisions, pinpointing the exact locations where the chain of errors propagated.
Simulation in Unreal Engine 5: smoke as a critical variable 🎮
With the digital twin ready, the reconstructed trajectories were imported into Unreal Engine 5. There, a simulation was run replicating the lighting and particle conditions of the original show. When activating the volumetric smoke effect, the virtual sensors of the drones showed a loss of precision in detecting nearby objects, validating the occlusion error hypothesis. The visualization of flight vectors and impact points in real-time confirmed that the saturation of the LiDAR sensor by suspended particles was the root cause of the chain reaction, offering a technical lesson for future swarm designs in environments with physical interference.
How to correctly model smoke propagation and sensor occlusion in a 3D digital twin to predict chain collisions during a drone show under reduced visibility conditions?
(PS: Simulating trajectories is like playing billiards, but without having to clean the table afterwards.)