Capturing Real-World Environments with Gaussian Splats
Gaussian splats have revolutionized real-time and offline rendering workflows, allowing for photo-realistic environments with flexible camera movement and interactive lighting. This technique enables high-quality scene reconstruction for vehicle configurators, cinematic renders, and virtual production, providing a cost-effective and scalable alternative to traditional CGI environments.
This page outlines my process for capturing and processing environments using Gaussian splats, breaking down site selection, scanning methodology, data processing, and integration into real-time or offline workflows.
Applications of Gaussian Splat Environments
1. Real-Time Vehicle Configurators
Gaussian splats provide lightweight, high-fidelity environments for interactive product configurators, such as those built in Unreal Engine. The goal is to capture environments that work well within an orbit-style camera, ensuring the background holds up from multiple angles while allowing users to move inside and around the vehicle seamlessly.
2. Virtual Cinematic Shoots
Gaussian splats also serve as a high-quality alternative to traditional location shoots, allowing for unrestricted virtual camera movement. Instead of physically rigging a chase car or setting up cranes, a pre-captured splat of the environment enables dynamic, realistic shots in post-production, providing full control over composition and camera angles without needing reshoots.
Step-by-Step Process
Location Selection
Choosing the right environment is critical to the final output. The location must:
Provide strong depth cues for the reconstruction process (e.g., environments with detailed textures and distinct objects work better than flat surfaces).
Match the brand identity of the vehicle (e.g., a luxury car should sit in a high-end, visually appealing setting).
Offer good lighting conditions to ensure accurate texture capture.
Environment Capture
Environment Capture
Once a location is selected, the next step is to scan the scene using video footage. The goal is to cover every possible angle the final render might require, ensuring complete 360-degree coverage.
Keep movements smooth and sequential, as small frame-to-frame differences help with alignment in later processing.
Walk through the space, capturing low, high, wide, and close-up shots.
Focus on areas where the camera is most likely to move, ensuring sufficient coverage for orbit cams or tracking shots.
Processing & Camera Alignment
After capturing the raw footage, the next stage is data processing and alignment:
- Convert video footage into image sequences, ensuring each frame is sequential with minimal variation.
- Import into camera alignment software, which calculates real-world camera positions and reconstructs the scene.
- Generate a sparse point cloud, serving as a rough preview of the environment before final reconstruction.
At this stage, the camera alignment is crucial—poor alignment leads to blurry or misaligned Gaussian splats.
Gaussian Splat Generation
Gaussian Splat Generation
Using the aligned data, we generate a Gaussian splat representation of the environment. This step involves:
Exporting the final Gaussian splat for use in Unreal Engine, offline rendering pipelines, or other visualization tools.
Feeding the sparse point cloud and aligned cameras into Gaussian splat processing software.
Refining the density and ensuring proper occlusion handling, so the environment appears natural from all angles.
Final Output & Use Cases
Once the Gaussian splat is generated, it can be deployed in different workflows:
Hybrid Workflows – Combined with traditional CGI elements or composited with live-action footage for seamless integration.
Unreal Engine / Real-Time Applications – Optimized for use in real-time vehicle configurators and interactive showrooms.
Cinematic Rendering / VFX – Used in virtual productions, allowing for fully adjustable camera movement within a captured real-world space.
Why This Matters
Gaussian splats represent a powerful new tool in rendering workflows, providing:
- Photo-realistic scene reconstruction with minimal computational overhead.
- Flexibility in camera movement, making them ideal for both real-time applications and pre-rendered sequences.
- Scalability, allowing multiple environments to be captured and used without requiring extensive manual cleanup.
This process is a game-changer for vehicle visualization, offering high-quality results at a fraction of the cost of traditional CG environments.