News

Abstract

We present an integrated, end-to-end system that enables photorealistic real-world objects reconstructed using 3D Gaussian Splatting (3DGS), to interact seamlessly with synthetic elements such as polygonal meshes, fluids, fabrics, and robotic systems within a unified simulation environment. By leveraging Material Point Method (MPM) simulation, our system ensures the compatibility of the 3DGS representation with established physics engines while remaining extensible.

Our workflow begins by capturing real-world scenes "in the wild" using 3DGS, from which we derive a simplified, appearance-agnostic particle proxy suitable for physics simulation. These particles, along with synthetic primitives, are imported into the system, where the simulator computes positions and deformation gradients for all bodies, including 3DGS-derived particles, at each timestep.

We validate our system through collision and deformation scenarios, and showcase a robotics application in which a manipulator plans and executes tasks involving both captured objects and synthetic elements. By selecting the most appropriate solver and constitutive model for each material, such as MPM for granular media and deformables, PBD for cloth, or SPH for fluids, our approach delivers: (i) high visual fidelity, (ii) accurate, material-specific physical behavior, and (iii) minimal performance overhead.

Our pipeline streamlines scene preparation, offering a significant advantage over traditional mesh-centric photogrammetry for time-sensitive reconstruction and emergency scenarios.

This combination of flexibility and realism makes our system well-suited for robot task planning, photorealistic multiview dataset generation for autonomous navigation, and other embodied AI applications.

Resources

Poster

If the poster does not render in your browser, open it directly here.

Results

PhysPLAT Voiceover

Mesh-3DGS interaction

Fabric-3DGS interaction

Robot arm - 3DGS simulation

Environment simulation

Liquid-3DGS interaction

Bibtex

@InProceedings{Alfonso-Arsuaga_2026_WACV,
    author    = {Alfonso-Arsuaga, Mario and Dominguez-Elvira, Henar and Casas-Guerrero, Jorge and Castiella-Aguirrezabala, Andrea and Dom{\'\i}nguez, Lorenzo Cost\'abile and Garc{\'\i}a-Gonz\'alez, Jorge and Naranjo-Almeida, Maria and Comino-Trinidad, Marc and Lopez-Moreno, Jorge},
    title     = {PHYSPLAT: a Framework for Photorealistic Hybrid Simulation of Real and Synthetic Elements using 3D Gaussian Splatting},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {March},
    year      = {2026},
    pages     = {1621-1631}
}