Hui Wang

As hardware and information technology continually advance, virtual reality (VR) has permeated numerous sectors, with applications becoming increasingly sophisticated. The evolution of VR systems has expanded from the seminal 3I characteristics—immersion, interaction, and imagination—to encompass 6I, incorporating intelligentization, interconnection, and iteration. The intelligentization of VR technology, an inevitable progression, has garnered heightened interest, particularly fueled by the emergence of artificial intelligence (AI) models and techniques like neural radiance fields, 3D Gaussian Splatting, neural rendering, generative adversarial networks, diffusion models, and large language models, which significantly propel the development of VR’s core and pivotal technologies. This survey offers a comprehensive assessment of these pivotal VR technologies, harnessing the latest AI advancements, aiming to provide fresh perspectives and assist new researchers in staying abreast of groundbreaking work. We commence by detailing the acquisition process of reviewed papers, outlining our taxonomy grounded in VR’s core elements and pivotal technological trajectories, and statistically analyzing the works within. Subsequently, we delve into the application of AI models, methodologies, and techniques across six research avenues: advanced AI-generated content representation, content rendering, content generation, physical simulation, virtual characters, and interaction, discussing their achievements. Concludingly, we summarize our findings, highlight existing challenges, and suggest potential avenues for future research.

Physically-based fluid animation in Virtual Reality (VR) significantly enhances the user experience through visually engaging flow motions. Nonetheless, such simulations are often limited by their substantial computational demands. A tailored adaptive simulation algorithm is important for high-performance VR fluid simulations, which dynamically allocate degrees of freedom (DoF) while accounting for user perception in VR. This paper proposes a novel scene-based gaze-contingent fluid simulation system for VR, featuring a highly adaptive fluid simulator integrated with a VR perceptual model that accounts for the foveation and geometry of fluid. Our method leverages an eccentricity and curvature-dependent perceptual model to dynamically allocate computational resources, improving the efficiency and maintaining spatio-temporal stability of fluid animation in VR. A user study was conducted to measure the simulation resolution thresholds for fluid animations in VR, considering various levels of eccentricity and curvature. Our findings indicate notable differences in perceptual thresholds based on these metrics. By incorporating these insights into our adaptive fluid simulator as a unified sizing function, we maintain perceptually optimal particle resolution, achieving up to a 3.62x performance improvement while delivering superior perceptual realism and user experience, as validated by a subjective evaluation study.

We present a novel particle-grid scheme for simulating bubble and foam flow. At the core of our approach lies a particle representation that combines the computational nature of moving least-squares particles and particle level-set methods. Specifically, we assign a dedicated particle system to each individual bubble, enabling accurate tracking of its interface evolution and topological changes in a foaming fluid system. The particles within each bubble’s particle system serve dual purposes. Firstly, they function as a surface discretization, allowing for the solution of surfactant flow physics on the bubble’s membrane. Additionally, these particles act as interface trackers, facilitating the evolution of the bubble’s shape and topology within the multiphase fluid domain. The combination of particle systems from all bubbles contributes to the generation of an unsigned level-set field, further enhancing the simulation of coupled multiphase flow dynamics. By seamlessly integrating our particle representation into a multiphase, volumetric flow solver, our method enables the simulation of a broad range of intricate bubble and foam phenomena. These phenomena exhibit highly dynamic and complex structural evolution, as well as interfacial flow details.

This paper presents a two-way coupling approach to simulate bouncing droplet phenomena by incorporating the lubricated thin aerodynamic gap between fluid volumes. At the heart of our framework lies a cut-cell representation of the thin air film between colliding liquid fluid volumes. The air pressures within the thin film, modeled using a reduced fluid model based on the lubrication theory, are coupled with the volumetric liquid pressures by the gradient across the liquid-air interfaces and solved in a monolithic two-way coupling system. Our method can accurately solve liquid-liquid interaction with air films without adaptive grid refinements, enabling accurate simulation of many novel surface-tension-driven phenomena such as droplet collisions, bouncing droplets, and promenading pairs.

Large-scale fluid simulation is widely useful in various Virtual Reality (VR) applications. While physics-based fluid animation holds the promise of generating highly realistic fluid details, it often imposes significant computational demands, particularly when simulating high-resolution fluid for VR. In this paper, we propose a novel foveated fluid simulation method that enhances both the visual quality and computational efficiency of physics-based fluid simulation in VR. To leverage the natural foveation feature of human vision, we divide the visible domain of the fluid simulation into foveal, peripheral, and boundary regions. Our foveated fluid system dynamically allocates computational resources, striking a balance between simulation accuracy and computational efficiency. We implement this approach using a multi-scale method. To evaluate the effectiveness of our approach, we have conducted subjective studies. Our findings show a significant reduction in computational resource requirements, resulting in a speedup of up to 2.27 times. It is crucial to note that our method preserves the visual quality of fluid animations at a level that is perceptually identical to full-resolution outcomes. Additionally, we investigate the impact of various metrics, including particle radius and viewing distance, on the visual effects of fluid animations. Our work provides new techniques and evaluations tailored to facilitate real-time foveated fluid simulation in VR, which can enhance the efficiency and realism of fluids in VR applications.

We propose a new Eulerian-Lagrangian approach to simulate the various surface tension phenomena characterized by volume, thin sheets, thin filaments, and points using Moving-Least-Squares (MLS) particles. At the center of our approach is a meshless Lagrangian description of the different types of codimensional geometries and their transitions using an MLS approximation. In particular, we differentiate the codimension-1 and codimension-2 geometries on Lagrangian MLS particles to precisely describe the evolution of thin sheets and filaments, and we discretize the codimension-0 operators on a background Cartesian grid for efficient volumetric processing. Physical forces including surface tension and pressure across different codimensions are coupled in a monolithic manner by solving one single linear system to evolve the surface-tension driven Navier-Stokes system in a complex non-manifold space. The codimensional transitions are handled explicitly by tracking a codimension number stored on each particle, which replaces the tedious meshing operators in a conventional mesh-based approach. Using the proposed framework, we simulate a broad array of visually appealing surface tension phenomena, including the fluid chain, bell, polygon, catenoid, and dripping, to demonstrate the efficacy of our approach in capturing the complex fluid characteristics with mixed codimensions, in a robust, versatile, and connectivity-free manner.

Solving a large-scale Poisson system is computationally expensive for most of the Eulerian fluid simulation applications. We propose a novel machine learning-based approach to accelerate this process. At the heart of our approach is a deep convolutional neural network (CNN), with the capability of predicting the solution (pressure) of a Poisson system given the discretization structure and the intermediate velocities as input. Our system consists of four main components, namely, a deep neural network to solve the large linear equations, a geometric structure to describe the spatial hierarchies of the input vector, a Principal Component Analysis (PCA) process to reduce the dimension of input in training, and a novel loss function to control the incompressibility constraint. We have demonstrated the efficacy of our approach by simulating a variety of high-resolution smoke and liquid phenomena. In particular, we have shown that our approach accelerates the projection step in a conventional Eulerian fluid simulator by two orders of magnitude. In addition, we have also demonstrated the generality of our approach by producing a diversity of animations deviating from the original datasets.

Abstract Eulerian-based smoke simulations are sensitive to the initial parameters and grid resolutions. Due to the numerical dissipation on different levels of the grid and the nonlinearity of the governing equations, the differences in simulation resolutions will result in different results. This makes it challenging for artists to preview the animation results based on low-resolution simulations. In this paper, we propose a learning-based flow correction method for fast previewing based on low-resolution smoke simulations. The main components of our approach lie in a deep convolutional neural network, a grid-layer feature vector and a special loss function. We provide a novel matching model to represent the relationship between low-resolution and high-resolution smoke simulations and correct the overall shape of a low-resolution simulation to closely follow the shape of a high-resolution down-sampled version. We introduce the grid-layer concept to effectively represent the 3D fluid shape, which can also reduce the input and output dimensions. We design a special loss function for the fluid divergence-free constraint in the neural network training process. We have demonstrated the efficacy and the generality of our approach by simulating a diversity of animations deviating from the original training set. In addition, we have integrated our approach into an existing fluid simulation framework to showcase its wide applications.