At SIGGRAPH fVDB, NVIDIA unveiled a new deep learning framework for generating AI-enabled virtual representations of the real world.
fVDB is built on OpenVDB, the industry standard library for simulating and rendering sparse volumetric data such as water, fire, smoke, and clouds.
Generative physics AI, such as autonomous cars and robots that inhabit the real world, requires “spatial intelligence” – the ability to understand and navigate 3D space.
Capturing both large-scale and microscopic details of the world around us is essential, but translating reality into a virtual representation to train AI is challenging.
Raw data of real-world environments can be collected through a variety of techniques, including Neural Radiance Fields (NeRF) and LIDAR. fVDB transforms this data into large-scale AI-enabled environments that are rendered in real-time.
Building on a decade of innovation in the OpenVDB standard, the introduction of fVDB at SIGGRAPH marks a major step forward in how industry benefits from real-world digital twins.
Real-scale virtual environments are used to train autonomous agents, city-scale 3D models are captured by drones for climate science and disaster planning, and today 3D generative AI is even used for urban spatial and smart city planning.
fVDB enables the industry to harness spatial intelligence at a larger scale and higher resolution than ever before, making physics AI even smarter.
The framework builds NVIDIA-accelerated AI operators on top of NanoVDB, a GPU-accelerated data structure for efficient 3D simulation. These operators include convolution, pooling, attention and meshing, all designed for high-performance 3D deep learning applications.
AI operators enable companies to build complex neural networks for spatial intelligence, such as large-scale point cloud reconstruction and 3D generative modeling.
fVDB is the result of a long-term effort by NVIDIA research teams and is already being used to support NVIDIA Research, NVIDIA DRIVE and NVIDIA Omniverse projects that require high-fidelity models of large, complex real-world spaces.
Key Benefits of fVDB
- Larger: 4x spatial scale than traditional frameworks
- Speedup: 3.5x faster than previous frameworks
- Interoperability: Enterprises can take full advantage of vast real-world datasets. fVDB loads VDB datasets into full-sized 3D environments that are AI-enabled and rendered in real time to build physics-based AI with spatial intelligence.
- More powerful: 10x more operators than previous frameworks. fVDB simplifies processes by combining functionality that previously required multiple deep learning libraries.
fVDB will soon be available as an NVIDIA NIM inference microservice. Three microservices enable companies to incorporate fVDB into their OpenUSD workflows and generate AI-ready OpenUSD geometry in NVIDIA Omniverse, a development platform for industrial digitalization and generative physics AI applications. These are:
- fVDB Mesh Generation NIM — Generate real-world digital 3D environments
- fVDB NeRF-XL NIM — Generate large scale NeRFs on OpenUSD using Omniverse Cloud API
- fVDB Physics Super-Res NIM — Performs super-resolution to generate high-resolution OpenUSD based physics simulations.
Over the past 10 years, OpenVDBhoused in Academy Software Foundation, It is a multiple Academy Award winner as a core technology used throughout the visual effects industry, and has since expanded beyond the entertainment realm into industrial and scientific applications such as industrial design and robotics.
NVIDIA continues to enhance its open-source OpenVDB library. Four years ago, the company introduced NanoVDB, which added GPU support to OpenVDB, delivering an order of magnitude speedup that enabled faster performance and easier development, opening the door to real-time simulation and rendering.
Two years ago, NVIDIA introduced NeuralVDB, which built machine learning on top of NanoVDB and compressed the memory footprint of VDB volumes by up to 100x, enabling creators, developers and researchers to work with extremely large, complex datasets.
fVDB builds AI operators on NanoVDB to enable spatial intelligence at real-world scale. Apply for the fVDB PyTorch extension early access program. fVDB will also be available as part of the OpenVDB GitHub repository.
Learn more about fVDB in this tech blog, and watch two SIGGRAPH fireside chats with NVIDIA founder and CEO Jensen Huang to learn how accelerated computing and generative AI are transforming industries and creating new opportunities for innovation and growth.
look News About software product information.