Hierarchical NeRF with JAX3D for Volumetric Rendering, Novel-View Synthesis, and 3D Reconstruction

In this tutorial, we build an end-to-end hierarchical Neural Radiance Field (NeRF) using JAX, Flax, Optax, and the volume-rendering primitives provided by jax3d.

Building a Synthetic Multi-View Dataset

We first construct a synthetic multi-view dataset from an analytic scene containing volumetric geometry and view-dependent radiance, using sample_along_rays and volume_rendering to establish the forward rendering process.

Implementing the NeRF Pipeline

We then implement a NeRF model, leveraging the hierarchical approach to represent the scene and enable volumetric rendering, novel-view synthesis, and 3D reconstruction.

Key Components and Libraries

• JAX for high-performance numerical computing
• Flax for defining and training neural network models
• Optax for gradient-based optimization
• jax3d for volume-rendering primitives, including sample_along_rays and volume_rendering