Uncertainty for SVBRDF Acquisition using Frequency Analysis

Ruben Wiersma1, 2, 3, Julien Philip2, 4, Milos Hasan2, Krishna Mullia2, Fujun Luan2, Elmar Eisemann3, Valentin Deschaintre2
1ETH Zürich, 2Adobe Research, 3TU Delft, 3Netflix Eyeline Studios
SIGGRAPH 2025
Paper (coming soon) Appendix Supplementary Code (to be released) Cite us
Overview image.

We quantify uncertainty for SVBRDF acquisition from multi-view captures (left) using entropy (right). We significantly accelerate the, otherwise heavy, computation in the frequency domain (Spherical Harmonics), yielding a practical, efficient method.

Abstract

This paper aims to quantify uncertainty for SVBRDF acquisition in multi-view captures. Under uncontrolled illumination and unstructured viewpoints, there is no guarantee that the observations contain enough information to reconstruct the appearance properties of a captured object. We study this ambiguity, or uncertainty, using entropy and accelerate the analysis by using the frequency domain, rather than the domain of incoming and outgoing viewing angles. The result is a method that computes a map of uncertainty over an entire object within a millisecond. We find that the frequency model allows us to recover SVBRDF parameters with competitive performance, that the accelerated entropy computation matches results with a physically-based path tracer, and that there is a positive correlation between error and uncertainty. We then show that the uncertainty map can be applied to improve SVBRDF acquisition using capture guidance, sharing information on the surface, and using a diffusion model to inpaint uncertain regions.

Problem statement

The reflections from an object reveal its material properties. Sometimes, it is difficult to know what material we are dealing with if we miss certain reflections or if a part of the object is in the shade. This can happen, for example, on a cloudy day where a glossy surface looks similar to a matte surface under certain viewing angles.

Which plane is glossy and which is matte? On the left, it's difficult to find out because of the lighting. On the right, it's immediately clear.

Two images that show the same matte and glossy surface in two lighting setups. The first image is rendered with a cloudy environment and the surfaces look similar. The second image is rendered with a sunny environment and the surfaces look very different.

Overview

The question whether we have the right lighting- and viewing conditions can be posed as a question of uncertainty. In this paper, we aim to quantify this uncertainty using entropy. Our work differs from recent contributions on uncertainty in inverse rendering by approaching the reflection equation from the signal processing framework on inverse rendering (Ramamoorthi and Hanrahan, 2001). Along the way, we find that an improved version of this framework

  • can match state-of-the-art results in a fraction of the time on the task of BRDF recovery,
  • while providing a practical way to measure uncertainty.

Uncertainty results

We show that our formulation for uncertainty using entropy matches with errors resulting from an optimization with Mitsuba.


Results showing a BRDF texture that was optimized with Mitsuba. On the right, the error is shown and a map of the uncertainty.

We observe that where there is error, entropy tends to be higher. We also show a mask for above-average MSE and entropy. The Mask Coverage column shows which regions of the MSE mask are not covered by the entropy mask and should, ideally, be completely black.

Relighting results

We optimize a PBR texture (base color, metallicity, roughness) on multi-view captures from Stanford ORB and relight the resulting object in Blender. More results in the supplemental material and the paper.

Method

Uncertainty as Entropy

We use statistical entropy on the likelihood of BRDF parameters to quantify uncertainty. While other methods test the likelihood function around a found optimum, our method takes a global perspective on all possible BRDF parameter combinations. This is tractable due to frequency analysis.

We show three examples of likelihood functions for different incoming and outgoing radiance. The red dot marks the ground-truth BRDF parameters and the title (H=...) shows the entropy.

An image showing three plots representing a likelihood and the corresponding entropy.

The left likelihood function shows incoming light for a dirac delta light source (constant in the spherical harmonic domain) and we see that entropy is low (H=0.25); we can be very certain about the BRDF recovery.

In the center likelihood function, the light only has amplitude in low frequencies and many roughness values are equally likely. The higher entropy (H=0.69) is associated with higher uncertainty, which is in line with the conclusions from Ramamoorthi and Hanrahan (2001).

The right likelihood function corresponds to a material with very low specular reflectance, resulting in high entropy (H=0.87) and thus high uncertainty.


Computing entropy can be extremely expensive, since we need to evaluate the entire parameter space for all viewpoints. Therefore, we propose to accelerate this evaluation in the frequency domain.

Frequency Analysis

Reflection as Convolution

The signal processing framework for inverse rendering (Ramamoorthi and Hanrahan, 2001) shows that the reflection equation can be approximated as a rotational convolution of the BRDF kernel with the incoming light over the incoming- and outgoing light directions at a point on the surface. This perspective allows us to study inverse rendering in the frequency domain, here the spherical harmonics domain.

An image that shows the reflection equation as a convolution of the BRDF kernel with the incoming light over the incoming- and outgoing light directions at a point on the surface.

Robust Spherical Harmonics Transform

Our first contribution is a method to estimate the spherical harmonics coefficients for the incoming- and outgoing radiance field, for sparse and irregularly distributed viewing positions. We achieve this by a least-squares fit with a weighted L2 regularization.

An image that represents a procedure to fit spherical harmonics coefficients to an irregularly distributed, sparse set of points.

BRDF Recovery with Shadowing and Masking

Our second contribution is to use these coefficients in a BRDF optimization pipeline to estimate parameters for an analytic microfacet BRDF (Disney principled BRDF). We improve the accuracy of the convolution model by incorporating shadowing and masking.

An image that represents shadowing and masking.

Power Spectrum Approximation

Our third contribution is to develop an extremely light-weight approximation of the convolution model that operates completely in the power spectrum of the spherical harmonics. This allows us to explore hundreds of BRDF parameter combinations in a couple of milliseconds. Moreover, the power spectrum is invariant to rotations of local coordinate frame (here, rotated normals).

An image showing three plots, representing power spectra. The first plot is the power spectrum of the incoming light, the second plot of the BRDF and the third of the outgoing light. On the second plot, the graph shows alternative plots for different parameter combinations.

Find out more

Paper (coming soon) Appendix Supplementary Code (to be released) Cite us

BibTeX


        @inproceedings{wiersma2025svbrdfuncertainty,
          author = {Wiersma, Ruben and Philip, Julien and Hašan, Miloš and Mullia, Krishna and Luan, Fujun and Eisemann, Elmar and Deschaintre, Valentin},
          title = {Uncertainty for SVBRDF Acquisition using Frequency Analysis},
          year = {2025},
          isbn = {979-8-4007-1540-2/2025/08},
          publisher = {Association for Computing Machinery},
          address = {New York, NY, USA},
          url = {https://doi.org/10.1145/3721238.3730592},
          doi = {10.1145/3721238.3730592},
          booktitle = {SIGGRAPH Conference Papers '25},
          location = {Vancouver, BC, CA},
          series = {SIGGRAPH Conference Papers '25}
        }