Abstract:
We introduce the concept of contractive appearance flow to address photometric stereo with general reflectance. Our solution is motivated by the fact that the shape intrinsics of an object are encoded by its Lambertian reflectance, based on which we design a neural network that maps a set of per-pixel general appearances to their Lambertian counterparts as if this process is carried by a flow in a field of vectors of pixel values. Our design has two features: (1) by introducing a transfer operator in the encoded latent space, we replace the typical workflow of a Variational AutoEncoder (VAE) with a more generic encode-transfer-decode procedure. For photometric stereo, we apply this procedure to produce consistent representations of the incident light fields and to eliminate the signal variation caused by material properties; (2) during training each sample of general reflectance is associated with its Lambertian-related template samples, and by minimizing the distance between these two types of signals in the latent space, we enforce the flow to contract in the subspace spanned by Lambertian appearances only. The proposed method learns reflectance measurements directly and does not need to parameterize material properties. Our design is simple, lightweight, and automatic, yet, experiments show that it is effective and yields accurate estimations.
