Shape from silhouette consensus and photo-consistency

Abstract

We propose a 3D reconstruction algorithm based on silhouettes and color images. It is robust to inconsistent silhouettes, often common in real applications due to occlusions, errors in the background subtraction, noise or even calibration errors. The recovery of the shape that best fits the available data is formulated as a continuous energy minimization problem. The energy is based on the error between the silhouettes and the shape plus a regularization term based on a photo-consistency measure that places the surface at photo-consistent locations. The visibility is modeled as a function of the shape. The proposed photo-consistency measure takes visibility into account, although the presented variational framework can use different photo-consistency computations.