Retinal noise emulation: a novel artistic tool for cinema that also improves compression efficiency

Zabaleta, Itziar; Cámara, Mateo; Díaz, César; Canham, Trevor; García, Narciso; Bertalmío, Marcelo

doi:http://dx.doi.org/10.1109/ACCESS.2020.2984994

Retinal noise emulation: a novel artistic tool for cinema that also improves compression efficiency

License

Citation

Zabaleta I, Cámara M, Díaz C, Canham T, García N, Bertalmío M. Retinal noise emulation: a novel artistic tool for cinema that also improves compression efficiency. IEEE Access. 2020 Apr 1;8:67263-76. DOI: 10.1109/ACCESS.2020.2984994

Abstract

In cinema it is standard practice to improve the appearance of images by adding noise that simulates film grain. This is computationally very costly, so it is only done in post-production and not on the set. It is also limiting because the artists are not able to really experiment with the noise nor introduce novel looks. Furthermore, video compression requires a higher bit rate when the source material has film grain or any other type of high frequency texture. In this work, we introduce a method for adding texture to digital cinema that aims to solve these problems. The proposed algorithm is based on modeling retinal noise, with which the images processed by our method have a natural appearance. This “retinal grain” serves a double purpose. One is aesthetic, as it has parameters that allow to vary widely the resulting texture appearance, which make it an artistic tool for cinematographers. Results are validated through psychophysical experiments in which observers, including cinema professionals, prefer our method over film grain synthesis methods from academia and the industry. The other purpose of the retinal noise emulation method is to improve the quality of compressed video by masking compression artifacts, which allows to lower the encoding bit rate while preserving image quality, and to improve image quality while keeping the bit rate fixed. The effectiveness of our approach for improving coding efficiency, with average bit rate savings of 22.5%, has been validated through psychophysical experiments using professional cinema content shot in 4K, color-graded and where the amount of retinal noise was selected by a motion picture specialist based solely on aesthetic preference.