Real-time highlight removal from a single image

Abstract

The problem of highlight removal from image data refers to an open problem in computer vision concerning the estimation of specular reflection components and the removal thereof. In recent applications, highlight removal methods have been employed for the reproduction of specular highlights on high dynamic range (HDR) displays; to increase glossiness of images in specular reflection control technologies; to improve image quality in display systems such as TVs; and to enhance the dynamic range of low dynamic range (LDR) images. However, the underlying processing required by state-of-the-art methods is computationally expensive and does not meet real-time operational requirements in image processing pipelines found in consumer electronics applications. In addition, these applications may require that methods work with a single frame in imaging or video streams. Thus, this work proposes a novel method for the real-time removal of specular highlights from a single image. The essence of the proposed method consists in matching the histogram of the luminance component of a pseudo-specular-free representation using as reference the luminance component of the input image. The operations performed by the proposed method have, at most, linear time complexity. In experimental evaluations, the proposed method is capable of matching or improving upon state-of-the-art results on the task of diffuse reflection component estimation from a single image, while being 5× faster than the method with the best computational time and 1500× faster than the method with the best results. The proposed method has high industrial applicability, and targeted use cases can take advantage of contributions of this work by incorporating the proposed method as a building block in image processing pipelines.