Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition, 1st Edition (2015)
Part I. Image and Signal Processing
Chapter 16. Night color image enhancement via statistical law and retinex
Huaxia Zhao; Chuangbai Xiao; Hongyu Zhao College of Computer Science and Technology, Beijing University of Technology, Beijing, China
Abstract
Due to the uneven distribution of light at night, the quality of the night color image is usually unsatisfactory. To solve this problem, in this article, we propose a new statistical method based on the retinex. The algorithm analyzes the transformation relationship between the nighttime image and illumination image by the algorithm of Michael Elad and MSRCR algorithm. Through this transformation, we can accurately and quickly get the illumination image. Then, we can get the resulting image successfully based on the retinex. Our algorithm can greatly enhance the image contrast and brightness, recover image details, eliminate the “halo effect” efficiently, and accelerate the computational speed remarkably. Experiments on different nighttime images demonstrate the effectiveness of our approach.
Keywords
Statistical law
Image enhancement
Night color image
Retinex
Illumination image
1 Introduction
The night color image enhancement is of great importance in both the computational photography and computer vision. First, it can effectively increase the visibility and surrealism of the scene. Second, artificial illumination light distributes unevenly at night leading to weakening the quality of monitoring photos and increasing the difficulty of surveillance. Thus, the night color image enhancement can promote the video surveillance. Last, the input images of most computer vision algorithms (e.g., the photometric analysis algorithm) are daytime images. Thus, the night color image enhancement can increase the scope of such algorithms by enhancing nighttime images.
However, the night color image enhancement is a challenging task. Currently, the main techniques for the night image enhancement are the image fusion and image enhancement. Image fusion techniques include two categories: one is the fusion of the nighttime image and visible image [1,2] and another is the fusion of the nighttime image and infrared image [3,4]. These methods require multiple different spectral images collected in the same scene and have high computational complexity. The main techniques for the image enhancement include contrast stretching, slicing, histogram equalization, and some algorithms based on the retinex [5–11], etc. Of all these algorithms, the algorithm based on the retinex has acceptable results, but it will produce the “halo effect” and high time complexity.
In this article, we propose a novel algorithm for enhancing the night color image based on the statistical law and the retinex. We assume that there is a transformation on the brightness components of the pixel values between the nighttime image and illumination image. Therefore, through this transformation, we can accurately and quickly get the illumination image. Then, we can get the resulting image successfully based on the retinex. The resulting image retains image details and exhibits higher brightness, so that the overall image looks more harmonious and natural. Our algorithm is simpler and faster compared to the other algorithms.
The rest of this article is organized as follows. In Section 2, overview of retinex theory is given. In Section 3, the proposed algorithm is described in detail, which contains two parts: analysis of the transformation law and enhancing the nighttime image. Finally, the experimental results are presented to demonstrate the efficiency of the proposed algorithm in Section 4. The conclusion is in Section 5.
2 Overview of Retinex Theory
2.1 The Basic Idea of Retinex Theory
The Retinex theory deals with the removal of unfavorable illumination effects from a given image. A commonly assumed model suggests that any given image S is the pixel-wise multiplication of two images, the reflection image R, and the illumination image L. This model is given in the following equation:
(1)
Therefore, if we can get the illumination image, we can quickly get the reflection image. In the actual calculation, a look-up-table log operation transfers this multiplication into an addition, resulting with s = log(S) = log(L) + log(R) = l + r.
2.2 The “halo effect”
The Retinex algorithm often results in the “halo effect.” It is mainly based on the position of surrounding pixels to give different weights to estimate the current pixel illumination in the calculation of the illumination, while ignoring the pixel itself. This often leads to mutual influence between different pixels in the intense chiaroscuro edge region: the estimated illumination of high pixels is lower affected by adjacent low pixels, the estimated illumination of low pixels is higher affected by adjacent high pixels. The distortion of the estimated illumination leads to the “halo effect.”
3 Analyzing the transformation law and enhancing the nighttime image
We first transform the original RGB space to HSV space, because processing the color image directly in RGB space will lead to color distortion. The HSV space is closer to human visual perception in color perception. Our transformation law is only used in brightness component of HSV space.
Currently, most algorithms often use the filtering method to estimate the illumination image, and achieve good results. In this article, we use the processing results of some algorithms (the algorithm of Michael Elad [12] and MSRCR [13,14]) as illumination images. Through these two algorithms, we get three images. One is the nighttime image and the other two are the corresponding illumination images. Their brightness components of HSV space are denoted as L, M, and N. For analyzing the transformation law, we get pixels which value is i (0 − 255) from L. Then, we have a set of coordinates through known pixels. In the same coordinates, we get two sets of pixel values from M and N. The average (j, k) of these two sets are the corresponding value to i. Figure 1 displays the correspondence between i and j, k. In order to facilitate observation, we add a linear which is y = x.
FIGURE 1 The corresponding graphs of pixel values of source images and illumination images obtained by the algorithm of Michael Elad and MSRCR algorithm. Panels (a–d) are the processing results of four different pictures.
By observing Figure 1, we find the curve of MSRCR on the figure can be represented by a circular arc. But it is too close to the linear which is y = x resulting in that the enhanced image is too bright and loses details seriously. The curve of Michael Elad is roughly like a circular arc except a small part. It is the reason that the resulting images processed by the algorithm of Michael Elad have a stronger noise. Overall, we can use a circular arc to represent the relationship between the input image and illumination image. Obviously, the fitting circular arc should pass the point (255,255). In order to facilitate the calculation, we use two parameters to represent the circular arc. One parameter is x-coordinate (x0) of the circular center. Another is the intersection (0, λ) of arc and y positive axle. According to the nature of the circle, the y-coordinate (y0) of the circular center can be expressed as the following equation:
(2)
The radius (r) of the circular arc can be expressed as follows:
(3)
The circular arc can be expressed as follows:
(4)
Using Equation (4), we can get the illumination image directly and quickly. The circular arc is shown in Figure 2.
FIGURE 2 Fitting circular arc.
According to Equation (4), we can know that the circular arc takes two parameters called x0 and λ. The smaller x0 is, the more obvious the brightness enhancement is. The greater λ is, the more obvious the brightness enhancement is. It can be seen that the darker the input image is, the smaller x0 is and the greater λ is, and vice versa. Therefore, we assign the average pixel value of the source image to λ. x0 is represented by the following formula:
(5)
Where round () is the rounding function which is used to improve the algorithm speed. The minimum size of x0 is limited to 127 to prevent image distortion. Therefore, we have got a no-argument function to show the relationship between the original image and the corresponding illumination image.
The algorithm is given below:
(1) Transform the original RGB data to the HSV data.
(2) Get the illumination image using Equations (4) and (5).
(3) Enhance the nighttime image through the retinex theory and the obtained illumination image.
(4) Transform the HSV data to the RGB data and show the enhanced image.
4 Comparison and results
In this section, the proposed algorithm of this article is compared with the algorithm of Michael Elad and MSRCR algorithm. Figures 3-8 show the experimental results of five scenes. (a) is the original image. (b) shows the enhanced image by the algorithm of Michael Elad. (c) is obtained by MSRCR algorithm. (d) displays the enhanced image by the proposed algorithm in this article.
FIGURE 3 Scene 1. (a) original image; (b) enhanced image by Michael Elad; (c) enhanced image by MSRCR; (d) enhanced image by the proposed algorithm.
FIGURE 4 Scene 2. (a) original image; (b) enhanced image by Michael Elad; (c) enhanced image by MSRCR; (d) enhanced image by the proposed algorithm.
FIGURE 5 Scene 3. (a) original image; (b) enhanced image by Michael Elad; (c) enhanced image by MSRCR; (d) enhanced image by the proposed algorithm.
FIGURE 6 Scene 4. (a) original image; (b) enhanced image by Michael Elad; (c) enhanced image by MSRCR; (d) enhanced image by the proposed algorithm.
FIGURE 7 Scene 5. (a) original image; (b) enhanced image by Michael Elad; (c) enhanced image by MSRCR; (d) enhanced image by the proposed algorithm.
FIGURE 8 Scene 6. (a) original image; (b) enhanced image by Michael Elad; (c) enhanced image by MSRCR; (d) enhanced image by the proposed algorithm.
It can be clearly seen from the figures that the three algorithms all have a certain enhancement effect on the nighttime images. The processing results of MSRCR algorithm are too bright leading to atomization phenomenon and loss of details, as shown in Figures 4(c), 5(c), 6(c), and 8(c). The processing results of Michael Elad’s algorithm tend to produce excessive sharpening phenomenon in highlighting edges and the “halo artifacts” phenomenon which is shown in Figures 6(b), 7(b), and 8(b). Moreover, it often leads to noisy amplification in dark areas, as shown in Figure 3(b), which is mainly due to the unsmooth curve (shown in Figure 1) between source images and illumination images. Compared with the two algorithms, the proposed algorithm can better recover details, eliminates the “halo effect” and suppress noise. Moreover, the results of our algorithm look more harmonious and natural.
For a more definite description of the experimental results, this article also uses objective evaluation criteria to test the effectiveness of our algorithm. We examine our algorithm in mean value, standard deviation and time-consuming (Computer configuration: CPU: Pentium(R) 3.00 GHz; Memory: 3.00 GB; Programming Language: Matlab). The image mean reflects the brightness level of the image; the standard deviation reflects the image contrast; the time-consuming reflects the time complexity of the algorithm. The results are shown in Tables 1–4.
Table 1
Criteria of Assessment of Figure 3
Table 2
Criteria of Assessment of Figure 4
Table 3
Criteria of Assessment of Figure 5
Table 4
Criteria of Assessment of Figure 6
As can be seen in Tables 1–6, in terms of the mean, MSRCR algorithm has the most significant effect of improving mean, but the enhanced images are too bright overall to protect details. Compared with the algorithm of Michael Elad, the proposed algorithm has a better enhancing effect of the mean which displays the brightness of the whole picture is consistent with human visual perception. In terms of standard deviation, the proposed algorithm is better than the other two algorithms. The proposed algorithm has remarkable enhancement of the image contrast and significant effect of image detail recovery. In terms of time consuming, the proposed algorithm has lower time complexity than the other two algorithms. Moreover, our algorithm does not need manual control parameters increasing the adaptability of our algorithm.
Table 5
Criteria of Assessment of Figure 7
Table 6
Criteria of Assessment of Figure 8
5 Application
Besides above applications, there are other applications. Because this algorithm uses the properties of the image itself. We process overexposed images by means of this algorithm and get good results. Our algorithm avails several applications due to its fundamentality and the special properties in processing natural images.
Overexposed images often have high brightness, low contrast, and invisible details. This algorithm can effectively restore the contrast and brightness to a suitable standard for human senses by means of the transformation between the original image and illumination image. The processing results of overexposed images are listed (seen in Figures 9–10). Panel (a) is the original image and (b) displays the enhanced image by the proposed algorithm in this article. It can be clearly seen from the figures that this algorithm has a good restoration of overexposed images, making the images more in line with human senses. This approach enhances significant edges, making structures visually pleasing.
FIGURE 9 Scene 6.
FIGURE 10 Scene 7.
6 The conclusion
In this article we present an effective algorithm for enhancing the nighttime image. In our algorithm, we propose a statistical law to present the relation of the original image and illumination image. Using this statistical law and retinex theory, we can accurately and quickly get the resulting image. The algorithm is validated through subjective and objective evaluation, which shows it can eliminate the “halo effect,” enhance the image contrast, recover image details and have low time complexity. In summary, our algorithm is effective to complete the challenging task of enhancing the nighttime image [15].
References
[1] Raskar R, Ilie A, Jingyi Y. Image fusion for context enhancement and video surrealism. In: Proceedings NPAR 2004—3rd international symposium on non-photorealistic animation and rendering, 2004; Annecy, France: Association for Computing Machinery; 2004:85–93.
[2] Yamasaki A, Takauji H, Kaneko S, et al. Denighting: enhancement of nighttime images for a surveillance camera. In: 2008 19th international conference on pattern recognition, ICPR 2008, 2008; Tampa, FL, United States: Institute of Electrical and Electronics Engineers Inc; 2008.
[3] Xiaopeng Z, Sim T, Xiaoping M. Enhancing photographs with near infrared images. In: 26th IEEE conference on computer vision and pattern recognition, CVPR, 2008; Anchorage, AK, United States: Institute of Electrical and Electronics Engineering Computer Society; 2008.
[4] Shaojie Z, Xiaopeng Z, Xiaoping M, et al. Enhancing low light images using near infrared flash images. In: 2010 17th IEEE international conference on image processing, ICIP 2010, 2010; Hong Kong: IEEE Computer Society; 2010:2537–2540.
[5] Brainard D, Wandell B. Analysis of the retinex theory of color vision. J Opt Soc Am. 1986;3:1651–1661.
[6] McCann J. Lessons learned from Mondrians applied to real images and color gamuts. In: Final program and proceedings of the 7th IS and T/SID color imaging conference: color science, systems and applications, 1999; Scottsdale, AZ, United States: Society for Imaging Science and Technology; 1999:1–8.
[7] Funt B, Ciurea F, McCann J. Retinex in Matlab. In: Final program and proceedings of the 8th IS and T/SID color imaging conference: color science, systems and applications, 2000; Scottsdale, AZ, United States: Society for Imaging Science and Technology; 2000:112–121.
[8] Jobson D, Rahman Z, Woodell GA. Properties and performance of a center/surround retinex. IEEE Trans Image Process. 1997;6(3):451–462.
[9] Rahman Z, Jobson DJ, Woodell GA. Multi-scale retinex for color image enhancement. In: Proceedings of the 1996 IEEE international conference on image processing, ICIP'96. 1996; Lausanne, Switz: IEEE; 1996:1003–1006.
[10] Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Proceedings of the 1998 IEEE 6th international conference on computer vision, 1998; Bombay, India: IEEE; 1998:839–846.
[11] Meylan L, Susstrunk S. High dynamic range image rendering with a retinex-based adaptive filter. IEEE Trans Image Process. 2006;15(9):2820–2830.
[12] Elad M. Retinex by two bilateral filters. In: Proceedings of the scale-space conference, vol. 9(7); 2005:217–229.
[13] Jobson DJ, Rahman Z, Woodell GA. Multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process. 1997;6(7):965–976.
[14] Rahman Z, Jobson DJ, Woodell GA. Retinex processing for automatic image enhancement. J Electron Imag. 2004;13(1):100–110.
[15] Huaxia Z, Chuangbai X, Hongyu Z. Night color image enhancement via statistical law and retinex. In: Proceedings of the 2014 world congress in computer science, computer engineering, and applied computing, IPCV 2014, 2014; Las vegas, Nevada, USA: WORLDCOMP'14 Steering Committee; 2014.