Single Image Super-Resolution Enhancement using Luminance Map and Atmospheric Light Removal

  • Mohammad Amin Shayegan Department of Computer, Shiraz Branch, Islamic Azad University, Shiraz, Iran http://orcid.org/0000-0001-7394-4772
  • Samira Poormajidi
Keywords: Single Image Super Resolution, Natural Images, Luminance Map, GAN, Convolutional Neural Network

Abstract

Image enhancement is used in many image processing applications such as medical diagnostics, satellite image analysis, surveillance cameras, etc. Super resolution attempts to reconstruct high resolution images from low resolution images and it can be considered as a preprocessing step for object recognition and image classification. Various algorithms have been introduced for single-image super resolution, but these algorithms often face important challenges such as poorly matching the reconstructed image with the original image, as well as the blurring of edges and texture details. The aim of this manuscript is to introduce a preprocessing operation to improve the performance of the super resolution process in natural images. In the proposed method, the low resolution input image is enhanced before entering the resolution change module. Calculating the brightness of the pixels in the image channels, creating the luminance map and removing atmospheric light, applying the transmittance map by using the luminance coefficients, and recovering the natural image in all three color channels are the above preprocessing steps. The proposed method succeeded in increasing the PSNR parameter by 4.35%, 10.62%, and 8.31%, as well as 0.23%, 3.10%, and 7.91% of the SSIM parameter for Set5, Set14, and BSD100 datasets compared to its closest state-of-the-art methods.

References

[1] Burger, W., & Burge, M. J. (2016). Digital image processing: an algorithmic introduction using Java. Springer.
[2] Huang, S., Sun, J., Yang, Y., Fang, Y., Lin, P., & Que, Y. (2018). Robust single-image super-resolution based on adaptive edge-preserving smoothing regularization. IEEE Transactions on Image Processing, 27(6), 2650-2663.
[3] Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016). Image super-resolution: the techniques, applications, and future. Signal Processing, 128, 389-408.
[4] Haris, M., Watanabe, T., Fan, L., Widyanto, M. R., & Nobuhara, H. (2017). Super resolution for uav images via adaptive multiple sparse representation and its application to 3-d reconstruction. IEEE Transactions on Geoscience and Remote Sensing, 55(7), 4047-4058.
[5] Ayas, S., & Ekinci, M. (2017, august). Learning based single image super resolution using discrete wavelet transform. In international conference on computer analysis of images and patterns (pp. 462-472). Springer, cham.
[6] Waleed Gondal, M., Scholkopf, B., & Hirsch, M. (2018). The unreasonable effectiveness of texture transfer for single image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 80-97, Springer, Cham.
[7] Lei, J., Zhang, S., Luo, L., Xiao, J., & Wang, H. (2018). Super-resolution enhancement of uav images based on fractional calculus and pocs. Geo-spatial information science, 21(1), 56-66.
[8] Liu, H., Fu, Z., Han, J., Shao, L., & Liu, H. (2018). Single satellite imagery simultaneous super-resolution and colorization using multi-task deep neural networks. Journal of visual communication and image representation, 53, 20-30.
[9] Han, W., Chu, J., Wang, L., & Pan, C. (2017). Edge-directed single image super-resolution via cross-resolution sharpening function learning. Multimedia tools and applications, 76(8), 11143-11155.
[10] Bei, Y., Damian, A., Hu, S., Menon, S., Ravi, N., & Rudin, C. (2018, may). New techniques for preserving global structure and denoising with low information loss in single-image super-resolution. In the IEEE conference on computer vision and pattern recognition (cvpr) workshops, pp. 874-881.
[11] Brifman, A., Romano, Y., & Elad, M. (2019). Unified Single-Image and Video Super-Resolution via Denoising Algorithms. IEEE Transactions on Image Processing, 28(12), 6063-6076.
[12] Ding, N., Liu, Y. P., Fan, L. W., & Zhang, C. M. (2019). Single Image Super-Resolution via Dynamic Lightweight Database with Local-Feature Based Interpolation. Journal of Computer Science and Technology, 34(3), 537-549.
[13] Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J. H., & Liao, Q. (2019). Deep learning for single image super-resolution: A brief review. IEEE Transactions on Multimedia, 21(12), 3106-3121.
[14] Lu, H., Li, Y., Nakashima, S., & Serikawa, S. (2016). Single image dehazing through improved atmospheric light estimation. Multimedia Tools and Applications, 75(24), 17081-17096.
[15] Song, J., Zhang, L., Shen, P., Peng, X., & Zhu, G. (2016, November). Single low-light image enhancement using luminance map. In Chinese Conference on Pattern Recognition (pp. 101-110). Springer, Singapore.
[16] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672-2680).
[17] Chang, H. W., Zhang, Q. W., Wu, Q. G., & Gan, Y. (2015). Perceptual image quality assessment by independent feature detector. Neurocomputing, 151, 1142-1152.
[18] Tuna, C., Unal, G., & Sertel, E. (2018). Single-frame super resolution of remote-sensing images by convolutional neural networks. International Journal of Remote Sensing, 39(8), 2463-2479.
[19] Li, F., Xin, L., Guo, Y., Gao, J., & Jia, X. (2017). A framework of mixed sparse representations for remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 55(2), 1210-1221.
[20] Fan, C., Wu, C., Li, G., & Ma, J. (2017). Projections onto convex sets super-resolution reconstruction based on point spread function estimation of low-resolution remote sensing images. Sensors, 17(2), 362.
[21] Lv, Z., Jia, Y., & Zhang, Q. (2017). Joint image registration and point spread function estimation for the super-resolution of satellite images. Signal processing: image communication, 58, 199-211.
[22] Cruz, C., Mehta, R., Katkovnik, V., & Egiazarian, K. O. (2018). Single image super-resolution based on wiener filter in similarity domain. IEEE Transactions on Image Processing, 27(3), 1376-1389.
[23] Lin, G., Wu, Q., Chen, L., Qiu, L., Wang, X., Liu, T., & Chen, X. (2018). Deep unsupervised learning for image super-resolution with generative adversarial network. Signal Processing: Image Commuication, 68, 88-100.
[24] Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., ... & Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 4681-4690).
[25] Zhang, X., Song, H., Zhang, K., Qiao, J., & Liu, Q. (2020). Single image super-resolution with enhanced Laplacian pyramid network via conditional generative adversarial learning. Neurocomputing, 398, 531-538.
[26] Qiao, J., Song, H., Zhang, K., & Zhang, X. (2021). Conditional generative adversarial network with densely-connected residual learning for single image super-resolution. Multimedia Tools and Applications, 80(3), 4383-4397.
[27] Chen, W., Liu, C., Yan, Y., Jin, L., Sun, X., & Peng, X. (2020). Guided Dual Networks for Single Image Super-Resolution. IEEE Access, 8, 93608-93620.
[28] Haris, M., Shakhnarovich, G., & Ukita, N. (2019). Deep Back-Projection Networks for Single Image Super-resolution. arXiv preprint arXiv:1904.05677.
[29] Hu, Y., Li, J., Huang, Y., & Gao, X. (2019). Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Transactions on Circuits and Systems for Video Technology, 30(11), 3911-3927.
[30] Xu, X., & Li, X. (2019). SCAN: Spatial Color Attention Networks for Real Single Image Super-Resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.
[31] Gao, S., & Zhuang, X. (2019). Multi-scale deep neural networks for real image super-resolution. In proceedings of the IEEE conference on computer vision and pattern recognition workshops.
[32] Xie, C., Liu, Y., Zeng, W., & Lu, X. (2019). An improved method for single image super-resolution based on deep learning. Signal, image and video processing, 13(3), 557-565.
[33] Yang, W., Wang, W., Zhang, X., Sun, S., & Liao, Q. (2019). Lightweight feature fusion network for single image super-resolution. IEEE Signal Processing Letters, 26(4), 538-542.
[34] Dai, T., Cai, J., Zhang, Y., Xia, S. T., & Zhang, L. (2019). Second-order attention network for single image super-resolution. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 11065-11074).
[35] Wang, Y., Wang, L., Wang, H., & Li, P. (2019). End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access, 7, 31959-31970.
[36] Xu, W., Chen, R., Huang, B., Zhang, X., & Liu, C. (2019). Single image super-resolution based on global dense feature fusion convolutional network. Sensors, 19(2), 316.
[37] Fang, F., Li, J., & Zeng, T. (2020). Soft-Edge Assisted Network for Single Image Super-Resolution. IEEE Transactions on Image Processing, 29, 4656-4668.
[38] Bevilacqua, M., Roumy, A., Guillemot, C., & Alberi-Morel, M. L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In Proceedings of the British Machine Vision Conference, (pp. 135.1–135.10). BMVA Press, 2012.
[39] Hui, Z., Wang, X., & Gao, X. (2018). Fast and accurate single image super-resolution via information distillation network. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 723-731).
[40] Choi, J. H., Zhang, H., Kim, J. H., Hsieh, C. J., & Lee, J. S. (2019). Evaluating robustness of deep image super-resolution against adversarial attacks. In Proceedings of the IEEE International Conference on Computer Vision, (pp. 303-311).
[41] Guo, X., Li, Y., & Ling, H. (2017). LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, 26(2), (pp. 982-993).
Published
2022-08-31
How to Cite
Shayegan, M., & Poormajidi, S. (2022). Single Image Super-Resolution Enhancement using Luminance Map and Atmospheric Light Removal. Majlesi Journal of Electrical Engineering. Retrieved from http://mjee.iaumajlesi.ac.ir/index/index.php/ee/article/view/4688
Section
Articles