Automatic Diagnosis of Breast Cancer in Histopathologic Images Based on Convolutional AutoEncoders and Reinforced Feature Selection

  • Ali Abdulhussain Fadhil
  • Miaad Adnan
  • Hamza Radhi
  • Mahmood Al-Mualm
  • Mahmood Hasen Alubaidy
  • Mohamed Salih
  • Sarah Jaafar Saadoon
Keywords: Histopathology, Reinforcement Learning, Feature Selection, Convolutional Autoencoders, Breast Cancer Detection


Breast cancer is one the most ubiquitous types of cancer which affect a considerable number of women around the globe. It is a malignant tumor, whose origin is in the glandular epithelium of the breast and causes serious health-related problems for patients. Although there is no known way of curing this disease, early detection of it can be very fruitful in terms of reducing the negative ramifications. Thus, accurate diagnosis of breast cancer based on automatic approaches is demanded immediately. Computer vision-based techniques in the analysis of medical images, especially histopathological images, have proved to be extremely performant. In this paper, we propose a novel approach for classifying malignant or non-malignant images. Our approach is based on the latent space embeddings learned by convolutional autoencoders. This network takes a histopathological image and learns to reconstruct it and by compressing the input into the latent space, we can obtain a compressed representation of the input. These embeddings are fed to a reinforcement learning-based feature selection module which extracts the best features for distinguishing the normal from the malicious images. We have evaluated our approach on a well-known dataset, named BreakHis, and used the K-Fold Cross Validation technique to obtain more reliable results. The accuracy, achieved by the proposed model, is 96.8% which exhibits great performance.


[1] Goodall GJ, Wickramasinghe VO. RNA in cancer. Nature Reviews Cancer. 2021 Jan;21(1):22-36.
[2] Waks AG, Winer EP, “Breast cancer treatment: a review,” Jama. 2019 Jan 22;321(3):288-300.
[3] Ferlay J, Colombet M, Soerjomataram I, Parkin DM, Piñeros M, Znaor A, Bray F, “Cancer statistics for the year 2020: An overview,” International journal of cancer. 2021 Aug 15;149(4):778-89.
[4] Elsharawy KA, Gerds TA, Rakha EA, Dalton LW, “Artificial intelligence grading of breast cancer: a promising method to refine prognostic classification for management precision,” Histopathology. 2021 Aug;79(2):187-99.
[5] Singh AV, Maharjan RS, Kanase A, Siewert K, Rosenkranz D, Singh R, Laux P, Luch A, “Machine-learning-based approach to decode the influence of nanomaterial properties on their interaction with cells,” ACS Applied Materials & Interfaces. 2020 Dec 29;13(1):1943-55.
[6] Sohail A, Khan A, Wahab N, Zameer A, Khan S, “A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images,” Scientific Reports. 2021 Mar 18;11(1):1-8.
[7] Budak Ü, Cömert Z, Rashid ZN, Şengür A, Çıbuk M, “Computer-aided diagnosis system combining FCN and Bi-LSTM model for efficient breast cancer detection from histopathological images,” Applied Soft Computing. 2019 Dec 1;85:105765.
[8] Saxena S, Gyanchandani M, “Machine learning methods for computer-aided breast cancer diagnosis using histopathology: a narrative review,” Journal of medical imaging and radiation sciences. 2020 Mar 1;51(1):182-93.
[9] Suganyadevi S, Seethalakshmi V, Balasamy K, “A review on deep learning in medical image analysis,” International Journal of Multimedia Information Retrieval. 2022 Mar;11(1):19-38.
[10] Zhang J, Xie Y, Wu Q, Xia Y, “Medical image classification using synergic deep learning,” Medical image analysis. 2019 May 1;54:10-9.
[11] Kiani A, Uyumazturk B, Rajpurkar P, Wang A, Gao R, Jones E, Yu Y, Langlotz CP, Ball RL, Montine TJ, Martin BA, “Impact of a deep learning assistant on the histopathologic classification of liver cancer,” NPJ digital medicine. 2020 Feb 26;3(1):1-8.
[12] Zhai J, Shen W, Singh I, Wanyama T, Gao Z, “A review of the evolution of deep learning architectures and comparison of their performances for histopathologic cancer detection,” Procedia Manufacturing. 2020 Jan 1;46:683-9.
[13] Houssein EH, Emam MM, Ali AA, Suganthan PN, “Deep and machine learning techniques for medical imaging-based breast cancer: A comprehensive review,” Expert Systems with Applications. 2021 Apr 1;167:114161.
[14] Bejnordi BE, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens G, Van Der Laak JA, Hermsen M, Manson QF, Balkenhol M, Geessink O, “Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer,” Jama. 2017 Dec 12;318(22):2199-210.
[15] Thuy MB, Hoang VT, “Fusing of deep learning, transfer learning and gan for breast cancer histopathological image classification,” InInternational Conference on Computer Science, Applied Mathematics and Applications 2019 Dec 19 (pp. 255-266). Springer, Cham.
[16] Bardou D, Zhang K, Ahmad SM, “Classification of breast cancer based on histology images using convolutional neural networks,” Ieee Access. 2018 May 1;6:24680-93.
[17] Boumaraf S, Liu X, Wan Y, Zheng Z, Ferkous C, Ma X, Li Z, Bardou D, “Conventional machine learning versus deep learning for magnification dependent histopathological breast cancer image classification: A comparative study with visual explanation,” Diagnostics. 2021 Mar 16;11(3):528.
[18] Song Y, Zou JJ, Chang H, Cai W, “Adapting fisher vectors for histopathology image classification,” In2017 IEEE 14th international symposium on biomedical imaging (ISBI 2017) 2017 Apr 18 (pp. 600-603). IEEE.
[19] Zhi W, Yueng HW, Chen Z, Zandavi SM, Lu Z, Chung YY, “Using transfer learning with convolutional neural networks to diagnose breast cancer from histopathological images,” InInternational Conference on Neural Information Processing 2017 Nov 14 (pp. 669-676). Springer, Cham.
[20] Zhan X, Cheng J, Huang Z, Han Z, Helm B, Liu X, Zhang J, Wang TF, Ni D, Huang K, “Correlation analysis of histopathology and proteogenomics data for breast cancer,” Molecular & Cellular Proteomics. 2019 Aug 9;18(8):S37-51.
[21] Li X, Shen X, Zhou Y, Wang X, Li TQ, “Classification of breast cancer histopathological images using interleaved DenseNet with SENet (IDSNet),” PloS one. 2020 May 4;15(5):e0232127.
[22] Al-Haija QA, Adebanjo A, “Breast cancer diagnosis in histopathological images using ResNet-50 convolutional neural network,” In2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) 2020 Sep 9 (pp. 1-7). IEEE.
[23] Parvin F, Hasan MA, “A comparative study of different types of convolutional neural networks for breast cancer histopathological image classification,” In2020 IEEE Region 10 Symposium (TENSYMP) 2020 Jun 5 (pp. 945-948). IEEE.
[24] Sharghi E, Nourani V, Najafi H, Molajou A, “Emotional ANN (EANN) and wavelet-ANN (WANN) approaches for Markovian and seasonal based modeling of rainfall-runoff process,” Water resources management. 2018 Aug;32(10):3441-56.
[25] Nourani V, Razzaghzadeh Z, Baghanam AH, Molajou A, “ANN-based statistical downscaling of climatic parameters using decision tree predictor screening method. Theoretical and Applied Climatology,” 2019 Aug;137(3):1729-46.
[26] Mehr AD, Nourani V, Hrnjica B, Molajou A, “A binary genetic programing model for teleconnection identification between global sea surface temperature and local maximum monthly rainfall events,” Journal of Hydrology. 2017 Dec 1;555:397-406.
[27] Wang J, Zhu H, Wang SH, Zhang YD, “A review of deep learning on medical image analysis. Mobile Networks and Applications,” 2021 Feb;26(1):351-80.
[28] Adegun AA, Viriri S, Ogundokun RO, “Deep learning approach for medical image analysis,” Computational Intelligence and Neuroscience. 2021 May 8;2021.
[29] Ahmed AS, El-Behaidy WH, Youssif AA, “Medical image denoising system based on stacked convolutional autoencoder for enhancing 2-dimensional gel electrophoresis noise reduction,” Biomedical Signal Processing and Control. 2021 Aug 1;69:102842.
[30] Nogales A, Garcia-Tejedor AJ, Monge D, Vara JS, Antón C, “A survey of deep learning models in medical therapeutic areas,” Artificial Intelligence in Medicine. 2021 Feb 1;112:102020.
[31] Mutabazi E, Ni J, Tang G, Cao W, “A review on medical textual question answering systems based on deep learning approaches,” Applied Sciences. 2021 Jan;11(12):5456.
[32] Dong S, Wang P, Abbas K, “A survey on deep learning and its applications,” Computer Science Review. 2021 May 1;40:100379.
[33] Souravlas S, Anastasiadou S, Katsavounis S, “A Survey on the Recent Advances of Deep Community Detection,” Applied Sciences. 2021 Aug 4;11(16):7179.
[34] Hammouche R, Attia A, Akhrouf S, Akhtar Z, “Gabor filter bank with deep autoencoder based face recognition system,” Expert Systems with Applications. 2022 Feb 26:116743.
[35] Ghojogh B, Ghodsi A, Karray F, Crowley M, “Generative adversarial networks and adversarial autoencoders: Tutorial and survey,” arXiv preprint arXiv:2111.13282. 2021 Nov 26.
[36] Baur C, Denner S, Wiestler B, Navab N, Albarqouni S, “Autoencoders for unsupervised anomaly segmentation in brain MR images: a comparative study,” Medical Image Analysis. 2021 Apr 1;69:101952.
[37] Liu K, Fu Y, Wu L, Li X, Aggarwal C, Xiong H. “Automated feature selection: A reinforcement learning perspective,” IEEE Transactions on Knowledge and Data Engineering. 2021 Sep 24.
[38] Feng J, Li D, Gu J, Cao X, Shang R, Zhang X, Jiao L, “Deep reinforcement learning for semisupervised hyperspectral band selection,” IEEE Transactions on Geoscience and Remote Sensing. 2021 Feb 19;60:1-9.
[39] Le N, Rathour VS, Yamazaki K, Luu K, Savvides M, “Deep reinforcement learning in computer vision: a comprehensive survey,” Artificial Intelligence Review. 2021 Sep 29:1-87.
[40] Kaur A, Guleria K, Trivedi NK, “Feature selection in machine learning: methods and comparison,” In2021 International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) 2021 Mar 4 (pp. 789-795). IEEE.
[41] Rasoul S, Adewole S, Akakpo A, “Feature selection using reinforcement learning,” arXiv preprint arXiv:2101.09460. 2021 Jan 23.
[42] Behura A, “The cluster analysis and feature selection: Perspective of machine learning and image processing. Data Analytics in Bioinformatics: A Machine Learning Perspective,” 2021 Feb 1:249-80.
[43] Obite F, Usman AD, Okafor E, “An overview of deep reinforcement learning for spectrum sensing in cognitive radio networks,” Digital Signal Processing. 2021 Jun 1;113:103014.
[44] Ebrahimi-Khusfi Z, Nafarzadegan AR, Dargahian F, “Predicting the number of dusty days around the desert wetlands in southeastern Iran using feature selection and machine learning techniques,” Ecological Indicators. 2021 Jun 1;125:107499.
[45] Pateria S, Subagdja B, Tan AH, Quek C, “Hierarchical reinforcement learning: A comprehensive survey,” ACM Computing Surveys (CSUR). 2021 Jun 5;54(5):1-35.
[46] Chakraborty A, Mitra S, De D, Pal AJ, Ghaemi F, Ahmadian A, Ferrara M, “Determining Protein–Protein Interaction Using Support Vector Machine: A Review,” IEEE Access. 2021 Jan 12;9:12473-90.
[47] Jueyendah S, Lezgy-Nazargah M, Eskandari-Naddaf H, Emamian SA, “Predicting the mechanical properties of cement mortar using the support vector machine approach,” Construction and Building Materials. 2021 Jul 12;291:123396.
How to Cite
Fadhil, A. A., Adnan, M., Radhi, H., Al-Mualm, M., Alubaidy, M. H., Salih, M., & Saadoon, S. J. (2022). Automatic Diagnosis of Breast Cancer in Histopathologic Images Based on Convolutional AutoEncoders and Reinforced Feature Selection. Majlesi Journal of Electrical Engineering. Retrieved from