Advertisement

Ultra-Attention: Automatic Recognition of Liver Ultrasound Standard Sections Based on Visual Attention Perception Structures

      Acquisition of a standard section is a prerequisite for ultrasound diagnosis. For a long time, there has been a lack of clear definitions of standard liver views because of physician experience. The accurate automated scanning of standard liver sections, however, remains one of ultrasonography medicine's most important issues. In this article, we enrich and expand the classification criteria of liver ultrasound standard sections from clinical practice and propose an Ultra-Attention structured perception strategy to automate the recognition of these sections. Inspired by the attention mechanism in natural language processing, the standard liver ultrasound views will participate in the global attention algorithm as modular local images in computer vision of ultrasound images, which will significantly amplify small features that would otherwise go unnoticed. In addition to using the dropout mechanism, we also use a Part-Transfer Learning training approach to fine-tune the model's rate of convergence to increase its robustness. The proposed Ultra-Attention model outperforms various traditional convolutional neural network-based techniques, achieving the best known performance in the field with a classification accuracy of 93.2%. As part of the feature extraction procedure, we also illustrate and compare the convolutional structure and the Ultra-Attention approach. This analysis provides a reasonable view for future research on local modular feature capture in ultrasound images. By developing a standard scan guideline for liver ultrasound-based illness diagnosis, this work will advance the research on automated disease diagnosis that is directed by standard sections of liver ultrasound.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Ultrasound in Medicine and Biology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Gangarosa LM.
        The practice of ultrasound: a step-by-step guide to abdominal scanning.
        Gastroenterology. 2005; 129: 1357
        • Spencer JK
        • Adler RS.
        Utility of portable ultrasound in a community in Ghana.
        J Ultrasound Med. 2008; 27: 1735-1743
        • Bamber JC
        • Miller NR
        • Tristam M.
        Diagnostic ultrasound.
        (editor)in: Flower M Webb's physics of medical imaging. 2nd ed. CRC Press, Boca Raton, FL2012
        • Wu B
        • Liu P
        • Wu H
        • Liu S
        • He S
        • Lv G.
        An Effective Machine-Learning Based Feature Extraction/Recognition Model for Fetal Heart Defect Detection from 2D Ultrasonic Imageries.
        Comput Model Eng Sci. 2022; 134: 1069-1089
        • Guo M
        • Wang K
        • Liu S
        • Du Y
        • Liu P
        • Su Q
        • Lv G.
        Recognition of thyroid ultrasound standard plane images based on residual network.
        Computational Intell Neurosci. 2021; 2021e5598001
        • Lacuna Y
        • Bengio Y
        • Hinton G.
        Deep learning.
        Nature. 2015; 521: 436-444
        • Halder A
        • Chatterjee S
        • Dey D.
        Adaptive morphology aided 2-pathway convolutional neural network for lung nodule classification.
        Biomed Signal Process Control. 2022; 72103347
        • Hu MK.
        Visual pattern recognition by moment invariants.
        IRE Trans Inf Theory. 1962; 8: 179-187
        • Huang YL
        • Chen DR
        • Jiang YR
        • Kuo SJ
        • Wu HK
        • Moon WK.
        Computer-aided diagnosis using morphological features for classifying breast lesions on ultrasound.
        Ultrasound Obstet Gynecol. 2008; 32: 565-572
        • Karimi B
        • Krzyżak A.
        A novel approach for automatic detection and classification of suspicious lesions in breast ultrasound images.
        J Artif Intell Soft Comput Res. 2013; : 265-276
        • Liu Y
        • Ren L
        • Cao X
        • Tong Y.
        Breast tumors recognition based on edge feature extraction using support vector machine.
        Biomed Signal Process Control. 2020; 58101825
        • Zhou S
        • Shi J
        • Zhu J
        • Cai Y
        • Wang R.
        Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image.
        Biomed Signal Process Control. 2013; 8: 688-696
        • Gao S
        • Peng Y
        • Guo H
        • Liu W
        • Gao T
        • Xu Y
        • Tang X.
        Texture analysis and classification of ultrasound liver images.
        Bio-Med Mater Eng. 2014; 24: 1209-1216
        • Hafizah WM
        • Supriyanto E
        • Yunus J.
        Feature extraction of kidney ultrasound images based on intensity histogram and gray level co-occurrence matrix.
        in: Proceedings, 2012 Sixth Asia Modelling Symposium, New York: IEEE2012: 115-120
        • Nugroho HA
        • Ramaswamy M
        • Triyani Y
        • Radiant I.
        Texture analysis for classification of thyroid ultrasound images.
        in: Proceedings, 2016 International Electronics Symposium (IES), New York: IEEE2016: 476-480
        • Djunaidi K
        • Agtriadi HB
        • Kuswardani D
        • Purwanto YS.
        Gray level co-occurrence matrix feature extraction and histogram in breast cancer classification with ultrasonographic imagery.
        Indonesian J Elect Eng Comput Sci. 2021; 22: 795
        • Ramamurthy B
        • Chandran KR.
        Content based medical image retrieval with texture content using gray level co-occurrence matrix and K-means clustering algorithms.
        J Comput Sci. 2012; 8: 1070-1076
        • Pradeep S
        • Nirmaladevi P.
        A review on speckle noise reduction techniques in ultrasound medical images based on spatial domain, transform domain and CNN methods.
        2021 IOP Conf Ser Mater Sci Eng. 2021; 1055012116
        • Wagih M
        • Abou-Chadi FEZ
        • El-Din H
        • Mekky N.
        Classification of ultrasound kidney images using PCA and neural networks.
        Int J Adv Comput Sci Appl. 2015; 6
        • Jégou H
        • Douze M
        • Schmid C
        • Pérez P.
        Aggregating local descriptors into a compact image representation.
        in: Proceedings, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York: IEEE2010: 3304-3311
        • Perronnin F
        • Sánchez J
        • Wensink T.
        • Daniilidis K
        • Maragos P
        • Paragios N
        Improving the Fisher kernel for large-scale image classification.
        Computer Vision—ECCV 2010. Springer, Berlin/Heidelberg2010: 143-156
        • Lei B
        • Tan EL
        • Chen S
        • Zhou L
        • Li S
        • Ni D
        • et al.
        Automatic recognition of fetal facial standard plane in ultrasound image via Fisher vector.
        PLoS One. 2015; 10e0121838
        • Byra M
        • Jarosik P
        • Szubert A
        • Galperin M
        • Ojeda-Fournier H
        • Olson L
        • et al.
        Breast mass segmentation in ultrasound with selective kernel U-Net convolutional neural network.
        Biomed Signal Process Control. 2020; 61102027
        • Rawat S
        • Rana KPS
        • Kumar V.
        A novel complex-valued convolutional neural network for medical image denoising.
        Biomed Signal Process Control. 2021; 69102859
        • Zhang Z
        • Li Y
        • Wu W
        • Chen H
        • Cheng L
        • Wang S.
        Tumor detection using deep learning method in automated breast ultrasound.
        Biomed Signal Process Control. 2021; 68102677
        • He K
        • Zhang X
        • Ren S
        • Sun J.
        Deep Residual learning for image recognition.
        in: Proceedings, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York: IEEE2016
        • Krizhevsky A
        • Sutskever I
        • Hinton GE.
        Imagenet classification with deep convolutional neural networks.
        in: Pereira F Burges CJ Bottou L Weinberger KQ Advances in neural information processing systems. Curran Associates, Red Hook, NY2012: 25
      1. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv 1409.1556. Available at: https://arxiv.org/abs/1409.1556.

        • Szegedy C
        • Liu W
        • Jia Y
        • Sermanet P
        • Reed S
        • Anguelov D
        • et al.
        Going deeper with convolutions.
        in: Proceedings, IEEE Conference on Computer Vision and Pattern Recognition, New York: IEEE2015: 1-9
        • Tan M
        • Le Q
        Efficientnet: Rethinking model scaling for convolutional neural networks.
        PMLR. 2019; : 6105-6114
        • Yu Z
        • Jiang X
        • Zhou F
        • Qin J
        • Ni D
        • Chen S
        • Lei B
        • Wang T.
        Melanoma recognition in dermoscopy images via aggregated deep convolutional features.
        IEEE Trans Biomed Eng. 2019; 66: 1006-1016
        • Chen H
        • Dou Q
        • Ni D
        • Cheng JZ
        • Qin J
        • Li S
        • et al.
        Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks.
        in: Proceedings, Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Springer, Cham2015: 507-514
        • Rhyou SY
        • Yoo JC.
        Cascaded deep learning neural network for automated liver steatosis diagnosis using ultrasound images.
        Sensors. 2021; 21: 5304
        • Ryu H
        • Shin SY
        • Lee JY
        • Lee KM
        • Kang H
        • Yi J.
        Joint segmentation and classification of hepatic lesions in ultrasound images using deep learning.
        Eur Radiol. 2021; 31: 8733-8742
        • Ahn JC
        • Qureshi TA
        • Singal AG
        • Li D
        • Yang JD.
        Deep learning in hepatocellular carcinoma: current status and future perspectives.
        World J Hepatol. 2021; 13: 2039-2051
        • Wu J
        • Zeng P
        • Liu P
        • Lv G.
        Automatic classification method of liver ultrasound standard plane images using pre-trained convolutional neural network.
        Connect Sci. 2022; 34: 975-989
      2. Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. arXiv 1412.6572. Available at: http://arxiv.org/abs/1412.6572.

        • Singh A
        • Sengupta S
        • Lakshminarayanan V.
        Explainable deep learning models in medical image analysis.
        J Imaging. 2020; 6: 52
      3. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv 1409.0473. Available at: http://arxiv.org/abs/1409.0473.

        • Vaswani A
        • Shazeer N
        • Parmar N
        • Uszkoreit J
        • Jones L
        • Gomez AN
        • et al.
        Advances in neural information processing systems. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA. Red Hook, NY: Curran Associates2017
      4. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16 x 16 words: transformers for image recognition at scale. arXiv 201011929. Available at: http://arxiv.org/abs/2010.11929.

        • Lin TY
        • Goyal P
        • Girshick R
        • He K
        • Dollar P.
        Focal loss for dense object detection.
        in: Proceedings, IEEE International Conference on Computer Vision (ICCV), New York: IEEE2017
        • Yu H
        • Li J
        • Sun J
        • Zheng J
        • Wang S
        • Wang G
        • Ding Y
        • Zhao J
        • Zhang J.
        Intelligent diagnosis algorithm for thyroid nodules based on deep learning and statistical features.
        Biomed Signal Process Control. 2022; 78103924
        • Amiri M
        • Brooks R
        • Rivaz H.
        Fine-tuning U-Net for ultrasound image segmentation: different layers, different outcomes.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2020; 67: 2510-2518
        • Dadoun H
        • Rousseau AL
        • de Kerviler E
        • Correas JM
        • Tissier AM
        • Joujou F
        • et al.
        Deep learning for the detection, localization, and characterization of focal liver lesions on abdominal US Images.
        Radiol Artif Intell. 2022; 4e210110
        • Chen X
        • Wang X
        • Zhang K
        • Fung KM
        • Thai TC
        • Moore K
        • et al.
        Recent advances and clinical applications of deep learning in medical image analysis.
        Med Image Anal. 2022; 79102444
        • Selvaraju RR
        • Cogswell M
        • Das A
        • Vedantam R
        • Parikh D
        • Batra D.
        Grad-CAM: visual explanations from deep networks via gradient-based localization.
        Int J Comput Vision. 2017; 128: 336-359