Abstract
The aim of the work described here was to evaluate the diagnostic performance of ultrasound
thyroid computer-aided diagnosis (CAD) software. This multicenter prospective study
included 494 patients (565 thyroid nodules) who underwent surgery or biopsy after ultrasonography
at four hospitals from January 2019 to September 2019. The diagnostic performance
metrics of different readers were calculated and compared with the pathologic results.
The sensitivity of CAD was outstanding and was equivalent to that of a senior radiologist
(90.51% vs. 88.47%, p > 0.05). The area under the curve of CAD was equivalent to that of a junior radiologist
(0.748 vs. 0.739, p > 0.05). However, the specificity was only 49.63%, which was lower than those of
the three radiologists (75.56%, 85.93% and 90.37% for the junior, intermediate and
senior radiologists, respectively). The diagnostic performance of the junior radiologist
was significantly improved with the aid of CAD (junior + CAD). The sensitivity and
area under the curve of junior + CAD were improved from 72.20% to 89.93% and from
0.739 to 0.816, respectively (both p values <0.05), and the positive predictive value, negative predictive value and κ
coefficient improved from 76.3% to 78.6%, 82.0% to 86.8% and 0.394 to 0.511, respectively.
Though specificity slightly decreased from 75.56% to 73.33%, the difference was not
statistically significant (p > 0.05). In general, the clinical application value of CAD is promising, and its
instrumental value for junior radiologists is significant.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Ultrasound in Medicine and BiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- ThyroScreen system: High resolution ultrasound thyroid image characterization into benign and malignant classes using novel combination of texture and discrete wavelet transform.Comput Methods Programs Biomed. 2012; 107: 233-241
- Classification of benign and malignant thyroid nodules using wavelet texture analysis of sonograms.J Ultrasound Med. 2015; 34: 1983-1989
- Computer-aided detection (CAD) in mammography: Does it help the junior or the senior radiologist?.Eur J Radiol. 2005; 54: 90-96
- Fine needle aspiration biopsy in the management of thyroid nodules.Endocrinologist. 1991; 1: 194-202
Chang KJ, Chen WH, Chen A, Chen CN, Ho MC, Tai HC, Wu MH, Tsai PW. Method for retrieving a tumor contour of an image processing system. 2013 [P]: China, CN 102156874 B. 2015-07-29.
- Computer-aided diagnosis of liver tumors on computed tomography images.Comput Methods Programs Biomed. 2017; 145: 45-51
- Computerized quantification of ultrasonic heterogeneity in thyroid nodules.Ultrasound Med Biol. 2014; 40: 2581-2589
- Computer-aided diagnosis with deep learning architecture: Applications to breast lesions in US images and pulmonary nodules in CT scans.Sci Rep. 2016; 6: 1-13
- Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules.Thyroid. 2010; 20: 167-172
- The 2017 Bethesda System for Reporting Thyroid Cytopathology.Thyroid. 2017; 27: 1341-1346
- A predictive model to distinguish papillary thyroid carcinomas from benign thyroid nodules using ultrasonographic features: A single-center, retrospective analysis.Med Sci Monit. 2019; 25: 9409
- Flow pattern and vascular resistive index as predictors of malignancy risk in thyroid follicular neoplasms.J Ultrasound Med. 2005; 24: 897-904
- Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach.Biometrics. 1988; 44: 837-845
- Computer‐aided system for diagnosing thyroid nodules on ultrasound: A comparison with radiologist‐based clinical assessments.Head Neck. 2018; 40: 778-783
- Fine-needle aspiration biopsy of the thyroid: An appraisal.Ann Intern Med. 1993; 118: 282-289
- American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules—2016 update.Endocr Practice. 2016; 22: 1-60
- Validation of three scoring risk-stratification models for thyroid nodules.Thyroid. 2017; 27: 1550-1557
- Computer-aided diagnosis system for thyroid nodules on ultrasonography: Diagnostic performance and reproducibility based on the experience level of operators.Eur Radiol. 2019; 29: 1978-1985
- Community of Population-Based Regional Cancer Registries. Cancer statistics in Korea: Incidence, mortality, survival, and prevalence in 2015.Cancer Res Treat. 2018; 50: 303-316
- Man to man training: Can it help improve the diagnostic performances and interobserver variabilities of thyroid ultrasonography in residents?.Eur J Radiol. 2012; 81: e352-e356
- Real-world performance of computer-aided diagnosis system for thyroid nodules using ultrasonography.Ultrasound Med Biol. 2019; 45: 2672-2678
- Preoperative diagnosis of the follicular variant of papillary carcinoma of the thyroid: discrepancy between image and cytologic diagnoses.Radiat Med. 1994; 12: 293-299
- Large scale deep learning for computer aided detection of mammographic lesions.Med Image Anal. 2017; 35: 303-312
- Value of US correlation of a thyroid nodule with initially benign cytologic results.Radiology. 2010; 254: 292-300
- Thyroid Imaging Reporting and Data System for US features of nodules: A step in establishing better stratification of cancer risk.Radiology. 2011; 260: 892-899
- Image reporting and characterization system for ultrasound features of thyroid nodules: multicentric Korean retrospective study.Korean J Radiol. 2013; 14: 110-117
- A curated mammography data set for use in computer-aided detection and diagnosis research.Sci Data. 2017; 4170177
- Diagnostic accuracy of digital screening mammography with and without computer-aided detection.JAMA Intern Med. 2015; 175: 1828-1837
- Large-scale comparative analysis reveals a simple model to predict the prevalence of thyroid nodules.Risk Manag Healthc Policy. 2019; 12: 225
- Value of computer software for assisting sonographers in the diagnosis of Thyroid Imaging Reporting and Data System grade 3 and 4 thyroid space‐occupying lesions.J Ultrasound Med. 2019; 38: 3291-3300
- Computer-aided medical image annotation: Preliminary results with liver lesions in CT.IEEE J Biomed Health Informatics. 2017; 22: 1561-1570
- Note on the sampling error of the difference between correlated proportions or percentages.Psychometrika. 1947; 12: 153-157
- Computer-aided diagnosis of cirrhosis and hepatocellular carcinoma using multi-phase abdomen CT.Int J Computer Assisted Radiol Surg. 2019; 14: 1341-1352
- Computer-aided diagnosis of lung nodule classification between benign nodule, primary lung cancer, and metastatic lung cancer at different image size using deep convolutional neural network with transfer learning.PloS One. 2018; 13e0200721
- Usefulness of ultrasonography for diagnosis of malignant lymphoma of the thyroid.Thyroid. 2006; 16: 983-987
- Comparison between linear and nonlinear machine-learning algorithms for the classification of thyroid nodules.Eur J Radiol. 2019; 113: 251-257
- Interobserver agreement in assessing the sonographic and elastographic features of malignant thyroid nodules.AJR Am J Roentgenol. 2009; 193: W416-W423
- A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of breast masses on ultrasound: added value for the inexperienced breast radiologist.Medicine (Baltimore). 2019; 98: e14146
- Worldwide increasing incidence of thyroid cancer: update on epidemiology and risk factors.J Cancer Epidemiol. 2013; 2013965212
Perros P, Boelaert K, Colley S, Evans C, Evans RM, Gerrard BG, Gilbert J, Harrison B, Johnson SJ, Giles TE. British Thyroid Association Guidelines for the Management of Thyroid Cancer. 2014;81:1–122.
- 2015 American Thyroid Association guidelines for thyroid nodules and differentiated thyroid cancer and their implementation in various care settings.Thyroid. 2016; 26: 319-321
- Diagnostic performance evaluation of a computer-assisted imaging analysis system for ultrasound risk stratification of thyroid nodules.AJR Am J Roentgenol. 2019; 213: 169-174
- Risk stratification of thyroid nodules on ultrasonography with the French TI-RADS: Description and reflections.Ultrasonography. 2016; 35: 25
- Ultrasound-based risk stratification for malignancy in thyroid nodules: A four-tier categorization system.Eur Radiol. 2015; 25: 2153-2162
- Detection of subsolid nodules in lung cancer screening: Complementary sensitivity of visual reading and computer-aided diagnosis.Invest Radiol. 2018; 53: 441-449
- ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White paper of the ACR TI-RADS committee.J Am Coll Radiol. 2017; 14: 587-595
- Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: A systematic review.Comput Methods Programs Biomed. 2018; 156: 25-45
Article info
Publication history
Published online: October 23, 2020
Accepted:
September 22,
2020
Received in revised form:
September 19,
2020
Received:
February 26,
2020
Identification
Copyright
© 2020 Published by Elsevier Inc. on behalf of World Federation for Ultrasound in Medicine & Biology.
