Advertisement

Coherence-Based Beamforming Increases the Diagnostic Certainty of Distinguishing Fluid from Solid Masses in Breast Ultrasound Exams

Open AccessPublished:February 29, 2020DOI:https://doi.org/10.1016/j.ultrasmedbio.2020.01.016

      Abstract

      Ultrasound is often used as a supplement for mammography to detect breast cancer. However, one known limitation is the high false-positive rates associated with breast ultrasound. We investigated the use of coherence-based beamforming (which directly displays spatial coherence) as a supplement to standard ultrasound B-mode images in 25 patients recommended for biopsy (26 masses in total), with the eventual goal of decreasing false-positive rates. Because of the coherent signal present within solid masses, coherence-based beamforming methods allow solid and fluid-filled masses to appear significantly different (p < 0.001). When presented to five board-certified radiologists, the inclusion of robust short-lag spatial coherence (R-SLSC) images in the diagnostic pipeline reduced the uncertainty of fluid-filled mass contents from 47.5% to 15.8% and reduced the percentage of fluid-filled masses unnecessarily recommended for biopsy from 43.3% to 13.3%. These results are promising for the potential introduction of R-SLSC (and related coherence-based beamforming methods) into the breast clinic to improve diagnostic certainty and reduce the number of unnecessary biopsies.

      Key Words

      Introduction

      According to the American Cancer Society, one in eight women will be diagnosed with breast cancer during their lifetimes (
      • Siegel R.L.
      • Miller K.D.
      • Jemal A.
      Cancer statistics, 2019.
      ), making early diagnosis increasingly important to improve prognosis. Currently, mammography is the most widely used screening modality, and it is considered one of the most important imaging methods to reduce the likelihood of death from breast cancer (
      • DeSantis C.
      • Siegel R.
      • Bandi P.
      • Jemal A.
      Breast cancer statistics, 2011.
      ). However, mammographic sensitivity (i.e., the ability of a mammogram to detect cancer) is significantly lower in patients with dense breast tissue (
      • Mandelson M.T.
      • Oestreicher N.
      • Porter P.L.
      • White D.
      • Finder C.A.
      • Taplin S.H.
      • White E.
      Breast density as a predictor of mammographic detection: Comparison of interval- and screen-detected cancers.
      ). When breast tissue is dense, the surrounding parenchyma can often obscure diagnostic features in the mammogram, necessitating a supplement to improve detection.
      Various studies have examined the impact of using other imaging modalities to supplement screening mammography, such as magnetic resonance imaging (MRI), molecular breast imaging (MBI), contrast-enhanced mammography (CEM) and ultrasound. A study by
      • Kuhl C.K.
      • Schrading S.
      • Leutner C.C.
      • Morakkabati-Spitz N.
      • Wardelmann E.
      • Fimmers R.
      • Kuhn W.
      • Schild H.H.
      Mammography, breast ultrasound, and magnetic resonance imaging for surveillance of women at high familial risk for breast cancer.
      found the sensitivity of MRI to be 90.7% alone and 93.0% when combined with mammography for women at high familial risk for breast cancer. However, breast MRI requires the use of gadolinium-based contrast agents, which carry potential risks of retention in the brain that are currently being investigated (
      • Kartamihardja A.A.P.
      • Nakajima T.
      • Kameo S.
      • Koyama H.
      • Tsushima Y.
      Distribution and clearance of retained gadolinium in the brain: Differences between linear and macrocyclic gadolinium based contrast agents in a mouse model.
      ;
      • Murata N.
      • Gonzalez-Cuyar L.F.
      • Murata K.
      • Fligner C.
      • Dills R.
      • Hippe D.
      • Mar- avilla K.R.
      Macrocyclic and other non-group 1 gadolinium contrast agents deposit low levels of gadolinium in brain and bone tissue: Preliminary results from 9 patients with normal renal function.
      ; Food and Drug Administration [

      Food and Drug Administration (FDA). FDA Drug Safety Communication: FDA warns that gadolinium-based contrast agents (GBCAs) are retained in body; requires new class warnings. 2017. Available at:https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-warns-gadolinium-based-contrast-agents-gbcas-are-retained-body.

      ;
      • McDonald R.J.
      • McDonald J.S.
      • Kallmes D.F.
      • Jentoft M.E.
      • Paolini M.A.
      • Murray D.L.
      • Williamson E.E.
      • Eckel L.J.
      Gadolinium deposition in human brain tissues after contrast-enhanced MR imaging in adult patients without intracranial abnormalities.
      ). MBI has been reported to find an additional 7.7 cancers per 1000 women, but there are concerns over breast radiation as well as patient comfort (
      • Shermis R.B.
      • Wilson K.D.
      • Doyle M.T.
      • Martin T.S.
      • Merryman D.
      • Kudrolli H.
      • Brenner R.J.
      Supplemental breast cancer screening with molecular breast imaging for women with dense breast tissue.
      ;
      • Hruska C.B.
      Molecular breast imaging for screening in dense breasts: State of the art and future directions.
      ). CEM, another emerging modality, offers sensitivity similar to that of MRI and MBI, but at a lower cost (
      • Patel B.K.
      • Gray R.J.
      • Pockaj B.A.
      Potential cost savings of contrast-enhanced digital mammography.
      ;
      • Covington M.F.
      • Pizzitola V.J.
      • Lorans R.
      • Pockaj B.A.
      • Northfelt D.W.
      • Appleton C.M.
      • Patel B.K.
      The future of contrast-enhanced mammography.
      ). Still, CEM requires the use of ionizing radiation, as well as iodinated contrast agent, which has the potential to cause an allergic reaction (
      • Lewis T.C.
      • Pizzitola V.J.
      • Giurescu M.E.
      • Eversman W.G.
      • Lorans R.
      • Robinson K.A.
      • Patel B.K.
      Contrast-enhanced digital mammography: A single-institution experience of the first 208 cases.
      ).
      As an alternative, ultrasound does not use ionizing radiation, does not require intravenous contrast injection and is painless, real time and highly portable. The major downside of ultrasound for use as a breast screening modality is its high false-positive rate, sometimes resulting in patients having to undergo unnecessary biopsies (
      • Berg W.A.
      • Bandos A.I.
      • Mendelson E.B.
      • Lehrer D.
      • Jong R.A.
      • Pisano E.D.
      Ultrasound as the primary screening test for breast cancer: Analysis from ACRIN 6666.
      ). In the American College of Radiology Imaging Network (ACRIN) 6666 trial, more false positives were seen with increasing breast density in ultrasound (e.g., 10% false positives with <25% density steadily increasing to 14.4% false positives with breast density >80%) (
      • Berg W.A.
      • Bandos A.I.
      • Mendelson E.B.
      • Lehrer D.
      • Jong R.A.
      • Pisano E.D.
      Ultrasound as the primary screening test for breast cancer: Analysis from ACRIN 6666.
      ). This trend is indicative of an interaction between dense breast tissue and the effectiveness of ultrasound. When tissue is dense, the speed of sound changes between tissue layers, causing a phenomenon called acoustic clutter (
      • Lediju M.A.
      • Pihl M.J.
      • Dahl J.J.
      • Trahey G.E.
      Quantitative assessment of the magnitude, impact and spatial extent of ultrasonic clutter.
      ), resulting in diagnostic uncertainty.
      To combat this high false-positive rate and reduce the number of unnecessary biopsies, investigators have utilized ultrasound to gain stiffness information on the tissue in question, including techniques such as acoustic streaming (
      • Nightingale K.R.
      • Kornguth P.J.
      • Trahey G.E.
      The use of acoustic streaming in breast lesion diagnosis: A clinical study.
      ), strain elastography (
      • Garra B.S.
      • Cespedes E.I.
      • Ophir J.
      • Spratt S.R.
      • Zuurbier R.A.
      • Magnant C.M.
      • Pennanen M.F.
      Elastography of breast lesions: Initial clinical results.
      ), acoustic radiation force impulse imaging (
      • Nightingale K.
      • Soo M.S.
      • Nightingale R.
      • Trahey G.
      Acoustic radiation force impulse imaging: In vivo demonstration of clinical feasibility.
      ;
      • Sharma A.C.
      • Soo M.S.
      • Trahey G.E.
      • Nightingale K.R.
      Acoustic radiation force impulse imaging of in vivo breast masses.
      ), shear wave elastography (SWE) (
      • Evans A.
      • Whelehan P.
      • Thomson K.
      • McLean D.
      • Brauer K.
      • Purdie C.
      • Jordan L.
      • Baker L.
      • Thompson A.
      Quantitative shear wave ultrasound elastography: Initial experience in solid breast masses.
      ;
      • Berg W.A.
      • Cosgrove D.O.
      • Doré C.J.
      • Schäfer F.K.
      • Svensson W.E.
      • Hooley R.J.
      • Ohlinger R.
      • Mendelson E.B.
      • Balu-Maestro C.
      • Locatelli M.
      • Tourasse C.
      • Cavanaugh B.C.
      • Juhan V.
      • Stavros A.T.
      • Tardivon A.
      • Gay J.
      • Henry J.P.
      • Cohen-Bacrie C.
      BE1 Investigators
      Shear-wave elastography improves the specificity of breast us: the BE1 multinational study of 939 masses.
      ;
      • Han J.
      • Li F.
      • Peng C.
      • Huang Y.
      • Lin Q.
      • Liu Y.
      • Cao L.
      • Zhou J.
      Reducing unnecessary biopsy of breast lesions: Preliminary results with combination of strain and shear-wave elastography.
      ) and supersonic shear wave imaging (SSI) (
      • Bercoff J.
      • Tanter M.
      • Fink M.
      Supersonic shear imaging: A new technique for soft tissue elasticity mapping.
      ;
      • Athanasiou A.
      • Tardivon A.
      • Tanter M.
      • Sigal-Zafrani B.
      • Bercoff J.
      • Deffieux T.
      • Gennisson J.L.
      • Fink M.
      • Neuenschwander S.
      Breast lesions: Quantitative elastography with supersonic shear imaging—Preliminary results.
      ), which each aim to measure the elastic response or properties of tissue by inducing small displacements. Specifically, for a preliminary study population of 21 indeterminate lesions,
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      reported 50% sensitivity and 100% specificity for cyst detection with acoustic streaming. In addition,
      • Berg W.A.
      • Cosgrove D.O.
      • Doré C.J.
      • Schäfer F.K.
      • Svensson W.E.
      • Hooley R.J.
      • Ohlinger R.
      • Mendelson E.B.
      • Balu-Maestro C.
      • Locatelli M.
      • Tourasse C.
      • Cavanaugh B.C.
      • Juhan V.
      • Stavros A.T.
      • Tardivon A.
      • Gay J.
      • Henry J.P.
      • Cohen-Bacrie C.
      BE1 Investigators
      Shear-wave elastography improves the specificity of breast us: the BE1 multinational study of 939 masses.
      reported improved specificity of 78.5% using SWE compared with B-mode, with implications for avoiding biopsy of complicated cysts. With the goal of reducing unnecessary biopsies,
      • Athanasiou A.
      • Tardivon A.
      • Tanter M.
      • Sigal-Zafrani B.
      • Bercoff J.
      • Deffieux T.
      • Gennisson J.L.
      • Fink M.
      • Neuenschwander S.
      Breast lesions: Quantitative elastography with supersonic shear imaging—Preliminary results.
      used SSI to modify the Breast Imaging Reporting and Data System (BI-RADS) score, demonstrating the potential avoidance of 46% of unnecessary biopsies.
      Overall, these elasticity-based methods have been successful at distinguishing solid from fluid-filled lesions; however, they are challenging for cysts with very thick fluid and small masses deep in tissue (
      • Nightingale K.R.
      • Kornguth P.J.
      • Trahey G.E.
      The use of acoustic streaming in breast lesion diagnosis: A clinical study.
      ;
      • Garra B.S.
      Imaging and estimation of tissue elasticity by ultrasound.
      ). In addition, acoustic radiation force-based and shear wave imaging methods require altered, high-energy transmit pulse sequences that are typically unable to be supported by miniaturized ultrasound systems, and they also depend on loading conditions introduced by the probe pressure required to maintain acoustic coupling and contact (
      • Barr R.G.
      Shear wave imaging of the breast still on the learning curve.
      ;
      • Barr R.G.
      • Zhang Z.
      Effects of precompression on elasticity imaging of the breast.
      ;
      • Balleyguier C.
      • Canale S.
      • Hassen W.B.
      • Vielh P.
      • Bayou E.
      • Mathieu M.
      • Uzan C.
      • Bourgier C.
      • Dromain C.
      Breast elasticity: Principles, technique, results: An update and overview of commercially available software.
      ), which complicates quantitative inter- and intra-patient longitudinal studies, as well as quantitative comparisons of results from different ultrasound system manufacturers (
      • Bell M.A.L.
      • Kumar S.
      • Kuo L.
      • Sen H.T.
      • Iordachita I.
      • Kazanzides P.
      Toward standardized acoustic radiation force (ARF)-based ultrasound elasticity measurements with robotic force control.
      ).
      Similar to elastography, quantitative ultrasound aims to discover information about the underlying microstructure of tissue by using spectral-based parameters, such as the backscatter coefficient, effective scatterer diameter and effective acoustic concentration, as well as envelope statistics through the fitting of models, such as the homodyned-K distribution (
      • Oelze M.L.
      • Mamou J.
      Review of quantitative ultrasound: Envelope statistics and backscatter coefficient imaging and contributions to diagnostic ultrasound.
      ). When combined, these methods have been successful in characterizing breast lesions (
      • Shankar P.M.
      • Dumane V.
      • Reid J.M.
      • Genis V.
      • Forsberg F.
      • Piccoli C.W.
      • Goldberg B.B.
      Classification of ultrasonic B-mode images of breast masses using Nakagami distribution.
      ;
      • Oelze M.L.
      • O'Brien Jr, W.D.
      • Zachary J.F.
      11B-4 quantitative ultrasound assessment of breast cancer using a multiparameter approach.
      ;
      • Nam K.
      • Zagzebski J.A.
      • Hall T.J.
      Quantitative assessment of in vivo breast masses using ultrasound attenuation and backscatter.
      ;
      • Sadeghi-Naini A.
      • Papanicolau N.
      • Falou O.
      • Zubovits J.
      • Dent R.
      • Verma S.
      • Trudeau M.
      • Boileau J.F.
      • Spayne J.
      • Iradji S.
      • Sofroni E.
      • Lee J.
      • Lemon-Wong S.
      • Yaffe M.
      • Kolios M.C.
      • Czarnota G.J.
      Quantitative ultrasound evaluation of tumor cell death response in locally advanced breast cancer patients receiving chemotherapy.
      ;
      • Trop I.
      • Destrempes F.
      • El Khoury M.
      • Robidoux A.
      • Gaboury L.
      • Allard L.
      • Chayer B.
      • Cloutier G.
      The added value of statistical modeling of backscatter properties in the management of breast lesions at us.
      ;
      • Sannachi L.
      • Tadayyon H.
      • Sadeghi-Naini A.
      • Tran W.
      • Gandhi S.
      • Wright F.
      • Oelze M.
      • Czarnota G.
      Non-invasive evaluation of breast cancer response to chemotherapy using quantitative ultrasonic backscatter parameters.
      ); however, they require careful calibration using reference phantoms, are computationally complex, and are often model based, which can cause gross estimation errors (
      • Oelze M.L.
      • Mamou J.
      Review of quantitative ultrasound: Envelope statistics and backscatter coefficient imaging and contributions to diagnostic ultrasound.
      ).
      Rather than inducing external forces or requiring custom calibrations, advanced beamforming methods aimed at reducing acoustic clutter are particularly well suited to improve image quality and offer the promise of improving the diagnostic power of ultrasound, as seen with spatial compounding (
      • Entrekin R.
      • Jackson P.
      • Jago J.
      • Porter B.
      Real time spatial compound imaging in breast ultrasound: Technology and early clinical experience.
      ;
      • Huber S.
      • Wagner M.
      • Medl M.
      • Czembirek H.
      Real-time spatial compound imaging in breast ultrasound.
      ). In addition, tissue harmonic imaging (THI), which displays higher-frequency harmonics of the transmitted ultrasound wave, is known to assist with characterization of simple cysts. One reason for this assistance is the removal of reverberation echoes in fluid masses when these echoes have the same frequency content as the transmitted ultrasound wave (
      • Hooley R.J.
      • Scoutt L.M.
      • Philpotts L.E.
      Breast ultrasonography: State of the art.
      ). However, existing literature is mixed on the overall benefit of THI for all patients. For example,
      • Mesurolle B.
      • Bining H.J.
      • Khoury M.E.
      • Barhdadi A.
      • Kao E.
      Contribution of tissue harmonic imaging and frequency compound imaging in interventional breast sonography.
      cites difficulty visualizing masses in tissue that is not purely fatty, and
      • Cha J.H.
      • Moon W.K.
      • Cho N.
      • Kim S.M.
      • Park S.H.
      • Han B.K.
      • Choe Y.H.
      • Park J.M.
      • Im J.G.
      Characterization of benign and malignant solid breast masses: Comparison of conventional us and tissue harmonic imaging.
      reports shadowing from normal structure in dense breasts, more conspicuous acoustic shadowing and decreased penetration depth with THI.
      Short-lag spatial coherence (SLSC) and robust short-lag spatial coherence (R-SLSC) beamforming can both improve ultrasound image quality by displaying the spatial coherence of backscattered pressure waves instead of traditional brightness information. Since its creation as a clutter reduction technique, SLSC has been applied to a variety of in vivo tissue types including thyroid (
      • Lediju M.A.
      • Trahey G.E.
      • Byram B.C.
      • Dahl J.J.
      Short-lag spatial coherence of backscattered echoes: Imaging characteristics.
      ), cardiac (
      • Bell M.A.L.
      • Goswami R.
      • Kisslo J.A.
      • Dahl J.J.
      • Trahey G.E.
      Short-lag spatial coherence imaging of cardiac ultrasound data: Initial clinical results.
      ), fetal (
      • Kakkad V.
      • Dahl J.
      • Ellestad S.
      • Trahey G.
      In vivo application of short-lag spatial coherence and harmonic spatial coherence imaging in fetal ultrasound.
      ;
      • Long W.
      • Hyun D.
      • Choudhury K.R.
      • Bradway D.
      • McNally P.
      • Boyd B.
      • Ellestad S.
      • Trahey G.E.
      Clinical utility of fetal short-lag spatial coherence imaging.
      ) and liver (
      • Jakovljevic M.
      • Trahey G.E.
      • Nelson R.C.
      • Dahl J.J.
      In vivo application of short-lag spatial coherence imaging in human liver.
      ) data. In addition, our previous work investigated the first known application of SLSC to breast data (
      • Wiacek A.
      • Falomo E.
      • Myers K.
      • Rindal O.M.H.
      • Fabrega-Foster K.
      • Harvey S.
      • Bell M.A.L.
      Clinical feasibility of coherence-based beamforming to distinguish solid from fluid hypoechoic breast masses.
      ,
      • Wiacek A.
      • Rindal O.M.H.
      • Falomo E.
      • Myers K.
      • Fabrega-Foster K.
      • Harvey S.
      • Lediju Bell M.A.
      Robust short-lag spatial coherence imaging of breast ultrasound data: Initial clinical results.
      ), reporting improved differentiation between solid and fluid masses. In this article, we present the clinical advantages of R-SLSC beamforming applied to breast data and assess its ability to increase diagnostic certainty and reduce the number of unnecessary biopsies.

      Methods

      Study population

      Twenty-five patients containing 26 masses in total were enrolled in our ongoing study after receiving informed consent and approval from the Johns Hopkins Medicine Institutional Review Board (Protocol No. IRB00127110). Patients ranged from 18–80 y in age, with a mean age of 55 y. Patients were identified as having at least one suspicious mass with diagnostic ultrasound and were scheduled for an ultrasound-guided core-needle biopsy. Patients were imaged prior to the biopsy, and pathology results were used as the ground truth for statistical calculations. In the case of surgical excision after core-needle biopsy, the ground truth pathology was updated to the surgical pathology found at excision. Masses that were successfully aspirated during the biopsy procedure were designated as fluid filled. Solid masses underwent core-needle biopsy and were further separated based on specific pathology.

      Data acquisition

      Each patient was scanned using an Alpinion ECUBE12R, which is a dual research and clinical ultrasound scanner (Alpinion, Seoul, Korea). This scanner was connected to an Alpinion L8-17 or Alpinion L3-8 ultrasound transducer with a transmit frequency of 12.5 or 8 MHz, respectively. We performed fundamental imaging rather than THI to prioritize higher transmit frequencies within the –6-dB transducer bandwidths (i.e., 8–17 or 3–8 MHz, respectively). Each lesion was insonified using 256 focused transmissions with a focus selected by our radiologist co-authors who performed the scans. The raw radiofrequency (RF) channel data to form a minimum of 40 frames were saved for offline processing post-procedure. In addition to the raw echo data, clinical screenshots of ultrasound B-mode images were concurrently acquired to match each RF channel data acquisition.

      Coherence-based beamforming

      SLSC beamforming

      SLSC (
      • Lediju M.A.
      • Trahey G.E.
      • Byram B.C.
      • Dahl J.J.
      Short-lag spatial coherence of backscattered echoes: Imaging characteristics.
      ) relies on the spatial coherence of backscattered pressure waves that are received across the ultrasound transducer. Once the data are delayed to compensate for time-of-arrival differences, normalized correlation measurements are calculated between equally spaced elements, or lags, resulting in the normalized spatial correlation
      R^(m)=1Nmi=1Nmn=n1n2si(n)si+m(n)n=n1n2si2(n)n=n1n2si+m2(n)
      (1)


      where N is the number of receive elements in the transducer, m is the lag and si(n) is a time-delayed, zero-mean signal received at element i from depth n.
      This resulting spatial coherence function is then summed up to a specific short-lag value, M, yielding the value of the SLSC pixel. M is commonly taken to be a value of ≤20 for in vivo breast images.
      Rsl=1MR^(m)dmm=1MR^(m)
      (2)


      This process is repeated for each lateral and axial position in the image, with an axial correlation kernel of size k=n2n1 (
      • Bell M.A.L.
      • Dahl J.J.
      • Trahey G.E.
      Resolution and brightness characteristics of short-lag spatial coherence (SLSC) images.
      ). Throughout this work, a kernel of 1.56 times the wavelength associated with the center frequency of the probe was used.

      Robust SLSC beamforming

      To take advantage of the sparse, high-frequency information contained at higher lag values, R-SLSC (
      • Nair A.A.
      • Tran T.D.
      • Bell M.A.L.
      Robust short-lag spatial coherence imaging.
      ) denoises the coherence information by using robust principal component analysis (RPCA). To accomplish this goal, lag images generated using the original SLSC algorithm were first vectorized and stacked to form a matrix, D. RPCA was then performed on this matrix to solve for A in the expression
      D=A+E
      (3)


      where A is the underlying low-rank matrix that estimates the ground truth, and E is an error matrix that is considered to be sparse with possible high-magnitude errors, which is characteristic of spatial coherence functions at higher lag values (when compared with the errors observed at lower lag values) (
      • Nair A.A.
      • Tran T.D.
      • Bell M.A.L.
      Robust short-lag spatial coherence imaging.
      ). After implementation of a series of simplification steps and a convex relaxation, this RPCA problem statement is formulated as a Lagrangian and solved by minimizing the Lagrangian objective function L(A, E, Y, p), which is known as the augmented Lagrange multiplier method (

      Lin Z, Chen M, Ma Y. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055, 2010.

      ):
      L(A,E,Y,μ)=A*+λE1+Y,DAE+μ2DAEF2
      (4)


      Here Y is a matrix of Lagrange multipliers, μ is a positive scalar, λ is the sparsity penalty parameter that is varied to smooth tissue texture, ‖ · ‖1 is the L1 norm, ‖ · ‖F is the Frobenius norm and ‖ · ‖* is the nuclear norm.
      After the application of RPCA, the resulting matrix, A, is then weighted according to a linearly decreasing weighting scheme where the lag 1 image is given a weight of 1 down to the lag 1 image given a weight of 1/M. After the weighted summation, the initial vectorization was reversed to form the final R-SLSC image.
      The M and λ values were selected by matching the R-SLSC signal-to-noise ratio (SNR) to the B-mode speckle SNR to present images that are quantitatively more similar. When multiple-parameter combinations were possible, contrast was used as a secondary metric to optimize the selected parameters. These metrics are defined in the next subsection.

      Performance metrics

      Contrast, SNR and contrast difference were measured and compared across matched B-mode and coherence-based images created with the same channel data according to the equations
      Contrast=20log(SiSo)
      (5)


      SNR=Soσo
      (6)


      ContrastDifference=ContrastBmodeContrastRSLSC
      (7)


      where Si and σi are the mean and standard deviation, respectively, within a region of interest (ROI) inside of the target prior to log-compression, and So and σo are the mean and standard deviation, respectively, of a ROI outside of the target prior to log-compression. The contrast subscripts in eqn (7) indicate the beamforming method used for the contrast measurement.

      Reader study classification system

      RF data from each patient were saved and processed offline to generate matched B-mode and R-SLSC images for each mass of clinical interest. The masses were then randomized and reviewed independently by five board-certified breast radiologists, including three co-authors of this paper (E.O., K.M. and L.M). Each reader had between 1 and 22 years of experience beyond residency. Two of the radiologists assisted with image acquisition (Readers 1 and 2). To include readers who had no prior knowledge of the specific patients enrolled in our study and no prior experience with the R-SLSC imaging technology, the remaining three readers (Readers 3, 4 and 5) did not participate in patient recruitment or imaging. Prior to reviewing the images, each radiologist was given a brief tutorial on R-SLSC image interpretation to understand the images presented.
      The masses were randomized and presented to the readers under radiology reading room conditions using the graphical user interface illustrated in Figure 1. The readers were instructed to assume that no prior imaging studies were available for each mass, for us to make the assumption that variations in responses were owing solely to observed differences in the presented images. For each patient, the readers were presented with two tasks (i.e., Task 1 followed by Task 2) prior to advancing to the next patient.
      Fig 1
      Fig. 1Annotated screenshot of the graphical user interface used by each reader to classify masses according to content and BI-RADS category. Task 1, outlined in red, indicates the first classification task using only the B-mode images; the regions outlined in blue and green are hidden during this task. Task 2, outlined in blue, indicates the second classification task using both the B-mode images from Task 1 and the added R-SLSC information. The green outline indicates the optional R-SLSC overlay. BI-RADS = Breast Imaging Reporting and Data System; R-SLSC = robust short-lag spatial coherence.
      For Task 1, only B-mode images were presented, including both the clinical screenshot (which includes proprietary non-linear filters) and the reconstructed B-mode image formed from the saved RF data, as illustrated in Figure 1. Both of these B-mode images were presented to avoid bias from the poorer-quality B-mode images created from saved RF data, which were processed without the non-linear filters that are present in most clinical systems. With only B-mode images, the readers were asked to select the content of the mass—(1) solid, (2) fluid, (3) mixed or (4) uncertain—and the BI-RADS (
      • Mendelson E.B.
      • Berg W.A.
      • Merritt C.R.
      Toward a standardized breast ultrasound lexicon, bi-rads: Ultrasound.
      ) category for each mass—(2) benign finding, (3) probably benign, (4) suspicious abnormality and (5) highly suggestive of malignancy. Subcategories were not used in this classification.
      For Task 2, after category selection using only the B-mode images, the readers were presented with the R-SLSC image for the same patient and asked to categorize the content and BI-RADS category of the presented mass given the original B-mode images and the added R-SLSC display. This selection using all available images is indicated as Task 2 in Figure 1. In addition, for Task 2, a duplex display (
      • Wiacek A.
      • Falomo E.
      • Myers K.
      • Rindal O.M.H.
      • Fabrega-Foster K.
      • Harvey S.
      • Bell M.A.L.
      Clinical feasibility of coherence-based beamforming to distinguish solid from fluid hypoechoic breast masses.
      ) was available if desired to further analyze the R-SLSC image (labeled as Optional in Figure 1). If chosen, the reader could select a region in the B-mode image, where the R-SLSC image would be overlaid in color, with blue representing low spatial coherence and red representing high spatial coherence.
      For Tasks 1 and 2, the readers had full control over tunable parameters, including dynamic range, short-lag value (i.e., M in eqn [2]) and lambda value (i.e.,  λ in eqn [4]) to aid decision making. The selected classifications for each mass during each task were saved for comparison and statistical analyses.

      Statistical analysis

      The difference between the mean contrast of 10 frames of B-mode images and 10 frames of R-SLSC images was calculated for each fluid-filled or solid mass using eqn (7). Differences between fluid and solid masses were compared using a repeated-measures analysis of variance to determine statistical significance (p < 0.05). In addition, the reader responses for both the BI-RADS category and content assessments were compared before and after introduction of the R-SLSC image using Cohen's κ to determine the similarity of each assessment to the truth (

      Cardillo G.Cohen's kappa: Compute the Cohen's kappa ratio on a 2x2 matrix, 2007a. Available at: http://www.mathworks.com/matlabcentral/fileexchange/15365.

      ). Fleiss’ κ for multiple observers was calculated to determine the level of inter-observer agreement among the content and BI-RADS classifications (

      Cardillo G.Fleiss'es kappa: compute the Fleiss'es kappa for multiple raters. 2007b. Available at: http://www.mathworks.com/matlabcentral/fileexchange/15426.

      ). Both Cohen's and Fleiss’ κ values <0.0 were interpreted as poor agreement; values between 0 and 0.20 were interpreted as slight agreement; values between 0.21 and 0.40 were interpreted as fair agreement; values between 0.41 and 0.60 were interpreted as moderate agreement; values between 0.61 and 0.80 were interpreted as substantial agreement; and values >0.81 were interpreted as perfect agreement, in accordance with
      • Landis J.R.
      • Koch G.G.
      The measurement of observer agreement for categorical data.
      .
      Sensitivity and specificity for correctly diagnosing cysts were measured as follows:
      Sensitivity=TPTP+FN
      (8)


      Specificity=TNTN+FP
      (9)


      Here a true positive (TP) was defined as a cyst that was not recommended for biopsy in Task 2 of the reader study (i.e., BI-RADS 2 or 3), a false negative (FN) was defined as a cyst that was recommended for biopsy in Task 2 (i.e., BI-RADS 4 or 5), a true negative (TN) was defined as a solid mass that was recommended for biopsy in Task 2 (i.e., BI-RADS 4 or 5) and a false positive (FP) was defined as a solid mass that was not recommended for biopsy in Task 2 (i.e., BI-RADS 2 or 3). The masses with a ground truth classified as mixed were omitted from this analysis, considering that they technically contain a mixture of TP and TN.
      We compared the sensitivity and specificity of our results with those of results from two of the few breast studies with data available and with a similar goal of distinguishing cysts from solid masses (
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      ) or evaluating cyst detection (
      • Chiorean A.R.
      • Duma M.M.
      • Dudea S.M.
      • Bolboaca S.
      • Dumitriu D.
      • Eniu D.
      • Sfrangeu S.A.
      Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
      ). To compare our results with the acoustic streaming results reported by
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      , TP was defined as a cyst in which acoustic streaming was generated, FN was defined as a cyst with no streaming detected, TP was defined as a solid mass with no streaming detected, and FN was defined as a solid mass with streaming. This analysis was performed for the study and control groups combined, resulting in 39 masses in total.
      To compare with the performance of B-mode alone from the same study (
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      ), any cyst in the study group was considered a FP and any solid mass in this group was considered a TN (considering that the study group consisted of patients with discrete breast lesions that were considered indeterminate for a cyst or solid mass with initial diagnostic sonography). Otherwise, TP for B-mode alone was defined as the control group cysts, and TN was defined as the control group solid masses.
      To compare our results with the elastography results reported by
      • Chiorean A.R.
      • Duma M.M.
      • Dudea S.M.
      • Bolboaca S.
      • Dumitriu D.
      • Eniu D.
      • Sfrangeu S.A.
      Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
      , TP was defined as all simple and complicated cysts that were downgraded after elastography (i.e., a final diagnosis of BI-RADS 2), and FN was defined as complicated cysts that were not altered in diagnosis as a result of elastography. Because no solid masses were included in this study, there were no TNs or FPs, and therefore, specificity could not be calculated. The results for complex masses (i.e., masses with mixed content according to our definition) were not included in this analysis, and only the results for simple and complicated cysts were included, resulting in a total of 92 masses.
      To compare with the performance of B-mode alone from the same study (
      • Chiorean A.R.
      • Duma M.M.
      • Dudea S.M.
      • Bolboaca S.
      • Dumitriu D.
      • Eniu D.
      • Sfrangeu S.A.
      Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
      ), any simple cyst was considered a TP and any complicated cyst was considered a FN. Because no solid masses were included in this study, there were no TNs or FPs, and therefore, specificity could not be calculated.

      Results

      Contrast difference across multiple masses

      Our previous work (
      • Wiacek A.
      • Rindal O.M.H.
      • Falomo E.
      • Myers K.
      • Fabrega-Foster K.
      • Harvey S.
      • Lediju Bell M.A.
      Robust short-lag spatial coherence imaging of breast ultrasound data: Initial clinical results.
      ) revealed improved contrast in R-SLSC images compared with B-mode images for fluid-filled regions such as simple and complicated cysts. Conversely, we also found that hypo-echoic solid masses contain low-amplitude internal echoes caused by coherent scatterers, which results in coherent signal in R-SLSC images. The difference in coherence within the fluid-filled and solid masses allows them to appear distinct from one another.
      Extending this analysis from 9 to 26 in vivo masses, Figure 2 illustrates the contrast difference between the respective B-mode image and R-SLSC image for all masses included in this study. In blue are the fluid-filled masses, which display a positive contrast difference, indicating an improvement in contrast. In gray are the solid masses, which display a negative contrast difference, indicating coherent content, and therefore confirming the presence of a solid mass. This difference in contrast differences between fluid-filled and solid masses is statistically significant (p < 0.001). Specific examples of masses are available in Figure 2, and contrast results are summarized in Table 1. For any masses with mixed solid and fluid content in this study, the ROI for contrast measurements was selected as the solid component of the mass. Therefore, the masses in Table 1 are categorized as fluid-filled, benign and malignant, where benign encompasses both benign solid and benign mixed masses.
      Fig 2
      Fig. 2Contrast difference between B-mode and robust short-lag spatial coherence for each of the masses included in this study.
      Table 1Summary of the 26 masses included in the reader study
      Mass typeNo. of massesMean contrast (dB)
      B-modeR-SLSCDifference
      FluidComplicated cyst4–13.56–16.402.85
      Simple cyst3–20.69–23.793.09
      BenignApocrine metaplasia3–8.52–4.31–4.21
      Fibroadenolipoma1–2.230.44–2.67
      Fibroadenoma4–9.79–1.52–8.27
      Fibrocystic change2–11.64–4.93–6.71
      Fibroepithelial lesion1–3.018.43–11.43
      Phyllodes tumor1–11.689.99–21.67
      MalignantDuctal carcinoma in situ3–8.92–1.43–7.49
      Invasive ductal carcinoma4–14.56–2.24–12.32
      BI-RADS = Breast Imaging Reporting and Data System; R-SLSC = robust short-lag spatial coherence.

      Fluid-filled masses

      In Figure 3 (a, f, k) is an example of a simple cyst. Figure 3a is the clinical screenshot representing the additional information that was accessible to the radiologists when making decisions during the reader study. Figure 3f was beamformed using standard delay-and-sum beamforming from the saved RF channel data, which were used for quantitative comparisons. Figure 3k was generated using R-SLSC beamforming where the M and λ values were selected to match the B-mode speckle SNR. Qualitatively, the R-SLSC image shows improved contrast and reduced acoustic clutter compared with the B-mode image created from raw data. The heterogeneous tissue texture of the B-mode image is also largely removed in the corresponding R-SLSC image. In addition, the R-SLSC image has a dark-field artifact present because of the use of focused transmits, which may be removed with synthetic transmit aperture imaging (
      • Bottenus N.
      • Byram B.C.
      • Dahl J.J.
      • Trahey G.E.
      Synthetic aperture focusing for short-lag spatial coherence imaging.
      ). Quantitatively, the contrast measurements match the qualitative observations. The contrast in the B-mode image in Figure 3f is –19.8 dB. For the same ROI, the contrast in the R-SLSC image in Figure 3k is improved to –22.0 dB. The average contrast improvement is 3 dB across all simple cysts in the study population.
      Fig 3
      Fig. 3(a–e) Clinical screenshots. (f–j) B-Mode images created from raw data. (k–o) Corresponding robust short-lag spatial coherence images of five different masses in the female breast. These masses appear hypo-echoic in the B-mode images, and the solid or fluid content is more readily distinguishable in the robust short-lag spatial coherence images.
      In Figure 3 (b, g, l) is an example of a complicated cyst. Figure 3b is the clinical screenshot for qualitative comparisons. Figure 3g is the B-mode image generated with the saved RF channel data for quantitative comparisons. Figure 3l was generated from the same raw data using R-SLSC beamforming. Similar to the simple cysts, the R-SLSC image shows improved contrast compared with its B-mode counterpart; however, the R-SLSC image shows some scattered internal coherence representative of proteinaceous material characteristic of complicated cysts. The contrast in the B-mode and R-SLSC images is –21.8 and –24.1 dB, respectively, reflecting improved contrast in the R-SLSC image. Overall, despite the minimal coherence within the mass in the R-SLSC image, the average contrast improvement is 2.8 dB across all complicated cysts in the study population. These results are summarized in Table 1.

      Masses with solid content

      In Figure 3 (c, h, m) is an example of a benign solid mass, specifically a fibroadenoma. Both Figure 3c, the clinical screenshot, and Figure 3h, the B-mode image generated using saved RF channel data, show the mass appearing as hypo-echoic with some internal echoes. Figure 3m, the corresponding R-SLSC image, appears to have isocoherent structures and begins to blend in with the background. This coherence is present because of the solid content of the mass. The contrast in the B-mode image created from raw data is –11.7 dB, representative of its hypo-echoic appearance. The R-SLSC image shows significantly worse contrast at -3.6 dB, which indicates the presence of a solid mass. The average contrast difference between B-mode images and R-SLSC images for the fibroadenomas in this study was –8.3 dB. These results and the results for other types of solid masses in this patient population are summarized in Table 1.
      In Figure 3 (d, i, n) is an example of a malignant solid mass, specifically an invasive ductal carcinoma. Similar to the benign solid mass, In Figure 3 (d, i), this malignant solid mass appears largely hypo-echoic in the B-mode image. Figure 3n is the respective R-SLSC, image which begins to blend in with the background because of the internal coherence, indicative of the solid content. The contrast for the B-mode image is –7.1 dB. Because of the coherent tissue content, the R-SLSC image has significantly worse contrast at –1.5 dB, indicating the presence of solid tissue. These results and the results for the other malignant masses are summarized in Table 1.
      Finally, in Figure 3 (e, j, o) is an example of a benign mixed mass, that is, a mass containing both solid and fluid tissue components, with biopsy pathology revealing fibrocystic changes. As illustrated in Figure 3 (e, j), this mass appears hypo-echoic, with some internal echoes in the B-mode image. Figure 3o is the R-SLSC image, which appears to have some coherent components in the middle of the mass, with less coherence on the right and left sides of the mass, indicative of both solid and fluid components. This mass is considered to have “mixed” content. The contrast of the B-mode image is –13.6 dB, which is reduced to –5.3 dB in the R-SLSC image using an ROI containing the solid components of the mass.

      Reader study results

      The results of the reader content classification are summarized in Figure 4. The leftmost bar of each grouping shows the ground truth for each type of mass (i.e., simple cyst, complicated cyst, benign solid, malignant solid and benign mixed). This ground truth is based on pathology results. The results from each reader appear to the right of the ground truth, and the left bar in each result pair indicates the reader decision using only B-mode images (i.e., Task 1), while the right bar of each result pair indicates the reader decision when combining both B-mode and R-SLSC for decision making (i.e., Task 2). Each color represents the percentage of the masses classified in each category.
      Fig 4
      Fig. 4Results of the content reader study for simple cysts, complicated cysts, benign solid masses, malignant solid masses and benign mixed masses. In the first column is the ground truth, followed by the results of Tasks 1 and 2 for each reader.
      The uncertainty surrounding the content of simple cysts ranged from 33%–100% with B-mode alone (specifically, 66.7%, 100%, 66.7%, 33.3% and 33.3% for Readers 1, 2, 3, 4 and 5, respectively). With R-SLSC included in the decision making, the uncertainty was removed for Readers 1–4, and all masses were correctly classified as fluid. However, Reader 5 remained uncertain about 33.3% of these masses.
      For the complicated cysts, when using only B-mode images, Readers 1–4 were uncertain or misclassified 50% of the masses, and Reader 5 was uncertain about 25% of the masses. With the inclusion of R-SLSC, the uncertainty was reduced for Readers 1 and 3, misclassifications were removed for Reader 2 and classifications remained the same for Readers 4 and 5. Two of the complicated cysts were classified as mixed, and possible reasons for this are presented in the Discussion.
      For the benign solid masses when using only B-mode imaging, Readers 1, 2 and 4 were uncertain or misclassified 33.3% of masses. Readers 3 and 5 were uncertain about the content of 66.7% and 55.6% of these masses, respectively. With the inclusion of R-SLSC, the uncertainty or misclassification was reduced to 11.1% for Readers 1, 2 and 3, and reduced to 0% for Reader 4. Reader 5 remained the same with 55.6% uncertainty.
      For the malignant solid and benign mixed masses, there was one case where uncertainty was reduced from 75% to 0% (Reader 3, benign mixed) and two cases where uncertainty increased with R-SLSC (Reader 3, malignant solid, and Reader 5, benign mixed). Otherwise, the classifications when using only B- mode and when adding R-SLSC remained approximately similar, with some discrepancies in the definition of mixed content.
      To assess possible connections between the quantitative metrics (which are reported in Fig. 2) and the uncertain or misclassified masses from the reader study (which are reported in Fig. 4), Figure 5 illustrates the contrast difference of each mass as a function of the number of readers uncertain about the content of that mass (top) and the number of misclassifications for that mass (bottom). For each mass, the filled circle represents the B-mode observations, and the open circle represents the R-SLSC observations. A left arrow indicates a decrease in uncertainty or misclassifications with the introduction of R-SLSC, and a right arrow indicates an increase in uncertainty or misclassifications with the introduction of R-SLSC. A data point without an arrow indicates no change with the introduction of R-SLSC.
      Fig 5
      Fig. 5Contrast difference as a function of the number of readers uncertain per mass (top) and the number of misclassifications per mass (bottom). The arrows indicate the direction of change when robust short-lag spatial coherence (R-SLSC) was added. R-SLSC = robust short-lag spatial coherence.
      Figure 5 illustrates an overall reduction in uncertainty and misclassifications with the introduction of R-SLSC, regardless of the magnitude of the contrast difference. In particular, the summary of uncertainty for fluid-filled masses (reported in the top left of Fig. 5) reveals that two or three readers were uncertain with B-mode alone, and in each of these cases, the uncertainty was minimized to no to two readers with the introduction of R-SLSC (with the exception of one complicated cyst that caused no changes in uncertainty by two readers). For the solid masses, overall the majority of masses decreased in uncertainty with R-SLSC, with the exception of two malignant solid masses moving from no readers being uncertain with B-mode to one reader being uncertain with R-SLSC included. The uncertainty in these two cases was associated with the same reader (i.e., Reader 3), and this uncertainty did not change the reader's recommendation for biopsy (i.e., BI-RADS 4) during Task 2. For the mixed masses, the uncertainty of the readers either remained the same or decreased.
      The summary of misclassifications of fluid-filled masses (reported in the bottom left of Figure 5) reveals that one complicated cyst resulted in two fewer misclassifications (from solid to the correct classification of fluid) with the inclusion of R-SLSC. In addition, one complicated cyst resulted in two misclassifications as mixed with the inclusion of R-SLSC. There were no additional misclassifications nor changes in misclassifications for these fluid masses. Similarly, for the solid masses, the number of misclassifications for the majority of these masses either remained the same or decreased, with the exception of one solid mass that was classified as mixed with the introduction of R-SLSC. For the mixed masses, the number of misclassifications remained the same regardless of the imaging method.
      The results of the reader BI-RADS classification for the simple cysts are summarized in the top of Figure 6. The first pair of bars show the ideal diagnosis on the left (based on the true pathology, which is BI-RADS 2, meaning the mass will neither be followed nor biopsied), and the actual diagnosis listed in the patients’ medical records is shown on the right. The following five pairs of results indicate the rating of each reader. The left bar in each pair indicates the reader decision using only B-mode images (i.e., Task 1), and the right bar indicates the reader decision when both B-mode and R-SLSC are available (i.e., Task 2).
      Fig 6
      Fig. 6Results of the Breast Imaging Reporting and Data System reader study for simple cysts, complicated cysts, benign solid masses, malignant solid masses and benign mixed masses. The first pair shows the ideal and diagnosed results, followed by the results of Tasks 1 and 2 for each reader. R-SLSC = robust short-lag spatial coherence.
      When B-mode alone was used, the top of Figure 6 illustrates that Readers 1, 3, 4 and 5 assigned BI-RADS 4 to 33%–100% of the simple cysts, indicating the need for a biopsy in these cases. With the inclusion of R-SLSC, 0% of the masses required biopsy for Readers 1–4. In addition, Reader 1 decided to follow 33.3% of the masses, and the diagnosis provided by Reader 5 remained the same (i.e., biopsy 33.3% of these masses).
      The remaining results of the reader BI-RADS classification are also summarized in Figure 6 in the same format as for the simple cysts. For the solid and mixed mass types (i.e., benign solid, malignant solid and benign mixed), the ideal diagnosis is a combination of two categories depending on additional imaging findings, and therefore, two colors are displayed in the bar representing the ideal choice for these two masses; either choice is considered correct.
      For the complicated cysts, using B-mode alone, Reader 1 required biopsy for 75% of the masses (i.e., BI-RADS 4), and Readers 2 and 3 required biopsy for 50% of the masses. Reader 5 recommended biopsy for 25% of the masses, and Reader 4 did not recommend biopsy for any of the masses. When R-SLSC was included, the number of biopsies was reduced for Readers 1, 2 and 3, requiring biopsies for 50%, 0% and 25% of the masses, respectively. The recommendations of Readers 4 and 5 remained the same for these complicated cysts.
      For the masses with solid content (i.e., the benign, malignant and mixed categories), the overall diagnostic decision remained similar when using B-mode only and when including R-SLSC for each reader, with the exception of two cases in which mixed diagnoses of BI-RADS 3 and 4 were modified to all BI-RADS 4 with the inclusion of R-SLSC (Reader 4, benign solid, and Reader 3, benign mixed).

      Statistical analysis results

      Figure 7 illustrates the results of the statistical analyses. For each reader, Cohen's κ was calculated between the truth (or ideal) diagnosis and either the B-mode selections or the selections when R-SLSC was added. When classifying the content of the mass, the κ value with B-mode only was 0.42, 0.50, 0.25, 0.48 and 0.43 for Readers 1, 2, 3, 4 and 5, respectively, as reported in Figure 7a. These results indicate fair agreement for Reader 3 and moderate agreement for the remaining readers. When R-SLSC was added, the κ value improved to the range 0.65–0.80 for Readers 1–4, representing substantial agreement with the truth. For Reader 5, the κ value was reduced to 0.38 when R-SLSC was added, indicating fair agreement with truth.
      Fig 7
      Fig. 7Results of Cohen's κ for (a) content classification and (b) BI-RADS classification. (c) Results of Fleiss’ κ for inter-reader variability. BI-RADS = Breast Imaging Reporting and Data System; R-SLSC = robust short-lag spatial coherence.
      When deciding the clinical path through BI-RADS, the κ values with B-mode alone were 0.17, 0.36, 0.28, 0.53 and 0.57 for Readers 1, 2, 3, 4 and 5, respectively, as reported in Figure 7b. These results indicate slight, fair and moderate agreement between the ideal diagnosis and the selections made with only B-mode image information by Readers 1, 2/3 and 4/5, respectively. When R-SLSC was included in the decision process, the κ values improved to the range 0.65–0.80 for Readers 1–4, indicating substantial agreement between the ideal diagnosis and the selections made with R-SLSC included in the diagnostic decision. In comparison, the κ value remained the same for Reader 5 (i.e., 0.57, which represents moderate agreement). In calculating these Cohen's κ values, BI-RADS 4 was used as the ideal diagnosis for any mass containing solid content (i.e., benign solid, malignant solid and benign mixed), as this is the standard diagnosis for baseline imaging at our clinic.
      Finally, the inter-reader agreement was evaluated using Fleiss’ κ. Figure 7c summarizes these results. For content classification, the κ value when using only B-mode was 0.33, indicating fair agreement; however, when R-SLSC is included, the κ value increases to 0.44, indicating moderate agreement. Similarly, for the BI-RADS classification, the κ value when using only B-mode was 0.30 and increased to 0.50 when R-SLSC was included, indicating fair and moderate agreement, respectively.
      Table 2 outlines the sensitivity and specificity of correctly diagnosing cysts for each reader. Sensitivity was improved when using R-SLSC combined with B-mode, compared with B-mode alone, for each reader except Reader 5. The specificity was improved when using R-SLSC combined with B-mode, compared with B-mode alone, for Readers 3 and 4. Otherwise, specificity remained the same for Readers 1, 2 and 5. The means ± standard deviation of sensitivity and specificity results for the five readers were 57% ± 29% and 90% ± 7%, respectively, with B-mode alone, and 86% ± 14% and 95% ± 5%, respectively, when using R-SLSC combined with B-mode, representing an overall increase in sensitivity and specificity with R-SLSC included.
      Table 2Sensitivity and specificity of cyst detection
      Sensitivity (%)Specificity (%)
      B-ModeReader
      114100
      27188
      34388
      48681
      57194
      All57 ± 2990 ± 7
      Soo et al.35100
      Chiorean et al.53-
      B-Mode + R-SLSCReader
      171100
      210088
      38694
      4100100
      57194
      All86 ± 1495 ± 5
      Acoustic streamingSoo et al.61100
      ElastographyChiorean et al.87-
      R-SLSC = robust short-lag spatial coherence.
      When compared with the acoustic streaming results, for which the sensitivity was 61% (
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      ), inclusion of R-SLSC resulted in greater sensitivity for each reader (and greater mean sensitivity for all readers). Similarly, when compared with the free-hand elastography sensitivity of 87% (
      • Chiorean A.R.
      • Duma M.M.
      • Dudea S.M.
      • Bolboaca S.
      • Dumitriu D.
      • Eniu D.
      • Sfrangeu S.A.
      Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
      ), the inclusion of R-SLSC had a similar mean sensitivity, with two readers achieving a sensitivity higher than that for free-hand elastography (i.e., Readers 2 and 4). The mean specificity with R-SLSC included was 5% lower than the comparative specificity of 100% for acoustic streaming (
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      ), with two readers achieving a specificity of 100% (i.e., Readers 1 and 4).

      Discussion

      Clinical implications

      The work described in this article indicates that the inclusion of R-SLSC images in clinical decision making has promising potential to improve both the certainty of mass contents and the BI-RADS classification accuracy, particularly for fluid-filled simple cysts, as illustrated in Figures 4, 5 and 6. Fluid-filled regions that appear hypo-echoic typically contain low-amplitude, incoherent echoes, which can be attributed to the presence of acoustic clutter. Because R-SLSC images display spatial coherence rather than traditional amplitude information, the signal caused by acoustic clutter is reduced with R-SLSC imaging, resulting in the positive contrast differences reported in Figure 2 (p < 0.001). This difference is suspected to be the reason for the improved certainty and the reduced need for biopsy or follow-up of the simple cysts; the top left plot of Figure 5 indicates that these benefits are achieved for contrast improvements as minimal as 1.2 dB.
      The improved contrast in the simple cysts allowed readers to be more confident in a fluid-filled diagnosis, resulting in no uncertainty and no recommendations for biopsies from four of the five readers. We also emphasize here that the recruited patients were recommended for biopsy. Thus, although our study population included a total of three simple cysts, these simple cysts are expected to represent a few of the most challenging simple cyst cases, which is also demonstrated by a minimum of three readers being uncertain of the content of each cyst in B-mode images (as observed in the top left plot of Figure 5). It is notable that R-SLSC reduced uncertainty in these challenging cases.
      Complicated cysts can be difficult to diagnose because of the presence of both acoustic clutter and proteinaceous material inside of the mass, causing confusion with hypo-echoic solid masses that require biopsy (
      • Meyer J.E.
      • Christian R.L.
      • Frenna T.H.
      • Sonnenfeld M.R.
      • Waitzkin E.D.
      • Shaffer K.
      Image-guided aspiration of solitary occult breast ‘cysts’.
      ). On examination of the R-SLSC image of a complicated cyst in Figure 3l, the fluid-filled content is evident by the black signal within the mass; however, the proteinaceous material can also be visualized within the mass. Therefore, we suspect that the complicated cysts classified as mixed by Readers 1 and 3 occurred because the content within complicated cysts was enhanced with R-SLSC imaging and because more guidelines regarding the definition of mixed content were needed. Otherwise, the majority of readers were more confident in the classification of complicated cysts in the fluid category, and assigned complicated cysts a less invasive diagnosis with R-SLSC included in the decision process.
      When the percentages of uncertain and biopsy-recommended masses for each reader are considered, the added information from R-SLSC images reduced the mean uncertainty about the content of fluid-filled masses (i.e., both simple and complicated cysts) from 47.5% to 15.8%, and reduced the mean percentage of fluid-filled masses recommended for biopsy from 43.3% to 13.3%. Decreasing the number of biopsies of fluid-filled masses helps to reduce patient discomfort and anxiety at the time of initial diagnosis, as well as save resources in the health care system. These results could be increasingly important in remote areas where the resources used to perform ultrasound-guided biopsies are scarce.
      The content selection and BI-RADS diagnosis provided by Reader 5 remained largely the same throughout the reader study. This reader cited difficulty in trusting the R-SLSC images without additional data or patient history, and as a result, this reader chose to keep most of the clinical decisions the same regardless of the imaging method. Therefore, our assumption that responses were based solely on observed differences in the presented images did not seem to hold true for Reader 5 because of these factors that we did not anticipate and were unable to control. Incorporation of this feedback will be the focus of future work (e.g., including readers in the patient recruitment and diagnostic retrieval process to gain more confidence and familiarity with the new technology prior to performing additional reader studies). When Reader 5 is removed from the inter-reader assessment, and only the agreement among readers who were more confident with the novel image presentation format is considered, the inter-reader agreement improved from κ = 0.44 to κ = 0.60 for the content classification; both values represent moderate agreement nonetheless.
      Finally, although the reader study indicates that the masses with solid content (i.e., benign solid, malignant solid and benign mixed) would follow similar clinical paths whether B-mode alone or B-mode paired with R-SLSC was used, the confidence in deciding the content of each mass was improved. In particular, when R-SLSC was used, the errors and uncertainty in classification were largely removed for the majority of readers, as illustrated in Figure 5.

      Study limitations

      The study described in this article focused on 26 masses in 25 different patients. While it is promising that statistically significant differences were seen between B-mode and R-SLSC images of solid and fluid-filled masses, larger-scale studies with increased patient numbers would be beneficial to determine the full clinical impact of R-SLSC. Despite the small number of masses, the comparison of results combining B-mode and R-SLSC with results from similar early-phase studies (
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      ;
      • Chiorean A.R.
      • Duma M.M.
      • Dudea S.M.
      • Bolboaca S.
      • Dumitriu D.
      • Eniu D.
      • Sfrangeu S.A.
      Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
      ) indicates improved sensitivity for all readers when compared with acoustic streaming results (
      • Soo M.S.
      • Ghate S.V.
      • Baker J.A.
      • Rosen E.L.
      • Walsh R.
      • Warwick B.N.
      • Ramachandran A.
      • Nightingale K.R.
      Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
      ), improved sensitivity for two readers when compared with elastography results (
      • Chiorean A.R.
      • Duma M.M.
      • Dudea S.M.
      • Bolboaca S.
      • Dumitriu D.
      • Eniu D.
      • Sfrangeu S.A.
      Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
      ) and specificity within one standard deviation of the mean specificity achieved by all readers, as reported in Table 2.
      In addition, two of our five readers (i.e., Readers 1 and 2) assisted with the acquisition of the ultrasound data. Although this could be considered a potential bias in the study design, a minimum of 154–228 days elapsed between the last imaged patient and the reader study, these two readers were not involved in the preparation of the reader study nor in analysis of the reader study results, and they both stated that they did not remember the diagnosis of the masses in question. In addition, the results from the additional three readers who did not participate in data collection were compared with the results from the remaining two readers (see Fig. 7c), resulting in moderate agreement among the five readers. This agreement indicates that potential bias was not a reason for the improvements obtained with R-SLSC images included.
      Additional limitations include the offline image formation and the static presentation of the masses to each reader. In addition, readers cited less confidence in their clinical decisions without patient medical history, other imaging findings and the ability of real-time imaging capabilities. Therefore, future studies will incorporate a summary of the patient medical history and imaging findings. Given our initial success with offline processing, future work will additionally focus on implementing real-time coherence-based imaging capabilities (
      • Hyun D.
      • Crowley A.L.C.
      • Dahl J.J.
      Efficient strategies for estimating the spatial coherence of backscatter.
      ) on our dual clinical and research ultrasound system to make larger-scale studies more feasible.

      Conclusions

      This work is the first to explore the effect of coherence-based beamforming in breast ultrasound to inform clinical decision making, highlighting the potential of this novel technique to reduce the number of unnecessary biopsies of breast masses. Solid breast masses produce spatial coherence images that appear distinctly different from those of fluid-filled masses. The information from R-SLSC images reduced the uncertainty of fluid mass content from 47.5% to 15.8%, and the number of fluid-filled masses recommended for biopsy was reduced from 43.3% to 13.3%. This benefit was achieved without affecting the clinical decisions for the solid masses, which remained approximately the same with and without the R-SLSC image information. These results provide promising support for additional investigations into the potential inclusion of R-SLSC in the breast clinic as a supplement to standard B-mode images to improve diagnostic certainty and reduce the number of unnecessary biopsies of fluid-filled masses.

      Acknowledgments

      This research was supported by a Ralph E. Powe Junior Faculty Enhancement Award from Oak Ridge Associated Universities (awarded to M.A.L.B.). The authors thank breast radiologists Emily Ambinder, M.D., and Philip Di Carlo, M.D., for participating in our reader study.

      Conflict of interest disclosure

      All authors declare no competing interests.

      References

        • Athanasiou A.
        • Tardivon A.
        • Tanter M.
        • Sigal-Zafrani B.
        • Bercoff J.
        • Deffieux T.
        • Gennisson J.L.
        • Fink M.
        • Neuenschwander S.
        Breast lesions: Quantitative elastography with supersonic shear imaging—Preliminary results.
        Radiology. 2010; 256: 297-303
        • Balleyguier C.
        • Canale S.
        • Hassen W.B.
        • Vielh P.
        • Bayou E.
        • Mathieu M.
        • Uzan C.
        • Bourgier C.
        • Dromain C.
        Breast elasticity: Principles, technique, results: An update and overview of commercially available software.
        Eur J Radiol. 2013; 82: 427-434
        • Barr R.G.
        Shear wave imaging of the breast still on the learning curve.
        J Ultrasound Med. 2012; 31: 347-350
        • Barr R.G.
        • Zhang Z.
        Effects of precompression on elasticity imaging of the breast.
        J Ultrasound Med. 2012; 31: 895-902
        • Bell M.A.L.
        • Goswami R.
        • Kisslo J.A.
        • Dahl J.J.
        • Trahey G.E.
        Short-lag spatial coherence imaging of cardiac ultrasound data: Initial clinical results.
        Ultrasound Med Biol. 2013; 39: 1861-1874
        • Bell M.A.L.
        • Dahl J.J.
        • Trahey G.E.
        Resolution and brightness characteristics of short-lag spatial coherence (SLSC) images.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2015; 62: 1265-1276
        • Bell M.A.L.
        • Kumar S.
        • Kuo L.
        • Sen H.T.
        • Iordachita I.
        • Kazanzides P.
        Toward standardized acoustic radiation force (ARF)-based ultrasound elasticity measurements with robotic force control.
        IEEE Trans Biomed Eng. 2016; 63: 1517-1524
        • Bercoff J.
        • Tanter M.
        • Fink M.
        Supersonic shear imaging: A new technique for soft tissue elasticity mapping.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2004; 51: 396-409
        • Berg W.A.
        • Cosgrove D.O.
        • Doré C.J.
        • Schäfer F.K.
        • Svensson W.E.
        • Hooley R.J.
        • Ohlinger R.
        • Mendelson E.B.
        • Balu-Maestro C.
        • Locatelli M.
        • Tourasse C.
        • Cavanaugh B.C.
        • Juhan V.
        • Stavros A.T.
        • Tardivon A.
        • Gay J.
        • Henry J.P.
        • Cohen-Bacrie C.
        • BE1 Investigators
        Shear-wave elastography improves the specificity of breast us: the BE1 multinational study of 939 masses.
        Radiology. 2012; 262: 435-449
        • Berg W.A.
        • Bandos A.I.
        • Mendelson E.B.
        • Lehrer D.
        • Jong R.A.
        • Pisano E.D.
        Ultrasound as the primary screening test for breast cancer: Analysis from ACRIN 6666.
        J Natl Cancer Inst. 2016; 108: djv367
        • Bottenus N.
        • Byram B.C.
        • Dahl J.J.
        • Trahey G.E.
        Synthetic aperture focusing for short-lag spatial coherence imaging.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2013; 60: 1816-1826
      1. Cardillo G.Cohen's kappa: Compute the Cohen's kappa ratio on a 2x2 matrix, 2007a. Available at: http://www.mathworks.com/matlabcentral/fileexchange/15365.

      2. Cardillo G.Fleiss'es kappa: compute the Fleiss'es kappa for multiple raters. 2007b. Available at: http://www.mathworks.com/matlabcentral/fileexchange/15426.

        • Cha J.H.
        • Moon W.K.
        • Cho N.
        • Kim S.M.
        • Park S.H.
        • Han B.K.
        • Choe Y.H.
        • Park J.M.
        • Im J.G.
        Characterization of benign and malignant solid breast masses: Comparison of conventional us and tissue harmonic imaging.
        Radiology. 2007; 242: 63-69
        • Chiorean A.R.
        • Duma M.M.
        • Dudea S.M.
        • Bolboaca S.
        • Dumitriu D.
        • Eniu D.
        • Sfrangeu S.A.
        Typical and unusual sonoelastographic patterns of breast cystic lesions: Impact on bi-rads classification.
        Ultraschall Med. 2012; 33: E138-E144
        • Covington M.F.
        • Pizzitola V.J.
        • Lorans R.
        • Pockaj B.A.
        • Northfelt D.W.
        • Appleton C.M.
        • Patel B.K.
        The future of contrast-enhanced mammography.
        AJR Am J Roentgenol. 2018; 210: 292-300
        • DeSantis C.
        • Siegel R.
        • Bandi P.
        • Jemal A.
        Breast cancer statistics, 2011.
        CA Cancer J Clin. 2011; 61: 408-418
        • Entrekin R.
        • Jackson P.
        • Jago J.
        • Porter B.
        Real time spatial compound imaging in breast ultrasound: Technology and early clinical experience.
        Medicamundi. 1999; 43: 35-43
        • Evans A.
        • Whelehan P.
        • Thomson K.
        • McLean D.
        • Brauer K.
        • Purdie C.
        • Jordan L.
        • Baker L.
        • Thompson A.
        Quantitative shear wave ultrasound elastography: Initial experience in solid breast masses.
        Breast Cancer Res. 2010; 12: R104
      3. Food and Drug Administration (FDA). FDA Drug Safety Communication: FDA warns that gadolinium-based contrast agents (GBCAs) are retained in body; requires new class warnings. 2017. Available at:https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-warns-gadolinium-based-contrast-agents-gbcas-are-retained-body.

        • Garra B.S.
        Imaging and estimation of tissue elasticity by ultrasound.
        Ultrasound Q. 2007; 23: 255-268
        • Garra B.S.
        • Cespedes E.I.
        • Ophir J.
        • Spratt S.R.
        • Zuurbier R.A.
        • Magnant C.M.
        • Pennanen M.F.
        Elastography of breast lesions: Initial clinical results.
        Radiology. 1997; 202: 79-86
        • Han J.
        • Li F.
        • Peng C.
        • Huang Y.
        • Lin Q.
        • Liu Y.
        • Cao L.
        • Zhou J.
        Reducing unnecessary biopsy of breast lesions: Preliminary results with combination of strain and shear-wave elastography.
        Ultrasound Med Biol. 2019; 45: 2317-2327
        • Hooley R.J.
        • Scoutt L.M.
        • Philpotts L.E.
        Breast ultrasonography: State of the art.
        Radiology. 2013; 268: 642-659
        • Hruska C.B.
        Molecular breast imaging for screening in dense breasts: State of the art and future directions.
        AJR Am J Roentgenol. 2017; 208: 275-283
        • Huber S.
        • Wagner M.
        • Medl M.
        • Czembirek H.
        Real-time spatial compound imaging in breast ultrasound.
        Ultrasound Med Biol. 2002; 28: 155-163
        • Hyun D.
        • Crowley A.L.C.
        • Dahl J.J.
        Efficient strategies for estimating the spatial coherence of backscatter.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2017; 64: 500-513
        • Jakovljevic M.
        • Trahey G.E.
        • Nelson R.C.
        • Dahl J.J.
        In vivo application of short-lag spatial coherence imaging in human liver.
        Ultrasound Med Biol. 2013; 39: 534-542
        • Kakkad V.
        • Dahl J.
        • Ellestad S.
        • Trahey G.
        In vivo application of short-lag spatial coherence and harmonic spatial coherence imaging in fetal ultrasound.
        Ultrasonic Imaging. 2015; 37: 101-116
        • Kartamihardja A.A.P.
        • Nakajima T.
        • Kameo S.
        • Koyama H.
        • Tsushima Y.
        Distribution and clearance of retained gadolinium in the brain: Differences between linear and macrocyclic gadolinium based contrast agents in a mouse model.
        Br J Radiol. 2016; 8920160509
        • Kuhl C.K.
        • Schrading S.
        • Leutner C.C.
        • Morakkabati-Spitz N.
        • Wardelmann E.
        • Fimmers R.
        • Kuhn W.
        • Schild H.H.
        Mammography, breast ultrasound, and magnetic resonance imaging for surveillance of women at high familial risk for breast cancer.
        J Clin Oncol. 2005; 23: 8469-8476
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; : 159-174
        • Lediju M.A.
        • Pihl M.J.
        • Dahl J.J.
        • Trahey G.E.
        Quantitative assessment of the magnitude, impact and spatial extent of ultrasonic clutter.
        Ultrason Imaging. 2008; 30: 151-168
        • Lediju M.A.
        • Trahey G.E.
        • Byram B.C.
        • Dahl J.J.
        Short-lag spatial coherence of backscattered echoes: Imaging characteristics.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2011; 58: 1377-1388
        • Lewis T.C.
        • Pizzitola V.J.
        • Giurescu M.E.
        • Eversman W.G.
        • Lorans R.
        • Robinson K.A.
        • Patel B.K.
        Contrast-enhanced digital mammography: A single-institution experience of the first 208 cases.
        Breast J. 2017; 23: 67-76
      4. Lin Z, Chen M, Ma Y. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055, 2010.

        • Long W.
        • Hyun D.
        • Choudhury K.R.
        • Bradway D.
        • McNally P.
        • Boyd B.
        • Ellestad S.
        • Trahey G.E.
        Clinical utility of fetal short-lag spatial coherence imaging.
        Ultrasound Med Biol. 2018; 44: 794-806
        • Mandelson M.T.
        • Oestreicher N.
        • Porter P.L.
        • White D.
        • Finder C.A.
        • Taplin S.H.
        • White E.
        Breast density as a predictor of mammographic detection: Comparison of interval- and screen-detected cancers.
        J Natl Cancer Inst. 2000; 92: 1081-1087
        • McDonald R.J.
        • McDonald J.S.
        • Kallmes D.F.
        • Jentoft M.E.
        • Paolini M.A.
        • Murray D.L.
        • Williamson E.E.
        • Eckel L.J.
        Gadolinium deposition in human brain tissues after contrast-enhanced MR imaging in adult patients without intracranial abnormalities.
        Radiology. 2017; 285: 546-554
        • Mendelson E.B.
        • Berg W.A.
        • Merritt C.R.
        Toward a standardized breast ultrasound lexicon, bi-rads: Ultrasound.
        Semin Roentgenol. 2001; 36: 217-225
        • Mesurolle B.
        • Bining H.J.
        • Khoury M.E.
        • Barhdadi A.
        • Kao E.
        Contribution of tissue harmonic imaging and frequency compound imaging in interventional breast sonography.
        J Ultrasound Med. 2006; 25: 845-855
        • Meyer J.E.
        • Christian R.L.
        • Frenna T.H.
        • Sonnenfeld M.R.
        • Waitzkin E.D.
        • Shaffer K.
        Image-guided aspiration of solitary occult breast ‘cysts’.
        Arch Surg. 1992; 127: 433-435
        • Murata N.
        • Gonzalez-Cuyar L.F.
        • Murata K.
        • Fligner C.
        • Dills R.
        • Hippe D.
        • Mar- avilla K.R.
        Macrocyclic and other non-group 1 gadolinium contrast agents deposit low levels of gadolinium in brain and bone tissue: Preliminary results from 9 patients with normal renal function.
        Invest Radiol. 2016; 51: 447-453
        • Nair A.A.
        • Tran T.D.
        • Bell M.A.L.
        Robust short-lag spatial coherence imaging.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2018; 65: 366-377
        • Nam K.
        • Zagzebski J.A.
        • Hall T.J.
        Quantitative assessment of in vivo breast masses using ultrasound attenuation and backscatter.
        Ultrason Imaging. 2013; 35: 146-161
        • Nightingale K.R.
        • Kornguth P.J.
        • Trahey G.E.
        The use of acoustic streaming in breast lesion diagnosis: A clinical study.
        Ultrasound Med Biol. 1999; 25: 75-87
        • Nightingale K.
        • Soo M.S.
        • Nightingale R.
        • Trahey G.
        Acoustic radiation force impulse imaging: In vivo demonstration of clinical feasibility.
        Ultrasound Med Biol. 2002; 28: 227-235
        • Oelze M.L.
        • Mamou J.
        Review of quantitative ultrasound: Envelope statistics and backscatter coefficient imaging and contributions to diagnostic ultrasound.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2016; 63: 336-351
        • Oelze M.L.
        • O'Brien Jr, W.D.
        • Zachary J.F.
        11B-4 quantitative ultrasound assessment of breast cancer using a multiparameter approach.
        Proc IEEE Int Ultrason Symp. 2007; : 981-984
        • Patel B.K.
        • Gray R.J.
        • Pockaj B.A.
        Potential cost savings of contrast-enhanced digital mammography.
        AJR Am J Roentgenol. 2017; 208: W231-W237
        • Sadeghi-Naini A.
        • Papanicolau N.
        • Falou O.
        • Zubovits J.
        • Dent R.
        • Verma S.
        • Trudeau M.
        • Boileau J.F.
        • Spayne J.
        • Iradji S.
        • Sofroni E.
        • Lee J.
        • Lemon-Wong S.
        • Yaffe M.
        • Kolios M.C.
        • Czarnota G.J.
        Quantitative ultrasound evaluation of tumor cell death response in locally advanced breast cancer patients receiving chemotherapy.
        Clin Cancer Res. 2013; 19: 2163-2174
        • Sannachi L.
        • Tadayyon H.
        • Sadeghi-Naini A.
        • Tran W.
        • Gandhi S.
        • Wright F.
        • Oelze M.
        • Czarnota G.
        Non-invasive evaluation of breast cancer response to chemotherapy using quantitative ultrasonic backscatter parameters.
        Med Image Anal. 2015; 20: 224-236
        • Shankar P.M.
        • Dumane V.
        • Reid J.M.
        • Genis V.
        • Forsberg F.
        • Piccoli C.W.
        • Goldberg B.B.
        Classification of ultrasonic B-mode images of breast masses using Nakagami distribution.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2001; 48: 569-580
        • Sharma A.C.
        • Soo M.S.
        • Trahey G.E.
        • Nightingale K.R.
        Acoustic radiation force impulse imaging of in vivo breast masses.
        Proc IEEE Int Ultrason Symp. 2004; 1: 728-731
        • Shermis R.B.
        • Wilson K.D.
        • Doyle M.T.
        • Martin T.S.
        • Merryman D.
        • Kudrolli H.
        • Brenner R.J.
        Supplemental breast cancer screening with molecular breast imaging for women with dense breast tissue.
        AJR Am J Roentgenol. 2016; 207: 450-457
        • Siegel R.L.
        • Miller K.D.
        • Jemal A.
        Cancer statistics, 2019.
        CA Cancer J Clin. 2019; 69: 7-34
        • Soo M.S.
        • Ghate S.V.
        • Baker J.A.
        • Rosen E.L.
        • Walsh R.
        • Warwick B.N.
        • Ramachandran A.
        • Nightingale K.R.
        Streaming detection for evaluation of indeterminate sonographic breast masses: A pilot study.
        AJR Am J Roentgenol. 2006; 186: 1335-1341
        • Trop I.
        • Destrempes F.
        • El Khoury M.
        • Robidoux A.
        • Gaboury L.
        • Allard L.
        • Chayer B.
        • Cloutier G.
        The added value of statistical modeling of backscatter properties in the management of breast lesions at us.
        Radiology. 2014; 275: 666-674
        • Wiacek A.
        • Falomo E.
        • Myers K.
        • Rindal O.M.H.
        • Fabrega-Foster K.
        • Harvey S.
        • Bell M.A.L.
        Clinical feasibility of coherence-based beamforming to distinguish solid from fluid hypoechoic breast masses.
        Proc IEEE Int Ultrason Symp. 2018; 8579846
        • Wiacek A.
        • Rindal O.M.H.
        • Falomo E.
        • Myers K.
        • Fabrega-Foster K.
        • Harvey S.
        • Lediju Bell M.A.
        Robust short-lag spatial coherence imaging of breast ultrasound data: Initial clinical results.
        IEEE Trans Ultrason Ferroelectr Freq Control. 2019; 66: 527-540