Data analysis for detection and localization of multiple abnormalities with application to mammography

Rationale and Objectives. In assessing diagnostic accuracy it is often essential to determine the reader's ability both to detect and to correctly locate multiple abnormalities per patient. The authors developed a new approach for the detection and localization of multiple abnormalities and compared it with other approaches. Materials and Methods. The new approach involves partitioning the image into multiple regions of interest (ROIs). The reader assigns a confidence score to each ROI. Statistical methods for clustered data are used to assess and compare reader accuracy. The authors applied this new method to a reader-performance study of conventional film images and digitized images used to detect and locate malignant breast cancer lesions. Results. The ROI-based approach, the free-response receiver operating characteristic (FROC) curve, and the patient-based approach handle the estimation of the false-positive rate (FPR) quite differently. These differences affect the measures of the respective areas under the curves. In the ROI-based approach the denominator is the number of ROIs without a malignant lesion. In the FROC approach the average number of false-positive findings per patient is plotted on the x axis of the curve. In contrast, the patient-based approach mishandles the FPR by ignoring multiple detection and/or localization errors in the same patient. The FROC approach does not lend itself easily to statistical evaluations. Conclusion. The ROI-based approach appropriately captures both the detection and localization tasks. The interpretation of the ROI-based accuracy measures is simple and clinically relevant. There are statistical methods for estimating and comparing ROI-based estimates of accuracy.