The interobserver agreement between expert ultrasound operators when differentiating between adnexal tumors using ultrasound pattern recognition

Objectives: To assess interobserver agreement and diagnostic accuracy of expert ultrasound operators using pattern recognition to assess the nature of adnexal masses. Methods: Static B-mode preoperative ultrasound images containing gray-scale and color Doppler information of a selected mix of difficult adnexal masses of 166 patients were examined independently by three expert ultrasound operators (A, B and C). They all had access to relevant clinical information, but none of the experts performed the original real-time scans. The experts were asked to classify tumors as benign or malignant and to subclassify tumors into one of 11 histological subgroups. The diagnostic performance of each expert was compared with the histological diagnosis of the respective specimen. Each operator s diagnoses were compared with the other two operators diagnoses using Cohen s kappa coefficient. Results: There were 36 invasive malignancies, 34 borderline ovarian tumors (BOT) and 96 benign tumors. The sensitivity and specificity with regard to malignancy (BOT and invasive) were: 86% (95% CI, 75 93) and 91% (95% CI, 83 96) for Expert A, 86% (95% CI, 75 93) and 80% (95% CI, 70 87) for Expert B, and 80% (95% CI, 68 89) and 85% (95% CI, 77 92) for Expert C. There was a good interobserver agreement when diagnosing ovarian tumors as benign, BOT and invasive (kappa = 0.69, 95% CI, 0.59 0.79 comparing Experts A and B; kappa = 0.71, 95% CI, 0.62 0.82, comparing Experts A and C; and kappa = 0.75, 95% CI, 0.66 0.84, comparing Experts B and C) and when predicting the histological subtypes (kappa = 0.63, 95% CI, 0.55 0.71, comparing Experts A and B; kappa = 0.67, 95% CI, 0.59 0.75, comparing Experts A and C; and kappa = 0.66, 95% CI, 0.58 0.76, comparing Experts B and C). Conclusions: Expert ultrasound operators have a good level of agreement when differentiating between difficult ovarian tumors using the ultrasound pattern recognition method. This shows that the high accuracy of this method is reproducible.