In a multi-institutional comparative effectiveness study, 24 independent radiologists and 20 independent oncologic providers compared AI Metrics to current manual methods. The purpose of this study was to compare the effectiveness of advanced cancer longitudinal imaging response evaluation using current practice versus artificial intelligence (AI)-assisted methods.

About the study

During the study, providers compared body CT images from 120 consecutive patients with multiple serial imaging exams and advanced cancer treated with systemic therapy using current-practice methods and AI-Assisted methods. Current practice methods included dictated text-based reports and separately categorized responses (CR, PR, SD, and PD). The AI-Metrics Platform used custom AI algorithms for tumor measurement, target and non-target location labeling, and tumor localization at follow up. The AI-assisted software was able to automatically categorize tumor response per RECIST 1.1 calculations and displayed longitudinal data in the form of a graph, table, and key images. The studies were read independently in triplicate for assessment of inter-observer agreement according to the metrics: major errors, time of image interpretation, and inter-observer agreement for final response category.

What they found

AI Metrics increased reporting accuracy by 25%, reduced errors by 99%, cut interpretation time in half, increased inter-observer agreement among oncologists by 58% and among radiologists by 45%, and was preferred by 96% of radiologists and 100% of oncologists compared to current practice with manual image assessments and text reports. In a study of the beta version of the AI Metrics Platform (called eMASS; included guided workflows and annotation tools, but no AI algorithms), eMASS reduced errors and time of evaluation was twice as fast, which indicated better overall effectiveness than standard of care, manual tumor response evaluation methods for three different therapy response criteria.

The Verdict

AI-assisted advanced cancer longitudinal imaging response evaluation significantly reduced major errors, was nearly twice as fast, and increased inter-observer agreement relative to the current-practice method. The decisive advantages of the AI Metrics Platform establishes a new and improved standard of care in oncology and radiology.