AI Cancer Tools May Rely on Visual Shortcuts; Stereotactic Radiation Improves Brain Metastases Outcomes

Artificial intelligence tools developed to predict cancer biology directly from microscope images may be relying on hidden shortcuts rather than genuine biological signals, according to new research from the University of Warwick published in Nature Biomedical Engineering. The findings raise concerns that some AI pathology tools are currently too unreliable for real-world patient care.

The researchers analysed more than 8,000 patient samples across four major cancer types—breast, colorectal, lung and endometrial—and compared the performance of leading machine learning approaches. While the models often achieved high headline accuracy, the team found this frequently came from statistical "shortcuts."

Instead of detecting mutations in the cancer-associated BRAF gene, a model might learn that BRAF mutations often occur alongside another clinical feature such as microsatellite instability (MSI). The system then learns to use this combination of cues to predict BRAF status rather than learning the causal BRAF signal itself, meaning accurate cancer predictions work only when these biomarkers co-occur and become unreliable when they do not.

When performance of AI models was assessed within stratified patient subgroups, such as only high-grade breast cancers or only MSI-positive tumours, accuracy fell substantially, revealing that the models were dependent on shortcut signals that disappear once confounding factors are controlled.

For certain prediction tasks, the performance advantage of deep learning over human-derived clinical information was modest. AI systems achieved accuracy scores of just over 80% when predicting biomarkers, compared with around 75% using tumour grade alone—a measure already assessed by pathologists.

Machine learning methods can still prove valuable for research, drug development candidate screening and for clinical triaging, screening, or supplementary decision support. However, the researchers argue that future AI tools must move beyond correlation-based learning and adopt approaches that explicitly model biological relationships and causal structure. They also call for stronger evaluation standards, including subgroup testing and comparison against simple clinical baselines, before looking at deployment in routine care.

In separate research addressing brain metastases treatment, a phase 3 randomized clinical trial conducted at 4 United States-based centers found that stereotactic radiation targeting individual tumors improved symptom severity and interference with daily functioning compared with hippocampal-avoidance whole brain radiation.

Of 196 randomized patients (mean age, 61 years; 129 [66%] female; 176 [90%] White; median number of brain metastases, 14 [IQR, 11-18]; 49 [25%] with prior neurosurgical resection), 83 (42%) completed the 6-month assessment. Eligible patients had 5 to 20 brain metastases and no prior brain-directed radiation. Enrollment occurred between April 11, 2017, and May 17, 2024 (final follow-up, March 18, 2025).

For the primary outcome, between baseline and postbaseline assessments through the 6-month follow-up, stereotactic radiation changed the weighted composite MD Anderson Symptom Inventory–Brain Tumor score from 2.69 to 2.37 (mean change, −0.32) and hippocampal-avoidance whole brain radiation changed the score from 2.29 to 3.03 (mean change, 0.74) (mean difference, −1.06 [95% CI, −1.54 to −0.58]; P < .001). The scale ranges from 0-10, with score change range of −10 to 10, where −10 represents the best outcome. A clinically meaningful difference was defined as 0.98.

Related grade 3-5 adverse events occurred in 12 patients (12%) in the stereotactic radiation group and 13 patients (13%) in the hippocampal-avoidance whole brain radiation group; grade 1-3 fatigue was most frequent (27 [28%] vs 43 [44%], respectively).

The trial was registered as ClinicalTrials.gov Identifier: NCT03075072.