AI can predict a response and still miss the interaction
A new preprint finds that plausible scientific predictions can conceal failure to recover how two factors interact, making the choice of test decisive.
Knowing what A does and what B does separately does not determine what they do together.
An October 14 preprint tests this problem in flow-matching models across physiological simulation, drug responses, cell perturbations and molecular conformations. It finds that overall prediction quality can mask errors in the combined effect. [1][2]
In the controlled simulation, models sometimes invented interactions where the true system was additive. Other interactions were recoverable when the training conditions supplied enough structure. The result is a warning about a particular prediction problem, rather than evidence that scientific AI cannot work. [1]
For a team evaluating a model, the practical test is to hold out combinations and score the interaction itself, alongside overall accuracy. The study is a preprint and does not establish clinical performance. [1]
Sources & details 5 sources
Discuss in your voice app
Take the reporting, visual explanations and original sources into a conversation. Copy the brief, paste it into your preferred assistant, then start its voice mode.
Paste the brief into your voice app to discuss this reporting.Reporting sources
- Gijsen, Seiler & Ritter · Original preprint
Full October 14 v1 read: interaction definition, four experimental settings, recoverable and failed cases, and two-factor/mean-contrast limitations. Results are not independently reproduced or peer reviewed.
- arXiv · Submission record
Version 1 submitted October 14, 2026. Establishes original date; not a separate experiment.
- Miller et al. · Metric calibration
Original Nature Biotechnology abstract in PubMed, published October 1, 2026. Across 14 datasets and 18 metrics, calibrated metrics can reveal deep-model gains. Authors disclose employment at Shift Bioscience and Xaira Therapeutics. Background, not a newly redated result.
- Ahlmann-Eltze et al. · Earlier independent benchmark
Peer-reviewed Nature Methods study, August 4, 2025. Original methods and double-perturbation results read. Different models and tasks; context for evaluating combinations, not replication of the October 14 preprint.
- Authors · Public evaluation code
Public code repository opened; code was inspected for availability, not run or independently reproduced.
Images & figures
Separate effects do not determine a combined response. These teaching values use a zero baseline; the interaction depends on the response scale.
Euangelion · original explanatory figure · Source · Usage terms