Shift plans a virtual-cell screen—and asks whether the benchmark can see the signal
Shift’s October 5 screening announcement draws on an October 1 paper showing that the measure used to score virtual-cell predictions can conceal meaningful differences.
Shift Bioscience says it will apply a new evaluation framework to laboratory and computational screens for inhibition targets, initially focused on fibrosis.
That October 5 announcement is a research plan. The accompanying Nature Biotechnology paper was published October 1; it reports benchmark analysis, not a new treatment. [1][2]
The central question is whether a scoring rule can distinguish useful predictions from a bland average. Across 14 datasets and 18 metrics, the authors find that some common scores are insensitive to changes affecting relatively few genes. They test the metrics with positive and negative controls before comparing models. [2]
This adds context to an independent 2025 benchmark in which tested deep-learning models did not beat simple baselines. The new results do not make every model reliable: performance still depends on the dataset, task and metric, and an additive baseline remains difficult to beat on one widely used combination dataset. [2][3]
Sources & details 4 sources
Discuss in your voice app
Take the reporting, visual explanations and original sources into a conversation. Copy the brief, paste it into your preferred assistant, then start its voice mode.
Paste the brief into your voice app to discuss this reporting.Revised 9 October 2026. Copy edited 9 October to remove commentary about graphic design; reported claims, qualifications and source dates preserved.
Reporting sources
- Shift Bioscience · October 5 announcement
Original announcement of planned screening; company statement, not verified experimental or clinical outcomes.
- Nature Biotechnology · Benchmark calibration paper
Original open-access paper published October 1, 2026. Dates, controls, 14 datasets/18 metrics, additive-baseline limitation and unseen-context limitation checked.
- Nature Methods · Independent 2025 benchmark
Original Ahlmann-Eltze, Huber and Anders analysis, August 4, 2025. Different models/tasks; context and methodological contrast, not replication of the new framework.
- Shift Bioscience · Reproduction repository
Original code documentation. Core MIT licence with component-specific restrictions; full benchmark recommends multiple GPUs. No execution or independent reproduction claimed.
Images & figures
Toy example: twenty positions represent hypothetical genes, three of which change. A flat prediction misses all three changes. Averaging errors across many unchanged genes can dilute that failure. Counts, amplitudes and spacing are illustrative, not results from Shift’s study; no clinical or model-performance gain is plotted.
Euangelion · original explanatory graphic · Source · Usage terms