Microsoft’s diagnostic system faces the next test: real patients
Microsoft reports strong sequential-diagnosis benchmark results and plans a prospective hospital study, which has not yet established patient benefit.
Microsoft’s October 22 update reports 83% diagnostic accuracy for MAI-DxO on complex NEJM cases, compared with 22% for physicians without reference resources and 36% with them.
On MIMIC-IV-ED cases, it reports 80% versus 47% for the physician comparator. Those are results in its sequential-diagnosis evaluation, not accuracy rates from routine care. [1]
The benchmark asks systems to request information and tests before committing a diagnosis. Its published methods reveal a consequential approximation: a gatekeeper supplies a realistic synthetic answer when the underlying case lacks a requested result. A separate judge scores the diagnosis. This design tests a richer workflow than a multiple-choice exam, while still differing from actual patients and clinical decisions. [2]
The next step is a planned prospective study with Beth Israel Deaconess Medical Center, in which patients will interact with the system and results will be compared with independent physician assessments. The announcement does not report completed prospective outcomes. Health-system buyers can use the benchmark to identify an evaluation candidate; evidence of improved care must come from the clinical setting. [1]
Take it further
Inspect SDBench’s evaluation and data-access requirements For clinical-evaluation teams. NEJM cases require licensing; MIMIC access requires PhysioNet credentialing, training and a data-use agreement. Bundled synthetic cases are not a clinical performance dataset.
Sources & details 2 sources
Discuss in your voice app
Take the reporting, visual explanations and original sources into a conversation. Copy the brief, paste it into your preferred assistant, then start its voice mode.
Paste the brief into your voice app to discuss this reporting.Reporting sources
- Microsoft AI · Diagnostic-system update
October 22, 2026. Author-reported benchmark outcomes and planned prospective evaluation. Linked Science paper returned 403 in this review.
- Microsoft · SDBench methods
Inspectably describes synthetic gatekeeper answers, judge scoring and data-access restrictions. Not an independent clinical validation.