GuideAdvancedRecently Added

MAST Is a Better Way to Ask Which Medical AI Model to Use

Ramez Kouzy 7 min

What you'll learn

  • What MAST measures across diagnostic reasoning, management, safety, imaging, and agentic work
  • Why the General and Clinical composites answer different questions
  • How to use MAST to form a shortlist rather than make a procurement decision
  • Which evaluated systems have public programmatic access, and which do not have public API documentation
  • Why benchmark transparency should lead to local validation

A clinician asks a deceptively simple question: Which model should I use today?

Most leaderboards do not help much. A vendor demo may show a strong answer to a benchmark you have not read. A general leaderboard may reward coding or math. A medical multiple-choice score does not tell you whether a model can handle uncertainty, images, a management decision, or a workflow that touches an EHR.

MAST, the Medical AI Superintelligence Test from ARISE, is a serious attempt to make that question less vague. It is not a universal score for medical intelligence. It is a public framework that combines several clinically relevant benchmarks, separates general-purpose models from clinician-facing products, and exposes the component scores rather than asking us to trust one number.

MAST general leaderboard showing models ranked across medical reasoning, safety, radiology, and medical images

MAST General leaderboard, captured July 29, 2026. Scores are a dated snapshot, not a permanent claim about a model. Source: ARISE MAST.

What MAST measures

Clinical usefulness is not one skill. A model can produce a good differential and still choose the wrong next step. It can be strong with text and weak with images. It can sound plausible while making a recommendation that is unsafe.

MAST separates performance across diagnostic reasoning, management reasoning, safety, radiology, other clinical images, and simulated agentic tasks. Its General Composite includes the image domains; its narrower Clinical Composite focuses on diagnostic reasoning, management, and safety. The distinction is useful. A strong multimodal model is not automatically the right comparison for a clinician-facing tool that does not accept images, and a high text-only score does not establish image competence.

MAST uses harmonic means rather than simple averages, which penalizes a weak link. It also publishes component scores and keeps agentic performance outside the composite for now. Those are sensible design choices. The composite is still orientation, not a procurement score.

The useful result is the split, not simply the winner

On the MAST page last updated July 25, 2026, GPT-5.5 led the General leaderboard at 61.6%. The clinician-facing leaderboard looked different: LiSA 2.5 from AMBOSS led the posted Clinical Composite at 81.0%, followed by Glass Health 5.6 at 79.1% and Doximity Ask 6.1 at 77.6%.

That does not mean the highest score is the tool to buy or use. It means specialized clinical products belong in the comparison set when the job is diagnostic or management support. The evidence layer, source transparency, privacy terms, model version, interface, and implementation boundary still matter.

The harder question is whether outsiders can independently test those products as they are actually used. The answer varies more than people often assume:

This is an important correction to the usual blanket claim that clinical products are inaccessible. OpenAlex is explicitly programmable, and Glass now advertises a Developer API. For the others above, I found no public API documentation, which is different from proving that no partner or enterprise integration exists. The practical point remains: researchers often cannot reproduce the proprietary retrieval layer, system prompt, content curation, model routing, policy logic, and interface behavior behind a clinician-facing answer.

Vendor demos and product announcements can still be useful signals. They are not independent evidence of broad clinical performance. Public leaderboards such as MAST are valuable because they create a common comparison point. The next step is versioned evaluation access and enough methodological detail for independent groups to test the actual system clinicians use.

Use MAST as a shortlist

How I would use MAST

Start with the job, then inspect the relevant dimensions. For patient-facing management support, safety and management reasoning matter more than an attractive overall rank. For image work, do not use the Clinical Composite alone. For an EHR agent, a Q&A benchmark is insufficient.

A practical workflow is simple:

  1. Define the job. Differential generation, evidence retrieval, note summarization, image interpretation, patient messaging, and EHR actions are different tasks.
  2. Choose the relevant MAST view. Use General for broad capability including images, and Clinical for the narrower diagnostic, management, and safety comparison.
  3. Read the component scores and benchmark methods. Ask what data the benchmark uses, who judged it, and whether it resembles the local task.
  4. Check access and provenance. Can you retrieve sources, document the model version, and independently test the product you plan to use?
  5. Run a governed local evaluation. Use representative cases, include edge cases and abstention behavior, and decide who reviews outputs and how errors are handled.

Benchmarks can saturate, leak into training data, and reflect the preferences of their designers. MAST is more transparent than many dashboards about those choices, but transparency should lead to local validation, not replace it.

The mature reading of MAST is not “the number-one model is the answer.” It is: this dashboard gives me a defensible first comparison, tells me which capabilities to inspect, and makes it harder to mistake a glossy demo for a clinical evaluation.

Sources

MAST Is a Better Way to Ask Which Medical AI Model to Use