Gemini 3.5 Flash on wrist X-rays

The question behind this post: what changes in a model's reading when only the instructions change.

This is the second post in our wrist X-ray series, and it uses the scoring method from How we evaluate AI for wrist fractures. Why we compare general-purpose models on X-rays at all is covered in Frontier, open-weight or medical models: which can read an X-ray?. Later posts will look at whether newer Gemini models do better, and at AI agents reading wrist X-rays together.

How much can a prompt change a wrist X-ray interpretation?

Recap: the measuring scale

Our previous post described how we evaluate AI for wrist fractures. We needed that work before testing a model. One score can hide a missed fracture or the wrong bone. A model may invent an injury and keep a high average.

We spend a lot of time discussing which AI model performs best in medical imaging. But there is another variable that perhaps deserves just as much attention: how we ask the model to look at the image.

Gemini 3.5 test run

We recently tested this with Gemini 3.5 Flash on wrist radiographs.

The question was fairly simple. If we give the model exactly the same X-rays, but change the instructions from a simple request to a structured radiology reading protocol, does its interpretation actually improve?

The answer was yes.

But, as is often the case with medical AI, the interesting part was not the improvement itself. It was how the model changed its mistakes.

First, how does a radiologist read a wrist?

A wrist X-ray may look like a relatively straightforward examination. It isn't always.

A radiologist does not simply look at the image and ask, "Is there a fracture?" They usually go through the anatomy systematically: distal radius, distal ulna, scaphoid and the remaining carpal bones. We look at the articular surfaces, search for small avulsion fragments and then check carpal alignment.

There are also several things that make these examinations more difficult:

• A fracture may be subtle.
• A cast can obscure the cortex.
• One of the expected projections may be missing.
• Some abnormalities are visible only on one view.

Finding a fracture is only part of the job. Where is it? Is it displaced? Does it extend into the joint? Is there associated carpal or distal radioulnar joint malalignment?

These details can influence what happens to the patient next.

So we wondered whether explicitly teaching an AI model to follow this type of search pattern would make a difference.

We tested it on 71 wrist studies

We selected 71 adult wrist radiograph studies from our internal dataset. The final expert radiology report was used as the reference.

Gemini received de-identified, normalised radiographic images. We intentionally did not provide clinical history or additional patient information.

We then ran every examination twice.

Run 1: the naive prompt. Deliberately simple. We asked the model to identify fractures, dislocations and alignment abnormalities.

Run 2: the structured protocol. Something much closer to a radiologist's reading checklist:

1. Establish which projections were available, laterality and whether image quality was adequate.

2. Systematically inspect the distal radius, distal ulna and individual carpal bones.

3. Evaluate scapholunate widening, Gilula's arcs, lunate and perilunate alignment, the distal radioulnar joint and overall carpal alignment.

4. Describe every abnormality with its location, laterality and confidence.

The model was the same. The images were the same. The output structure was the same. Only the instructions changed.

So, did it work?

Somewhat.

Average rubric score across the 71 studies: naive prompt 58.60, structured protocol 60.77.

That is not a dramatic improvement.

But looking only at the average score hides something more interesting. Of the 71 studies:

So the structured approach was clearly changing the model's behaviour. But then we looked at how it was changing.

We gained sensitivity but paid for it with specificity

With the structured protocol, Gemini detected two additional fractures that were documented in the reference reports. That sounds good.

Unfortunately, it also called three additional fractures in studies where the reference report did not describe a fracture.

This is an important trade-off. In radiology, a false negative and a false positive are not simply two numbers that cancel each other out.

A missed fracture can delay treatment. But an unsupported fracture diagnosis can also send a patient down an unnecessary pathway of immobilisation, additional imaging, specialist consultation or even treatment. Perhaps more importantly for clinical adoption, repeated false-positive calls quickly reduce a radiologist's trust in an AI system.

The individual cases were more interesting than the averages

This is probably the most important lesson from the experiment. A very professional-looking radiology report can still be wrong. The language may be convincing. The anatomy may sound appropriate. The report may even contain a confidence score. None of those things guarantee that the underlying observation is correct.

Would I let it report wrist X-rays autonomously?

Not based on these results.

The structured protocol still missed four fractures that were present in the reference reports. It also produced ten fracture calls in report-negative studies. Using our review criteria, 68 of the 71 structured outputs would still have been sent to an expert for review.

That tells us quite clearly where this technology sits today. At least for this experiment, Gemini is much more interesting as a radiologist's copilot than as an autonomous reporter.

It could draft an initial report. It could force a second look at the scaphoid or triquetrum. It could point out an alignment abnormality that deserves another few seconds of attention. But every one of those observations still needs to be verified by the radiologist before the report is signed.

There is another limitation worth mentioning

The more useful observation was this:

Changing the prompt changed the model's clinical behaviour. A more systematic prompt encouraged the model to find more fractures. But it also encouraged it to invent more fractures.

Prompt engineering in medical imaging therefore isn't simply about improving a benchmark score. It can change the model's operating point (sensitivity, specificity and ultimately the kinds of mistakes a clinician will have to deal with).

What do we try next?

This brings us to something I find much more interesting: agentic radiology workflows.

Instead of asking one model to inspect the entire wrist and produce the final interpretation in a single pass, what happens if we break the problem apart?

• One agent evaluates the distal radius and ulna.
• Another inspects the carpal bones.
• Another concentrates entirely on alignment.
• A final agent reconciles those observations, challenges contradictory findings and generates the report.

What we test next: the same reading, broken into focused jobs and then reconciled.

This approach is already being explored extensively in coding, where a complex problem can sometimes be solved more reliably by decomposing it into smaller tasks. Whether the same principle works for medical images is a different question.

It may improve accuracy. Or we may simply create more opportunities for AI agents to confidently agree with each other's mistakes.

That is what we are testing next. And I suspect the answer will again be more interesting than the average score.

For anyone deciding how AI should sit in a reading room, the practical takeaway from this experiment is that the prompt is part of the product. Change it and you change the kinds of mistakes a radiologist will have to catch, so the radiologist has to stay in the loop and the output has to be easy to check. That is the same position Dr. Vishwanath argued in AI is with the radiologist, and it is how we think about AI inside SOMATIQ's reporting workflow.

Related reading

• How we evaluate AI for wrist fractures — the rubric behind every score in this post.
• Frontier, open-weight or medical models: which can read an X-ray? — why we are testing general-purpose models on X-rays.
• Structured reporting in radiology — why structure in the report itself matters for checking AI output.

FAQ

Did a structured reading protocol improve Gemini 3.5 Flash on wrist X-rays?

Somewhat. On the same 71 adult studies, the structured protocol performed better in 21 cases, the naive prompt in 10, and the remaining 40 were essentially ties. It detected two additional fractures documented in the reference reports but also called three additional fractures in studies where the reference did not describe one.

Could Gemini 3.5 Flash report wrist X-rays on its own?

Not based on these results. The structured protocol still missed four fractures present in the reference reports and produced ten fracture calls in report-negative studies, and 68 of the 71 structured outputs would still have been sent to an expert for review. In this experiment it was more useful as a radiologist's copilot than as an autonomous reporter.

What are the limits of this experiment?

It was an internal benchmark of 71 studies from a single dataset, not a multicentre clinical trial. Clinical history was withheld, and the final expert report was the reference standard, which is not necessarily equivalent to an independent multi-reader ground truth.