DICOM SEG and PET/CT lesion segmentation
By Naveen K, Co-Founder and Chief of AI ·
Medically reviewed by Dr. Vishwanath R S, MD, CIIP
Outlining a lesion slice by slice is some of the slowest work in imaging. On a PET/CT with several avid lesions, it can take longer than the read itself, and in radiotherapy planning the same manual contouring was for years a routine part of the job. That is why segmentation is worth getting right, even though Herman Oosterwijk estimates it is only about 10% of AI applications, much less than detection or measurement.
His deep dive into DICOM SEG output (April 2026) explains the format well. So, let's go over the short version of what a SEG object is, then how we do PET/CT segmentation in the browser, and why we chose a threshold rather than something cleverer.
What DICOM SEG is
A DICOM Segmentation object (modality SEG) records which voxels belong to which structure. It is one of the four object types the IHE AI Results profile expects a viewer to display (see The AI results checklist we hold our viewer to). Its main properties are as follows.
- It is one enhanced multi-frame file, not one file per slice.
- In BINARY and FRACTIONAL SEG, each frame is a 2D slice of one segment, such as "liver" or "lesion 3".
- One file can hold many segments, each in its own set of frames.
- Segments are binary (in or out) or fractional (a probability or occupancy); the newer LABELMAP type instead carries several segments in one frame as label numbers.
- Each segment carries a coded label, such as a SNOMED code, and a display colour.
- It references the source series and shares its frame of reference, so it lines up in 3D.
| SEG type | What a frame or voxel holds | Overlap between segments |
|---|---|---|
| BINARY | In or out of the segment | Possible: each segment is its own mask |
| FRACTIONAL | A probability or occupancy value | Possible: each segment is its own mask |
| LABELMAP | A label number, so several segments share one frame | Not possible: one label per voxel |
The information that applies to every frame sits in a "shared" group, and what differs per frame sits in a "per-frame" group; each per-frame item says which segment that frame belongs to. Herman's example makes the case against the alternative: a lesion stored in 149 frames of 1-bit data in one SEG file. As Secondary Capture, the same result would be 149 more images. It is the same argument we made for overlays in Stop burning AI into the image, only stronger, because a region is a volume and a burned-in copy duplicates every slice it touches.
PET/CT segmentation in the SOMATIQ viewer
In the SOMATIQ viewer, a radiologist segments PET/CT lesions in 3D in the browser. There is no separate workstation and no install. The steps are these.
Threshold-based 3D lesion segmentation on fused PET/CT, step by step.
- Open the PET/CT study. PET and CT load fused in the browser.
- Place a region around the lesion.
- The viewer applies a threshold to the PET values inside that region, across every slice the region spans.
- The voxels above the threshold become a 3D lesion segment, drawn over both the PET and CT.
- Review, adjust and repeat for the next lesion.
A threshold is deliberate. It is fast and predictable. The same region and threshold give the same volume every time, and the radiologist can see exactly why a voxel is in or out. That matters for follow-up scans, where the change in volume is the finding. The disadvantage is equally clear: a threshold does not know anatomy, so a lesion next to the bladder or the myocardium, both of which take up tracer, needs a tighter region or a manual correction. We think that trade-off is the right one for now, because it keeps the radiologist in charge of the result, which is the principle we set out in AI is with the radiologist.
Everything runs in the radiologist's browser (see What is a zero-footprint DICOM viewer for why that is possible). That means the same tool is available at the centre, at home and at a teleradiology desk, without a dedicated PET workstation at each.
In conclusion, DICOM SEG is the right container for any segmentation result, from AI or from a radiologist, and the format is simple enough that there is little excuse for sending regions as screenshots. For PET/CT, a threshold inside a radiologist-placed region gives a reproducible 3D segment in the browser, which is what response assessment needs. We will cover the PET/CT reading workflow itself in a later post.
Related reading
- The AI results checklist we hold our viewer to — how AIR expects a viewer to display regions.
FAQ
What is DICOM SEG?
A DICOM object that stores segmentations as labelled voxel masks referenced to the source images. It is the recommended format for AI segmentation output.
DICOM SEG or RTSTRUCT?
SEG stores voxel masks; RTSTRUCT stores contours and is standard in radiotherapy planning.
Why threshold-based PET segmentation?
It is reproducible and transparent: the same input always gives the same volume. Comparing volumes across scans also needs consistent acquisition, reconstruction and SUV inputs, but a deterministic method removes one source of variation.
Do I need to install software?
No. Segmentation runs in a modern browser.
Ready to see your workflow on one platform?
We will walk you through the whole workflow, from registration to signed report.