Poor segmentation for sct_deepseg gray matter mouse model (affine issues, low axial resolution)

[mariamcoris]

Dear SCT team:

I am experiencing a problem while segmenting the grey matter of mouse spinal cord MRIs (T1 and T2). I am using the seg_mice_gm task of sct_deepseg on SCT 6.2 :

sct_deepseg -i medulas/sub-MEDULA2_ses-1_acq-tr_T2rare.nii -task seg_mice_gm

The resulting segmentation only covers a section of the whole MRI image. This region is consistent throughout all images, and happens both for T1 and T2.
I have tried changing the orientation from LSP to LPI using fslswapdim but the error prevails.
I attach a screenshot of the segmentation on ITK-Snap and the T1 image i’m using.

vol0001.nii (106.4 KB)

Thank you in advance.

[joshuacwnewton]

Hi @mariamcoris,

Thank you so much for sharing your question on the forum.

Firstly, I notice that your input data appears to have some orientation issues:

1. If you look at the axis labels in the image viewer, the “AP” and “SI” labels don’t match the orientation of the data array.
2. Additionally, I notice that the AP axis (and possibly the LR axis) of the data array is flipped relative to the header information. (In other words, instead of being Posterior-to-Anterior, the data array is Anterior-to-Posterior.)

Once I correct for these two issues, the data instead looks like this in my image viewer:

• The SI axis label now properly reflects the data array.
• The AP axis has been flipped.

However, it turns out that these orientation issues are not actually consequential for the “empty slices” issue that you noticed. Re-running the segmentation command still produces this “empty slice” behavior, specifically in the axis labeled “AP”:

(Note: While the segmentation looks different than before, this is only because I re-oriented the image, such that the “AP” label now refers to a different axis. But, the “empty slices” issue is still present.)

So, I will investigate this “empty slices” issue separately, and follow-up with a new reply once I figure out why this is happening.

Kind regards,
Joshua

[joshuacwnewton]

When I examine the header of the original image using sct_image -header, I see:

dim		    [3, 39, 29, 12, 1, 1, 1, 1]
pixdim		[1.0, 0.078, 0.078, 1.0, 1.0, 1.0, 1.0, 1.0]
sform_name	Aligned Anat
sform_code	2
sto_xyz:1	0.078000 0.000000 0.000000 -3.523000 
sto_xyz:2	0.000000 0.000000 1.000000 -3.204000 
sto_xyz:3	0.000000 -0.078000 0.000000 5.270000 
sto_xyz:4	0.000000 0.000000 0.000000 1.000000 

In other words, each axial slice is [39, 29] with [0.078, 0.078] mm resolution.

However, the actual affine is a bit strange: In the top-left 3x3 matrix, the zooms are not diagonal, as one might expect?

As a result, when I look at the internal state of sct_deepseg (after the images are loaded by our ivadomed deep learning package), I see:

Suspiciously, the “zooms” do not match the axis order of the “data_shape”. I think this could be caused by the affine matrix, and I think this might explain the “empty slices” behavior (wrong zooms → wrong preprocessing → only part of the image is processed).

As a quick, hacky test, I tried to manually edit the affine (not recommended), and I was able to eliminate the “cropping” issues. So, I believe the affine is indeed the root cause of this behavior.

Before I continue further with manually editing the file, though, I want to stop and suggest that you look back to your image acquisition pipeline (DICOM → NIFTI conversion, etc.) and see if you have any steps that modify the affine/data array/header. Ideally, we would want to fix the data issue at the source, rather than performing these manual fixes to the data after the fact.

Kind regards,
Joshua

[mariamcoris]

Hello.
Thanks for your quick response.
As you say, when I tweak the affine so that the zooms are in the diagonal I am able to overcome the issue of it not segmenting all slices.
The data I am currently working with I had received already in nifti format. I will ask for the dicom files so I can do the conversion myself to ensure that there are no problems.
With the manual modification of the affine matrix (I left the translation column untouched), although it segments everything, it seems to do so in a very poor way.

I’m now using the new mouse gm/wm model from the 6.2 update and the results are, although not perfect, much better than before.

Until I can get the dicom files, I will continue working by modifying the affine matrices and using this new model to segment.

Thanks again for your assistance.

Greetings,

Mariam

[joshuacwnewton]

I am very glad to hear that you are seeing improved results with the modified affine plus the new model.

There is one additional note that a colleague of mine shared:

I am thinking that the poor performances might be due to the low axial resolution. The model was trained on high axial resolution images. This might be a feature to improve.

I’m not sure what actions we can take from this observation, but I thought I would share it regardless, maybe as a clue for anyone else who comes across this discussion thread.

---

As an aside: If you are able to find out what happened in the NIfTI conversion process, I would love to hear. (It may help me to diagnose issues for other users in the future if I can understand how affine issues such as this can arise.)

Kind regards,
Joshua