Registration tips

Hi @Evelyne,

I am another SCT developer, and I’m helping @jcohenadad to debug this issue. :slight_smile:

I tried out @Evelyne’s script, and the necessary fixes are quite simple:

  1. Rearrange the files to have the following structure:
    files
  2. Change data_pre to point to the parent folder for the newly-created sub-020 folder.
  3. Remove the - character that is causing the syntax error.

After doing this, I am indeed able to run the script and reproduce the black image issue.


This is the likely cause of your issues. In order to solve this, you could copy the sform into the qform and then re-run the segmentation.

I tried your suggestion (though I had to format the output name differently to match the expectations of the script):

sct_image -i s302300909_sub020-0018-00001-000192-01.nii -set-sform-to-qform \
          -o s302300909_qform_sub020-0018-00001-000192-01.nii

Then, I deleted the previously-generated intermediate files (everything but anat), and re-ran the script with various settings turned on (to regenerate the segmentation):

# ANATOMICAL T1
first_segment_t1 = 1
smooth = 1
second_segment_t1 = 1  # Correct
label_t1 = 1
register_t1_temp = 1  # NOT CORRECT

This requires some manual input for sct_deepseg_sc -centerline-viewer and the disc 3/7 labeling, so I tried to recreate the labels as faithfully as possible.

With the qform/sform fix, I can confirm that the PAM50 template is warped to the subject image and vice-versa. However, the quality of the registration is somewhat poor (mismatched vertebrae), so the registration parameters will likely need to be tweaked:

[animate output image]

[animate output image]

I will now try to investigate why this qform/sform mismatch causes the registration failure in the first place.

Kind regards,
Joshua

1 Like

Thankyou. It works now.

When I try to concatenate the warping fields from t1-temp and t2-t1, I come across the following problem: The dimensions are not the same. I already tried to cut the images with no success. I have now overcome this issue by not doing the concatenation, but just selecting the three warping fields in the sct_apply_transfo function. However, this restricts me from adding grey matter segmentation to improve registration. Any thoughts on this? This is my code and two examples:

    concatenated_warping_field = '/project/3023009.09/evefra/sub-024/spinalcord//project/3023009.09/evefra/sub-024/spinalcord/concat_t2_t1_temp/warp.nii.gz'
    
command = f"sct_image -i /project/3023009.09/evefra/sub-024/spinalcord/register_t1_temp/warp_anat2template.nii.gz /project/3023009.09/evefra/sub-024/spinalcord/register_t2_t1/warp_s302300909_sub024-0009-00001-000001-012s302300909_sub024-0016-00001-000192-01.qform.nii.gz'
    -concat t -o {concatenated_warping_field}"
    process = subprocess.Popen(command, shell=True)
    process.wait()

This is the error:

--
Spinal Cord Toolbox (5.8)

sct_image -i /project/3023009.09/evefra/sub-024/spinalcord/register_t1_temp/warp_anat2template.nii.gz /home/affneu/evefra/Spinalcord/scripts/cropped_image.nii.gz -concat t -o /project/3023009.09/evefra/sub-024/spinalcord//project/3023009.09/evefra/sub-024/spinalcord/concat_t2_t1_temp/warp.nii.gz
--

Traceback (most recent call last):
  File "/home/affneu/evefra/sct_5.8/spinalcordtoolbox/scripts/sct_image.py", line 610, in <module>
    main(sys.argv[1:])
  File "/home/affneu/evefra/sct_5.8/spinalcordtoolbox/scripts/sct_image.py", line 254, in main
    im_out = [concat_data(im_in_list, dim)]
  File "/home/affneu/evefra/sct_5.8/spinalcordtoolbox/image.py", line 909, in concat_data
    data_concat = np.concatenate(dat_list, axis=dim)
  File "<__array_function__ internals>", line 180, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 2, the array at index 0 has a size of 991 and the array at index 1 has a size 192

Data: SURFdrive

Is it for example possible to select multiple -initwarp in the sct_register_multimodal function?

Hi @Evelyne,

My deepest apologies for the late reply.

I don’t believe this is currently possible. I think you are taking the correct approach (by concatenating the warping fields).

Instead of using the sct_image -concat t function, I would recommend using the sct_concat_transfo function instead. ((Warping fields are actually 5D images (as described on this page of SCT’s documentation), so regular image concatenation via the t dimension will not work in this case. Hence, why SCT provides a specialized script for concatenating warping fields.))

Kind regards,
Joshua

1 Like

Thankyou @joshuacwnewton

Did you already figure out why a blank screen occurred? I get another blank screen when improving my registration using sct_multimodal_transfo and gm segmentations. Without the addition of gm, the warping field is working. Some data (t2, initial warp, my result): SURFdrive

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Wednesday 24 may 2023

@author: evefra
"""


# import libraries
import os
import shutil
import subprocess
import fnmatch
import nibabel as nib
import numpy as np
# work in conda env, where SCT is installed. For me: conda activate venv_sct

######################################################################################
# INDICATE WHAT TO DO#
######################################################################################



# COREGISTERING
register_t2_t1 = 0
register_fmri_t2 = 0
t2_to_t1_mult_t1_to_temp = 0
correct_qform_sc = 0
gm_segment = 0
gm_segment_add = 0
epi_to_t2_mult_t2_to_temp = 0
final_warp_apply = 0

# LOOP OVER SUB NUMBERS
# ------------------------------------------------------------------------------

sub_list = [10]
# [1, 2, 4, 5, 6, 8, 9, 10, 11, 13, 14, 16, 17, 18, 24]
for sub in sub_list:

# CORRECT QFORM ANATOMICAL
# -----------------------------------------------------------------------------

    if correct_qform_sc == 1:

        # Define the input
        input_directory_d = os.path.join(
            data_pre, subdirectory, "anat_sc")
        wildcard_pattern_d = 's3*01.nii'
        input_file_d = fnmatch.filter(os.listdir(
            input_directory_d), wildcard_pattern_d)
        input_path_des = os.path.join(input_directory_d, input_file_d[0])

        # define output
        output_directory = os.path.join(
            data_pre, subdirectory, "anat_sc")
        output_file = input_file_d[0].replace("nii", "qform.nii")
        output_path = os.path.join(output_directory, output_file)

        # run command
        command = f"sct_image -i {input_path_des} -set-sform-to-qform -o {output_path}"
        process = subprocess.Popen(command, shell=True)
        process.wait()

        print('qform fix anat sc completed for ', subdirectory)
        
# CALC SEGMENTATION GM
# ----------------------------------------------------------------------------

    if gm_segment == 1:

        # define input
        input_directory_d = os.path.join(
            data_pre, subdirectory, "anat_sc")
        wildcard_pattern_d = 's3*01.qform.nii'
        input_file_d = fnmatch.filter(os.listdir(
            input_directory_d), wildcard_pattern_d)
        input_path_des = os.path.join(input_directory_d, input_file_d[0])
       
        # define output
        output_directory = os.path.join(
            data_pre, subdirectory, "spinalcord", "gm_segment_t2_anat")
        output_file_seg = 'gm_t2.nii'
        output_path = os.path.join(output_directory, output_file_seg)

        # Check if the removed volumes directory exists, create it if necessary
        if not os.path.exists(output_directory):
            os.makedirs(output_directory)

        # define qc
        qc_directory = os.path.join(output_directory, 'qc')

        # Run gm segment
        command = f"sct_deepseg_gm -i {input_path_des} -qc {qc_directory} -o {output_path}"
        process = subprocess.Popen(command, shell=True)
        process.wait()
        print('segment gm completed for ', subdirectory)


# ADD SEGMENTATION GM
# ----------------------------------------------------------------------------
    if gm_segment_add == 1:

        # define input template (always the same)
        input_directory = '/home/affneu/evefra/sct_5.8/data/PAM50/template/PAM50_t1.nii.gz'

        # define input gm segmentation (always the same)
        input_directory_iseg = '/home/affneu/evefra/sct_5.8/data/PAM50/template/PAM50_gm.nii.gz'

        # define destination (T2)
        input_directory_d = os.path.join(
            data_pre, subdirectory, "anat_sc")
        wildcard_pattern_d = 's3*01.nii'
        input_file_d = fnmatch.filter(os.listdir(
            input_directory_d), wildcard_pattern_d)
        input_path_des = os.path.join(input_directory_d, input_file_d[0])

        # define destination segmentation (T2 GM)
        input_directory_des_seg = os.path.join(
            data_pre, subdirectory, "spinalcord", "gm_segment_t2_anat")
        wildcard_pattern_des_seg = 'gm*.nii'
        input_file_des_seg = fnmatch.filter(os.listdir(
            input_directory_des_seg), wildcard_pattern_des_seg)
        input_path_des_seg = os.path.join(
            input_directory_des_seg, input_file_des_seg[0])

        # define init warp
        input_directory_warp = os.path.join(
            data_pre, subdirectory, "spinalcord", "concat_t2_t1_temp")
        wildcard_pattern_warp = 'warp_t22temp.nii.gz'
        input_file_warp = fnmatch.filter(os.listdir(
            input_directory_warp), wildcard_pattern_warp)
        input_path_des_warp = os.path.join(
            input_directory_warp, input_file_warp[0])

        # define output warp
        output_directory = os.path.join(
            data_pre, subdirectory, "spinalcord", "concat_t2_t1_temp")
        if not os.path.exists(output_directory):
            os.makedirs(output_directory)
        output_file = 'gm_warp_t22temp.nii.gz'
        output_path_warp = os.path.join(output_directory, output_file)
        
        print(input_directory)
        print(input_directory_iseg)
        print(input_path_des)
        print(input_path_des_seg)
        print(input_path_des_warp)
        print(output_path_warp)        

        # run warping calculation
        command = f"sct_register_multimodal -i {input_directory} -iseg {input_directory_iseg} -d {input_path_des} -dseg {input_path_des_seg} -initwarp {input_path_des_warp} -owarp {output_path_warp}"
        process = subprocess.Popen(command, shell=True)
        process.wait()
        print('add gm to t2-temp warp completed for ', subdirectory)

Yes, we did! :slightly_smiling_face: The earlier blank image was caused by the mismatched qform and sform matrices in the input image. If you are interested, there is a more technical description of the internal cause of the error here. But, in short, the best way to avoid this is by making sure that the sform and qform matrices match in your input anat image before producing the segmentation/labels/registration/etc.

However, looking at the sample images you shared, it looks like you already solved the sform/qform issues. So, I will begin investigating the new issue with sct_register_multimodal now. :slight_smile:

Kind regards,
Joshua

Hi @Evelyne,

It looks like the script you shared in this comment is only partially copied and pasted. Some of the steps listed below are missing:

# COREGISTERING
register_t2_t1 = 0
register_fmri_t2 = 0
t2_to_t1_mult_t1_to_temp = 0
correct_qform_sc = 0
gm_segment = 0
gm_segment_add = 0
epi_to_t2_mult_t2_to_temp = 0
final_warp_apply = 0

Could you please share the entire processing script by uploading your “.py” file in your reply? (I’d like to try running the full pipeline (segmentation, labeling, registration, etc.) from start to finish, just to make sure I can reproduce the issue on my machine.)

Kind regards,
Joshua

Dear @joshuacwnewton,

Thank you for helping out.
Hereby the script:
sct_preproces.py (37.7 KB)

1 Like

I tried out your script with “gm_segment = 1” and “gm_segment_add = 1” plus the input files you provided. I did have to make 1 small change to your script, though:

    if gm_segment_add == 1:

        # define destination (T2)
        input_directory_d = os.path.join(
            data_pre, subdirectory, "anat_sc")
-        wildcard_pattern_d = 's3*01.nii'
+        wildcard_pattern_d = 's3*01.qform.nii'

After making this change, I got the following results:

  • I got an empty/blank “PAM50_T1_reg.nii.gz” file, just as you describe.
  • The GM-informed warping field “gm_warp_t22temp.nii.gz” is also empty (all voxels are 3.4028234663852886e+38.)
  • The input GM segmentation looks good. :tada:
  • However, the concatenated -initwarp warping field “warp_t22temp.nii.gz” is empty (all voxels are 3.4028234663852886e+38).

So, I think the “sct_concat_transfo” command (that produced the file “warp_t22temp.nii.gz”) might be the issue here. However, I’m unable to troubleshoot the “t2_to_t1_mult_t1_to_temp” step in your script because I don’t have the necessary files to reproduce your steps. I was mistaken. Please see this comment.

Could you please provide all of the input files for sub-010 (both T1 and T2, and anything else that may be necessary, if possible) so that I can follow each step in the script from start to finish?

Thank you kindly,
Joshua

Dear @joshuacwnewton

Thank you for helping out.
I have also uploaded a T1 and one EPI image.

I do not think the problem is in warp_t22temp.nii.gz. Whenever I apply this warp, I can successfully register the T2 to PAM50. I think the problem is in the next step when the initial warp is combined with the GM segmentation.

My deepest apologies! You are absolutely correct. I had forgotten to adjust my contrast settings on my image viewer (FSLeyes), and so warp_t22temp.nii.gz appeared empty, when it was indeed not.

Looking into this further now. I will reply when I have a further update. :slight_smile:

Hi @Evelyne,

I took a closer look at the final sct_register_multimodal command inside the “gm_segment_add” step, and compared it to the command used in the GM-informed registration between the PAM50 template and T2* data - Spinal Cord Toolbox documentation page of the SCT tutorials.

I noticed the following differences:

  1. For GM-informed registration, the WM segmentation should be used (to retain the shape of both the cord and the GM)
  2. The version of the PAM50 template used should match the contrast of the -d image (in this case, I think PAM50_t2.nii.gz should be used instead of PAM50_t1.nii.gz, specifically because we are using a T2 anat image here).
  3. A different set of registration parameters are used for -param, due to this being a “fine-tuning” registration thanks to the use of -initwarp (rather than a more coarse registration).
  4. In sct_register_multimodal, -initwarp gets applied to the source image (-i), and -initwarpinv gets applied to the dest image (-d).
    • In your command, warp_t22temp.nii.gz (i.e. anat->template) is passed to -initwarp, so it will be applied to the source image.
    • However, the source image (-i) in this case is the template image. So, I believe the wrong initial warping field is being applied to the PAM50 template.

I fixed the 1st, 2nd, and 3rd discrepancies. However, this still resulted in a blank registration warping field.

I then fixed the 4th discrepancy by swapping the source and dest images, that way warp_t22temp.nii.gz gets applied to the T2 image and not the PAM50 template image. This successfully generated a new GM-informed gm_warp_t22temp.nii.gz:

However, one small detail to mention regarding my fix is that, because only -initwarp was provided, only the anat->template warping field will be generated. If you want to also produce the inverse (template->anat) warping field, you will need to supply a second initial warping field (i.e. warp_temp2t2.nii.gz), which I think will mean going back and adding a second sct_concat_transfo command to your earlier step.

Anyway, here are the total changes I made to your script: Changes made to sct_preprocess.py - Diff Checker (Note: Please disregard the filepath and subprocess.Popen changes I made! I needed to make those changes to get the script to work on my machine.)

Kind regards,
Joshua

1 Like

Dear @joshuacwnewton,

Thank you for all your help.
I managed to complete the registration.
During quality checks, I noticed the following:

  • Registration is always off/less good in caudal parts of the cervical spine
    example_1

  • When I compare the images warped with the current approach (warping field from EPI - T2 concatenated with WM improved warping field from T2 to PAM50), with a simpler approach (only applying the three warping fields (EPI-T2, T2-T1, T1-PAM50), the current approach gives smaller images. It seems that the more rostral parts are partly cut off.

(red = current approach, green = simple approach)
example_2

Is there any way to solve these issues?
Best,
Evelyne

Hi @Evelyne,

I’m sorry to hear about these strange registration results!

Could you please share the full updated “sct_preprocess.py” script, just so we can see exactly the commands you are using for both the current approach and the simpler approaches?

Kind regards,
Joshua

Dear @joshuacwnewton ,

Hereby the script:
sct_preproces.py (42.3 KB)

The simple approach is called with final_warp_apply_old.

One note, I wanted to use the WM segmentation, but sct_maths -sub was not working for me. To get a quick workaround, I used the MATLAB image calculator for this. I will upload one of these pictures, so you can take a look. SURFdrive

Thank you in advance.
Best,
Evelyne

Dear @joshuacwnewton,

Did you have any time to look into the registration?

Best,
Evelyne

Dear @Evelyne,

My apologies for the delay in responding. I did take a look, but admittedly registration parameter adjustment is not my strongest skill (I am a software developer for SCT, rather than a researcher).

So, I have asked my colleagues for assistance on this issue, and @sandrinebedard has been taking a look. As well, @jcohenadad is now back from vacation, so he may also have some feedback.

Thank you kindly for your patience,
Joshua

Ah! It turns out that @sandrinebedard has been working on a solution, and has shared with me a summary of her conclusions, quoted below:

  • -l flag was used instead of -ldisc for registration to PAM50 template (since discs labels are used here) → so T1 reg to PAM50 was poor
  • Registration T1 → T2 star was poor → use the segmentation and centerofmass instead
  • Registration EPI → T2star was poor → use the segmentation and centerofmass instead
  • I would suggest here to also test out using the T2star warping field to initialize the warping field and directly register EPI to the template
  • As a general suggestion, I would suggest to always check the intermediate steps of each registration before putting them together – using a lot of intermediate registrations can accumulate errors too.

She also shared a modified script with her changes: sct_preproces.py (45.7 KB)

The changes that were made are as follows: https://www.diffchecker.com/x9ti3koj/

Could you please test out these suggestions, then let us know how it goes? :slight_smile:

Kind regards,
Joshua

1 Like

Dear @joshuacwnewton,

Thank you for your help.

The registration is overall better, but not perfect yet.
I applied the inverse warping field to the atlases and still see a lot of mismatches at the caudal cervical parts. Also, for some parts of the spinal cord, the masks are not warped and are just blank (see missing parts in images below). I also tried to directly estimate the EPI-temp registration with initialization of the GM-informed T2s-temp warping field (as suggested in previous messages), but this did improve the registration. Overall, the individual differences are big, which leads to no results in my group-level seed-based analysis (with spinal cord horns/hemicords as seeds). Any thoughts on this?