Using isct_ functions

Alexandra_Tinnermann · March 23, 2020, 12:53pm

Hi, to better test the function, I copied one single file to my home directory which is also my working directory :

tinnermann@isn0068ebea3a78:~$ pwd 
/home/tinnermann
tinnermann@isn0068ebea3a78:~$ ls
fmri_moco_mean.nii  scripts  sct

Then I tried to register this image to itself which of course doesn’t make sense but for testing purposes I thought it is ok and I eliminated argument by argument to end up with these two calls:

isct_antsRegistration --metric "MeanSquares[fmri_moco_mean.nii,fmri_moco_mean.nii]"

isct_antsRegistration --float 0

None of that gave any error message or any output at all. So I misspelled the filename:

 isct_antsRegistration --metric "MeanSquares[fmri_moco_mean.nii,fmri_moco_meann.nii]"

Also no error message.

To clarify some arguments: the mask is of course optional and in previous testings I already deleted that argument but it didn’t change anything. The script I provided is a script that I used for my sct preprocessing. So this paticular script was used to smooth my epi images. Instead of the sct funtion within the loop that loops over all files, I replaced it by the isct function but kept the file names (again, doesn’t make sense here but it was just for testing purposes).

anon33887296 · March 23, 2020, 3:18pm

I checked into isct_antsRegistration -h and noticed there’s a --verbose option. If I run what you ran I also get nothing:

$ isct_antsRegistration --metric "MeanSquares[fmri_moco_mean.nii,fmri_moco_mean.nii]"

there is this hidden output in the return code:

$ isct_antsRegistration --metric "MeanSquares[fmri_moco_mean.nii,fmri_moco_mean.nii]"; echo $?
1

However if I add --verbose I see the error:

$ isct_antsRegistration --verbose --metric "MeanSquares[fmri_moco_mean.nii,fmri_moco_mean.nii]"; echo $?
All_Command_lines_OK
Image dimensionality not specified.  See command line option --dimensionality
1

If I try running your original command with this I see the immediate problem:

$   isct_antsRegistration \
>         --verbose \
>         --dimensionality 3 \
>         --float 0 \
>         --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]' \
>         --output '[X_${files}_,X_${files}_Warped.nii.gz]' \
>         --interpolation 'BSpline' \
>         --winsorize-image-intensities '[0.005,0.995]' \
>         --use-histogram-matching 0 \
>         --transform 'Rigid[1]' \
>         --shrink-factors 1 \
>         --smoothing-sigmas 0 \
>         -x mask_fmri_65mm.nii
All_Command_lines_OK
Using double precision for computations.
ERROR: the convergence option ('-c') must be specified.  See help menu.

Alexandra_Tinnermann · March 23, 2020, 3:35pm

Ahhh, thank you very much! That should solve all problems at least in the sense that I know what is wrong with my code!

anon33887296 · March 23, 2020, 3:39pm

If I add that argument with some value it gets further but there’s more problems, because it doesn’t have access to the right files:

$ files=0
$ isct_antsRegistration \
>         --verbose \
>         --dimensionality 3 \
>         --float 0 \
>         --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]' \
>         --output '[X_${files}_,X_${files}_Warped.nii.gz]' \
>         --interpolation 'BSpline' \
>         --winsorize-image-intensities '[0.005,0.995]' \
>         --use-histogram-matching 0 \
>         --transform 'Rigid[1]' \
>         --shrink-factors 1 \
>         --smoothing-sigmas 0 \
>         -c 10 \
>         -x mask_fmri_65mm.nii
All_Command_lines_OK
Using double precision for computations.
  Reading mask(s).
    Registration stage 0
 file mask_fmri_65mm.nii does not exist . 
      No fixed mask
  number of levels = 1
  fixed image: fmri_moco_norm_T0000.nii
  moving image: fmri_moco_norm_T`printf %04d ${files}`.nii
 file fmri_moco_norm_T0000.nii does not exist . 
 file fmri_moco_norm_T`printf %04d ${files}`.nii does not exist . 
Segmentation fault (core dumped)

This particular error is demonstrating Using isct_ functions because it’s complaining about a file with “%04d” in the filename. We can fix that first:

$   files=0
(venv_sct) [kousu@requiem spinalcordtoolbox]$   isct_antsRegistration \
>         --verbose \
>         --dimensionality 3 \
>         --float 0 \
>         --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T'$(printf %04d ${files})'.nii,1,32,Regular,0.2]' \
>         --output '[X_'${files}'_,X_'${files}'_Warped.nii.gz]' \
>         --interpolation 'BSpline' \
>         --winsorize-image-intensities '[0.005,0.995]' \
>         --use-histogram-matching 0 \
>         --transform 'Rigid[1]' \
>         --shrink-factors 1 \
>         --smoothing-sigmas 0 \
>         -c 10 \
>         -x mask_fmri_65mm.nii
All_Command_lines_OK
Using double precision for computations.
  Reading mask(s).
    Registration stage 0
 file mask_fmri_65mm.nii does not exist . 
      No fixed mask
  number of levels = 1
  fixed image: fmri_moco_norm_T0000.nii
  moving image: fmri_moco_norm_T0000.nii
 file fmri_moco_norm_T0000.nii does not exist . 
 file fmri_moco_norm_T0000.nii does not exist . 
Segmentation fault (core dumped)

Now I need to find some real files to test with. When I ran ./install_sct I ended up with sct_testing_data/ with a bunch of sample files.
I’ll pick some files out to try to analyse. I don’t know if these files make any sense for the task but they should be enough to get the program to run.
I’ll also do this in a fresh working directory to avoid confusion.

$ mkdir issue346
$ cd issue346/
$ cp ../sct_testing_data/fmri/fmri_r_T0000.nii.gz .
$ ls -l
total 48
-rw-r--r-- 1 kousu kousu 17410 Mar 23 11:32 fmri_r_T0000.nii.gz
$ sha256sum *.nii.gz  # just in case you want to verify this work precisely
53693deeecd15ceab11b210c9de10516a186291a18bc3da3cbfa37b2b16bfbff  fmri_r_T0000.nii.gz

Now let’s try that command again:


$ files=0
$ isct_antsRegistration \
>         --verbose \
>         --dimensionality 3 \
>         --float 0 \
>         --metric 'MeanSquares[fmri_r_T0000.nii.gz,fmri_r_T0000.nii.gz,1,32,Regular,0.2]' \
>         --output '[output_issue346,output_issue346_Warped.nii.gz]' \
>         --interpolation 'BSpline' \
>         --winsorize-image-intensities '[0.005,0.995]' \
>         --use-histogram-matching 0 \
>         --transform 'Rigid[1]' \
>         --shrink-factors 1 \
>         --smoothing-sigmas 0 \
>         -c 10 \
>         -x mask_fmri_65mm.nii
All_Command_lines_OK
Using double precision for computations.
  Reading mask(s).
    Registration stage 0
 file mask_fmri_65mm.nii does not exist . 
      No fixed mask
  number of levels = 1
  fixed image: fmri_r_T0000.nii.gz
  moving image: fmri_r_T0000.nii.gz
Dimension = 3
Number of stages = 1
Use Histogram Matching false
Winsorize image intensities true
Lower quantile = 0.005
Upper quantile = 0.995
Stage 1 State
   Image metric = MeanSquares
     Fixed image = Image (0x4f4cbd0)
  RTTI typeinfo:   itk::Image<double, 3u>
  Reference Count: 2
  Modified Time: 644
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 0
  UpdateMTime: 466
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [34, 34, 6]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [34, 34, 6]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [34, 34, 6]
  Spacing: [1.91176, 1.91176, 16.8]
  Origin: [-29.9549, 34.5494, -23.8665]
  Direction: 
0.999561 8.49015e-08 0.0296327
0.00222132 -0.997187 -0.0749259
-0.0295494 -0.0749588 0.996749

  IndexToPointMatrix: 
1.91093 1.62312e-07 0.49783
0.00424664 -1.90639 -1.25876
-0.0564915 -0.143304 16.7454

  PointToIndexMatrix: 
0.522847 0.00116192 -0.0154566
4.44724e-08 -0.521605 -0.0392092
0.00176385 -0.00445988 0.0593303

  Inverse Direction: 
0.999561 0.00222132 -0.0295494
8.50207e-08 -0.997187 -0.0749588
0.0296327 -0.0749259 0.996749

  PixelContainer: 
    ImportImageContainer (0x4f42c10)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, double>
      Reference Count: 1
      Modified Time: 463
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0x4f55230
      Container manages memory: true
      Size: 6936
      Capacity: 6936

     Moving image = Image (0x4f50be0)
  RTTI typeinfo:   itk::Image<double, 3u>
  Reference Count: 2
  Modified Time: 645
  Debug: Off
  Object Name: 
  Observers: 
    none
  Source: (none)
  Source output name: (none)
  Release Data: Off
  Data Released: False
  Global Release Data: Off
  PipelineMTime: 0
  UpdateMTime: 642
  RealTimeStamp: 0 seconds 
  LargestPossibleRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [34, 34, 6]
  BufferedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [34, 34, 6]
  RequestedRegion: 
    Dimension: 3
    Index: [0, 0, 0]
    Size: [34, 34, 6]
  Spacing: [1.91176, 1.91176, 16.8]
  Origin: [-29.9549, 34.5494, -23.8665]
  Direction: 
0.999561 8.49015e-08 0.0296327
0.00222132 -0.997187 -0.0749259
-0.0295494 -0.0749588 0.996749

  IndexToPointMatrix: 
1.91093 1.62312e-07 0.49783
0.00424664 -1.90639 -1.25876
-0.0564915 -0.143304 16.7454

  PointToIndexMatrix: 
0.522847 0.00116192 -0.0154566
4.44724e-08 -0.521605 -0.0392092
0.00176385 -0.00445988 0.0593303

  Inverse Direction: 
0.999561 0.00222132 -0.0295494
8.50207e-08 -0.997187 -0.0749588
0.0296327 -0.0749259 0.996749

  PixelContainer: 
    ImportImageContainer (0x4f3ecd0)
      RTTI typeinfo:   itk::ImportImageContainer<unsigned long, double>
      Reference Count: 1
      Modified Time: 639
      Debug: Off
      Object Name: 
      Observers: 
        none
      Pointer: 0x4f65d00
      Container manages memory: true
      Size: 6936
      Capacity: 6936

     Weighting = 1
     Sampling strategy = regular
     Number of bins = 32
     Radius = 4
     Sampling percentage  = 0.2
   Transform = Rigid
     Gradient step = 1
     Update field sigma (voxel space) = 0
     Total field sigma (voxel space) = 0
     Update field time sigma = 0
     Total field time sigma  = 0
     Number of time indices = 0
     Number of time point samples = 0
Registration using 1 total stages.

Stage 0
  iterations = 10
  convergence threshold = 1e-06
  convergence window size = 10
  number of levels = 1
  using the MeanSquares metric (weight = 1)
  preprocessing:  winsorizing the image intensities
  Shrink factors (level 1 out of 1): [1, 1, 1]
  smoothing sigmas per level: [0]
  regular sampling (percentage = 0.2)

*** Running Euler3DTransform registration ***

DIAGNOSTIC,Iteration,metricValue,convergenceValue,ITERATION_TIME_INDEX,SINCE_LAST
 2DIAGNOSTIC,     1, 0.000000000000e+00, 1.797693134862e+308, 6.6512e-03, 6.6512e-03, 
 2DIAGNOSTIC,     2, 0.000000000000e+00, 1.797693134862e+308, 8.3532e-03, 1.7021e-03, 
 2DIAGNOSTIC,     3, 0.000000000000e+00, 1.797693134862e+308, 1.0092e-02, 1.7390e-03, 
 2DIAGNOSTIC,     4, 0.000000000000e+00, 1.797693134862e+308, 1.1875e-02, 1.7829e-03, 
 2DIAGNOSTIC,     5, 0.000000000000e+00, 1.797693134862e+308, 1.3513e-02, 1.6379e-03, 
 2DIAGNOSTIC,     6, 0.000000000000e+00, 1.797693134862e+308, 1.5179e-02, 1.6661e-03, 
 2DIAGNOSTIC,     7, 0.000000000000e+00, 1.797693134862e+308, 1.6851e-02, 1.6720e-03, 
 2DIAGNOSTIC,     8, 0.000000000000e+00, 1.797693134862e+308, 1.8378e-02, 1.5271e-03, 
 2DIAGNOSTIC,     9, 0.000000000000e+00, 1.797693134862e+308, 1.9923e-02, 1.5450e-03, 
  Elapsed time (stage 0): 2.2582e-02


Total elapsed time: 2.3342e-02

This produced two new files:

$ ls -l
total 80
-rw-r--r-- 1 kousu kousu 17410 Mar 23 11:32 fmri_r_T0000.nii.gz
-rw-r--r-- 1 kousu kousu   193 Mar 23 11:35 output_issue3460GenericAffine.mat
-rw-r--r-- 1 kousu kousu 27998 Mar 23 11:35 output_issue346_Warped.nii.gz

It seems to have ignored the mask, like you suggested.

anon33887296 · March 23, 2020, 3:43pm

Right now I am thinking the lessons here are:

Several of the errors cases are currently hidden by verbose = 0, and that’s a mistake on our end
The -x mask option is non-mandatory, which is confusing.
It’s important to pay attention to single vs double quotes and your working directory when in shell, and to double check each subpiece of a command or a script, which you can do with set -x and by using echo ....

Let me know if that solves your problem and if I can help any more, and thanks for using SCT

Alexandra_Tinnermann · March 24, 2020, 1:49pm

Just to let you know, with the following code it still breaks with an error message but that seems to be an ANTs problem and is not related to SCT and I thought I still share it because syntax-wise it seems to be fine right now:

isct_antsRegistration \
	--verbose \
        --dimensionality 3 \
        --float 0 \
        --metric "MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]" \
        --output "[X_,Warped.nii.gz]" \
        --interpolation "BSpline" \
        --winsorize-image-intensities "[0.005,0.995]" \
        --use-histogram-matching 0 \
        --transform "Rigid[1]" \
        --smoothing-sigmas "[0]" \
	--shrink-factors "[1]" \
	--convergence "[1.e-8]" \ 
	echo "RESULT = $?"

The error message says “Segmentation fault” and in the ANTs helpline a similar error was discussed in relation to image size which shouldn’t be any problem with spinal data. I will play around with parameters to see if that they are problematic.

Thanks a lot for your help!

anon33887296 · March 24, 2020, 2:26pm

Hey Alexandra,

I get a different error with that command than you:

$ isct_antsRegistration \
> --verbose \
>         --dimensionality 3 \
>         --float 0 \
>         --metric "MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]" \
>         --output "[X_,Warped.nii.gz]" \
>         --interpolation "BSpline" \
>         --winsorize-image-intensities "[0.005,0.995]" \
>         --use-histogram-matching 0 \
>         --transform "Rigid[1]" \
>         --smoothing-sigmas "[0]" \
> --shrink-factors "[1]" \
> --convergence "[1.e-8]" \
> echo "RESULT = $?"
All_Command_lines_OK
Using double precision for computations.
Exception Object caught: 

itk::ExceptionObject (0x3e71670)
Location: "unknown" 
File: /home/brain/ANTs/Utilities/antsCommandLineParser.h
Line: 126
Description: itk::ERROR: CommandLineParser(0x3e52960): ERROR: Parse error occured during command line argument processing
ERROR: Unable to convert 'RESULT = 0' to type 'j' as unsigned int

You need a ; or a newline on that second-last line, not a \ because \ means continue the current command but echo .. is a separate command.

If I fix that I get

$ isct_antsRegistration \
> --verbose \
>         --dimensionality 3 \
>         --float 0 \
>         --metric "MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]" \
>         --output "[X_,Warped.nii.gz]" \
>         --interpolation "BSpline" \
>         --winsorize-image-intensities "[0.005,0.995]" \
>         --use-histogram-matching 0 \
>         --transform "Rigid[1]" \
>         --smoothing-sigmas "[0]" \
> --shrink-factors "[1]" \
> --convergence "[1.e-8]"
All_Command_lines_OK
Using double precision for computations.
Exception Object caught: 

itk::ExceptionObject (0x48eff20)
Location: "unknown" 
File: /home/brain/ANTs/Utilities/antsCommandLineParser.h
Line: 126
Description: itk::ERROR: CommandLineParser(0x48d1870): ERROR: Parse error occured during command line argument processing
ERROR: Unable to convert '1.e-8' to type 'j' as unsigned int



$ echo "RESULT = $?"
RESULT = 1

The help for -c says

$ isct_antsRegistration -h
...
     -c, --convergence MxNxO
                       [MxNxO,<convergenceThreshold=1e-6>,<convergenceWindowSize=10>]
...

so you need to give two arguments if you want to set a threshold. For example, I got this to run for me, using the sample file from before:

isct_antsRegistration \
	--verbose \
        --dimensionality 3 \
        --float 0 \
        --metric "MeanSquares[fmri_r_T0000.nii.gz,fmri_r_T0000.nii.gz,1,32,Regular,0.2]" \
        --output "[X_,Warped.nii.gz]" \
        --interpolation "BSpline" \
        --winsorize-image-intensities "[0.005,0.995]" \
        --use-histogram-matching 0 \
        --transform "Rigid[1]" \
        --smoothing-sigmas "[0]" \
	--shrink-factors "[1]" \
	--convergence "[30x40x9,1.e-8]";
	echo "RESULT = $?"

anon33887296 · March 24, 2020, 2:29pm

By the way I have noticed that this program will segfault for lots of reasons: for not being able to find a file, for misparsing some arguments, etc. Sometimes there’s a clean error message and sometimes it’s the environment.

anon33887296 · March 24, 2020, 2:30pm

Also, the -c arguments seem to be, in order, iterations, convergence, and window size, for example I saw in the --verbose output:

  iterations = 30x40x9
  convergence threshold = 1e-08
  convergence window size = 10

Alexandra_Tinnermann · March 24, 2020, 2:58pm

I used Ants once in python and there arguments are a little easier to understand because as in your example it would be three seperate arguments and not several ones in one. So thanks for pointing that out. If you say 30x40x9, several other arguments such as smoothing sigmas also need three numbers. This code finally runs:

isct_antsRegistration \
	--verbose \
        --dimensionality 3 \
        --float 0 \
        --metric "MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]" \
        --output "[X_,Warped.nii.gz]" \
        --interpolation "BSpline" \
        --winsorize-image-intensities "[0.005,0.995]" \
        --use-histogram-matching 0 \
        --transform "Rigid[1]" \
        --smoothing-sigmas 0 \
	--shrink-factors 1 \
	--convergence "[10,1.e-8]"