Using isct_ functions

Hello there,

I would like to use antsRegistration within a script that otherwise uses sct functions. If I call the isct_antsRegistration function with parameters nothing happens, not even an error message or anything.

I call the function within a loop and after the isct function is called the loop is exited after the first iteration. Even calling the function without any arguments would lead to the same results.

Is there something I am missing or doing fundamentally wrong while using isct functions? Is it possible to use the isct functions from a shell script?

Any help or hint would be appreciated!

Thanks,
Alexandra

Here is the code that I used:

/usr/local/sct_4.2.1/bin/isct_antsRegistration
–dimensionality 3
–float 0
–metric ‘MeanSquares[imgX.nii,img${files}.nii,1,32,Regular,0.2]’
–output ‘[img${files}_, img${files}_Warped.nii.gz]’
–interpolation ‘BSpline’
–winsorize-image-intensities ‘[0.005,0.995]’
–use-histogram-matching 0
–transform ‘Rigid[1]’
–shrink-factors 1
–smoothing-sigmas 0
-x mask.nii

Hi Alexandra,
you don’t need to specify the absolute PATH to ANTs function. Can you try running this and let me know the output:

isct_antsRegistration

Hi Julien,

thank you very much for your reply! Today and tomorrow I am helping other people with spinal preprocessing so I cannot run any code for now because data and sct installation are on their system and we finished for today but in our first versions we did not use the absolute path. We added that after figuring out how the toolbox calls this function.
We also tried to just call isct_antsRegistration without any arguments and in this case it outputed the help/documentation but funnily only once and not as many times as the supposed number of loop iterations.

We also tried to just call isct_antsRegistration without any arguments and in this case it outputed the help/documentation but funnily only once and not as many times as the supposed number of loop iterations.

hum, strange… but this is really specific to your batch script so I can’t really help unless I can reproduce your code on my end.

Ok, I will try the code on my computer and see what happens there.
Thanks for your help. At least now I know that it should work in theory :slight_smile:

Hi Alexandra,

Just following up on this issue: is it solved on your end or do you still need assistance?

Hi Julien,
sorry with all that stuff going on, I didn’t find the time to check. I know ran isct_antsRegistration in a for loop with and without arguments on our server and the result is the same. Without arguments, the help is printed once and with arguments basically nothing happens.

It is not that urgent for now but do you think it would help if I sent you the bash script or post it here?

Best,
Alexandra

sure, if you send me the exact syntax with terminal output, i can try on my end to see if i can reproduce the behaviour on your end.

Here is my script

test_isct.sh (1.0 KB)

that generates the following output:

tinnermann@revelations:~/sct_scripts $ ~/sct_scripts/test_isct.sh
Sub4
Run1
File0

COMMAND:
     antsRegistration
          This program is a user-level registration application meant to utilize
          ITKv4-only classes. The user can specify any number of "stages" where a stage
          consists of a transform; an image metric; and iterations, shrink factors, and
          smoothing sigmas for each level. Note that explicitly setting the
          dimensionality, metric, transform, output, convergence, shrink-factors, and
          smoothing-sigmas parameters is mandatory.

OPTIONS:
     --version
          Get Version Information.

     -d, --dimensionality 2/3
          This option forces the image to be treated as a specified-dimensional image. If
          not specified, we try to infer the dimensionality from the input image.

     -o, --output outputTransformPrefix
[outputTransformPrefix,<outputWarpedImage>,<outputInverseWarpedImage>]
          Specify the output transform prefix (output format is .nii.gz ). Optionally, one
          can choose to warp the moving image to the fixed space and, if the inverse
          transform exists, one can also output the warped fixed image. Note that only the
          images specified in the first metric call are warped. Use antsApplyTransforms to
          warp other images using the resultant transform(s).

     -j, --save-state saveSateAsTransform
          Specify the output file for the current state of the registration. The state
          file is written to an hdf5 composite file. It is specially usefull if we want to
          save the current state of a SyN registration to the disk, so we can load and
          restore that later to continue the next registration process directly started
          from the last saved state. The output file of this flag is the same as the
          write-composite-transform, unless the last transform is a SyN transform. In that
          case, the inverse displacement field of the SyN transform is also added to the
          output composite transform. Again notice that this file cannot be treated as a
          transform, and restore-state option must be used to load the written file by
          this flag.

     -k, --restore-state restoreStateAsATransform
          Specify the initial state of the registration which get immediately used to
          directly initialize the registration process. The flag is mutually exclusive
          with other intialization flags.If this flag is used, none of the
          initial-moving-transform and initial-fixed-transform cannot be used.

     -a, --write-composite-transform 1/(0)
          Boolean specifying whether or not the composite transform (and its inverse, if
          it exists) should be written to an hdf5 composite file. This is false by default
          so that only the transform for each stage is written to file.
          <VALUES>: 0

     -p, --print-similarity-measure-interval <unsignedIntegerValue>
          Prints out the CC similarity metric measure between the full-size input fixed
          and the transformed moving images at each iteration a value of 0 (the default)
          indicates that the full scale computation should not take placeany value greater
          than 0 represents the interval of full scale metric computation.
          <VALUES>: 0

     --write-interval-volumes <unsignedIntegerValue>
          Writes out the output volume at each iteration. It helps to present the
          registration process as a short movie a value of 0 (the default) indicates that
          this option should not take placeany value greater than 0 represents the
          interval between the iterations which outputs are written to the disk.
          <VALUES>: 0

     -z, --collapse-output-transforms (1)/0
          Collapse output transforms. Specifically, enabling this option combines all
          adjacent transforms wherepossible. All adjacent linear transforms are written to
          disk in the forman itk affine transform (called xxxGenericAffine.mat).
          Similarly, all adjacent displacement field transforms are combined when written
          to disk (e.g. xxxWarp.nii.gz and xxxInverseWarp.nii.gz (if available)).Also, an
          output composite transform including the collapsed transforms is written to the
          disk (called outputCollapsed(Inverse)Composite).
          <VALUES>: 1

     -i, --initialize-transforms-per-stage (1)/0
          Initialize linear transforms from the previous stage. By enabling this option,
          the current linear stage transform is directly intialized from the previous
          stage's linear transform; this allows multiple linear stages to be run where
          each stage directly updates the estimated linear transform from the previous
          stage. (e.g. Translation -> Rigid -> Affine).
          <VALUES>: 0

     -n, --interpolation Linear
                         NearestNeighbor
MultiLabel[<sigma=imageSpacing>,<alpha=4.0>]
Gaussian[<sigma=imageSpacing>,<alpha=1.0>]
                         BSpline[<order=3>]
                         CosineWindowedSinc
                         WelchWindowedSinc
                         HammingWindowedSinc
                         LanczosWindowedSinc
          Several interpolation options are available in ITK. These have all been made
          available. Currently the interpolator choice is only used to warp (and possibly
          inverse warp) the final output image(s).

     -g, --restrict-deformation PxQxR
          This option allows the user to restrict the optimization of the displacement
          field, translation, rigid or affine transform on a per-component basis. For
          example, if one wants to limit the deformation or rotation of 3-D volume to the
          first two dimensions, this is possible by specifying a weight vector of '1x1x0'
          for a deformation field or '1x1x0x1x1x0' for a rigid transformation.
          Low-dimensional restriction only works if there are no preceding
          transformations.

     -q, --initial-fixed-transform initialTransform
[initialTransform,<useInverse>]
[fixedImage,movingImage,initializationFeature]
          Specify the initial fixed transform(s) which get immediately incorporated into
          the composite transform. The order of the transforms is stack-esque in that the
          last transform specified on the command line is the first to be applied. In
          addition to initialization with ITK transforms, the user can perform an initial
          translation alignment by specifying the fixed and moving images and selecting an
          initialization feature. These features include using the geometric center of the
          images (=0), the image intensities (=1), or the origin of the images (=2).

  -r, --initial-moving-transform initialTransform
[initialTransform,<useInverse>]
[fixedImage,movingImage,initializationFeature]
          Specify the initial moving transform(s) which get immediately incorporated into
          the composite transform. The order of the transforms is stack-esque in that the
          last transform specified on the command line is the first to be applied. In
          addition to initialization with ITK transforms, the user can perform an initial
          translation alignment by specifying the fixed and moving images and selecting an
          initialization feature. These features include using the geometric center of the
          images (=0), the image intensities (=1), or the origin of the images (=2).

     -m, --metric CC[fixedImage,movingImage,metricWeight,radius,<samplingStrategy={None,Regular,Random}>,<samplingPercentage=[0,1]>]
MI[fixedImage,movingImage,metricWeight,numberOfBins,<samplingStrategy={None,Regular,Random}>,<samplingPercentage=[0,1]>]
Mattes[fixedImage,movingImage,metricWeight,numberOfBins,<samplingStrategy={None,Regular,Random}>,<samplingPercentage=[0,1]>]
MeanSquares[fixedImage,movingImage,metricWeight,radius=NA,<samplingStrategy={None,Regular,Random}>,<samplingPercentage=[0,1]>]
Demons[fixedImage,movingImage,metricWeight,radius=NA,<samplingStrategy={None,Regular,Random}>,<samplingPercentage=[0,1]>]
GC[fixedImage,movingImage,metricWeight,radius=NA,<samplingStrategy={None,Regular,Random}>,<samplingPercentage=[0,1]>]
ICP[fixedPointSet,movingPointSet,metricWeight,<samplingPercentage=[0,1]>,<boundaryPointsOnly=0>]
PSE[fixedPointSet,movingPointSet,metricWeight,<samplingPercentage=[0,1]>,<boundaryPointsOnly=0>,<pointSetSigma=1>,<kNeighborhood=50>]
JHCT[fixedPointSet,movingPointSet,metricWeight,<samplingPercentage=[0,1]>,<boundaryPointsOnly=0>,<pointSetSigma=1>,<kNeighborhood=50>,<alpha=1.1>,<useAnisotropicCovariances=1>]
IGDM[fixedImage,movingImage,metricWeight,fixedMask,movingMask,<neighborhoodRadius=0x0>,<intensitySigma=0>,<distanceSigma=0>,<kNeighborhood=1>,<gradientSigma=1>]
          These image metrics are available--- CC: ANTS neighborhood cross correlation,
          MI: Mutual information, Demons: (Thirion), MeanSquares, and GC: Global
          Correlation. The "metricWeight" variable is used to modulate the per stage
          weighting of the metrics. The metrics can also employ a sampling strategy
          defined by a sampling percentage. The sampling strategy defaults to 'None' (aka
          a dense sampling of one sample per voxel), otherwise it defines a point set over
          which to optimize the metric. The point set can be on a regular lattice or a
          random lattice of points slightly perturbed to minimize aliasing artifacts.
          samplingPercentage defines the fraction of points to select from the domain. In
          addition, three point set metrics are available: Euclidean (ICP), Point-set
          expectation (PSE), and Jensen-Havrda-Charvet-Tsallis (JHCT).

     -t, --transform Rigid[gradientStep]
                     Affine[gradientStep]
                     CompositeAffine[gradientStep]
                     Similarity[gradientStep]
                     Translation[gradientStep]
                     BSpline[gradientStep,meshSizeAtBaseLevel]
GaussianDisplacementField[gradientStep,updateFieldVarianceInVoxelSpace,totalFieldVarianceInVoxelSpace]
BSplineDisplacementField[gradientStep,updateFieldMeshSizeAtBaseLevel,totalFieldMeshSizeAtBaseLevel,<splineOrder=3>]
TimeVaryingVelocityField[gradientStep,numberOfTimeIndices,updateFieldVarianceInVoxelSpace,updateFieldTimeVariance,totalFieldVarianceInVoxelSpace,totalFieldTimeVariance]
TimeVaryingBSplineVelocityField[gradientStep,velocityFieldMeshSize,<numberOfTimePointSamples=4>,<splineOrder=3>]
SyN[gradientStep,updateFieldVarianceInVoxelSpace,totalFieldVarianceInVoxelSpace]
BSplineSyN[gradientStep,updateFieldMeshSizeAtBaseLevel,totalFieldMeshSizeAtBaseLevel,<splineOrder=3>]
Exponential[gradientStep,updateFieldVarianceInVoxelSpace,velocityFieldVarianceInVoxelSpace,<numberOfIntegrationSteps>]
BSplineExponential[gradientStep,updateFieldMeshSizeAtBaseLevel,velocityFieldMeshSizeAtBaseLevel,<numberOfIntegrationSteps>,<splineOrder=3>]
          Several transform options are available. The gradientStep or learningRate
          characterizes the gradient descent optimization and is scaled appropriately for
          each transform using the shift scales estimator. Subsequent parameters are
          transform-specific and can be determined from the usage. For the B-spline
          transforms one can also specify the smoothing in terms of spline distance (i.e.
          knot spacing).

     -c, --convergence MxNxO
[MxNxO,<convergenceThreshold=1e-6>,<convergenceWindowSize=10>]
          Convergence is determined from the number of iterations per level and is
          determined by fitting a line to the normalized energy profile of the last N
          iterations (where N is specified by the window size) and determining the slope
          which is then compared with the convergence threshold.

     -s, --smoothing-sigmas MxNxO...
          Specify the sigma of gaussian smoothing at each level. Units are given in terms
          of voxels ('vox') or physical spacing ('mm'). Example usage is '4x2x1mm' and
          '4x2x1vox' where no units implies voxel spacing.

     -f, --shrink-factors MxNxO...
          Specify the shrink factor for the virtual domain (typically the fixed image) at
          each level.

     -u, --use-histogram-matching
          Histogram match the images before registration.

     -l, --use-estimate-learning-rate-once
          turn on the option that lets you estimate the learning rate step size only at
          the beginning of each level. * useful as a second stage of fine-scale
          registration.

     -w, --winsorize-image-intensities [lowerQuantile,upperQuantile]
          Winsorize data based on specified quantiles.

     -x, --masks [fixedImageMask,movingImageMask]
          Image masks to limit voxels considered by the metric. Two options are allowed
          for mask specification: 1) Either the user specifies a single mask to be used
          for all stages or 2) the user specifies a mask for each stage. With the latter
          one can select to which stages masks are applied by supplying valid file names.
          If the file does not exist, a mask will not be used for that stage. Note that we
          handle the fixed and moving masks separately to enforce this constraint.

     --float
          Use 'float' instead of 'double' for computations.
          <VALUES>: 0

     --minc
          Use MINC file formats for transformations.
          <VALUES>: 0

     -v, --verbose (0)/1
          Verbose output.

     -h
          Print the help menu (short version).

     --help
          Print the help menu. Will also print values used on the current command line
          call.

your script includes absolute hard-coded path so I cannot run it entirely. However, if I grab only the bottom portion of the script (line 36 and below), it works as expected, i.e., it shows “FileX” (X=0:158) followed by the antsRegistration help.

What happens if you just run this?

for files in {0..3};do
  echo "File${files}"
  isct_antsRegistration 
done

Another comment, you should not have to define SCT’s path in your script (line 8: PATH=~/sct/bin:$PATH), as it should already be defined in your environment variable when you start your Terminal (bashrc or equivalent). By doing so, you run the risk of running another sets of binaries in case you re-install SCT somewhere else (I’ve already seen that in other users-- very frustrating to debug)

For me it prints File0 and the help and that’s it. So no File1 to File3 and help again. That is really mysterious.

About the path: on my laptop the path is defined in my environment variable but if I work on our servers that is somehow different. I don’t remember exactely because it has been a while but I think that was not possible and I had to come up with that solution. Or at least that was suggested by our IT person. I can ask again…

That loop quits early because the script has set -e set, and passing no arguments is considered an error (isct_antsRegistration is returning 1, not 0)

The first version of the code

is missing \s at the end of each line, so each line appears to be a separate command; and because of set -e that first isct_antsRegistration terminates the whole process.

In the full script, you have

	isct_antsRegistration 
	#~ \
        #~ --dimensionality 3 \
        #~ --float 0 \
        #~ --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]' \
        #~ --output '[X_${files}_,X_${files}_Warped.nii.gz]' \
        #~ --interpolation 'BSpline' \
        #~ --winsorize-image-intensities '[0.005,0.995]' \
        #~ --use-histogram-matching 0 \
        #~ --transform 'Rigid[1]' \
        #~ --shrink-factors 1 \
        #~ --smoothing-sigmas 0 \
        #~ -x mask_fmri_65mm.nii

so I can tell you tackled that, but there’s an small but crucial oversight: the missing \ on the first line.

So is this what you actually want?

	isct_antsRegistration \
        --dimensionality 3 \
        --float 0 \
        --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]' \
        --output '[X_${files}_,X_${files}_Warped.nii.gz]' \
        --interpolation 'BSpline' \
        --winsorize-image-intensities '[0.005,0.995]' \
        --use-histogram-matching 0 \
        --transform 'Rigid[1]' \
        --shrink-factors 1 \
        --smoothing-sigmas 0 \
        -x mask_fmri_65mm.nii

Thanks for giving SCT a shakedown cruise. Let us know how it goes.

If it helps you out in the future, I always start my bash scripts with

set -e -o pipefail
#set -x

-o pipefail means that a pipeline (e.g. cat something | grep something_else | sort) will fail if any of the pieces fail; without it the failure code is just taken from the right-most piece.

set -x traces my code; I uncomment that line when debugging to see the exact bash commands being run.

Thank you very much for your detailed response! With the set -e that makes a lot of sense to me now and if I comment that line Julien’s test code does what I thought it should do.

Now the but… I tried your code with all the \ s (weird that my first posted code and my script differ in that sense because I thought I copied the same code but well apparently I didn’t) and with set -e commented, it runs through all the loops and echos what it should echo without executing antsRegistration but if I uncomment that line the script breaks in the first loop interation without any error message. I assume that some of the arguments are wrong?? Or the files are not found? But shouldn’t I get an error message in that case?

Try adding ; echo "RESULT = $?" after the isct_antsRegistration call. That will show you the return code. If its not 0, it’s an error. I would also turn on set -x so you can see exactly what line is dying.

These are C programs so they don’t have as much obvious error handling as python or bash programs. It’s possible your combination of data is hitting some weird corner case that is killing them without being caught.

I’ve made the edits here:
test_isct.sh (1012 Bytes)

We’re debugging somewhat blindly over here since we don’t have your dataset and your script is tuned to your particular folders. But if you could start a fresh terminal, run that, and copy the entire backlog in to here that would help. You can use the Code quote button like this to make it clear to us:

This is some sample code, written using triple backticks: ``` ... ```

That is the output I get:

+ PATH=/home/tinnermann/sct/bin:/home/tinnermann/bin:/home/tinnermann/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/local/bin
+ basedir=/projects/crunchie/remi3/
+ export TMPDIR=/projects/crunchie/remi3/tmp
+ TMPDIR=/projects/crunchie/remi3/tmp
+ listofSubs=(4)
+ for subject in ${listofSubs[@]}
++ printf %02d 4
+ subdir=/projects/crunchie/remi3/Sub04_test
+ echo Sub4
Sub4
+ for run in {1..8}
+ rundir=/projects/crunchie/remi3/Sub04_test/Run1/sct/
+ echo Run1
Run1
+ cd /projects/crunchie/remi3/Sub04_test/Run1/sct/
+ for files in {0..152}
+ echo File0
File0
+ isct_antsRegistration --dimensionality 3 --float 0 --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]' --output '[X_${files}_,X_${files}_Warped.nii.gz]' --interpolation BSpline --winsorize-image-intensities '[0.005,0.995]' --use-histogram-matching 0 --transform 'Rigid[1]' --shrink-factors 1 --smoothing-sigmas 0 -x mask_fmri_65mm.nii

That is about what I’m seeing too. I assumed the reason it stops is that I don’t have the data, but maybe it doesn’t like the arguments.

cd /projects/crunchie/remi3/Sub04_test/Run1/sct/
files=0

Then experiment with changing

isct_antsRegistration --dimensionality 3 --float 0 --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]' --output '[X_${files}_,X_${files}_Warped.nii.gz]' --interpolation BSpline --winsorize-image-intensities '[0.005,0.995]' --use-histogram-matching 0 --transform 'Rigid[1]' --shrink-factors 1 --smoothing-sigmas 0 -x mask_fmri_65mm.nii

until you discover what is wrong.

What is -x mask_fmri_65mm.nii about? Possibly the cd made that filename is invalid. I’ll try to experiment with my copy as best I can.

Oh I see one problem: there are single quotes where you want double quotes. You have to write

isct_antsRegistration \
  --dimensionality 3 \
  --float 0 \
  --metric "MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T`printf %04d ${files}`.nii,1,32,Regular,0.2]" \
  --output "[X_${files}_,X_${files}_Warped.nii.gz]" \
  --interpolation BSpline \
  --winsorize-image-intensities '[0.005,0.995]' \
  --use-histogram-matching 0 \
  --transform 'Rigid[1]' \
  --shrink-factors 1 \
  --smoothing-sigmas 0 \
  -x mask_fmri_65mm.nii

for the string interpolations (the $s and the backticks) to work. Single quotes mean a literal string in shell.

You can also use the fact that strings run together like egg yolk and instead write:

isct_antsRegistration \
  --dimensionality 3 \
  --float 0 \
  --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T'`printf %04d ${files}`'.nii,1,32,Regular,0.2]' \
  --output '[X_'${files}'_,X_'${files}'_Warped.nii.gz]' \
  --interpolation BSpline \
  --winsorize-image-intensities '[0.005,0.995]' \
  --use-histogram-matching 0 \
  --transform 'Rigid[1]' \
  --shrink-factors 1 \
  --smoothing-sigmas 0 \
  -x mask_fmri_65mm.nii

which is a little bit more reliable because it avoids potentially deeply nesting quotes.

In addition, shell is a fragile language, so I would add a layer of quoting around the interpolations to prevent accidental whitespace (or worse, shell code) in their values from getting interpreted, and prefer $(..) to `` because it’s more readable:

isct_antsRegistration \
  --dimensionality 3 \
  --float 0 \
  --metric 'MeanSquares[fmri_moco_norm_T0000.nii,fmri_moco_norm_T'"$(printf %04d "${files}")"`'.nii,1,32,Regular,0.2]' \
  --output '[X_'"${files}"'_,X_'"${files}"'_Warped.nii.gz]' \
  --interpolation BSpline \
  --winsorize-image-intensities '[0.005,0.995]' \
  --use-histogram-matching 0 \
  --transform 'Rigid[1]' \
  --shrink-factors 1 \
  --smoothing-sigmas 0 \
  -x mask_fmri_65mm.nii

Finally, fmri_moco_norm_T0000.nii sounds like a filename, just like mask_fmri_65mm.nii, so check that both of those exist in the folder the program is running from. It’s likely that the cd makes that untrue.

It’s often easier to build up pathnames than to try to cd to the right places. So, change

-  --output '[X_'"${files}"'_,X_'"${files}"'_Warped.nii.gz]' \
+  --output '['"${subdir}/"'X_'"${files}"'_,'"${subdir}/"'X_'"${files}"'_Warped.nii.gz]' \

and remove the cd.