How to use the updated sct_propseg (v5.3.0) for CentOS/RHEL 6?

Which OS are you using on your work machine?

@joshuacwnewton RedHatEnterpriseServer 6.10

Tried copying isct_propseg from 3.2.3 to 5.3.0 bin folder, still no luck. It might be silly, but was worth a shot I guess.

Aha! It turns out that there is a big difference between the versions. v3.2.3 had a version of “isct_propseg” that was specifically compiled for CentOS/RHEL 6.

SCT ended support for CentOS 6 last year because it had reached end-of-life status. So, we no longer distribute the CentOS 6-specifc binaries. That is why v5.3.0 does not work on your machine. :sweat:

As an alternative to this, there are some steps you can take to improve the performance of sct_propseg on v3.2.3. Please see my later post for more details: How to use the updated sct_propseg (v5.3.0) for CentOS/RHEL 6? - #40 by joshuacwnewton

@joshuacwnewton ah, I understand. Thank you so much!

When you have time, could you please let me know why I cannot download data when executing ./install_sct. I am getting errors such as:

sct_download_data -d PAM50 -o /mnt/algo/sct_5.3.0/data/PAM50
--

Removing existing destination folder '/mnt/algo/sct_5.3.0/data/PAM50'
Trying URL: https://github.com/sct-data/PAM50/releases/download/r20201104/PAM50-r20201104.zip
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream
Link download error, trying next mirror (error was: HTTPSConnectionPool(host='github-releases.githubusercontent.com', port=443): Max retries exceeded with url: /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)))
```

@joshuacwnewton Just to let you know, copying isct_propseg from SCT-3.2.3/bin to SCT-5.3.0/bin PARTIALLY worked. Here is what I mean, result is NOT AS GOOD AS when using ORIGINAL SCT-5.3.0, but it is MUCH closer then using plain SCT-3.2.3:

1 Like

Of course! I would be happy to. I am just looking into this now. :slightly_smiling_face:

I’m very glad to see this! Thank you for trying this.

For some reason C1, C2, C3 were not segmented. Could this be because SCT is relying on some of the data I was mentioning that was not downloaded? Or its due to the other improvements you mentioned erlier that are not directly related to sct_propseg?

And once again @joshuacwnewton , REALLY appreciate your fast and informative responses!

1 Like

Thank you for the kind words. :smile:

Truthfully, it’s difficult to speculate given that this is quite “uncharted territory” here… :sweat_smile:

After some searching, the “sct_download_data” issues may be due to an outdated version of OpenSSL. On your work machine, could you please run the following command and share the output?

openssl version

Kind regards,
Joshua

Hahaha, I understand :smiley:

Here it is: OpenSSL 1.1.1.h Sep 2020

Thank you!

Could you please try running the following command before using sct_download_data?

export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt

For transparency, my suggestion is based off of this recommendation and this information for RHEL 6.

@joshuacwnewton still the same issue :confused:

@joshuacwnewton it seems that data HAS been downloaded.

Well, then I am out of ideas why do we get 3x different results with these 3x approaches that should be the same since sct_propseg did not change from 3.2.3 - 5.3.0 :smiley:

Here is my current understanding for why there are three different results:

  • v3.2.3 with CentOS6 “isct_propseg” binary: Baseline performance (not satisfactory).
  • v5.3.0 with CentOS6 “isct_propseg” binary copied over: Better performance (even though the “isct_propseg” binary is the same) because it gets the improvements from years worth of updates to other parts of SCT.
  • v5.3.0 with the non-CentOS6 version of the “isct_propseg” binary: Best performance, but isn’t compatible with RHEL6 because of GLIBC requries.

Sorry for the trouble here, and thank you for the patience. :sweat_smile:

1 Like

@joshuacwnewton is it possible to download code for SCT v5.3.0 and compile it for CentOS6? :smiley:

With the current “hybrid” setup, you should already have most of SCT v5.3.0 – the only differences here are the binaries (“isct_propseg” and others). And, it’s difficult for me to guarantee that recompiling the binaries would have any effect.

I do have some alternate suggestions, though:

1. Tweaking the parameters of sct_propseg

Because you now have an improved segmentation, it might be a better use of your time to try to tweak the parameters of the original “sct_propseg” command. Our documentation currently has a tutorial for fixing segmentations called: Correcting sct_propseg, so some of the tips there may be of use.

We are also in the process of updating the tutorial parts of our documentation, so if anything is unclear, please ask! We will be happy to guide you accordingly, and incorporate any feedback into the documentation. :slightly_smiling_face:

2. Trying sct_deepseg_sc

sct_propseg was developed in 2015, and in the time since, SCT has also developed an improved spinal cord segmentation method called sct_deepseg_sc. It was developed using PyTorch (not Keras), so it may be possible to use it on your RHEL6 machine.

The corresponding command would be, for example:

sct_deepseg_sc -i {input} -c t2 -ofolder {ofolder}

Kind regards,
Joshua

1 Like

The code is in https://github.com/spinalcordtoolbox/spinalcordtoolbox/tree/master/dev/isct_propseg. You could do something like

git clone https://github.com/spinalcordtoolbox/spinalcordtoolbox/
cd spinalcordtoolbox
cd dev/isct_propseg
cmake .
cmake --build .

However we haven’t tested this process in years and we don’t know stable this is. Fixing that up is an ongoing project. If you do manage to make this work, you could help us out a lot by sharing your experiences with us on Github, it might accelerate making this more reliable for other upgraders in the future!

We dropped CentOS 6 support when it was officially dropped 8 months ago, so we can’t really vouch for the quality of any results obtained if you do manage to get it running there.

You might have better luck with a virtual machine. You can

yum install qemu-kvm

or maybe virtualbox (with this http://download.virtualbox.org/virtualbox/rpm/rhel/virtualbox.repo)

and then install Ubuntu 20.04 or newer in that, and then install SCT in that.

Hi @anon33887296 @joshuacwnewton! I created a Docker image with CentOS6 where I am trying to recompile SCT’s propseg (as @anon33887296 described) with GLIBC 2.12. Athough, I am running into some issues:

mnt/algo/spinalcordtoolbox-5.3.0/dev/isct_propseg/Image3D.cpp:34:25: error: OrientImage.h: No such file or directory
/mnt/algo/spinalcordtoolbox-5.3.0/dev/isct_propseg/Image3D.cpp: In member function ‘void Image3D::TransformMeshToBinaryImage(Mesh*, std::string, OrientationType, bool, bool, CVector3*, CVector3*, CVector3*, CVector3*)’:
/mnt/algo/spinalcordtoolbox-5.3.0/dev/isct_propseg/Image3D.cpp:435: error: ‘OrientImage’ was not declared in this scope
/mnt/algo/spinalcordtoolbox-5.3.0/dev/isct_propseg/Image3D.cpp:435: error: expected primary-expression before ‘>’ token
/mnt/algo/spinalcordtoolbox-5.3.0/dev/isct_propseg/Image3D.cpp:435: error: ‘orientationFilter’ was not declared in this scope
gmake[2]: *** [CMakeFiles/isct_propseg.dir/Image3D.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/isct_propseg.dir/all] Error 2
gmake: *** [all] Error 2

This occurs when I try to execute cmake --build .

To come clean here, at this point you’re as educated about how to compile isct_propseg as any of us. The person who wrote it hasn’t been actively part of the team for some years now.

If you can figure it out that would be a helpful contribution and we would certainly merge updated build instructions into SCT. I think it requires building all of ITK too, first, which is a fairly large project that is difficult in its own way.

At this point, installing a virtual machine on that server is going to be as much trouble as figuring out how to build it. You can still benefit from all the CPUs, so long as you’re using qemu-kvm, with hardware-assisted virtualization.

Or, is there a reason your IT team can’t upgrade the server itself? RHEL6 ended 8 months ago: https://endoflife.software/operating-systems/linux/red-hat-enterprise-linux-rhel. SCT is probably not the only software your team is having trouble with on that server.