How to use the updated sct_propseg (v5.3.0) for CentOS/RHEL 6?

propseg_log_5.3.0.txt (1.8 KB)

@joshuacwnewton ah, seems to be GLIBC issue. :frowning:

1 Like

Indeed…

This is a bit confusing to me. The error for v5.3.0 occurred inside “isct_propseg”, but that should be identical to the “isct_propseg” that was inside v3.2.3. (See the screenshots in my earlier comment.)

In other words, if the GLIBC on your work machine is causing issues for v5.3.0, then in theory you would experience this error on any version of SCT (v3.2.3 included)… :thinking:

@joshuacwnewton strange. I think I already sent you SCT 3.2.3 output from the same machine. Btw, here we dont have any GLIBC issues, just INCORRECT segmentation.

propseg_log_3.2.3.txt (6.0 KB)

@joshuacwnewton here it is. This is generated by the same machine with the same data, only different version of SCT. In this case, 3.2.3.

Which OS are you using on your work machine?

@joshuacwnewton RedHatEnterpriseServer 6.10

Tried copying isct_propseg from 3.2.3 to 5.3.0 bin folder, still no luck. It might be silly, but was worth a shot I guess.

Aha! It turns out that there is a big difference between the versions. v3.2.3 had a version of “isct_propseg” that was specifically compiled for CentOS/RHEL 6.

SCT ended support for CentOS 6 last year because it had reached end-of-life status. So, we no longer distribute the CentOS 6-specifc binaries. That is why v5.3.0 does not work on your machine. :sweat:

As an alternative to this, there are some steps you can take to improve the performance of sct_propseg on v3.2.3. Please see my later post for more details: How to use the updated sct_propseg (v5.3.0) for CentOS/RHEL 6? - #40 by joshuacwnewton

@joshuacwnewton ah, I understand. Thank you so much!

When you have time, could you please let me know why I cannot download data when executing ./install_sct. I am getting errors such as:

sct_download_data -d PAM50 -o /mnt/algo/sct_5.3.0/data/PAM50
--

Removing existing destination folder '/mnt/algo/sct_5.3.0/data/PAM50'
Trying URL: https://github.com/sct-data/PAM50/releases/download/r20201104/PAM50-r20201104.zip
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream
Link download error, trying next mirror (error was: HTTPSConnectionPool(host='github-releases.githubusercontent.com', port=443): Max retries exceeded with url: /269508010/0a8f1980-1e1c-11eb-84b0-fa146c143d89?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210606T200404Z&X-Amz-Expires=300&X-Amz-Signature=1efc1d9363803537dee13f262c09a1c905fc7d32e01bb17a5aaca61d4cfafebf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=269508010&response-content-disposition=attachment%3B%20filename%3DPAM50-r20201104.zip&response-content-type=application%2Foctet-stream (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)))
```

@joshuacwnewton Just to let you know, copying isct_propseg from SCT-3.2.3/bin to SCT-5.3.0/bin PARTIALLY worked. Here is what I mean, result is NOT AS GOOD AS when using ORIGINAL SCT-5.3.0, but it is MUCH closer then using plain SCT-3.2.3:

1 Like

Of course! I would be happy to. I am just looking into this now. :slightly_smiling_face:

I’m very glad to see this! Thank you for trying this.

For some reason C1, C2, C3 were not segmented. Could this be because SCT is relying on some of the data I was mentioning that was not downloaded? Or its due to the other improvements you mentioned erlier that are not directly related to sct_propseg?

And once again @joshuacwnewton , REALLY appreciate your fast and informative responses!

1 Like

Thank you for the kind words. :smile:

Truthfully, it’s difficult to speculate given that this is quite “uncharted territory” here… :sweat_smile:

After some searching, the “sct_download_data” issues may be due to an outdated version of OpenSSL. On your work machine, could you please run the following command and share the output?

openssl version

Kind regards,
Joshua

Hahaha, I understand :smiley:

Here it is: OpenSSL 1.1.1.h Sep 2020

Thank you!

Could you please try running the following command before using sct_download_data?

export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt

For transparency, my suggestion is based off of this recommendation and this information for RHEL 6.

@joshuacwnewton still the same issue :confused:

@joshuacwnewton it seems that data HAS been downloaded.

Well, then I am out of ideas why do we get 3x different results with these 3x approaches that should be the same since sct_propseg did not change from 3.2.3 - 5.3.0 :smiley:

Here is my current understanding for why there are three different results:

  • v3.2.3 with CentOS6 “isct_propseg” binary: Baseline performance (not satisfactory).
  • v5.3.0 with CentOS6 “isct_propseg” binary copied over: Better performance (even though the “isct_propseg” binary is the same) because it gets the improvements from years worth of updates to other parts of SCT.
  • v5.3.0 with the non-CentOS6 version of the “isct_propseg” binary: Best performance, but isn’t compatible with RHEL6 because of GLIBC requries.

Sorry for the trouble here, and thank you for the patience. :sweat_smile:

1 Like

@joshuacwnewton is it possible to download code for SCT v5.3.0 and compile it for CentOS6? :smiley:

With the current “hybrid” setup, you should already have most of SCT v5.3.0 – the only differences here are the binaries (“isct_propseg” and others). And, it’s difficult for me to guarantee that recompiling the binaries would have any effect.

I do have some alternate suggestions, though:

1. Tweaking the parameters of sct_propseg

Because you now have an improved segmentation, it might be a better use of your time to try to tweak the parameters of the original “sct_propseg” command. Our documentation currently has a tutorial for fixing segmentations called: Correcting sct_propseg, so some of the tips there may be of use.

We are also in the process of updating the tutorial parts of our documentation, so if anything is unclear, please ask! We will be happy to guide you accordingly, and incorporate any feedback into the documentation. :slightly_smiling_face:

2. Trying sct_deepseg_sc

sct_propseg was developed in 2015, and in the time since, SCT has also developed an improved spinal cord segmentation method called sct_deepseg_sc. It was developed using PyTorch (not Keras), so it may be possible to use it on your RHEL6 machine.

The corresponding command would be, for example:

sct_deepseg_sc -i {input} -c t2 -ofolder {ofolder}

Kind regards,
Joshua

1 Like