Fail to install SCT on Ubuntu 22.04

MRI Processing & SCT
1

Hello,

I’m trying to install SCT (the plb/4995_new_ms_model branch), but it seems to have an issue after installing dipy.
I’m running on Ubuntu 22.04, I just cloned the repo and checked out the branch, then ran ./install_sct -y

I don’t know what changed, but I believed few weeks ago, I managed to install it successfully, but not anymore.

I attached the log file.
install_sct_log.txt (68.2 KB)

Thanks.

Edit: I believe this installation is not working as expected because when trying to import numpy from the created environment, it doesn’t find it:

lchauvin@imaging-gpu-05:/scratch/02/lchauvin/workspace/projects/spinalcordtoolbox$ ./python/envs/venv_sct/bin/python -c 'import numpy as np'

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

Edit 2: I also tried to follow the same steps as this test: New multi-fold MS lesion model · spinalcordtoolbox/spinalcordtoolbox@debc01f · GitHub

But I’m still getting the same error.

Edit 3: My understanding is that the installation of dipy is trying to run this code

  ../dipy/meson.build:106:17: ERROR: Command `/scratch/02/lchauvin/workspace/projects/spinalcordtoolbox/python/envs/venv_sct/bin/python3.10 -c '
      import numpy as np
      try:
        incdir = os.path.relpath(np.get_include())
      except Exception:
        incdir = np.get_include()
      print(incdir)
        '` failed with status -6.

However, numpy has not been installed yet.
If I look at the tests, we can see:

Collecting dipy==1.8.0 (from -r requirements.txt (line 12))
  Downloading dipy-1.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.6 kB)
Collecting ivadomed (from -r requirements.txt (line 16))
  Downloading ivadomed-2.9.10-py3-none-any.whl.metadata (6.4 kB)
Collecting matplotlib (from -r requirements.txt (line 17))
  Downloading matplotlib-3.10.6-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (11 kB)
Collecting matplotlib-inline (from -r requirements.txt (line 22))
  Downloading matplotlib_inline-0.1.7-py3-none-any.whl.metadata (3.9 kB)
Collecting nibabel (from -r requirements.txt (line 30))
  Downloading nibabel-5.3.2-py3-none-any.whl.metadata (9.1 kB)
Collecting nilearn (from -r requirements.txt (line 31))
  Downloading nilearn-0.12.1-py3-none-any.whl.metadata (9.9 kB)
Collecting nnunetv2==2.6.2 (from -r requirements.txt (line 43))
  Downloading nnunetv2-2.6.2.tar.gz (211 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting numpy<2 (from -r requirements.txt (line 45))
  Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)

Numpy is installed after dipy.

It seems the problem is coming from GraalPy somehow that uses a different version of pip:

<frozen graalpy.pip_hook>:48: RuntimeWarning: You are using an untested version of pip. GraalPy provides patches and workarounds for a number of packages when used with compatible pip versions. We recommend to stick with the pip version that ships with this version of GraalPy.
WARNING: Disabling truststore because platform isn't supported

But I don’t understand why.

Edit 4: A collegue of mine find out that installing a new conda is messing up with the existing path. So we modified the install_sct script to use our already existing micromamba installation, and it seems to work that way. But it’s a workaround. There should be a better way to fix this.

2

Dear @lchauvin,

My deepest apologies for the late reply. I somehow missed the notification for your post.

The issue stems from the fact that "dipy" is being built from source:

Collecting dipy==1.8.0 (from -r requirements.txt (line 12))
  Using cached dipy-1.8.0.tar.gz (6.5 MB)

Typically, “source builds” (.tar.gz) will be more error-prone, so it is preferable to install from a “wheel” instead, which looks like this in SCT’s test suite:

 Collecting dipy==1.8.0 (from -r requirements.txt (line 14))
  Downloading dipy-1.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.6 kB)

If a package is being installed from a source distribution (instead of a wheel), it usually means that there is some sort of OS or system incompatibility. We do test for Ubuntu 22.04 in our test suite, so in theory there shouldn’t be any incompatibility, but in spite of that you are experiencing this issue. (Strange!)

If I compare your installation log to the one from the passing test (difference), we can see that the installations are largely the same during the conda environment creation stages, but there is something that stands out to me:

  • During your install, the package “graalpy-23.0.0” gets installed. This also throws a number of warnings:
    <frozen graalpy.pip_hook>:48: RuntimeWarning: You are using an untested version of pip. GraalPy provides patches and workarounds for a number of packages when used with compatible pip versions. We recommend to stick with the pip version that ships with this version of GraalPy.
WARNING: Disabling truststore because platform isn't supported
  • I have never heard of GraalPy before, but from what I understand, it is a replacement runtime for the default Cython
  • I also notice that its README contains the following message:

    Packages with C code usually do not provide binaries for GraalPy, so they will be automatically compiled during installation.

  • This would explain why dipy is being built from source instead of installed via a wheel.

I do not know why your environment has graalpy, nor do I understand why it is interfering with the installation of SCT, but I think this is the source of the issue.

If possible, can you tell me more about your environment? Had you installed GraalPy at an earlier date? Have you configured your system to insert GraalPy into normal Python installations?

Kind regards,
Joshua

3

Hello,

Thank you.

I don’t why GraalPy is there neither. However, like I said, a coworker of mine find out that installing MininForge during installation, when other conda (or micromamba) already exists on the system mess up with some environment variable.

I tried to install SCT on a singularity image with a clean installation, and it works. So it seems it has to do with some conflicts between existing conda / micromamba installation on the system, and the conda installed with SCT.

We modified the installation script to use an existing conda / micromamba and it worked. The vanilla installation scripts works in a clean container though.

Hope this could help.

Thanks.

4

I am glad to hear that it is working for you under the conditions you describe.

I’m sorry to hear that the conda installations are clashing. In theory, SCT’s conda installation should be isolated – if it is not, then we would consider that a bug to fix.

If you have any more information about what environment variable gets changed upon the installation of SCT’s conda environment, please let me know and I would be happy to investigate further.

Kind regards,
Joshua

5

Hi, I had the same issue:
During the installation, it had downloaded graal and the log showed the following

<frozen graalpy.pip_hook>:48: RuntimeWarning: You are using an untested version of pip.

You confirmed my suspicions that it was detritus from an old ***conda install that was causing the problem, as I had no problem when performing exactly the same steps with a local, new account (and that install did NOT download graalpy)

So I did a little digging, and for the account with the failed install, I removed some files:
I first performed a find for “conda” type files

find [theusername]/ -iname "*conda*"

I then removed likely candidates

 rm -rf .condarc
 rm -rf .cache/conda/notices/notices.cach

(I’m not sure all were necessary, oh well)

Then the install worked

I hope that helps.

J