Spacy installation isn't installing correct pytorch version for gpu support on windows

[erotavlas]

On Windows when I install spacy using this command

pip install -U spacy[cuda111,transformers,lookups]

torch doesn't end up being installed with cuda support

torch                     1.8.1                    pypi_0    pypi

Instead I have to start from a new environment, I FIRST have to install the correct torch version for my cuda version from the commands provided on pytorch website

pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

Then After torch installation is complete I can run the install command for spacy

pip install -U spacy[cuda111,transformers,lookups]

torch                     1.8.1+cu111              pypi_0    pypi
torchaudio                0.8.1                    pypi_0    pypi
torchvision               0.9.1+cu111              pypi_0    pypi

And then it will work, if I do it the other way around (install spacy first, then reinstall torch, I get an error when I try to import spacy

[polm]

Sorry that doesn't work, and thanks for the detailed explanation.

I think we don't require torch directly, but instead it's pulled in via Transformers, so there may not be much we can do about this beyond documenting it better, but we're aware that installing is prone to problems and always looking for ways to improve it.

[adrianeboyd]

Yes, it's best to follow the pytorch instructions because pip can't really detect the right CUDA settings. We'd also recommend using the pytorch conda instructions in a conda venv, since that may be a better match for the rest of the environment.

Here are our general recommendations: https://spacy.io/usage/embeddings-transformers#transformers-installation

You can also install spacy from conda-forge instead, but we don't have conda-forge packages for every single extra package like spacy-transformers or spacy-lookups-data, in particular not for the pure python packages. I think transformers recently added support for conda, but similarly with their own custom directions requiring a specific conda channel.