Error: Can't find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory

[simonrmortensen]

I'm trying to load the en_core_web_sm spaCy model, but I have been unsuccessful in doing so.

The error that occurs is the following:

OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory.

I'm working in a Anaconda virtual environment. The following checkboxes are ticked:

• Did conda activate gcp-env prior to installing spaCy and the english language model
• Have run conda install -c conda-forge spacy while on the right environment
• Then, have run python -m spacy download en, still while on the right environment
• Also tried adding spacy to the requirements.txt , and installing dependencies via that route, after first attempts failed

spacy info produces this output:

spacy info

============================== Info about spaCy ==============================

spaCy version    3.3.0                         
Location         /Users/simonmortensen/opt/anaconda3/envs/gcp-env/lib/python3.10/site-packages/spacy
Platform         macOS-11.6.5-x86_64-i386-64bit
Python version   3.10.4                        
Pipelines        en_core_web_sm (3.3.0)

python -m spacy validate produces this output:

================= Installed pipeline packages (spaCy v3.3.0) =================
ℹ spaCy installation:
/Users/simonmortensen/opt/anaconda3/envs/gcp-env/lib/python3.10/site-packages/spacy

NAME             SPACY                 VERSION                            
en_core_web_sm   >=3.3.0.dev0,<3.4.0   3.3.0   ✔

For additional context, pip list on the environment contains both

spacy                         3.3.0
spacy-legacy                  3.0.9
spacy-loggers                 1.0.2

and

en-core-web-sm                3.3.0

Even so, import en_core_web_sm also doesn't work:

import en_core_web_sm
Traceback (most recent call last):

  Input In [65] in <cell line: 1>
    import en_core_web_sm

ModuleNotFoundError: No module named 'en_core_web_sm'

I've been through several previous StackOverflow posts on the same topic. Those have often been solved, but my issue remains. So I've also posted this same question to StackOverflow, but have not been able to find an answer (so far...).

Any advice would be very much appreciated. Thanks in advance!

Simon

[koaning]

This might be a virtual environment thing.

There could be a difference between:

python -m spacy download en_core_web_sm

and

spacy download en_core_web_sm

I couldn't help but notice some of your commands omit python -m, which might cause some of the issues we're seeing. If you're curious about details, I've attempted to make a tutorial on what might go wrong here.

Could you try running:

# Use python explicitly to grab info. 
python -m spacy info

# Just confirm the versions via pip too
python -m pip freeze | grep spacy

# Download the en_core_web_sm model explicitly 
python -m spacy download en_core_web_sm

Once that's downloaded, you should be able to load in the model via:

import spacy 

nlp = spacy.load("en_core_web_sm")

[koaning]

Could you try running the commands from a terminal? I understand it might be new to you, but today is as good a time as any to maybe get started with it.

[koaning]

Actually, because you said you were unfamiliar, you may appreciate following along with the entire course on calmcode. I'm the maintainer of that project, and I don't want to toot my own horn here, but I can imagine just taking the time through all the steps there might also just help shape your understanding.

[simonrmortensen]

Happy to learn, and thanks for all the help! 😄

I've run all commands inside the terminal, including python testing.py. They all work (or at least, the import statements are no longer the error), so as you suggest, I'm assuming that Spyder is doing something funky.

I'm researching right now, and at first glance it seems as though you actually need to install Spyder on every virtual environment, just like any other package. Trying this now.

[simonrmortensen]

As you suggested - Spyder was the villain.

All packages were correctly installed on the virtual environment, but Spyder was not running that environment (even if the IDE was launched with the spyder command from a terminal where the environment was in fact activated).

In order to make Spyder run the correct environment, you needed to change the Python interpreter in the Spyder preferences:

... and then restart the kernel.

I got an error prompting me to pip install spyder-kernels==2.1.*, but once that was done (make sure to do it on the right env) and I restarted Spyder, it finally worked!

@koaning, thanks a ton for hopping on this thread, you've been a great help!

[koaning]

Happy to hear there's more calm now. 🙂

[ClaudioAlejandroTorre]

Hello everyone. I worked with nlp = spacy.load("en_core_web_sm") having generated .exe with PyInstaller, considering that in the python interpreter environment the software worked well. When executing the .exe the following error occurred: Error: Can't find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory. I was able to solve it by watching the video published in:

https://ar.video.search.yahoo.com/search/video;_ylt=AwrJ_GuHE1lmbjIAzFmr9Qt.;_ylu=Y29sbwNiZjEEcG9zAzEEdnRpZAMEc2VjA3BpdnM-?p=OSError%3A+%5BE050%5D+Can%27t+find+model+%27en_core_web_s m%27. +It+doesn%27t+seem+to+be+a+Python+package+or+a+valid+path+to+a+data+directory.&fr2=piv-web&type=E210AR739G0&fr=mcafee#id=2&vid=33eb7c89f0a4a4d380f2fb6865a978cc&action =view

It was necessary to replace the statement nlp = spacy.load("en_core_web_sm") in my source program with nlp = spacy.load("C:\Users\claud\OneDrive\Desktop\Python312.venv\Lib\site-packages\en_core_web_sm \en_core_web_sm-3.7.1")
nlp = spacy.load("C:/Users/claud/OneDrive/Desktop/Python312/.venv/Lib/site-packages/en_core_web_sm/en_core_web_sm-3.7.1").

That is, as the video explains, I directly replaced "en_core_web_sm" with the address on my Windows 11 notebook where the spacy library (python -m spacy download en). Thank you so much

[ClaudioAlejandroTorre]

Hello everyone. I worked with nlp = spacy.load("en_core_web_sm") having generated .exe with PyInstaller, considering that in the python interpreter environment the software worked well. When executing the .exe the following error occurred: Error: Can't find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory. I was able to solve it by watching the video published in: https://ar.video.search.yahoo.com/search/video;_ylt=AwrJ_GuHE1lmbjIAzFmr9Qt.;_ylu=Y29sbwNiZjEEcG9zAzEEdnRpZAMEc2VjA3BpdnM-?p=OSError%3A+%5BE050%5D+Can%27t+find+model+%27en_core_web_s m%27. +It+doesn%27t+seem+to+be+a+Python+package+or+a+valid+path+to+a+data+directory.&fr2=piv-web&type=E210AR739G0&fr=mcafee#id=2&vid=33eb7c89f0a4a4d380f2fb6865a978cc&action =view

It was necessary to replace the statement nlp = spacy.load("en_core_web_sm") in my source program with nlp = spacy.load("C:\Users\claud\OneDrive\Desktop\Python312.venv\Lib\site-packages\en_core_web_sm \en_core_web_sm-3.7.1")
nlp = spacy.load("C:/Users/claud/OneDrive/Desktop/Python312/.venv/Lib/site-packages/en_core_web_sm/en_core_web_sm-3.7.1"). That is, as the video explains, I directly replaced "en_core_web_sm" with the address on my Windows 11 notebook where the spacy library (python -m spacy download en). The .exe run ok. Thank you so much

[XYLxiria]

Now my code run perfectly! Thanks, i took solution#3 and now there is no bug about spacy

[kd65541]

referring to solution 3, you can also copy paste the en_core_web_sm folders from the installed path (stated in cmd prompt) to the path your IDE said was missing