Trouble running Spacy - Cannot set unknown dimension 'nO' for model 'layernorm'.

[raj-trustlab]

I'm trying to build and run some team code. There is a line in the code that does:

nlp = spacy.load('en_core_web_sm')

which is erroring with this:

  File "/usr/local/lib/python3.9/site-packages/spacy/__init__.py", line 47, in load
    return util.load_model(name, disable=disable, exclude=exclude, config=config)
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 322, in load_model
    return load_model_from_package(name, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 355, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)
  File "/usr/local/lib/python3.9/site-packages/en_core_web_sm/__init__.py", line 10, in load
    return load_model_from_init_py(__file__, **overrides)
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 514, in load_model_from_init_py
    return load_model_from_path(
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 390, in load_model_from_path
    return nlp.from_disk(model_path, exclude=exclude)
  File "/usr/local/lib/python3.9/site-packages/spacy/language.py", line 1851, in from_disk
    util.from_disk(path, deserializers, exclude)
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 1174, in from_disk
    reader(path / key)
  File "/usr/local/lib/python3.9/site-packages/spacy/language.py", line 1845, in <lambda>
    deserializers[name] = lambda p, proc=proc: proc.from_disk(
  File "spacy/pipeline/trainable_pipe.pyx", line 336, in spacy.pipeline.trainable_pipe.TrainablePipe.from_disk
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 1174, in from_disk
    reader(path / key)
  File "spacy/pipeline/trainable_pipe.pyx", line 327, in spacy.pipeline.trainable_pipe.TrainablePipe.from_disk.load_model
  File "/usr/local/lib/python3.9/site-packages/thinc/model.py", line 526, in from_bytes
    return self.from_dict(msg)
  File "/usr/local/lib/python3.9/site-packages/thinc/model.py", line 549, in from_dict
    node.set_dim(dim, value)
  File "/usr/local/lib/python3.9/site-packages/thinc/model.py", line 182, in set_dim
    raise KeyError(
KeyError: "Cannot set unknown dimension 'nO' for model 'layernorm'."

Versions:

• Python 3.9.5
• spacy 3.0.3
• spacy-langdetect 0.1.2
• spacy-legacy 3.0.5

Downloaded / installed "en_core_web_sm" using

python3 -m spacy download en_core_web_sm
pip3 install /path/to/en_core_web_sm-3.0.0.tar.gz

Anyone know what might be the issue?

[polm]

Ah, I thought I recognized this question, it was on StackOverflow.

I commented there, but for completeness:

1. The way you are installing the model is a bit strange. Just the spacy download command should install the model, you should not need to call pip. Can you try your code again in a clean environment?
2. spacy-langdetect is an unofficial third-party package that hasn't been updated in years and only works with spaCy 2. Maybe your code only works with spaCy v2?
3. Can you load the spaCy model outside this particular project you're working on? If you can't this is an environment issue, if you can this is an issue with your code; knowing which it is will help us help you.