Windows DLL Weirdness

[polm]

Sometimes this error happens on Windows:

ImportError: DLL load failed while importing fugashi: the specified module could not be found

From experience, this can always be fixed by using a virtualenv. If you use a virtualenv and it does not work, please let me know. Using a virtualenv is the officially recommended workaround. Other environment managers should also work.

Normally when this happens, the dll has been installed, but Python is not looking in the right path for it. You can copy it to the right path to fix it, but I would recommend using a virtualenv instead.

The current status of this issue (2024-11-13) is that I cannot reproduce it, and I do not know what steps can be taken, if any, to prevent it from happening at the package level.

What follows is how I have tried to investigate this.

• Asked pip devs DLLs / wheel data not installed on Windows sometimes pypa/pip#10605: They indicated that the way I am packaging the dll is wrong, but are not sure what the right thing is, and did not have a concrete suggestion.
• Asked setuptools devs How to specify the install path for DLLs? pypa/setuptools#2830: I got one response saying people don't normally package DLLs, but no recommendation on what to do.
• See issues tagged missing-dll , also in the mecab-python3 repo

In order to resolve this issue, I will need either:

1. a process I can use to reproduce it OR
2. cooperation from someone experiencing the issue

Ideally I would be able to test it myself, but if that is difficult, someone who has the issue and can test new package builds would be a big help.

---

What follows is the original text of this issue from 2021.

Via email I have a report of a Windows user who installed fugashi via pip without errors, but didn't get libmecab.dll in their site-packages/fugashi directory, which led to errors at import time like this:

ImportError: DLL load failed while importing fugashi: the specified module could not be found

For what it's worth, the dll is definitely in the wheel file, and when I install it on Windows the dll ends up in the site-packages/fugashi package as expected.

This thread has some info on DLLs and Python on Windows:

Toblerity/Fiona#851

One thing that we could potentially do is check for ImportErrors, and if the code is being executed on Windows, check if libmecab.dll is present and give a very specific error if not. On the other hand, since it's not clear how this happened in the first place, maybe just having an FAQ entry (or this issue) is enough for now.

[kinow]

Tested on my Win 10 pro, with Python 3.9 (from Windows Store) and a venv. Worked with no issues, both with or without wheel package 👍

[polm]

Closing because no action is required for the time being.

[alinacoding]

I also encountered this error while using Python 3.8 (from Windows Store). Where can I find the fugashi DLL to manually download it?

[polm]

@alinacoding You can download it from PyPI.

https://pypi.org/project/fugashi/#files

Can you explain how you installed fugashi? Did you use pip, was it in Powershell or something else, etc. This really shouldn't happen so any hints are helpful.

[polm]

Note that maybe installing in a venv works, but installing globally does not. Not really sure though. (In general I would recommend always using venvs.)

[alinacoding]

I used pip to install it on a Windows 64-bit machine, after I got the following error trace:

Traceback (most recent call last):
  File "chatbot.py", line 341, in <module>
    main()
  File "chatbot.py", line 292, in main
    bjsa = BertJapaneseSentimentAnalyzer()
  File "chatbot.py", line 48, in __init__
    self.tokenizer = BertJapaneseTokenizer.from_pretrained('cl-tohoku/bert-base-japanese-whole-word-masking')
  File "C:\Users\avoic\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\transformers\tokenization_utils_base.py", line 1719, in from_pretrained
    return cls._from_pretrained(
  File "C:\Users\avoic\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\transformers\tokenization_utils_base.py", line 1792, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "C:\Users\avoic\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\transformers\models\bert_japanese\tokenization_bert_japanese.py", line 151, in __init__
    self.word_tokenizer = MecabTokenizer(
  File "C:\Users\avoic\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\transformers\models\bert_japanese\tokenization_bert_japanese.py", line 231, in __init__
    import fugashi
  File "C:\Users\avoic\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\fugashi\__init__.py", line 1, in <module>
    from .fugashi import *
ImportError: DLL load failed while importing fugashi: The specified module could not be found.

On a Ubuntu 16.04 64-bit machine the installation via pip went smoothly.
On Windows I am also having trouble installing the mecab-python-windows package, since it complains about missing mecab.h file.

Thank you for your suggestion, I will try using a venv.

[polm]

Thanks for the extra info! It definitely looks like pip is just being weird, no idea why that would happen.

On Windows I am also having trouble installing the mecab-python-windows package, since it complains about missing mecab.h file.

That package is not maintained - the author moved development to mecab on PyPI. They mention the error you saw in their announcement post here. In general there is not much difference between their package and mecab-python3 except that mecab-python3 includes the MeCab binary on all platforms so it shouldn't require an extra install, though that might not help if you keep having this pip issue.

[Pugnator]

Have the same issue, installed python 3.12 from the official site, default settings, Windows 10.
The problem is pip installs the mecab.dll together with fugashi into c:\users\user\appdata\roaming\python\lib\site-packages\fugashi\libmecab.dll
But the library is checked against C:\Users\user\AppData\Roaming\Python\Python312\site-packages\fugashi during the import from python.
If I copy the lib - everything works like a charm.

[polm]

Glad you figured it out, and sorry you had to deal with this.

To be clear, while I still don't understand this very well, as far as I am aware it never happens if you use a virtualenv.

Let me update the top of this ticket to summarize what I know.

[Pugnator]

Glad you figured it out, and sorry you had to deal with this.

To be clear, while I still don't understand this very well, as far as I am aware it never happens if you use a virtualenv.

Let me update the top of this ticket to summarize what I know.

While it's easy to reproduce at least in my environment, I'll try to debug the installation this weekend to find out the reason. I'll keep you updated.