Failing on M1 Mac - OSError: Cannot load native module 'Crypto.Hash._SHA256'

[krisun09]

I am trying to follow the installation guide for PyCryptodome. But when I try to run Cryptodome / Crypto modules in my code it is giving me the following error.

With python3.9, I've tried installing pycryptodome 3.14.1, removing pycryptodome 3.14.1 then install pycryptodomex 3.14.1, removing pycryptodome x3.14.1 then install pycryptodome 3.10.1,

Then I tried to update my python to python3.10 using ARM, and install pycryptodome 3.14.1, which is still giving me the same error.

Should I try reinstalling python? ;( Any help is appreciated. Thanks in advance.

[Varbin]

As I do not have an ARM Mac (or any Mac at all), I can only be of little help.

The error message indicates the compiled extensions and your Python installation have different architectures. Do you mind telling which architecture your Python and the installed extension has? You should be able to get it with file /opt/homebrew/lib/python3.10/site-packages/Crypto/Hash/_SHA256.abi3.so and file $(which python3).

[Will-Gaines]

As I do not have an ARM Mac (or any Mac at all), I can only be of little help.

The error message indicates the compiled extensions and your Python installation have different architectures. Do you mind telling which architecture your Python and the installed extension has? You should be able to get it with file /opt/homebrew/lib/python3.10/site-packages/Crypto/Hash/_SHA256.abi3.so and file $(which python3).

I am having a near identical issue (but failing to import _raw_ecb native module). PyCryptoDome was installed as part of other project dependencies (using Poetry over pip), and have the same errors coming back as the OP.

Outputs of your suggested commands:
file /Users/XXXX/environments/grid-actions/lib/python3.9/site-packages/Cryptodome/Cipher/_raw_ecb.abi3.so

/Users/XXXX/environments/grid-actions/lib/python3.9/site-packages/Cryptodome/Cipher/_raw_ecb.abi3.so: Mach-O 64-bit bundle x86_64

file $(which python3) (Run inside my virtualenv)
/Users/XXXX/environments/grid-actions/bin/python3: Mach-O 64-bit executable arm64

[xilopaint]

Some users of my application are complaining about the same problem.

[j1nma]

I also have this problem with Python 3.8.12, Poetry, on Apple M1

[Luke7389]

same problem here, python 3.10

[Luke7389]

resolved by downgrading to python 3.9, in my case I was using a function that uses a module called pyteal, downgrading forced me to use a different version of petal and now everything works as expected

[xilopaint]

resolved by downgrading to python 3.9

This is the traceback from one of them:

Traceback (most recent call last):
  File "/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/alfred_pdf_tools.py", line 48, in <module>
    from PyPDF2 import PdfReader, PdfWriter, PdfMerger, PageRange, PageObject, errors
  File "/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/PyPDF2/__init__.py", line 10, in <module>
    from ._encryption import PasswordType
  File "/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/PyPDF2/_encryption.py", line 60, in <module>
    from Crypto.Cipher import AES, ARC4  # type: ignore[import]
  File "/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/Crypto/Cipher/__init__.py", line 27, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher
  File "/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/Crypto/Cipher/_mode_ecb.py", line 35, in <module>
    raw_ecb_lib = load_pycryptodome_raw_lib("Crypto.Cipher._raw_ecb", """
  File "/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/Crypto/Util/_raw_api.py", line 309, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Not found '_raw_ecb.cpython-39-darwin.so', Cannot load '_raw_ecb.abi3.so': dlopen(/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/Crypto/Util/../Cipher/_raw_ecb.abi3.so, 0x0006): tried: '/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/Crypto/Util/../Cipher/_raw_ecb.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/Users/xyz/Downloads/Alfred.alfredpreferences/workflows/user.workflow.F5626124-7D80-45A1-A349-E413ADB69042/Crypto/Cipher/_raw_ecb.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), Not found '_raw_ecb.so'

[Luke7389]

@xilopaint although they might be related, you are now receiving a different error, which looks like the one Will is receiving, in my case it worked simply downgrading. I used a clean python installation

[pyrito]

Hi, I'm currently running into this issue myself. Is there a clear idea on why the x86 shared object is being installed instead of the arm64 one? Is there a way to force the install to use the arm64 version? It seems that this isn't version specific to Pycryptodome, but python. Things are working somehow when I am on python 3.9 vs python 3.10. @Legrandin your help would be much appreciated here!

[jeugregg]

Hi, I have the same error.
For you investigation, I managed to use 3.15.0 version with conda from a fresh new env, after installing web3 lib : pip install web3
pip uninstall pycryptodome(v3.16.0)
conda install pycryptodome (v3.15.0)
It was not the last version... why?

[Rikuto7]

I am also having the same problem.
I get an error in lambda.
I installed the following.
docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.9" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.9/site-packages/; exit"
I also tried using docker with amazonlinux:latest but it does not work.

Cannot load native module 'Crypto.Cipher._raw_ecb': Not found '_raw_ecb.cpython-39-x86_64-linux-gnu.so', Cannot load '_raw_ecb.abi3.so': /opt/python/lib/python3.9/site-packages/Crypto/Util/../Cipher/_raw_ecb.abi3.so: cannot open shared object file: No such file or directory, Not found '_raw_ecb.so'

Does anyone know anything about this?

[EnkhAmar]

I have same issue here. I am trying to make layer for aws sam project. I am using macbook m1.
"Cannot load native module 'Crypto.Hash._MD5': Not found '_MD5.cpython-39-aarch64-linux-gnu.so', Cannot load '_MD5.abi3.so': /opt/python/Crypto/Util/../Hash/_MD5.abi3.so: invalid ELF header, Not found '_MD5.so'"

[gorosz]

I also have the same problem as @Rikuto7 . I have an application which depends on pycryptodome and I try to run it in a docker container.
I use ubuntu focal as the base of my docker image.
I install pycryptodome with apt. Once I start my application I get the following error:

from Cryptodome.Hash import MD4
  File "/usr/lib/python3/dist-packages/Cryptodome/Hash/MD4.py", line 54, in <module>
    _raw_md4_lib = load_pycryptodome_raw_lib(
  File "/usr/lib/python3/dist-packages/Cryptodome/Util/_raw_api.py", line 259, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Cryptodome.Hash._MD4': 

What I figured out is that when pycryptodome tries to collect the C file extensions suffixes the .cpython-38-x86_64-linux-gnu.so suffix is not showing up.

This is the 3.6.1 pycryptodome part which handles the lib loading:

if sys.version_info[0] <= 3 or \
   (sys.version_info[0] == 3 and sys.version_info[1] <= 3):

    import imp
    extension_suffixes = []
    for ext, mod, typ in imp.get_suffixes():
        if typ == imp.C_EXTENSION:
            extension_suffixes.append(ext)

else:
    from importlib import machinery
    extension_suffixes = machinery.EXTENSION_SUFFIXES

It doesn't matter which branch get executed the .cpython-38-x86_64-linux-gnu.so extension suffix is missing, although when I start the interpreter and run imp.get_suffixes() or machinery.EXTENSION_SUFFIXES the suffixes is there.

When I modify the lib and include the suffix in the initial extension_suffixes or append to it the .so get loaded fine

I don't know if the two cases (docker, Mac M1) is related but very similar.
I also know that my lib version a bit out dated but I have strict version rules and cannot update but the lib loading logic hasn't changed that much as I can see.

python version is 3.8.10
pycryptodome is 3.6.1