[BUG] installation with pip fails on MacOS

[alessandrofelder]

Describe the bug
Can't pip install on Silicon Mac - fails with HDF5 related problem. At its root this is a problem with pip install tables.

To Reproduce

• create and activate a clean conda env on a MacOS with M2
• run pip install brainglobe-segmentation or try to install via the napari plugin installer.
• command line output is:

Collecting tables>=3.7.0 (from pandas[hdf5]->brainglobe-segmentation)
  Using cached tables-3.9.1.tar.gz (4.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      /var/folders/gw/g898mw1d2319ncb69pn6gvjc0000gn/T/H5closeobvdxyoq.c:1:10: fatal error: 'H5public.h' file not found
      #include "H5public.h"
               ^~~~~~~~~~~~
      1 error generated.
      cpuinfo failed, assuming no CPU features: 'flags'
      * Using Python 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:37:07) [Clang 15.0.7 ]
      * Found cython 3.0.5
      * USE_PKGCONFIG: False
      * Found conda env: ``/Users/alessandrofelder/mambaforge/envs/tutorials``
      .. ERROR:: Could not find a local HDF5 installation.
         You may need to explicitly state where your local HDF5 headers and
         library can be found by setting the ``HDF5_DIR`` environment
         variable or by using the ``--hdf5`` command-line option.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behaviour
Ideally pip install brainglobe-segmentation would just work (this would also make the napari installer work, I think).
But alternatively we could specify the extra step in the installation docs.

Computer used (please complete the following information):

• OS: MacOS on M2
• Python: clean conda env with Python 3.11

[alessandrofelder]

From a superficial skim of PyTables/PyTables#1050 there's some hope that the next patch of pytables (3.9.2) will fix this.

[adamltyson]

I just tried to reproduce this and noticed something that might be relevant for us as devs.

If you try to install tables on M1 it currently fails due to a lack of the HDF5 library. This is fixed by running conda install -c conda-forge hdf5 first. What I didn't know is that the pip cache doesn't just store wheels that have been downloaded, it also stores wheels created locally. So if you try to install tables again in a fresh environment, it will work fine.

TLDR; to fully reproduce, create a new env and run pip cache purge

[adamltyson]

Just to confirm this issue still exists.