Investigate making a brainglobe metapackage on conda-forge

[dstansby]

No description provided.

[dstansby]

I think this should be possible - here's a proof of concept: conda-forge/staged-recipes#22687

[adamltyson]

Awesome. I was hoping it would be straightforward.

[sfmig]

from a conversation with @willGraham01:
right now when creating a conda environment, we need to install hdf5 manually for pip install brainglobe to work (tables requires it). In conda, we should be able to add hdf5 explicitly to the recipe.

[adamltyson]

@sfmig is this only on macOS though? I assumed it was only a problem when there aren't pre-compiled tables wheels, i.e. only on Apple Silicon.

See discussion at brainglobe/brainglobe-segmentation#145. Hopefully this will be fixed with pip installs sometime soon.

[sfmig]

oh probs yes! I can't test in other platforms

[willGraham01]

Thought I'd revive this thread now that we're over the majority of the reorganisation of BrainGlobe version 1 and our packages have stable releases on PyPI.

This issue is currently blocked by:

• Merging atlasapi and atlasgen, renaming in the process: Merge bg-atlasapi and bg-atlasgen BrainGlobe#43
• Replacing tensorflow with keras in our cellfinder dependency. (Although we can get quite far ignoring this actually).
• morphapi is currently a) broken and b) needs a particularly tricky conda-forge recipe.

Packages currently on conda-forge:

• brainglobe-utils
• brainglobe-napari-io
• brainglobe-segmentation
• bg-space OUTDATED, replace with brainglobe-space
• bg-atlasapi OUTDATED, replace with brainglobe-atlasapi
• imio OUTDATED, absorbed into brainglobe-utils-feedstock.
• brainreg

Of these, we can retain the utils, napari-io, and segmentation feedstocks, but they will need updating to the latest version as we carry out the process of updating our packages. To that point:

• bg-atlasapi will need to be marked as deprecated. We'll need a new conda recipe for brainglobe-atlasapi, as per Merge bg-atlasapi and bg-atlasgen BrainGlobe#43
• bg-space will also need to be marked as deprecated, since we'll be needing the new brainglobe-space package instead.
• imio will suffer the same fate as the above.

Otherwise, we can follow the dependency tree to upload packages in a sensible order. conda-forge is able to add multiple packages at once - even with interdependencies - so long as all packages can be installed through conda channels.

The Plan

Presented in the order that should be attempted due to dependency issues, etc.
However some tasks can be done in parallel.

• Upload brainglobe-space: Add brainglobe-space conda-forge/staged-recipes#25292
• Upload brainglobe-atlasapi | Add brainglobe-atlasapi and pymcubes dependency conda-forge/staged-recipes#25533 | Needs pymcubes - awaiting author permission.

From here, we can split into two parallel streams:

morphapi and dependencies

• Upload morphapi, brainrender, and brainglobe-heatmap in one PR
  • BLOCKED by morphapi problems.
  • Depends on brainglobe-altasapi being available on conda-forge.
  • morphapi
  • brainrender
  • brainglobe-heatmap

Building up to brainreg

• Upload brainrender-napari
• Update brainglobe-utils-feedstock to ship a brainglobe-utils version that depends on brainglobe-space and brainglobe-atlasapi rather than the bg-named packages.
• Upload brainglobe-napari-io and cellfinder
  • Can be done simultaneously
  • cellfinder is BLOCKED by the shift from tensorflow to keras.
  • brainglobe-napari-io
  • cellfinder
• Deprecate imio-feedstock
• Update brainglobe-segment-feedstock if necessary
• Update brainreg-feedsotck

Meta-package

Once both streams are complete, we can upload the meta-package to conda-forge.
Note: we might be able to do this sooner if we want to forgo the morphapi-dependencies, but this would be a manual conda-forge exclusion for the time being.

• Upload brainglobe-meta and brainglobe-workflows
  • Can be done simultaneously
  • brainglobe-meta
  • brainglobe-workflows

[willGraham01]

brainglobe-atlasapi depends on pymcubes, which is not on conda-forge. This is almost certainly because it now includes the old atlasgen package.

I've asked the creator of pymcubes if they're OK with us uploading it to conda-forge, and if they want to be added as a maintainer. I have a (seemingly working) pull request that uploads both packages to conda-forge over in conda-forge/staged-recipes#25533

[adamltyson]

If this doesn't work, we don't necessarily need all the atlasgen code in the package, so it might be better to either move it out, or just add some optional atlas generation dependencies.

[willGraham01]

If this doesn't work, we don't necessarily need all the atlasgen code in the package, so it might be better to either move it out, or just add some optional atlas generation dependencies.

Recipe seems to work - had to suppress the Windows build tests because vtk is broken on conda-forge (doesn't show up in pip check but is there).

[willGraham01]

brainglobe-atlasapi is now available on conda-forge 🥳 So the rest of the plan can start to be enacted.