RFC: Resolvers and environment management in ASV

[HaoZeke]

There are now a bunch of front-end (CLI) tools which handle dependency resolution and installation of environments, including rip, rye (both already dropped out¹), and uv.

A key issue is that most of these tools (including mamba) don't actually expose the functionality which asv really needs², that is the dependency resolution / a Python API (e.g. see astral-sh/uv#7345) itself. This is accomplished via pubgrub-rs in uv, resolvo in pixi / rip and the venerable libsolv in mamba.

Somewhat related are these issues around the ecosystem:

• Request: Add uv as an environment type #1433
• Cooperation with PubGrub-rs. mamba-org/resolvo#2
• RFC — Towards a new solver for libmamba mamba-org/mamba#3387
• What about pixi and rip? astral-sh/uv#1572
• https://prefix.dev/blog/uv_in_pixi
• Better uv integration astral-sh/rye#668
• Rye and uv astral-sh/rye#1342

Basically, moving forward, it seems like the best approach is to generate bindings to uv internals like the dependency resolution while extracting the environment management out further. Or by using py-rattler (here). Although the CycloneDX SBOM might also be of interest as an intermediate file format ³.

Another option is to use subprocess to drive calls to uv or something similar, but in the long run this is more problematic, since parsing CLI is never a great idea, and also leaves the project vulnerable to changes in the in internal lock file representation.

Footnotes

1. e.g. this comment ↩

2. e.g. the trouble with the libmambapy dependency which needed a new release since part of the API was removed. ↩

3. https://github.com/CycloneDX/cyclonedx-python ↩

[HaoZeke]

For some more context, this matters to asv because when run with existing environments, we still need to know the changes to versions relative to the last known configuration.

[lucascolley]

@HaoZeke with mamba 2.0 it seems like some of the things we need have been made public. gh-1438 reports that the current mamba plugin is incompatible with mamba 2.0. Just to check, do you have any plans to move forwards with an alternative like uv or rattler and deprecate support for new versions of mamba? If not, I've started a very rough draft of changing the mamba plugin for 2.0 compatibility at #1439. The mamba docs aren't great here, so I've been asking for some clarifications at mamba-org/mamba#3567.

[HaoZeke]

Great questions.

Just to check, do you have any plans to move forwards with an alternative like uv or rattler and deprecate support for new versions of mamba?

I'm not planning that, although if I get this correct then I'd spin out all the environment managers except virtualenv into their own packages, so asv itself will not have a release schedule tied to mamba or conda.

If not, I've started a very rough draft of changing the mamba plugin for 2.0 compatibility at #1439.

This is fantastic contribution in any case and would be happy to accept that and review it whenever its ready.

The mamba docs aren't great here, so I've been asking for some clarifications at mamba-org/mamba#3567.

Like pixi, they really move too quickly for good documentation of the internals.

The overall goal here is not to switch to one environment manager over the other, but to incorporate a sort of software bill of materials (SBOM) into ASV. This SBOM can then be satisfied by any of the environment managers (even system ones) and since ASV in this scenario will be able to determine the required / changed requirements better support using existing environments.

[HaoZeke]

After discussions with @rgommers, it seems like one pragmatic approach is to simply focus on offloading to the stable CLI and generating lockfiles of say, uv for pip or pixi for the conda-ecosystem.

[pauloxnet]

I would love to see uv support in ASV. Thanks for pushing this forward.