[DISCUSSION] possibly use conda-recipe-manager, bring souschef into incubation, or vendor souschef?

[beckermr]

Is your feature request related to a problem? Please describe.

There is no problem, but we may want to try and unify dev of these tools into a single spot to get increased efficiencies in maintenance. souschef is fantastic and is fundamental to this package.

Describe the solution you'd like / Describe alternatives you've considered

Some possible options:

• leave things as they are which is 100% fine
• move to using conda-recipe-manager
• bring souschef into incubation
• vendor souschef into grayskull

Additional context

It appears that conda-recipe-manager is moving towards being more than just a parser (conda-incubator/conda-recipe-manager#129). I do wonder how we'll square more features there with the already official garyskull project.

[marcelotrevisani]

Hi, thanks for opening this discussion! I appreciate the opportunity to share my perspective:

• Souschef was mainly created to manage the complex logic required to handle the various selectors we use as YAML comments in v1 recipes. Dealing with them was becoming quite cumbersome. However, my intention has always been to eventually phase it out as the ecosystem adopts v2 recipes more widely. With v2, the need for a dedicated parser significantly diminishes since everything will essentially be YAML + Jinja2.

• Currently, I don't have the interest or time to replace Souschef for conda-recipe-manager without a clear, tangible benefit. I’m trying to be pragmatic here and avoid changes that don't provide meaningful improvements.

• That said, I’m fine with bringing Souschef into incubation if the community feels it’s the right path. Alternatively, I could vendor it as an independent module, though I’m not sure there’s much advantage in doing so since a few projects appear to rely on it.

I’m open to further discussion and happy to align with whatever direction the conda-forge community feels would be best. Flexibility is key, and I trust the community's judgment! :)

[beckermr]

These are all great points @marcelotrevisani!

The main possible benefits of moving to conda-recipe-mansger from my perspective are

• less maintenance for devs (e.g., possibly you!) by centralizing around one parsing package
• built-in conversion utilities from the old format (v0) to the new format (v1)
• conda-recipe-manager is getting a lot of attention from various folks at anaconda
• FWIW, I suspect even though the new format is formally YAML and so requires no special parser, using a dedicated library will help with certain manipulations of the recipe and so might be attractive from a maintenance point-of-view. This remains to be seen.

I am happy to contribute work moving grayskull over to CRM if people are in agreement that this is not a horrible idea.

[schuylermartin45]

I was just forwarded this thread by a colleague.

I want to be 100% clear that conda-recipe-manager is not intended to replace anything. We have business needs at Anaconda that warrant advanced parsing and editing capabilities of conda recipe files. We are doing this work in the open so it may benefit the community. We might be developing some tools that do things that are similar to Grayskull. I get paid by Anaconda, so I must build what the company needs.

Again, I'm not trying to dictate what conda-forge does or bifurcate the ecosystem, but there are trade-offs at play. conda-recipe-manager (CRM) is backed by a corporate entity, so there will be full time engineering support going forward.

Now that being said, I have V1 support on my roadmap. The CRM parser is already semi capable of editing V1 files, but there are limitations.

Is it silly to use CRM's parsing tools on pure YAML files that are parseable in other ways? Maybe. But there are a few main reasons I think we should continue using CRM:

1. Maintenance. We're going to be in a V0 --> V1 transition period for some amount of time. If we invest in tools that use CRM, we should have less maintenance cost in the long run when a full switch occurs.
2. CRM has a lot of tooling specific to recipe files that could make it more valuable than just using a pure YAML parser.
3. To @beckermr 's point, centralizing around one library will pay future dividends. Using a YAML parser of your choosing per tool would likely incur multiple solutions to the same problems.

Am I fan that this project is all in Python? No. I kind of wish we went with Rust, but at the time I started this, I needed to prove a parser was worth the engineering time with a quick turnaround. Even if it is flawed in some aspects, I think the tooling available in CRM is a net positive for the community. A rising tide lifts all boats, after all.

@wolfv @jezdez for visibility.