Statiq does not see pipelines after engine.Pipelines.Clear()

[juchom]

I have made a simple pipeline to better understand how Statiq works and I have an issue with this code :

This code works fine, Statiq use my pipeline

await Bootstrapper
    .Factory
    .CreateDefault(args)
    .AddPipeline<MyPipeline>()
    .RunAsync();

This code does not work, Statiq does not see it

[WARN] No pipelines are configured or specified for execution.

await Bootstrapper
    .Factory
    .CreateDefault(args)
    .ConfigureEngine(engine =>
    {
        engine.Pipelines.Clear();
    })
    .AddPipeline<MyPipeline>()
    .RunAsync();

Basically, I was trying to remove all default pipelines from Statiq.Web and see how my pipeline was doing.

[daveaglick]

At first glance I think what's going on here is that the fluent pipeline configuration methods like AddPipeline work by adding the specified pipeline to the dependency injection container, which the engine then uses to get pipelines when it's created. The confusing part here is that the fluent methods aren't necessarily called in the order they're specified in the chain (I.e. they're all lazy). All the AddPipeline() and related methods do their work before the ConfigureEngine() call, even if they're called later in the fluent chain. So in your second example, the MyPipeline pipeline does get added, it just gets removed again inside the ConfigureEngine() call.

To do what you want, the trick here is to call engine.Pipelines.Add() inside ConfigureEngine(). I can totally see where that's confusing and would welcome any documentation or other suggestions on how to make it clearer. Another thing I could look at doing is enhancing the fluent API so that the order of the chain and the order of execution is enforced a little bit (I.e. don't allow a AddPipeline() call after a ConfigureEngine() call. I'm a little torn on that though because there are legitimate scenarios where you might want or need to add the fluent methods out of execution order, knowing when they'll be executed.

Did all that make sense?

[juchom]

Hello @daveaglick,

Thanks for this very detailed response, after digging in the source I suspected something like this.

Before you take any technical decision, it's important that you know that these are my first days using Statiq and I have not read the doc in details.

I'm learning from this repo : https://github.com/AnkurSheel/StatiqHelpers and this one https://github.com/spectreconsole/spectre.console/tree/main/docs

Now concerning my mistake, I didn't know that pipelines where added to a dependency injection container. When I saw the AddPipeline I assumed it was a shortcut to engine.Pipelines.Add. I have expected the same behaviour like the AddProcess in Spectre.Console doc.

And here is a screenshot of the intellisense of the AddPipeline

The first thing, the doc could be something like that Adds a pipeline of type TPipeline to the dependency injection container.

The second thing, that may make things clearer is to rename AddPipeline to RegisterPipeline.

I was really expecting the same behaviour like AddProcess which is basically : I add the process to the bootstrapper, I see it during the execution.

Again, thanks a lot for this great answer, and thanks a lot for this great tool !

[daveaglick]

The suggestions for Intellisense you made are right on - if that had been better you probably would have realized what was going on without too much trouble. Already made some changes for the next release on that code documentation.

As for renaming the methods - I really like your suggestion and wish I had thought of Register... vs. Add... initially. Backwards compatibility being what it is, and with the widespread use of those methods, I'm not sure it's worth a break at this point though.

[juchom]

Why not tagging the AddPipeline with Obsolote with a forwarding message to RegisterPipeline ?

You could then enforce the Obsolete to break the build in after some time and after that you could remove AddPipeline gently from the basecode.

[daveaglick]

Oh - I like that idea! Even if I never remove the Add... versions, the ObsoleteAttribute should let folks know something is going on. That's especially helpful since not all the AddPipeline methods actually work the same way - that overloads that add pipelines directly (I.e. not through a class you write but by defining module collections inline) use a different pattern that works a little differently. Changing the name of the ones that actually register with the DI container is a nice way to distinguish. I'll make the change for the next release - thanks!