Asynchronous dependency resolution

[danielcondemarin]

Some backend servers act as API aggregators with many external dependencies downstream. The order in which dependencies are called is not always simple, for example some could be called in parallel using goroutines, some others might need to wait for one or more dependencies to complete before they can run, etc. For a large application this can be hard to maintain. Such problem can be modelled with a DAG and the order in which external services are called defined by topologically sorting the graph.

Wire implements a large part of the above, except the code emitted is designed for synchronous execution. Essentially what I wanted is for Wire to do the following,

type Bar int

func execute(ctx context.Context) int {
	wire.Build(
		wire.AsyncFunc(provideBar),
		wire.AsyncFunc(provideFoo),
	)
	return 0
}

func provideFoo(b1 Bar) (int, error) {
	return int(b1), nil
}

func provideBar(ctx context.Context) (Bar, error) {
        time.Sleep(1 * time.Second)
	return 1, nil
}

where the emitted code is,

func execute(ctx context.Context) (int, error) {
	g, ctx := errgroup.WithContext(ctx)
	barChan := make(chan Bar, 1)
	g.Go(func() error {
		bar := provideBar(ctx)
		select {
		  case barChan <- bar:
		  case <-ctx.Done():
			  return ctx.Err()
		  }
		return nil
	})
	int2Chan := make(chan int, 1)
	g.Go(func() error {
		var bar Bar
		select {
		  case bar = <-barChan:
		  case <-ctx.Done():
			  return ctx.Err()
		  }
		int2 := provideFoo(bar)
		select {
		  case int2Chan <- int2:
		  case <-ctx.Done():
			  return ctx.Err()
		  }
		return nil
	})
	if err := g.Wait(); err != nil {
          return 0, err
	}
	int2 := <-int2Chan
	return int2, nil
}

Granted this example doesn't really shine but imagine for large complex aggregations this can be really useful.

I actually went ahead and tried implementing it, with some nice results: https://github.com/deliveroo/async-wire

I was wondering if this has come up in the past and thoughts on merging this kind of functionality upstream?

PS. Initally I implemented a reflection based solution using https://github.com/twitter/nodes API as a guide, but I really wanted compile time safety and realised Wire did a lot of what I needed already.