Support multiple DbContexts in integration tests

[jakubfijalkowski]

Currently, we do this in DbContextsInitializer:

foreach (var ctx in getContexts())
{
    await CreatePolicy.ExecuteAsync(async () =>
    {
        await ctx.Database.EnsureDeletedAsync(); // This is the culprit
        await ctx.Database.EnsureCreatedAsync();
    });
}

Unfortunately, if you add multiple DbContexts, every one loop will delete and re-create the database (with different schema!) because we use single connection string. This makes it unusable in multi-context systems. There is a workaround that just overrides the registered context (in TestOverridesPostModule pattern), like:

public override void ConfigureServices(IServiceCollection services)
{
    var oldCtx = services
        .Where(c =>
            c.ServiceType == typeof(ClientsDbContext) ||
            c.ServiceType == typeof(DbContextPool<ClientsDbContext>) ||
            c.ServiceType == typeof(DbContextPool<ClientsDbContext>.Lease) ||
            c.ServiceType == typeof(DbContextOptions<ClientsDbContext>))
        .ToList();
    foreach (var c in oldCtx)
    {
        services.Remove(c);
    }

    var dbConnStr = Api.Config.ConnectionStrings.Database(config);
    var builder = new SqlConnectionStringBuilder(dbConnStr);
    builder.InitialCatalog += "_clients";
    services.AddDbContext<ClientsDbContext>(options =>
    {
        options.UseSqlServer(builder.ToString(), sqlOpts => sqlOpts.MigrationsAssembly("XYZ.Migrations"));
        options.EnableSensitiveDataLogging();
    });
}

but that is suboptimal.

[jakubfijalkowski]

Here is a rough idea on how we can address that systematically:

Project problem

Now, most of the projects depend on SqlServer__ConnectionString env variable even if it has multiple contexts in it. This is rather too much of a simplification - if we have multiple DbContexts in a project, each should be configured with a separate connection string. This would allow us to juggle databases with ConfigurationOverrides and use separate (physical) database for each and every contest.

CoreLibrary problem

The main culprit is here, in ConfigurationOverrides. It just assumes that it provides a single connection string. That is very limiting. Forcing overriding the addresses and log level is also limiting - maybe you don't want that or you have a different configuration scheme.

To fix that, we should generalize the overrides.

Proposed solution

Instead of having ConfigurationOverrides look like it looks now (so a predefined set of config vars), it should accept a set of variables as a dictionary and inject it at runtime. We don't rely on any dynamic behavior there (nor should we), so it will suffice. To handle the connection strings, we just need a static factory method - we already rely on env variable there, that is accessible using other static methods.

Code skeleton

public class ConfigurationOverrides : IConfigurationSource
{
    public const string ConnectionStringBaseDefault = "SqlServer__ConnectionStringBase";

    private readonly Dictionary<string, string> data;

    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        return new Provider(data);
    }

    private class Provider : ConfigurationProvider
    {
        private readonly Dictionary<string, string> data;

        public override void Load()
        {
            Data = data;
        }
    }

    public static string OverrideDatabaseConnectionString(string dbPrefix, string sourceEnv = ConnectionStringBaseDefault, string varName = "Database")
    {
        var dbName = $"{dbPrefix}_{Guid.NewGuid():N}";
        var rest = Environment.GetEnvironmentVariable(sourceEnv);
        return $"{varName}={dbName};" + rest;
    }
}

Of course, it needs tests and probably some DX improvements (builder pattern with two methods on ConfigurationOverrides, one for plain value, the other for connection strings). Plus some documentation in docs (we should use this issue to start documenting how we do integration testing).