Intermittent NullReferenceException in Unit Tests

[NathanTurnbow]

In many of the draft unit tests we create a new testing context like this:

using (var http = new HttpTest()) {
    http.RespondWithJson(Fixtures.Directory.DefaultResponse);

Intermittently, the call to RespondWithJson will throw a NullReferenceException.

in flurl, the source code for this call is (currently) this:

public HttpTest RespondWithJson(object body, int status = 200, object headers = null, object cookies = null, bool replaceUnderscoreWithHyphen = true) {
	var content = new CapturedJsonContent(Settings.JsonSerializer.Serialize(body));
	return RespondWith(content, status, headers, cookies, replaceUnderscoreWithHyphen);
}

https://github.com/tmenier/Flurl/blob/80cf699ef8b153c0c0fd2324a426f54f39197fd0/src/Flurl.Http/Testing/HttpTest.cs#L92-L95

The only case where we can possibly throw a null reference exception is if either Settings is null or Settings.JsonSerializer is null. In this case it is Settings.JsonSerializer, but Settings actually has a public setter so if you want to deliberately cause a similar error you could call http.Settings = null, but we're not doing that in draft tests.

When Settings.JsonSerializer is called, it first attempts to find a local value for JsonSerializer, and if it fails to find a local setting it then looks for a local default, and ultimately falls through to the globally configured static defaults accessed at FlurlHttp.GlobalSettings. In all of the test cases, we fall all the way through to the static global settings searching for an instance of JsonSerializer that can be used to configure the JsonResponse for the test.

Accessing settings in FlurlHttp.GlobalSettings is not thread-safe to be used across tests because it has the following publicly exposed method:

public override void ResetDefaults() {
	base.ResetDefaults();
	Timeout = TimeSpan.FromSeconds(100); // same as HttpClient
	JsonSerializer = new NewtonsoftJsonSerializer(null);
	UrlEncodedSerializer = new DefaultUrlEncodedSerializer();
	FlurlClientFactory = new PerHostFlurlClientFactory();
	HttpClientFactory = new DefaultHttpClientFactory();
}

https://github.com/tmenier/Flurl/blob/80cf699ef8b153c0c0fd2324a426f54f39197fd0/src/Flurl.Http/Configuration/FlurlHttpSettings.cs#L212-L219

Here, base.ResetDefaults() empties the global settings before they are reset to the hard-coded defaults

public virtual void ResetDefaults() {
	_vals.Clear();
}

https://github.com/tmenier/Flurl/blob/80cf699ef8b153c0c0fd2324a426f54f39197fd0/src/Flurl.Http/Configuration/FlurlHttpSettings.cs#L131-L133

So there exists a period of time where any of the global settings could be null, including JsonSerializer, if ResetDefaults() is called.

We have a set of tests which call FlurlHttp.GlobalSettings.ResetDefaults() explicitly (see

Draft/tests/Draft.Tests/Tests/VerificationStrategies/BaseVerificationStrategyTests.cs

Line 61 in 2eddfaf

FlurlHttp.GlobalSettings.ResetDefaults();

).

When all of the unit tests are run in parallel, we have a chance for the global defaults to be reset at the same time that we are trying to configure a json response, possibly resulting in the observed NullReferenceException.

When this offending line is removed (below) I no longer can reproduce any intermittent test failures.

Draft/tests/Draft.Tests/Tests/VerificationStrategies/BaseVerificationStrategyTests.cs

Line 61 in 2eddfaf

FlurlHttp.GlobalSettings.ResetDefaults();