Best practices for tests that involve databases

[ej-shafran]

I have a very complex function with some defined behavior that I want to use property-based testing to ensure is correct (or find issues with). However, this function makes several database calls. Up to now I've been using a simple function with Faker to save a bunch of fake data to the database, then calling the function with some randomized input. However, I'm missing a lot of the features I like in fast-check, such as shrinking, replayability, etc.

What's the recommended way to write a property-based test that deals with a database? Is it to reset the database between each property run? That seems like it would consume a lot of time... Additionally - how would you create an arbitrary for values that depend on database returns (i.e. a user needs a department ID, which requires saving the department to the database...)? I would really appreciate some help understanding what other people have done.

[gruhn]

#5022 is a similar question on how to randomize database relations. I think this also addresses this question if I understand you correctly:

Additionally - how would you create an arbitrary for values that depend on database returns (i.e. a user needs a department ID, which requires saving the department to the database...)?

Regarding this question:

What's the recommended way to write a property-based test that deals with a database? Is it to reset the database between each property run? That seems like it would consume a lot of time...

There is a strong assumption that every test run starts on a clean slate and that no relevant state is carried over. Otherwise shrinking can give bogus results. For example, let's say your test fails (for whatever reason) when there is an email address in the database that contains a "+". One test run inserts [email protected] provoking the issue. Now the shrinker tries to find a simpler counterexample. But if the database is not reset, then the faulty email address stays in there. So the issue is suddenly reproducible in any test run and the shrinker will give you a counterexample that has nothing to do with the actual problem.

Maybe it's fine to reset the database before every run. If it's too slow or cumbersome, is mocking away the database calls is an option?

[ej-shafran]

That definitely answers the dependency part of my question, thanks!

I kinda want to check that the database is properly updated, so I was trying my hardest to avoid mocking it, but I might have to try... Will play around with it a bit and see if I have any neat ideas.