Allow tuning of size distribution introduced in 2.10.0

[glchapman]

From my experience with other QuickCheck libraries, I was surprised by the prevalence of ("", 0) in this:

> Gen.sample 10 10 Arb.from<(string * int) list>.Generator;;
val it : (string * int) list list =
  [[("", 0); ("", 4)];
   [(null, 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0);
    ("", 0); ("", 0)]; [("lb", 1)];
   [("", 0); ("", 0); (null, 0); ("", 0); ("", 4); ("", 0)];
   [("", 0); ("3", 1); ("", 0); (null, 0); ("", 0); ("", 0); ("", 0); ("", 0);
    ("", 0)]; [("", -1); ("", 0); ("", 0); ("", 0)]; [("", 0); ("", -1)];
   [(null, 0); ("", 0); ("", 0); ("", 0); ("", 0); ("", 0)];
   [("", 0); (null, 0); ("^", -1)]; [("QAz", 0)]]

I've briefly scanned the paper you reference in the release notes for 2.10.0, and, if I understand correctly, FSCheck is only using the fixedSize Distributor described in the paper. I think it would be useful to implement the other Distributors described, and provide some mechanism for selecting them when generating data.

[kurtschelfthout]

I became convinced that the way the original quickcheck distributed size is less than ideal; what we're used to is not necessarily better.

Also, my sense is that adding distributors causes more complexity than it's worth.

The options available in FsCheck today, and likely for the forseeable future, are:

• for coarse control, simply increase the size;
• for fine-grained control over how size is distributed, write a custom generator using the size related functions on Gen.

[kurtschelfthout]

Just to let you know I've changed my mind about what I said above. I now think we should somehow allow either the old or the new behavior for product-like types, and that it should be configurable. I need to think a bit of how to fit it into the API though, and it will go into v3.

[witoldsz]

I am having similar issue, i.e.: having such a signature:
rates: Map<ProviderId, RateObject list> and MaxTest=5 the FsCheck is generating, almost always, for 3 of 5 times the map with empty list.

On the other hand, if I change rates to:
rates: RateObject list and MaxTest=5
then FsCheck is generating 5 times a very long lists. Not even once the list is empty.

The RateObject looks like this:

type ProviderId = ProviderId of string
type Currency = Currency of string
type Pair = Currency * Currency
type Rate = Rate of decimal
type RateObject = { Pair: Pair; Rate: Rate; ValidFrom: Instant }

What is wrong? Why the generated data is so bad? On one side the lists are empty 3/5 times, on the other I cannot even get an empty list to see if the code handles it well. Please advise.

[AlexeyRaga]

I am having this problem too.

When generating a record with a few string fields, in a reasonable (100-200) amount of tests the strings are not getting longer than 1, rarely 2 characters and are very often null.

This is problematic because this way the actual coverage is very small.

Has anyone found a workaround? Except to tuning startSize, which is not a greatest idea?

[kurtschelfthout]

Has anyone found a workaround? Except to tuning startSize, which is not a greatest idea?

Why not? Increasing StartSize and EndSize seems fine to me.