README.md

Randomized Tests with JUnit5

You can use Datagen + JUnit5 integration to facilitate randomization in parameterized tests. Examples:

@Alphanumeric(length = 1, name = "min boundary")
@Alphanumeric(min = 2, max = 29, name = "middle value")
@Alphanumeric(length = 30, name = "max boundary")
@English(max=30)
@Unicode(max=30)
@Numeric(max=30)
void invokedForEachAnnotation(String value, String name) {
    assertTrue(value.length() >= 1 && value.length() <= 31, "Failed case: " + name);
}

This will run the test 6 times with different parameters according to the annotations. These tests will run 2 times each:

@RandomInt(min = 1, name = "greater than zero")
@RandomInt(max = -1, name = "less than zero")
void zeroIsNotPassed(int param, String name) {
    assertNotEquals(0, param, "Failed case: " + name);
}

@RandomLong(min = 1, name = "greater than zero")
@RandomLong(max = -1, name = "less than zero")
void zeroIsNotPassed(long value, String name) {
    assertNotEquals(0, value, "Failed case: " + name);
}

@RandomDouble(min = 1, name = "greater than zero")
@RandomDouble(max = -1, name = "less than zero")
void zeroIsNotPassed(double value, String name) {
    assertFalse(value > -1 && value < 1, "Failed case: " + name);
}

@English(max=1)
@BlankString
void canAlsoPassNullsOrBlankStrings(String value, String name) {
    assertTrue(value == null || value.trim().length() < 2, name);
}

In the end results look like this:

Though if you need to run a test only once and you want to use randomization - it's going to be more concise to use Datagen API directly. More examples are available in the test.

Seeds

Seed is a number that determines which random values are generated. By default it's initialized with System.nanoTime() but it's possible to hardcode it, in that case every time the test is run the same random data is going to be generated:

@Test @Seed(123)
void explicitSeed_generatesSameDataEveryTime() {
    assertEquals("56847945", numeric(integer(1, 10)));
    assertEquals("0o2V9KpUJc6", alphanumeric(integer(1, 20)));
    assertEquals("lfBi", english(1, 10));
    assertEquals(1876573356364443993L, Long());
    assertEquals(-8.9316016195567002E18, Double());
    assertEquals(false, bool());
}

This will work for parameterized tests as well. It's possible to set the @Seed per class which would make all the test methods generate the same values over and over again.

Usually you don't need to set the seed, but if your test fails it's nice if you could reproduce it exactly again - that's the primary use case for setting seeds manually. If you add this extension to your test classes it will put the method and class seeds into logs (use SLF4J implementations) if a test fails:

@ExtendWith(DatagenSeedExtension.class)
class Junit5ParameterizedTest {}

According to JUnit5 docs you can pass system argument to enable auto-registration instead of marking each class with @ExtendWith:

-Djunit.extensions.autodetection.enabled=true

The output may look something like this:

Random Seeds:  testMethod[162024700321388] NestedTestClass[162024700321388] EnclosingTestClass[286157404280696]

Seed Caveats

1. Right now JUnit5 doesn't have a deterministic order of tests. This means that if you run your tests twice the order can change. If you put @Seed on your class and that actually happens - it will generate different data in some cases. This is especially possible if your tests start with the same name and then are followed by underscores. So if you use @Seed you may need to run the test class multiple times before the order repeats. Or just put the annotation on methods instead.

2. If you use @MethodSource the @Seed is applied only after the data came from the data methods. This is due to current restrictions of JUnit5 which doesn't have any callbacks to impact @MethodSource. So this seed will not work:

@Seed(1234)
@ParameterizedTest
@MethodSource("numericsMethod")
void test(String value) {
    assertFalse(value.contains(english(1)));
}
private static Stream<? extends Arguments> numericsMethod() {
    return Stream.of(numeric(10)).map(Arguments::of);
}

Add to your project

Choose JUnit5 dependencies and versions suitable for your project, this is just an example:

<dependencies>
    <dependency>
        <groupId>io.qala.datagen</groupId>
        <artifactId>qala-datagen-junit5</artifactId>
        <version>2.3.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-params</artifactId>
        <version>5.0.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.0.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-commons</artifactId>
        <version>1.0.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-surefire-provider</artifactId>
        <version>1.0.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.platform</groupId>
                    <artifactId>junit-platform-surefire-provider</artifactId>
                    <version>1.0.1</version>
                </dependency>
                <dependency>
                    <groupId>org.junit.jupiter</groupId>
                    <artifactId>junit-jupiter-engine</artifactId>
                    <version>5.0.1</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>