Am i doing something wrong?

[dmirica-gpsw]

I'm trying to setup a simple/proof of concept project to see the new features regarding the maximization of parallelization.
Im encountering the following error after what seems to be the "first" batch of data-provider cases.

[ERROR] tests.DemoTest.testStrings -- Time elapsed: 0.070 s <<< FAILURE!
java.lang.ClassCastException: class java.util.concurrent.CompletableFuture$AsyncSupply cannot be cast to class java.lang.Comparable (java.util.concurrent.CompletableFuture$AsyncSupply and java.lang.Comparable are in module java.base of loader 'bootstrap')
        at java.base/java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:349)
        at java.base/java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:475)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at java.base/java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1782)
        at java.base/java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:2005)
        at org.testng.internal.invokers.MethodRunner.runInParallel(MethodRunner.java:143)
        at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:950)
        at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:201)
        at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
        at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.testng.internal.thread.graph.TestNGFutureTask.run(TestNGFutureTask.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

More details:
The test class:

package tests;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.List;

public class DemoTest {

    @DataProvider(parallel = true)
    public static Object[] parallelDpStrings() {
        List<String> l = new ArrayList<>();
        for (int i = 0; i < 100; i++) {
            l.add(STR."string \{i}");
        }
        return l.toArray();
    }

    @Test(dataProvider = "parallelDpStrings")
    public void testStrings(String s) throws InterruptedException {
        print(s);
        Thread.sleep(3000);
    }

    public static void print(String s) {
        System.out.println(Thread.currentThread().threadId() + "  " + Thread.currentThread().getStackTrace()[2].getMethodName() + " " + s);
    }
}

testng.xml file:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.1.dtd">
<suite name="testng.xml"
       parallel="instances"
       use-global-thread-pool="true"
       share-thread-pool-for-data-providers="true"
>

    <test name="tng">
        <packages>
            <package name="tests.*"/>
        </packages>
    </test>
</suite>

maven command:

mvn clean test -DthreadCount=10

more console output:

[INFO] --- surefire:3.2.1:test (default-test) @ tng-gtc ---
[INFO] Using auto detected provider org.apache.maven.surefire.testng.TestNGProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
42  testStrings string 8
40  testStrings string 6
35  testStrings string 1
41  testStrings string 7
38  testStrings string 4
36  testStrings string 2
39  testStrings string 5
34  testStrings string 0
37  testStrings string 3
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.457 s <<< FAILURE! -- in TestSuite
[ERROR] tests.DemoTest.testStrings -- Time elapsed: 0.070 s <<< FAILURE!
java.lang.ClassCastException: class java.util.concurrent.CompletableFuture$AsyncSupply cannot be cast to class java.lang.Comparable (java.util.concurrent.CompletableFuture$AsyncSupply and java.lang.Comparable are in module java.base of loader 'bootstrap')
        at java.base/java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:349)
        at java.base/java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:475)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at java.base/java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1782)
        at java.base/java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:2005)
        at org.testng.internal.invokers.MethodRunner.runInParallel(MethodRunner.java:143)
        at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:950)
        at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:201)
        at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
        at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.testng.internal.thread.graph.TestNGFutureTask.run(TestNGFutureTask.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   DemoTest.testStrings » ClassCast class java.util.concurrent.CompletableFuture$AsyncSupply cannot be cast to class java.lang.Comparable (java.util.concurrent.CompletableFuture$AsyncSupply and java.lang.Comparable are in module java.base of loader 'bootstrap')
[INFO] 
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.317 s
[INFO] Finished at: 2024-02-29T15:59:43+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.2.1:test (default-test) on project tng-gtc: There are test failures.

Did i miss something?

[krmahadevan]

@dmirica-gpsw - Please share a sample project which also includes how you are configuring your surefire plugin. The stacktrace definitely suggests that its a bug, but am not able to reproduce that with a sample that I created at my end. Please refer to the sample I used
discussion_3080.zip

[dmirica-gpsw]

Hi, thanks for the reply!
I attached a zip with the project, meanwhile i'm gonna have a look on the one you attached
tng-gtc.zip

[dmirica-gpsw]

After doing a fast comparison with your sample and some dependencies commenting it seems is related to

        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-testng</artifactId>
            <version>2.25.0</version>
        </dependency>

from my pom.
Right now is late on my time zone, i will continue to investigate tomorrow.
(hopefully i will not have to choose between testng and allure :)) )

[krmahadevan]

This is a bug in TestNG with respect to the newly added shared thread pool feature. Please see #3081 for mor details