Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Spark] Native test TestPrestoSparkNativeAggregations.testAggregations fails in CI #24254

Open
czentgr opened this issue Dec 12, 2024 · 1 comment
Labels
bug flaky-test spark Apache Spark related

Comments

@czentgr
Copy link
Contributor

czentgr commented Dec 12, 2024

The CircleCI tests only tests a subset of the Spark native E2E tests.

It only tests these three test classes (56 total tests):

TESTCLASSES=TestPrestoSparkExpressionCompiler,TestPrestoSparkNativeJoinQueries,TestPrestoSparkSqlFunctions

Meanwhile, when migrating to github actions a larger number of tests is run revealing test issues.

The following set of test classes is run:

TESTCLASSES = TestPrestoSparkExpressionCompiler,TestPrestoSparkNativeBitwiseFunctionQueries,TestPrestoSparkNativeAggregations,TestPrestoSparkNativeTpchConnectorQueries,TestPrestoSparkNativeSimpleQueries,TestPrestoSparkSqlFunctions,TestPrestoSparkNativeJoinQueries,TestPrestoSparkNativeWindowQueries,TestPrestoSparkNativeTpchQueries,TestPrestoSparkNativeArrayFunctionQueries,TestPrestoSparkNativeGeneralQueries

TestPrestoSparkNativeAggregations.testAggregations fails:

2024-12-12T20:02:17.6606016Z [ERROR] Tests run: 29, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 347.388 s <<< FAILURE! - in com.facebook.presto.spark.TestPrestoSparkNativeAggregations
2024-12-12T20:02:17.6607466Z [ERROR] com.facebook.presto.spark.TestPrestoSparkNativeAggregations.testAggregations[with_row_wise_exchange_encoding](2)  Time elapsed: 24.913 s  <<< FAILURE!
2024-12-12T20:02:17.6608434Z java.lang.NullPointerException
2024-12-12T20:02:17.6609186Z 	at com.facebook.presto.spark.util.PrestoSparkFailureUtils.toPrestoSparkFailure(PrestoSparkFailureUtils.java:59)
2024-12-12T20:02:17.6610417Z 	at com.facebook.presto.spark.execution.AbstractPrestoSparkQueryExecution.execute(AbstractPrestoSparkQueryExecution.java:395)
2024-12-12T20:02:17.6611611Z 	at com.facebook.presto.spark.PrestoSparkQueryRunner.executeWithStrategies(PrestoSparkQueryRunner.java:548)
2024-12-12T20:02:17.6612597Z 	at com.facebook.presto.spark.PrestoSparkQueryRunner.execute(PrestoSparkQueryRunner.java:522)
2024-12-12T20:02:17.6613732Z 	at com.facebook.presto.spark.PrestoSparkQueryRunner.execute(PrestoSparkQueryRunner.java:515)
2024-12-12T20:02:17.6614890Z 	at com.facebook.presto.nativeworker.AbstractTestNativeAggregations.assertQueryResultCount(AbstractTestNativeAggregations.java:401)
2024-12-12T20:02:17.6616263Z 	at com.facebook.presto.nativeworker.AbstractTestNativeAggregations.testAggregations(AbstractTestNativeAggregations.java:108)
2024-12-12T20:02:17.6617529Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-12-12T20:02:17.6618490Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2024-12-12T20:02:17.6619507Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-12-12T20:02:17.6620304Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2024-12-12T20:02:17.6621054Z 	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135)
2024-12-12T20:02:17.6621912Z 	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673)
2024-12-12T20:02:17.6622653Z 	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220)
2024-12-12T20:02:17.6623412Z 	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
2024-12-12T20:02:17.6624065Z 	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945)
2024-12-12T20:02:17.6624690Z 	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193)
2024-12-12T20:02:17.6625344Z 	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
2024-12-12T20:02:17.6625950Z 	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
2024-12-12T20:02:17.6626543Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2024-12-12T20:02:17.6627171Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2024-12-12T20:02:17.6627639Z 	at java.base/java.lang.Thread.run(Thread.java:829)
2024-12-12T20:02:17.6627846Z 
2024-12-12T20:02:18.5127455Z [INFO] Running com.facebook.presto.spark.TestPrestoSparkNativeArrayFunctionQueries
2024-12-12T20:02:18.8742040Z 2024-12-12T14:02:18.793-0600	INFO	TestNG-test=Surefire test-1	com.facebook.airlift.log.Logging	Logging to stderr
2024-12-12T20:02:20.4462581Z WARNING: An illegal reflective access operation has occurred
2024-12-12T20:02:20.4467872Z WARNING: Illegal reflective access by org.apache.hadoop.fs.HadoopExtendedFileSystemCache (file:/root/.m2/repository/com/facebook/presto/presto-hdfs-core/0.291-SNAPSHOT/presto-hdfs-core-0.291-SNAPSHOT.jar) to field java.lang.reflect.Field.modifiers
2024-12-12T20:02:20.4470169Z WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.fs.HadoopExtendedFileSystemCache
2024-12-12T20:02:20.4471412Z WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
2024-12-12T20:02:20.4472532Z WARNING: All illegal access operations will be denied in a future release

See:
https://github.com/czentgr/presto/actions/runs/12303617334/job/34339523632

Your Environment

N/A

Expected Behavior

Test succeeds.

Current Behavior

Test fails.

Possible Solution

Steps to Reproduce

Screenshots (if appropriate)

Context

@czentgr
Copy link
Contributor Author

czentgr commented Dec 13, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug flaky-test spark Apache Spark related
Projects
Status: 🆕 Unprioritized
Status: 🆕 Unprioritized
Development

No branches or pull requests

1 participant