Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Projection/TPC issue: MatchError substring(input_1#6269, 1, 2) (Query 22) #350

Open
wgip opened this issue Nov 23, 2021 · 16 comments
Open

Projection/TPC issue: MatchError substring(input_1#6269, 1, 2) (Query 22) #350

wgip opened this issue Nov 23, 2021 · 16 comments
Assignees
@carlos-reyes-123

Comments

@wgip
Copy link
Contributor

wgip commented Nov 23, 2021 

Query 22 *** FAILED ***
(Hide Details)
Message:	substring(input_1#6269, 1, 2) AS cntrycode#6518 (of class org.apache.spark.sql.catalyst.expressions.Alias)
Exception:	scala.MatchError
com.nec.spark.agile.SparkExpressionToCExpression$.evalString(SparkExpressionToCExpression.scala:161)
com.nec.spark.planning.VERewriteStrategy.$anonfun$apply$2(VERewriteStrategy.scala:134)
@wgip wgip mentioned this issue Nov 23, 2021
Merged
wgip added a commit that referenced this issue Nov 23, 2021
@wgip
Project on VE (#347)
3de47d9 
It works mostly correctly. There are some non-implemented things.

https://www.sparkcyclone.io/tpc-html/20211123160932/


Out of the issues, 2 of these are issues with the test/rounding (#352); the other 5 are to do with string projection (#350, #351) missing.

![image](https://user-images.githubusercontent.com/52247468/143087317-6eff8216-4c60-4ea6-a91c-5bb17d8a6991.png)
@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 18, 2022 •
edited
Loading

https://www.sparkcyclone.io/tpc-html/20211123160932/

com.nec.spark.agile.SparkExpressionToCExpression$.evalString(SparkExpressionToCExpression.scala:161)
com.nec.spark.planning.VERewriteStrategy.$anonfun$apply$2(VERewriteStrategy.scala:134)
scala.collection.immutable.List.map(List.scala:286)
com.nec.spark.planning.VERewriteStrategy.res$1(VERewriteStrategy.scala:131)
com.nec.spark.planning.VERewriteStrategy.apply(VERewriteStrategy.scala:386)
org.apache.spark.sql.catalyst.planning.QueryPlanner.$anonfun$plan$1(QueryPlanner.scala:63)
scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:484)
scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:490)
scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:489)
org.apache.spark.sql.catalyst.planning.QueryPlanner.plan(QueryPlanner.scala:93)
org.apache.spark.sql.execution.SparkStrategies.plan(SparkStrategies.scala:67)
org.apache.spark.sql.catalyst.planning.QueryPlanner.$anonfun$plan$3(QueryPlanner.scala:78)

@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 18, 2022 •
edited
Loading 

// SparkCyclone/src/main/scala/com/nec/spark/agile/SparkExpressionToCExpression.scala(143)
case _ => sys.error(s"Could not handle the expression ${expression}, type ${expression.getClass}")

@carlos-reyes-123 carlos-reyes-123 self-assigned this Jan 19, 2022
@carlos-reyes-123
Copy link
Contributor

tests/tpchbench/src/main/scala/sparkcyclone/tpch/TPCHBenchmark.scala(1161)

@carlos-reyes-123
Copy link
Contributor

SparkCyclone - new GitHub branch
carlos-gh350-add-match-else

@carlos-reyes-123
Copy link
Contributor 

      case _ => sys.error(s"SparkExpressionToCExpression.evalString: " +
        "Could not match ${expression}, " +
        "class ${expression.getClass}")

@carlos-reyes-123 carlos-reyes-123 mentioned this issue Jan 20, 2022
Closed
@carlos-reyes-123
Copy link
Contributor

#441

@carlos-reyes-123
Copy link
Contributor

Failed toString() invocation on an object of type [org.apache.spark.sql.execution.QueryExecution]

@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 20, 2022 •
edited
Loading 

java.lang.RuntimeException: SparkExpressionToCExpression.evalString:
    Could not match Success(substring(input_1#57879, 1, 2) AS cntrycode#59510), 
        class class org.apache.spark.sql.catalyst.expressions.Alias

@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 20, 2022 •
edited
Loading 

import scala.util.Try

  case _ => sys.error(s"SparkExpressionToCExpression.evalString: " +
    s"Could not match ${Try(expression.toString)}, " +
    s"type ${expression.getClass}")

@carlos-reyes-123
Copy link
Contributor 

select
    substring(c_phone from 1 for 2) as cntrycode,
    c_acctbal
  from
    customer

@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 20, 2022 •
edited
Loading 

class Alias
   child: Expression
   name: String
override def toString: String = s"$child AS $name#${exprId.id}$typeSuffix$delaySuffix"

@carlos-reyes-123
Copy link
Contributor

new branch
carlos-gh350-query22

@carlos-reyes-123 carlos-reyes-123 mentioned this issue Jan 20, 2022
Closed
@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 20, 2022 •
edited
Loading 

SLF4J: Failed toString() invocation on an object of type [org.apache.spark.sql.execution.QueryExecution]
   │ Reported exception:
   │ java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.AttributeReference
   │ cannot be cast to org.apache.spark.unsafe.types.UTF8String
   │    at org.apache.spark.sql.catalyst.expressions.Substring.nullSafeEval(stringExpressions.scala:1728)
   │    at com.nec.spark.agile.SparkExpressionToCExpression$.evalString(SparkExpressionToCExpression.scala:173)

@carlos-reyes-123
Copy link
Contributor

carlos-gh350-query22-2

@carlos-reyes-123
Copy link
Contributor

carlos-gh350-query22-3

@carlos-reyes-123
Copy link
Contributor

carlos-reyes-123 commented Jan 21, 2022 •
edited
Loading

https://github.com/XpressAI/SparkCyclone/actions/runs/1726602754

 SLF4J: Failed toString() invocation on an object of type [org.apache.spark.sql.execution.QueryExecution]
 Reported exception:
 java.lang.RuntimeException: Could not map substring(input_4#57879, 1, 2) IN (13,31,23,29,30,18,17) (class org.apache.spark.sql.catalyst.
       │ expressions.In None); input is List(c_custkey#57875L, c_name#57876, c_address#57877, c_nationkey#57878L, c_phone#57879, c_acctbal#57880, c_mktsegment#57881, c_commen
       │ t#57882), condition is (substring(c_phone#57879, 1, 2) IN (13,31,23,29,30,18,17) AND (c_acctbal#57880 > scalar-subquery#59508 []))
at scala.sys.package$.error(package.scala:30)
at com.nec.spark.planning.VERewriteStrategy.$anonfun$apply$19(VERewriteStrategy.scala:305)
at scala.util.Either.fold(Either.scala:192)
at com.nec.spark.planning.VERewriteStrategy.res$1(VERewriteStrategy.scala:307)
at com.nec.spark.planning.VERewriteStrategy.apply(VERewriteStrategy.scala:692)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@carlos-reyes-123 carlos-reyes-123

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@carlos-reyes-123 @wgip