scala.MatchError: ArrayType(StructType(List(StructField(date,StringType, true ), StructField(n_reachat,IntegerType, true ))),true) (of class org.apache.spark.sql.catalyst.types.ArrayType)

nabble.com | 9 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    Apache Spark User List - scala.MatchError on SparkSQL when creating ArrayType of StructType

    nabble.com | 9 months ago
    scala.MatchError: ArrayType(StructType(List(StructField(date,StringType, true ), StructField(n_reachat,IntegerType, true ))),true) (of class org.apache.spark.sql.catalyst.types.ArrayType)

    Root Cause Analysis

    1. scala.MatchError

      ArrayType(StructType(List(StructField(date,StringType, true ), StructField(n_reachat,IntegerType, true ))),true) (of class org.apache.spark.sql.catalyst.types.ArrayType)

      at org.apache.spark.sql.catalyst.expressions.Cast.cast$lzycompute()
    2. Spark Project Catalyst
      InterpretedMutableProjection.apply
      1. org.apache.spark.sql.catalyst.expressions.Cast.cast$lzycompute(Cast.scala:247)
      2. org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:247)
      3. org.apache.spark.sql.catalyst.expressions.Cast.eval(Cast.scala:263)
      4. org.apache.spark.sql.catalyst.expressions.Alias.eval(namedExpressions.scala:84)
      5. org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(Projection.scala:66)
      6. org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(Projection.scala:50)
      6 frames
    3. Scala
      Iterator$$anon$11.next
      1. scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
      2. scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
      2 frames
    4. Spark Project Hive
      InsertIntoHiveTable$$anonfun$saveAsHiveFile$1.apply
      1. org.apache.spark.sql.hive.execution.InsertIntoHiveTable.org$apache$spark$sql$hive$execution$InsertIntoHiveTable$$writeToFile$1(InsertIntoHiveTable.scala:149)
      2. org.apache.spark.sql.hive.execution.InsertIntoHiveTable$$anonfun$saveAsHiveFile$1.apply(InsertIntoHiveTable.scala:158)
      3. org.apache.spark.sql.hive.execution.InsertIntoHiveTable$$anonfun$saveAsHiveFile$1.apply(InsertIntoHiveTable.scala:158)
      3 frames
    5. Spark
      Executor$TaskRunner.run
      1. org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
      2. org.apache.spark.scheduler.Task.run(Task.scala:54)
      3. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
      3 frames
    6. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      3. java.lang.Thread.run(Thread.java:744)
      3 frames