java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary

GitHub | ash211 | 3 months ago
tip
Do you find the tips below useful? Click on the to mark them and say thanks to poroszd . Or join the community to write better ones.
  1. 0

    Support reading DECIMAL(18,2) columns from Parquet

    GitHub | 3 months ago | ash211
    java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary
  2. 0
    samebug tip
    You should use java.sql.Timestamp or Date to map BsonDateTime from mongodb.
  3. 0

    Spark: error reading DateType columns in partitioned parquet data

    Stack Overflow | 3 months ago | capitalistpug
    java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Working on 1.6.2, broken on 2.0 {code} select * from logs.a where year=2016 and month=9 and day=14 limit 100 {code} {code} java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary at org.apache.parquet.column.Dictionary.decodeToInt(Dictionary.java:48) at org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.getInt(OnHeapColumnVector.java:233) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:370) at org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:246) at org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:240) at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803) at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319) at org.apache.spark.rdd.RDD.iterator(RDD.scala:283) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70) at org.apache.spark.scheduler.Task.run(Task.scala:86) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) {code}

    Apache's JIRA Issue Tracker | 6 months ago | Egor Pahomov
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4, ip-10-1-101-18.ec2.internal): java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.UnsupportedOperationException

      org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary

      at org.apache.parquet.column.Dictionary.decodeToLong()
    2. org.apache.parquet
      Dictionary.decodeToLong
      1. org.apache.parquet.column.Dictionary.decodeToLong(Dictionary.java:52)[parquet-column-1.7.0.jar:1.7.0]
      1 frame
    3. org.apache.spark
      ColumnVector.getDecimal
      1. org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.getLong(OnHeapColumnVector.java:274)[spark-sql_2.11-2.0.1.jar:2.0.1]
      2. org.apache.spark.sql.execution.vectorized.ColumnVector.getDecimal(ColumnVector.java:588)[spark-sql_2.11-2.0.1.jar:2.0.1]
      2 frames
    4. Spark Project Catalyst
      GeneratedClass$GeneratedIterator.processNext
      1. org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)[na:na]
      1 frame
    5. Spark Project SQL
      SparkPlan$$anonfun$4.apply
      1. org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)[spark-sql_2.11-2.0.1.jar:2.0.1]
      2. org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:370)[spark-sql_2.11-2.0.1.jar:2.0.1]
      3. org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:246)[spark-sql_2.11-2.0.1.jar:2.0.1]
      4. org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:240)[spark-sql_2.11-2.0.1.jar:2.0.1]
      4 frames
    6. Spark
      Executor$TaskRunner.run
      1. org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)[spark-core_2.11-2.0.1.jar:2.0.1]
      2. org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)[spark-core_2.11-2.0.1.jar:2.0.1]
      3. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)[spark-core_2.11-2.0.1.jar:2.0.1]
      4. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)[spark-core_2.11-2.0.1.jar:2.0.1]
      5. org.apache.spark.rdd.RDD.iterator(RDD.scala:283)[spark-core_2.11-2.0.1.jar:2.0.1]
      6. org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)[spark-core_2.11-2.0.1.jar:2.0.1]
      7. org.apache.spark.scheduler.Task.run(Task.scala:86)[spark-core_2.11-2.0.1.jar:2.0.1]
      8. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)[spark-core_2.11-2.0.1.jar:2.0.1]
      8 frames