org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.NullPointerException

GitHub | razaba | 2 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    GitHub comment 928#280332030

    GitHub | 2 months ago | razaba
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.NullPointerException
  2. 0

    SparkSql scala.MatchError when inserting single-value object in nested type

    GitHub | 9 months ago | juyttenh
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 7.0 failed 1 times, most recent failure: Lost task 1.0 in stage 7.0 (TID 17, localhost): scala.MatchError: [joe,15] (of class org.elasticsearch.spark.sql.ScalaEsRow)
  3. 0

    Scala - Spark - How to transform a dataframe containing one string column to a DF with columns with the rigth type?

    Stack Overflow | 6 months ago | wymeka
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 38.0 failed 1 times, most recent failure: Lost task 0.0 in stage 38.0 (TID 32, localhost): scala.MatchError: ["20295930","20295930"] (of class java.lang.String)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    geoip fields trigger error

    GitHub | 1 month ago | cvjones17
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 16.0 failed 4 times, most recent failure: Lost task 0.3 in stage 16.0 (TID 38, 172.31.30.96, executor 0): java.lang.IndexOutOfBoundsException: 1
  6. 0

    GitHub comment 951#288760587

    GitHub | 1 month ago | cvjones17
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 4 in stage 20.0 failed 4 times, most recent failure: Lost task 4.3 in stage 20.0 (TID 201, 172.31.30.96, executor 0): java.lang.IndexOutOfBoundsException: 1

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.apache.spark.SparkException

      Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.NullPointerException

      at org.elasticsearch.spark.sql.ScalaEsRow.values$lzycompute()
    2. Elasticsearch Spark
      ScalaEsRow.length
      1. org.elasticsearch.spark.sql.ScalaEsRow.values$lzycompute(ScalaEsRow.scala:27)
      2. org.elasticsearch.spark.sql.ScalaEsRow.values(ScalaEsRow.scala:27)
      3. org.elasticsearch.spark.sql.ScalaEsRow.length(ScalaEsRow.scala:34)
      3 frames
    3. Spark Project SQL
      Row$class.size
      1. org.apache.spark.sql.Row$class.size(Row.scala:124)
      1 frame
    4. Elasticsearch Spark
      ScalaEsRow.size
      1. org.elasticsearch.spark.sql.ScalaEsRow.size(ScalaEsRow.scala:25)
      1 frame
    5. Spark Project Catalyst
      CatalystTypeConverters$ArrayConverter$$anonfun$toCatalystImpl$2.apply
      1. org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:258)
      2. org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:251)
      3. org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:103)
      4. org.apache.spark.sql.catalyst.CatalystTypeConverters$ArrayConverter$$anonfun$toCatalystImpl$2.apply(CatalystTypeConverters.scala:164)
      4 frames
    6. Scala
      AbstractTraversable.map
      1. scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      2. scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      3. scala.collection.Iterator$class.foreach(Iterator.scala:893)
      4. scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
      5. scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
      6. scala.collection.AbstractIterable.foreach(Iterable.scala:54)
      7. scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
      8. scala.collection.AbstractTraversable.map(Traversable.scala:104)
      8 frames
    7. Spark Project Catalyst
      CatalystTypeConverters$$anonfun$createToCatalystConverter$2.apply
      1. org.apache.spark.sql.catalyst.CatalystTypeConverters$ArrayConverter.toCatalystImpl(CatalystTypeConverters.scala:164)
      2. org.apache.spark.sql.catalyst.CatalystTypeConverters$ArrayConverter.toCatalystImpl(CatalystTypeConverters.scala:154)
      3. org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:103)
      4. org.apache.spark.sql.catalyst.CatalystTypeConverters$$anonfun$createToCatalystConverter$2.apply(CatalystTypeConverters.scala:403)
      4 frames
    8. Spark Project SQL
      RDDConversions$$anonfun$rowToRowRdd$1$$anonfun$apply$2.apply
      1. org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1$$anonfun$apply$2.apply(ExistingRDD.scala:67)
      2. org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1$$anonfun$apply$2.apply(ExistingRDD.scala:64)
      2 frames
    9. Scala
      Iterator$$anon$11.next
      1. scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
      1 frame
    10. Spark Project Catalyst
      GeneratedClass$GeneratedIterator.processNext
      1. org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)
      1 frame
    11. Spark Project SQL
      SparkPlan$$anonfun$4.apply
      1. org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
      2. org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:370)
      3. org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:246)
      4. org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:240)
      4 frames
    12. Spark
      Executor$TaskRunner.run
      1. org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
      2. org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
      3. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      4. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
      5. org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
      6. org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
      7. org.apache.spark.scheduler.Task.run(Task.scala:86)
      8. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
      8 frames
    13. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      3. java.lang.Thread.run(Thread.java:745)
      3 frames