org.apache.spark.SparkException: Task not serializable

Stack Overflow | Nilesh | 4 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    org.apache.spark.SparkException: Task not serializable?

    Stack Overflow | 4 months ago | Nilesh
    org.apache.spark.SparkException: Task not serializable

    Root Cause Analysis

    1. org.apache.spark.SparkException

      Task not serializable

      at org.apache.spark.util.ClosureCleaner$.ensureSerializable()
    2. Spark
      RDD$$anonfun$foreach$1.apply
      1. org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:298)
      2. org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:288)
      3. org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:108)
      4. org.apache.spark.SparkContext.clean(SparkContext.scala:2037)
      5. org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:874)
      6. org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:873)
      6 frames