org.apache.spark.SparkException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • newAPIHadoopRDD Task not serializable
    via Stack Overflow by D.Asare
    ,
  • How to read textual lzo seq with scala spark?
    via Stack Overflow by Jas
    ,
    • org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:304) at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:294) at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:122) at org.apache.spark.SparkContext.clean(SparkContext.scala:2021) at org.apache.spark.rdd.RDD$$anonfun$map$1.apply(RDD.scala:314) at org.apache.spark.rdd.RDD$$anonfun$map$1.apply(RDD.scala:313) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) at org.apache.spark.rdd.RDD.withScope(RDD.scala:306) at org.apache.spark.rdd.RDD.map(RDD.scala:313) at org.apache.spark.api.java.JavaRDDLike$class.map(JavaRDDLike.scala:93) at org.apache.spark.api.java.AbstractJavaRDDLike.map(JavaRDDLike.scala:47) at com.hbfinance.DataframeExample.run(DataframeExample.java:54) at com.hbfinance.DataframeExample.main(DataframeExample.java:88) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.io.NotSerializableException: com.hbfinance.DataframeExample Serialization stack: - object not serializable (class: com.hbfinance.DataframeExample, value: com.hbfinance.DataframeExample@1f3165e7) - field (class: com.hbfinance.DataframeExample$1, name: this$0, type: class com.hbfinance.DataframeExample) - object (class com.hbfinance.DataframeExample$1, com.hbfinance.DataframeExample$1@1866da85) - field (class: org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1, name: fun$1, type: interface org.apache.spark.api.java.function.Function) - object (class org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1, <function1>) at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40) at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47) at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84) at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:301) ... 22 more

    Users with the same issue

    Unknown visitor
    Unknown visitor1 times, last one,
    rprp
    1 times, last one,
    bandocabandoca
    1 times, last one,
    johnxflyjohnxfly
    14 times, last one,
    Nikolay RybakNikolay Rybak
    15 times, last one,
    7 more bugmates