org.apache.spark.SparkException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • how to solve this sparkstreaming error
    via Stack Overflow by Samy Nathan
    ,
  • GitHub comment 1051#142827688
    via GitHub by sgomezg
    ,
  • GitHub comment 1077#142882450
    via GitHub by danielcsant
    ,
  • GitHub comment 1051#142827688
    via GitHub by sgomezg
    ,
  • GitHub comment 1077#142882450
    via GitHub by danielcsant
    ,
    • org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:304) at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:294) at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:122) at org.apache.spark.SparkContext.clean(SparkContext.scala:2030) at org.apache.spark.streaming.dstream.DStream$$anonfun$map$1.apply(DStream.scala:528) at org.apache.spark.streaming.dstream.DStream$$anonfun$map$1.apply(DStream.scala:528) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) at org.apache.spark.SparkContext.withScope(SparkContext.scala:709) at org.apache.spark.streaming.StreamingContext.withScope(StreamingContext.scala:266) at org.apache.spark.streaming.dstream.DStream.map(DStream.scala:527) at org.apache.spark.streaming.api.java.JavaDStreamLike$class.map(JavaDStreamLike.scala:157) at org.apache.spark.streaming.api.java.AbstractJavaDStreamLike.map(JavaDStreamLike.scala:43) at com.bt.product.sparkstreaming.data.Sparkstreaming.run(Sparkstreaming.java:251) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.NotSerializableException: edu.stanford.nlp.ie.crf.CRFClassifier Serialization stack: - object not serializable (class: edu.stanford.nlp.ie.crf.CRFClassifier, value: edu.stanford.nlp.ie.crf.CRFClassifier@52cfd138) - field (class: com.bt.product.sparkstreaming.data.Sparkstreaming, name: classifier, type: class edu.stanford.nlp.ie.AbstractSequenceClassifier) - object (class com.bt.product.sparkstreaming.data.Sparkstreaming, com.bt.product.sparkstreaming.data.Sparkstreaming@54859767) - field (class: com.bt.product.sparkstreaming.data.Sparkstreaming$1, name: this$0, type: class com.bt.product.sparkstreaming.data.Sparkstreaming) - object (class com.bt.product.sparkstreaming.data.Sparkstreaming$1, com.bt.product.sparkstreaming.data.Sparkstreaming$1@48c9c466) - field (class: org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1, name: fun$1, type: interface org.apache.spark.api.java.function.Function) - object (class org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1, <function1>) at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40) at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47) at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84) at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:301) ... 14 more

    Users with the same issue

    rp
    1 times, last one,
    bandoca
    1 times, last one,
    johnxfly
    14 times, last one,
    Nikolay Rybak
    15 times, last one,
    tyson925
    22 times, last one,
    7 more bugmates