org.apache.spark.SparkException: Task not serializable

Searched on Google with the first line of a JAVA stack trace?

We can recommend more relevant solutions and speed up debugging when you paste your entire stack trace with the exception message. Try a sample exception.

Recommended solutions based on your search

Solutions on the web

via GitHub by AndreasKostler
, 1 year ago
Task not serializable
via gitbooks.io by Unknown author, 1 year ago
via Stack Overflow by Arun
, 2 weeks ago
via Stack Overflow by Anji
, 1 year ago
java.io.NotSerializableException: shapeless.package$$anon$2 Serialization stack: - object not serializable (class: shapeless.package$$anon$2, value: shapeless.package$$anon$2@166b21ea) - field (class: commbank.grimlock.spark.ReducibleMatrix$$anonfun$melt$1, name: ev3$19, type: interface shapeless.package$$eq$colon$bang$eq) - object (class commbank.grimlock.spark.ReducibleMatrix$$anonfun$melt$1, <function1>)
at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40)
at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47)
at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:84)
at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:301)
at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:294)
at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:122)
at org.apache.spark.SparkContext.clean(SparkContext.scala:2021)
at org.apache.spark.rdd.RDD$$anonfun$map$1.apply(RDD.scala:314)
at org.apache.spark.rdd.RDD$$anonfun$map$1.apply(RDD.scala:313)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
at org.apache.spark.rdd.RDD.map(RDD.scala:313)
at commbank.grimlock.spark.ReducibleMatrix$class.melt(Matrix.scala:749)
at commbank.grimlock.spark.Matrix5D.melt(Matrix.scala:1438)
at commbank.grimlock.spark.examples.Conditional$.main(Conditional.scala:65)
at commbank.grimlock.spark.examples.Conditional.main(Conditional.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)

Users with the same issue

Once, 4 months ago
Once, 7 months ago
14 times, 9 months ago
15 times, 11 months ago
30 times, 1 year ago

Write tip

Know the solutions? Share your knowledge to help other developers to debug faster.