scheduler.TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1, localhost): java.lang.NoClassDefFoundError: com/esotericsoftware/kryo/KryoSerializable

Stack Overflow | apolak | 6 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    Error with simple Spark Rdd

    Stack Overflow | 6 months ago | apolak
    scheduler.TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1, localhost): java.lang.NoClassDefFoundError: com/esotericsoftware/kryo/KryoSerializable

    Root Cause Analysis

    1. java.lang.ClassNotFoundException

      com.esotericsoftware.kryo.KryoSerializable

      at java.net.URLClassLoader.findClass()
    2. Java RT
      ClassLoader.loadClass
      1. java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      2. java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      3. sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
      4. java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      5. java.lang.ClassLoader.defineClass1(Native Method)
      6. java.lang.ClassLoader.defineClass(ClassLoader.java:760)
      7. java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
      8. java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
      9. java.net.URLClassLoader.access$100(URLClassLoader.java:73)
      10. java.net.URLClassLoader$1.run(URLClassLoader.java:368)
      11. java.net.URLClassLoader$1.run(URLClassLoader.java:362)
      12. java.security.AccessController.doPrivileged(Native Method)
      13. java.net.URLClassLoader.findClass(URLClassLoader.java:361)
      14. java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      15. sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
      16. java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      16 frames
    3. Spark
      Executor$TaskRunner.run
      1. org.apache.spark.rdd.SqlNewHadoopRDDState$.setInputFileName(SqlNewHadoopRDDState.scala:37)
      2. org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:218)
      3. org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208)
      4. org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
      5. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
      6. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
      7. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      8. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
      9. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
      10. org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
      11. org.apache.spark.scheduler.Task.run(Task.scala:89)
      12. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
      12 frames
    4. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      3. java.lang.Thread.run(Thread.java:745)
      3 frames