org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 6, legio.intec.ugent.be): java.lang.ClassNotFoundException: org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$33

Stack Overflow | Dries Decap | 9 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    ClassNotFoundException using Spark-Notebook on Yarn when reading a file

    Stack Overflow | 9 months ago | Dries Decap
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 6, legio.intec.ugent.be): java.lang.ClassNotFoundException: org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$33
  2. 0

    GitHub comment 12#115366372

    GitHub | 2 years ago | bmabey
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 51 in stage 3.0 failed 4 times, most recent failure: Lost task 51.3 in stage 3.0 (TID 1131, minvertica1): java.lang.ClassNotFoundException: cmd11$$user$$anonfun$1$$anonfun$apply$1

    Root Cause Analysis

    1. org.apache.spark.SparkException

      Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 6, legio.intec.ugent.be): java.lang.ClassNotFoundException: org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$33

      at org.apache.spark.repl.ExecutorClassLoader.findClass()
    2. Spark REPL
      ExecutorClassLoader.findClass
      1. org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:88)
      1 frame
    3. Java RT
      Class.forName
      1. java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      2. java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      3. java.lang.Class.forName0(Native Method)
      4. java.lang.Class.forName(Class.java:348)
      4 frames
    4. Spark
      JavaDeserializationStream$$anon$1.resolveClass
      1. org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:68)
      1 frame