java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.DEFAULT_DRIVER_MEM_MB()I

hortonworks.com | 9 months ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    Getting Stack trace: ExitCodeException exitCode=255 during MapReduce

    Stack Overflow | 9 months ago | Abhay Singh
    java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JVMId.(Lorg/apache/hadoop/mapred/JobID;ZJ)V

    Root Cause Analysis

    1. java.lang.NoSuchMethodError

      org.apache.spark.util.Utils$.DEFAULT_DRIVER_MEM_MB()I

      at org.apache.spark.deploy.yarn.ClientArguments.<init>()
    2. Spark Project YARN Stable API
      Client.main
      1. org.apache.spark.deploy.yarn.ClientArguments.<init>(ClientArguments.scala:49)
      2. org.apache.spark.deploy.yarn.Client$.main(Client.scala:1120)
      3. org.apache.spark.deploy.yarn.Client.main(Client.scala)
      3 frames
    3. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:497)
      4 frames
    4. Spark
      SparkSubmit.main
      1. org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
      2. org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
      3. org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      3 frames
    5. org.apache.oozie
      SparkMain.main
      1. org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:104)
      2. org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:95)
      3. org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
      4. org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)
      4 frames
    6. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:497)
      4 frames
    7. org.apache.oozie
      LauncherMapper.map
      1. org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
      1 frame
    8. Hadoop
      YarnChild$2.run
      1. org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      2. org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
      3. org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      4. org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
      4 frames
    9. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:422)
      2 frames
    10. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
      1 frame
    11. Hadoop
      YarnChild.main
      1. org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
      1 frame