ava.lang.IllegalArgumentException: Attempt to add (hdfs://ip-172-20-10-53.ec2.internal:8020/user/oozie/share/lib/lib_20170208121307/oozie/apacheds-kerberos-codec-2.0.0-M15.jar) multiple times to the distributed cache.

Stack Overflow | Rinin | 2 weeks ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    I am getting the below error while trying to execute spark submit using oozie on emr

    Stack Overflow | 2 weeks ago | Rinin
    ava.lang.IllegalArgumentException: Attempt to add (hdfs://ip-172-20-10-53.ec2.internal:8020/user/oozie/share/lib/lib_20170208121307/oozie/apacheds-kerberos-codec-2.0.0-M15.jar) multiple times to the distributed cache.
  2. 0

    Re: Spark Submit - java.lang.IllegalArgumentException: requirement failed

    spark-user | 1 year ago | Jean-Baptiste Onofré
    yarn.Client: Deleting staging directory .sparkStaging/application_1447442727308_0126 Exception in thread "main" java.lang.IllegalArgumentException: requirement failed
  3. 0

    Spark cluster computing framework

    gmane.org | 1 year ago
    yarn.Client: Deleting staging directory .sparkStaging/application_1447442727308_0126 Exception in thread "main" java.lang.IllegalArgumentException: requirement failed
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. ava.lang.IllegalArgumentException

      Attempt to add (hdfs://ip-172-20-10-53.ec2.internal:8020/user/oozie/share/lib/lib_20170208121307/oozie/apacheds-kerberos-codec-2.0.0-M15.jar) multiple times to the distributed cache.

      at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$11$$anonfun$apply$8.apply()
    2. Spark Project YARN Stable API
      Client$$anonfun$prepareLocalResources$11$$anonfun$apply$8.apply
      1. org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$11$$anonfun$apply$8.apply(Client.scala:608)
      2. org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$11$$anonfun$apply$8.apply(Client.scala:599)
      2 frames
    3. Scala
      ArraySeq.foreach
      1. scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
      1 frame
    4. Spark Project YARN Stable API
      Client$$anonfun$prepareLocalResources$11.apply
      1. org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$11.apply(Client.scala:599)
      2. org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$11.apply(Client.scala:598)
      2 frames
    5. Scala
      List.foreach
      1. scala.collection.immutable.List.foreach(List.scala:381)
      1 frame
    6. Spark Project YARN Stable API
      Client.main
      1. org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:598)
      2. org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
      3. org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
      4. org.apache.spark.deploy.yarn.Client.run(Client.scala:1154)
      5. org.apache.spark.deploy.yarn.Client$.main(Client.scala:1213)
      6. org.apache.spark.deploy.yarn.Client.main(Client.scala)
      6 frames
    7. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:498)
      4 frames
    8. Spark
      SparkSubmit.main
      1. org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
      2. org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
      3. org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
      4. org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
      5. org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      5 frames
    9. org.apache.oozie
      SparkMain.main
      1. org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:338)
      2. org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:257)
      3. org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
      4. org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:78)
      4 frames
    10. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:498)
      4 frames
    11. org.apache.oozie
      LauncherMapper.map
      1. org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
      1 frame
    12. Hadoop
      LocalContainerLauncher$EventHandler$1.run
      1. org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      2. org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
      3. org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
      4. org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
      5. org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
      6. org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
      7. org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
      7 frames
    13. Java RT
      Thread.run
      1. java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      2. java.util.concurrent.FutureTask.run(FutureTask.java:266)
      3. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      4. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      5. java.lang.Thread.run(Thread.java:745)
      5 frames