java.lang.IllegalArgumentException: requirement failed: pyspark.zip not found; cannot run pyspark application in YARN mode.

Stack Overflow | Pari Margu | 3 months ago
  1. 0

    PySpark job fails in AWS EMR cluster

    Stack Overflow | 3 months ago | Pari Margu
    java.lang.IllegalArgumentException: requirement failed: pyspark.zip not found; cannot run pyspark application in YARN mode.
  2. 0

    Save object in cassandra using spark and java

    Stack Overflow | 1 year ago | chris
    java.lang.IllegalArgumentException: requirement failed: Columns not found in class com.app.models.User: [username]
  3. 0

    Apache Spark User List - java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute

    nabble.com | 5 months ago
    java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Identifiers that differ only in case are handled incorrectly in non-case sensitive filesystems

    GitHub | 2 years ago | denisrosset
    java.lang.IllegalArgumentException: requirement failed: Source file '/Users/rossetd0/Documents/phd/spire/core/target/scala-2.11/classes/spire/optional/optionAction.class' does not exist.
  6. 0

    ClassfileManager crashes when name hashing is enabled

    GitHub | 3 years ago | gkossakowski
    java.lang.IllegalArgumentException: requirement failed: Source file '/home/alois/timeout/jvm/target/classes.bak/sbt3697008749647665633.class' does not exist.
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.IllegalArgumentException

    requirement failed: pyspark.zip not found; cannot run pyspark application in YARN mode.

    at scala.Predef$.require()
  2. Scala
    Predef$.require
    1. scala.Predef$.require(Predef.scala:233)
    1 frame
  3. Spark Project YARN Stable API
    Client$$anonfun$findPySparkArchives$2.apply
    1. org.apache.spark.deploy.yarn.Client$$anonfun$findPySparkArchives$2.apply(Client.scala:1052)
    2. org.apache.spark.deploy.yarn.Client$$anonfun$findPySparkArchives$2.apply(Client.scala:1049)
    2 frames
  4. Scala
    Option.getOrElse
    1. scala.Option.getOrElse(Option.scala:120)
    1 frame
  5. Spark Project YARN Stable API
    Client.submitApplication
    1. org.apache.spark.deploy.yarn.Client.findPySparkArchives(Client.scala:1049)
    2. org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:722)
    3. org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:142)
    3 frames
  6. Spark
    JavaSparkContext.<init>
    1. org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
    2. org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
    3. org.apache.spark.SparkContext.<init>(SparkContext.scala:530)
    4. org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
    4 frames
  7. Java RT
    Constructor.newInstance
    1. sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    2. sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    3. sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    4. java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    4 frames
  8. Py4J
    GatewayConnection.run
    1. py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234)
    2. py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
    3. py4j.Gateway.invoke(Gateway.java:214)
    4. py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79)
    5. py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68)
    6. py4j.GatewayConnection.run(GatewayConnection.java:209)
    6 frames
  9. Java RT
    Thread.run
    1. java.lang.Thread.run(Thread.java:745)
    1 frame