java.lang.IllegalStateException: Spark context stopped while waiting for backend

Stack Overflow | aks | 5 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    How to know what is the reason for ClosedChannelExceptions with spark-shell in YARN client mode?

    Stack Overflow | 5 months ago | aks
    java.lang.IllegalStateException: Spark context stopped while waiting for backend
  2. 0

    Spark Shell - __spark_libs__.zip does not exist

    Stack Overflow | 3 months ago | Ryan Nel
    java.lang.IllegalStateException: Spark context stopped while waiting for backend

    Root Cause Analysis

    1. java.lang.IllegalStateException

      Spark context stopped while waiting for backend

      at org.apache.spark.scheduler.TaskSchedulerImpl.waitBackendReady()
    2. Spark
      SparkContext$.getOrCreate
      1. org.apache.spark.scheduler.TaskSchedulerImpl.waitBackendReady(TaskSchedulerImpl.scala:581)
      2. org.apache.spark.scheduler.TaskSchedulerImpl.postStartHook(TaskSchedulerImpl.scala:162)
      3. org.apache.spark.SparkContext.<init>(SparkContext.scala:549)
      4. org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2256)
      4 frames
    3. Spark Project SQL
      SparkSession$Builder$$anonfun$8.apply
      1. org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:831)
      2. org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:823)
      2 frames
    4. Scala
      Option.getOrElse
      1. scala.Option.getOrElse(Option.scala:121)
      1 frame
    5. Spark Project SQL
      SparkSession$Builder.getOrCreate
      1. org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:823)
      1 frame
    6. Spark REPL
      Main$.createSparkSession
      1. org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
      1 frame