org.apache.spark.SparkException: Job cancelled because SparkContext was shut down

Apache's JIRA Issue Tracker | Tathagata Das | 2 years ago
  1. 0

    Test fails with SparkContext has been shutdown

    GitHub | 2 years ago | jamborta
    org.apache.spark.SparkException: Job cancelled because SparkContext was shut down
  2. 0

    Spark : OutOfMemoryError: Java heap space

    Stack Overflow | 1 year ago | Amine CHERIFI
    org.apache.spark.SparkException: Job cancelled because SparkContext was shut down
  3. 0

    [SPARK-2892] Socket Receiver does not stop when streaming context is stopped - ASF JIRA

    apache.org | 1 year ago
    org.apache.spark.SparkException: Job cancelled because SparkContext was shut down
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Running NetworkWordCount with {quote} ssc.start(); Thread.sleep(10000); ssc.stop(stopSparkContext = false); Thread.sleep(60000) {quote} gives the following error {quote} 14/08/06 18:37:13 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 10047 ms on localhost (1/1) 14/08/06 18:37:13 INFO DAGScheduler: Stage 0 (runJob at ReceiverTracker.scala:275) finished in 10.056 s 14/08/06 18:37:13 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 14/08/06 18:37:13 INFO SparkContext: Job finished: runJob at ReceiverTracker.scala:275, took 10.179263 s 14/08/06 18:37:13 INFO ReceiverTracker: All of the receivers have been terminated 14/08/06 18:37:13 WARN ReceiverTracker: All of the receivers have not deregistered, Map(0 -> ReceiverInfo(0,SocketReceiver-0,null,false,localhost,Stopped by driver,)) 14/08/06 18:37:13 INFO ReceiverTracker: ReceiverTracker stopped 14/08/06 18:37:13 INFO JobGenerator: Stopping JobGenerator immediately 14/08/06 18:37:13 INFO RecurringTimer: Stopped timer for JobGenerator after time 1407375433000 14/08/06 18:37:13 INFO JobGenerator: Stopped JobGenerator 14/08/06 18:37:13 INFO JobScheduler: Stopped JobScheduler 14/08/06 18:37:13 INFO StreamingContext: StreamingContext stopped successfully 14/08/06 18:37:43 INFO SocketReceiver: Stopped receiving 14/08/06 18:37:43 INFO SocketReceiver: Closed socket to localhost:9999 {quote}

    Apache's JIRA Issue Tracker | 2 years ago | Tathagata Das
    org.apache.spark.SparkException: Job cancelled because SparkContext was shut down
  6. 0

    GitHub comment 54#71801487

    GitHub | 2 years ago | velvia
    org.apache.spark.SparkException: Job cancelled because SparkContext was shut down

  1. tyson925 1 times, last 6 days ago
  2. Nikolay Rybak 2 times, last 1 month ago
  3. tyson925 12 times, last 2 months ago
3 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. org.apache.spark.SparkException

    Job cancelled because SparkContext was shut down

    at org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply()
  2. Spark
    DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply
    1. org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:702)
    2. org.apache.spark.scheduler.DAGScheduler$$anonfun$cleanUpAfterSchedulerStop$1.apply(DAGScheduler.scala:701)
    2 frames
  3. Scala
    HashSet.foreach
    1. scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
    1 frame
  4. Spark
    DAGSchedulerEventProcessActor.postStop
    1. org.apache.spark.scheduler.DAGScheduler.cleanUpAfterSchedulerStop(DAGScheduler.scala:701)
    2. org.apache.spark.scheduler.DAGSchedulerEventProcessActor.postStop(DAGScheduler.scala:1428)
    2 frames
  5. Akka Actor
    Actor$class.aroundPostStop
    1. akka.actor.Actor$class.aroundPostStop(Actor.scala:475)
    1 frame
  6. Spark
    DAGSchedulerEventProcessActor.aroundPostStop
    1. org.apache.spark.scheduler.DAGSchedulerEventProcessActor.aroundPostStop(DAGScheduler.scala:1375)
    1 frame
  7. Akka Actor
    ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec
    1. akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210)
    2. akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172)
    3. akka.actor.ActorCell.terminate(ActorCell.scala:369)
    4. akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
    5. akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
    6. akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
    7. akka.dispatch.Mailbox.run(Mailbox.scala:219)
    8. akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
    8 frames
  8. Scala
    ForkJoinWorkerThread.run
    1. scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    2. scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    3. scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    4. scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
    4 frames