java.lang.AssertionError: assertion failed

Apache's JIRA Issue Tracker | Thomas Graves | 3 years ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. Speed up your debug routine!

    Automated exception search integrated into your IDE

  2. 0

    When running on spark on yarn and accessing an HDFS file (like in the SparkHdfsLR example) while using the event logging configured to write logs to HDFS, it throws an exception at the end of the application. SPARK_JAVA_OPTS=-Dspark.eventLog.enabled=true -Dspark.eventLog.dir=hdfs:///history/spark/ 14/04/03 13:41:31 INFO yarn.ApplicationMaster$$anon$1: Invoking sc stop from shutdown hook Exception in thread "Thread-41" java.io.IOException: Filesystem closed at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:398) at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:1465) at org.apache.hadoop.hdfs.DFSOutputStream.sync(DFSOutputStream.java:1450) at org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:116) at org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:137) at org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:137) at scala.Option.foreach(Option.scala:236) at org.apache.spark.util.FileLogger.flush(FileLogger.scala:137) at org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:69) at org.apache.spark.scheduler.EventLoggingListener.onApplicationEnd(EventLoggingListener.scala:101) at org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$13.apply(SparkListenerBus.scala:67) at org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$13.apply(SparkListenerBus.scala:67) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.SparkListenerBus$class.postToAll(SparkListenerBus.scala:67) at org.apache.spark.scheduler.LiveListenerBus.postToAll(LiveListenerBus.scala:31) at org.apache.spark.scheduler.LiveListenerBus.post(LiveListenerBus.scala:78) at org.apache.spark.SparkContext.postApplicationEnd(SparkContext.scala:1081) at org.apache.spark.SparkContext.stop(SparkContext.scala:828) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$1.run(ApplicationMaster.scala:460)

    Apache's JIRA Issue Tracker | 3 years ago | Thomas Graves
    java.lang.AssertionError: assertion failed
  3. 0

    spark在那里指定master URL呢? - Spark - 知乎

    zhihu.com | 2 years ago
    java.lang.AssertionError: assertion failed

  1. jstrayer 9 times, last 2 weeks ago
  2. tyson925 1 times, last 5 months ago
  3. rp 1 times, last 5 months ago
  4. poroszd 1 times, last 8 months ago
18 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.AssertionError

    assertion failed

    at scala.Predef$.assert()
  2. Scala
    Predef$.assert
    1. scala.Predef$.assert(Predef.scala:165)
    1 frame
  3. Spark Project YARN Stable API
    ApplicationMaster.main
    1. org.apache.spark.deploy.yarn.ApplicationMaster.waitForSparkContextInitialized(ApplicationMaster.scala:234)
    2. org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:107)
    3. org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:480)
    4. org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
    4 frames