java.lang.AssertionError: assertion failed

Apache's JIRA Issue Tracker | Thomas Graves | 3 years ago
  1. 0

    [SPARK-1407] EventLogging to HDFS doesn't work properly on yarn - ASF JIRA

    apache.org | 1 year ago
    java.lang.AssertionError: assertion failed
  2. Speed up your debug routine!

    Automated exception search integrated into your IDE

  3. 0

    When running on spark on yarn and accessing an HDFS file (like in the SparkHdfsLR example) while using the event logging configured to write logs to HDFS, it throws an exception at the end of the application. SPARK_JAVA_OPTS=-Dspark.eventLog.enabled=true -Dspark.eventLog.dir=hdfs:///history/spark/ 14/04/03 13:41:31 INFO yarn.ApplicationMaster$$anon$1: Invoking sc stop from shutdown hook Exception in thread "Thread-41" java.io.IOException: Filesystem closed at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:398) at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:1465) at org.apache.hadoop.hdfs.DFSOutputStream.sync(DFSOutputStream.java:1450) at org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:116) at org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:137) at org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:137) at scala.Option.foreach(Option.scala:236) at org.apache.spark.util.FileLogger.flush(FileLogger.scala:137) at org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:69) at org.apache.spark.scheduler.EventLoggingListener.onApplicationEnd(EventLoggingListener.scala:101) at org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$13.apply(SparkListenerBus.scala:67) at org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$13.apply(SparkListenerBus.scala:67) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.SparkListenerBus$class.postToAll(SparkListenerBus.scala:67) at org.apache.spark.scheduler.LiveListenerBus.postToAll(LiveListenerBus.scala:31) at org.apache.spark.scheduler.LiveListenerBus.post(LiveListenerBus.scala:78) at org.apache.spark.SparkContext.postApplicationEnd(SparkContext.scala:1081) at org.apache.spark.SparkContext.stop(SparkContext.scala:828) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$1.run(ApplicationMaster.scala:460)

    Apache's JIRA Issue Tracker | 3 years ago | Thomas Graves
    java.lang.AssertionError: assertion failed
  4. 0

    spark在那里指定master URL呢? - Spark - 知乎

    zhihu.com | 1 year ago
    java.lang.AssertionError: assertion failed

  1. tyson925 1 times, last 5 days ago
  2. rp 1 times, last 1 week ago
  3. poroszd 1 times, last 4 months ago
  4. rp 1 times, last 9 months ago
17 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.AssertionError

    assertion failed

    at scala.Predef$.assert()
  2. Scala
    Predef$.assert
    1. scala.Predef$.assert(Predef.scala:165)
    1 frame
  3. Spark Project YARN Stable API
    ApplicationMaster.main
    1. org.apache.spark.deploy.yarn.ApplicationMaster.waitForSparkContextInitialized(ApplicationMaster.scala:234)
    2. org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:107)
    3. org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:480)
    4. org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
    4 frames