java.io.IOException: Filesystem closed

spark-user | Akhil Das | 2 years ago
  1. Speed up your debug routine!

    Automated exception search integrated into your IDE

  2. 0

    Re: java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]

    spark-user | 2 years ago | Akhil Das
    java.io.IOException: Filesystem closed
  3. 0

    Re: java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]

    spark-user | 2 years ago | Akhil Das
    java.io.IOException: Filesystem closed

    1 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.IOException

      Filesystem closed

      at org.apache.hadoop.hdfs.DFSClient.checkOpen()
    2. Apache Hadoop HDFS
      DFSInputStream.close
      1. org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:707)
      2. org.apache.hadoop.hdfs.DFSInputStream.close(DFSInputStream.java:619)
      2 frames
    3. Java RT
      FilterInputStream.close
      1. java.io.FilterInputStream.close(FilterInputStream.java:155)
      1 frame
    4. Hadoop
      LineReader.close
      1. org.apache.hadoop.util.LineReader.close(LineReader.java:150)
      1 frame
    5. Hadoop
      LineRecordReader.close
      1. org.apache.hadoop.mapred.LineRecordReader.close(LineRecordReader.java:244)
      1 frame
    6. Spark
      TaskContext$$anonfun$executeOnCompleteCallbacks$1.apply
      1. org.apache.spark.rdd.HadoopRDD$$anon$1.close(HadoopRDD.scala:211)
      2. org.apache.spark.util.NextIterator.closeIfNeeded(NextIterator.scala:63)
      3. org.apache.spark.rdd.HadoopRDD$$anon$1$$anonfun$1.apply$mcV$sp(HadoopRDD.scala:196)
      4. org.apache.spark.TaskContext$$anonfun$executeOnCompleteCallbacks$1.apply(TaskContext.scala:63)
      5. org.apache.spark.TaskContext$$anonfun$executeOnCompleteCallbacks$1.apply(TaskContext.scala:63)
      5 frames
    7. Scala
      ArrayBuffer.foreach
      1. scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
      2. scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
      2 frames
    8. Spark
      Executor$TaskRunner.run
      1. org.apache.spark.TaskContext.executeOnCompleteCallbacks(TaskContext.scala:63)
      2. org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:204)
      3. org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
      4. org.apache.spark.scheduler.Task.run(Task.scala:51)
      5. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183)
      5 frames
    9. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      3. java.lang.Thread.run(Thread.java:662)
      3 frames