org.apache.spark.shuffle.FetchFailedException: Error in opening FileSegmentManagedBuffer{file=/tmp/spark-b5507e64-ccaf-4082-8c7e-373bd7945133/executor-690303d2-011f-4a03-a21f-62479c2b9cb2/blockmgr-42f641bb-db6e-4fd8-913b-c343e792a704/31/shuffle_0_78_0.data, offset=608, length=16}

GitHub | car2008 | 3 months ago
  1. 0

    GitHub comment 197#244256304

    GitHub | 3 months ago | car2008
    org.apache.spark.shuffle.FetchFailedException: Error in opening FileSegmentManagedBuffer{file=/tmp/spark-b5507e64-ccaf-4082-8c7e-373bd7945133/executor-690303d2-011f-4a03-a21f-62479c2b9cb2/blockmgr-42f641bb-db6e-4fd8-913b-c343e792a704/31/shuffle_0_78_0.data, offset=608, length=16}
  2. 0

    Spark Job failed on YARN -

    Stack Overflow | 12 months ago | Shankar
    org.apache.spark.shuffle.FetchFailedException: Error in opening FileSegmentManagedBuffer{file=/hdfs1/yarn/nm/usercache/phdpentcustcdibtch/appcache/application_1449986083135_60217/blockmgr-34a2e882-6b36-42c6-bcff-03d9bc5ef80b/0c/shuffle_4_42_0.data, offset=5899394, length=46751}
  3. 0

    GitHub comment 200#106711720

    GitHub | 2 years ago | linxixiong
    org.apache.spark.shuffle.FetchFailedException: Error in opening FileSegmentManagedBuffer{file=/data2/hadoop/hd_space/tmp/nm-local-dir/usercache/linxixiong/appcache/application_1432021926412_9808/spark-82b1829f-7662-4480-82ca-9abf7cc6618d/10/shuffle_5_106_0.data, offset=17462106, length=2529501}
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    GitHub comment 200#88290674

    GitHub | 2 years ago | thanigaiv
    org.apache.spark.shuffle.FetchFailedException: Error in opening FileSegmentManagedBuffer{file=/data/yarn/nm/usercache/root/appcache/application_1426812029369_0893/spark-local-20150331165623-6233/19/shuffle_5_0_0.data, offset=3437, length=532}
  6. 0

    Intermittent java.io.FileNotFoundException on release

    GitHub | 12 months ago | samthebest
    java.io.FileNotFoundException: /var/lib/jenkins/workspace/altmetrics/target/scala-2.10/classes/com/MYCLASS$$anonfun$main$5$$anonfun$apply$3$$anonfun$apply$5.class (No such file or directory)

  1. gehel 2 times, last 1 week ago
  2. Kialandei 100 times, last 1 week ago
  3. Reich 1 times, last 2 weeks ago
  4. bluebunnty 1 times, last 2 weeks ago
  5. Aria Pahlavan 1 times, last 2 weeks ago
43 more registered users
52 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.io.FileNotFoundException

    /tmp/spark-b5507e64-ccaf-4082-8c7e-373bd7945133/executor-690303d2-011f-4a03-a21f-62479c2b9cb2/blockmgr-42f641bb-db6e-4fd8-913b-c343e792a704/31/shuffle_0_78_0.data (there is no file or directory)

    at java.io.FileInputStream.open0()
  2. Java RT
    FileInputStream.<init>
    1. java.io.FileInputStream.open0(Native Method)
    2. java.io.FileInputStream.open(FileInputStream.java:195)
    3. java.io.FileInputStream.<init>(FileInputStream.java:138)
    3 frames
  3. Spark
    FileSegmentManagedBuffer.createInputStream
    1. org.apache.spark.network.buffer.FileSegmentManagedBuffer.createInputStream(FileSegmentManagedBuffer.java:98)
    1 frame
  4. Spark
    ShuffleBlockFetcherIterator.next
    1. org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:304)
    2. org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:51)
    2 frames
  5. Scala
    Iterator$$anon$11.hasNext
    1. scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
    2. scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
    3. scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    3 frames
  6. Spark
    Executor$TaskRunner.run
    1. org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:32)
    2. org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)
    3. org.apache.spark.util.collection.ExternalAppendOnlyMap.insertAll(ExternalAppendOnlyMap.scala:152)
    4. org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:45)
    5. org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:89)
    6. org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:98)
    7. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    8. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    9. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
    10. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    11. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    12. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
    13. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    14. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    15. org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:87)
    16. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    17. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    18. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
    19. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    20. org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    21. org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
    22. org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
    23. org.apache.spark.scheduler.Task.run(Task.scala:89)
    24. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
    24 frames
  7. Java RT
    Thread.run
    1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    3. java.lang.Thread.run(Thread.java:745)
    3 frames