org.apache.spark.shuffle.FetchFailedException: Error in opening FileSegmentManagedBuffer{file=/tmp/spark-b5507e64-ccaf-4082-8c7e-373bd7945133/executor-690303d2-011f-4a03-a21f-62479c2b9cb2/blockmgr-42f641bb-db6e-4fd8-913b-c343e792a704/31/shuffle_0_78_0.data, offset=608, length=16}

Searched on Google with the first line of a JAVA stack trace?

We can recommend more relevant solutions and speed up debugging when you paste your entire stack trace with the exception message. Try a sample exception.

Recommended solutions based on your search

Samebug tips

,

Make sure the file resides in the project folder if you're referencing it via a relative path. If not, use the absolute path to the file with the file extension.

Solutions on the web

via GitHub by car2008
, 1 year ago
Error in opening FileSegmentManagedBuffer{file=/tmp/spark-b5507e64-ccaf-4082-8c7e-373bd7945133/executor-690303d2-011f-4a03-a21f-62479c2b9cb2/blockmgr-42f641bb-db6e-4fd8-913b-c343e792a704/31/shuffle_0_78_0.data, offset=608, length=16}
via Stack Overflow by shad
, 2 years ago
Failed to connect to <executor>
via Google Groups by Cuong Tham, 2 years ago
/tmp/spark-90507c1d-e983-422d-9e01-74ff0a5a2806/executor-360151d5-6b83-4e3e-a0c6-6ddc955cb16c/blockmgr-bca2bde9-212f-4219-af8b-ef0415d60bfa/31/shuffle_16_0_0.index (No such file or directory)
via Google Groups by Digambar Bhat, 7 months ago
Error in opening FileSegmentManagedBuffer{file=/tmp/spark-a01777ba-3b4a-44c6-aeff-44f30a9d4f3d/executor-f085563e-fe3b-4716-9938-fe726ff3d317/blockmgr-69cb01c0-2fc5-4194-8b2f-ec8e799ebe57/15/shuffle_0_1_0.data, offset=0, length=81610299}
via Stack Overflow by Faisal R
, 7 months ago
Adjusted frame length exceeds 2147483647: 2505744379 - discarded
via Stack Overflow by theMadKing
, 1 year ago
/tmp/spark-e4238a07-bf89-4a7d-9de3-176cba0a076d/executor-93a11b25-1cb1-4e13-b1b6-b1d64d3a9602/blockmgr-c11cd046-7c37-429c-9137-936d391d3cbc/30/shuffle_0_0_0.index (No such file or directory)
java.io.FileNotFoundException: /tmp/spark-b5507e64-ccaf-4082-8c7e-373bd7945133/executor-690303d2-011f-4a03-a21f-62479c2b9cb2/blockmgr-42f641bb-db6e-4fd8-913b-c343e792a704/31/shuffle_0_78_0.data (there is no file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:304)
at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:51)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:32)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)
at org.apache.spark.util.collection.ExternalAppendOnlyMap.insertAll(ExternalAppendOnlyMap.scala:152)
at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:45)
at org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:89)
at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:98)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:87)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Users with the same issue

3 times, 2 days ago
Once, 3 days ago
Once, 1 week ago
3 times, 1 week ago

Write tip

Know the solutions? Share your knowledge to help other developers to debug faster.