java.io.FileNotFoundException: /tmp/spark-local-20140324074221-b8f1/01/temp_1ab674f9-4556-4239-9f21-688dfc9f17d2 (Too many open files)

nabble.com | 4 months ago
  1. 0

    Apache Spark User List - distinct on huge dataset

    nabble.com | 4 months ago
    java.io.FileNotFoundException: /tmp/spark-local-20140324074221-b8f1/01/temp_1ab674f9-4556-4239-9f21-688dfc9f17d2 (Too many open files)
  2. 0

    Spark Executor: Managed memory leak detected

    Stack Overflow | 1 year ago | auxdx
    java.io.FileNotFoundException: /tmp/blockmgr-3acec504-4a55-4aa8-a3e5-dda97ce5d055/03/temp_shuffle_cb37f147-c055-4014-a6ae-fd505cb49f57 (Too many open files)
  3. 0

    What could have caused this error - Java.io.FileNotFoundException: /tmp/spark...?

    Google Groups | 1 year ago | Cuong Tham
    java.io.FileNotFoundException: /tmp/spark-90507c1d-e983-422d-9e01-74ff0a5a2806/executor-360151d5-6b83-4e3e-a0c6-6ddc955cb16c/blockmgr-bca2bde9-212f-4219-af8b-ef0415d60bfa/26/temp_shuffle_98deadd9-f7c3-4a12-9a30-7749f097b5c8 (No such file or directory)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Scala Spark FileNotFoundException in windows shared folder

    Stack Overflow | 10 months ago | Paner
    java.io.FileNotFoundException: /media/Ji-share/tmp/blockmgr-2860ab59-1d38-4265-a709-c8372895ace6/13/shuffle_11_49_0.index (Input/output error)
  6. 0

    How to achieve Hadoop/HDFS Streaming

    codegur.net | 8 months ago
    java.io.FileNotFoundException: /tmp/blockmgr-3acec504-4a55-4aa8-a3e5-dda97ce5d055/03/temp_shuffle_cb37f147-c055-4014-a6ae-fd505cb49f57 (Too many open files)

  1. tyson925 49 times, last 1 week ago
  2. tyson925 95 times, last 1 month ago
  3. eti22 3 times, last 2 weeks ago
  4. MoYapro 1 times, last 1 month ago
  5. musketyr 3 times, last 1 month ago
23 more registered users
71 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.io.FileNotFoundException

    /tmp/spark-local-20140324074221-b8f1/01/temp_1ab674f9-4556-4239-9f21-688dfc9f17d2 (Too many open files)

    at java.io.FileOutputStream.openAppend()
  2. Java RT
    FileOutputStream.<init>
    1. java.io.FileOutputStream.openAppend(Native Method)
    2. java.io.FileOutputStream.<init>(FileOutputStream.java:192)
    2 frames
  3. Spark
    Executor$TaskRunner.run
    1. org.apache.spark.storage.DiskBlockObjectWriter.open(BlockObjectWriter.scala:113)
    2. org.apache.spark.storage.DiskBlockObjectWriter.write(BlockObjectWriter.scala:174)
    3. org.apache.spark.util.collection.ExternalAppendOnlyMap.spill(ExternalAppendOnlyMap.scala:191)
    4. org.apache.spark.util.collection.ExternalAppendOnlyMap.insert(ExternalAppendOnlyMap.scala:141)
    5. org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:59)
    6. org.apache.spark.rdd.PairRDDFunctions$$anonfun$1.apply(PairRDDFunctions.scala:95)
    7. org.apache.spark.rdd.PairRDDFunctions$$anonfun$1.apply(PairRDDFunctions.scala:94)
    8. org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:471)
    9. org.apache.spark.rdd.RDD$$anonfun$3.apply(RDD.scala:471)
    10. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:34)
    11. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:241)
    12. org.apache.spark.rdd.RDD.iterator(RDD.scala:232)
    13. org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:161)
    14. org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:102)
    15. org.apache.spark.scheduler.Task.run(Task.scala:53)
    16. org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:213)
    17. org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:49)
    18. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178)
    18 frames
  4. Java RT
    Thread.run
    1. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    3. java.lang.Thread.run(Thread.java:662)
    3 frames