java.lang.OutOfMemoryError: error while calling spill() on org.apache.spark.shuffle.sort.ShuffleExternalSorter@6438a928 : No space left on device

Hot problem!
This exception has a lot of traffic. You can help 25 users by writing a short tip.

Searched on Google with the first line of a JAVA stack trace?

We can recommend more relevant solutions and speed up debugging when you paste your entire stack trace with the exception message. Try a sample exception.

Recommended solutions based on your search

Solutions on the web

via Stack Overflow by Adetiloye Philip Kehinde
, 1 year ago
error while calling spill() on org.apache.spark.shuffle.sort.ShuffleExternalSorter@6438a928 : No space left on device
via Apache's JIRA Issue Tracker by Devaraj K, 11 months ago
error while calling spill() on org.apache.spark.shuffle.sort.ShuffleExternalSorter@714b17b0 : null
java.lang.OutOfMemoryError: error while calling spill() on org.apache.spark.shuffle.sort.ShuffleExternalSorter@6438a928 : No space left on device	at org.apache.spark.memory.TaskMemoryManager.acquireExecutionMemory(TaskMemoryManager.java:179)	at org.apache.spark.memory.TaskMemoryManager.allocatePage(TaskMemoryManager.java:245)	at org.apache.spark.memory.MemoryConsumer.allocatePage(MemoryConsumer.java:121)	at org.apache.spark.shuffle.sort.ShuffleExternalSorter.acquireNewPageIfNecessary(ShuffleExternalSorter.java:364)	at org.apache.spark.shuffle.sort.ShuffleExternalSorter.insertRecord(ShuffleExternalSorter.java:387)	at org.apache.spark.shuffle.sort.UnsafeShuffleWriter.insertRecordIntoSorter(UnsafeShuffleWriter.java:235)	at org.apache.spark.shuffle.sort.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:162)	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)	at org.apache.spark.scheduler.Task.run(Task.scala:85)	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)	at java.lang.Thread.run(Thread.java:745)