java.lang.OutOfMemoryError: GC overhead limit exceeded

Stack Overflow | user2459075 | 8 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    I'm having a "GC overhead limit exceeded" on Spark 1.5.2 (reproductible every ~20 hours)

    Stack Overflow | 8 months ago | user2459075
    java.lang.OutOfMemoryError: GC overhead limit exceeded
  2. 0

    OOM error

    GitHub | 11 months ago | jpdna
    java.lang.OutOfMemoryError: Java heap space
  3. 0

    Re: Spark Application Master on Yarn client mode - Virtual memory limit

    spark-user | 1 year ago | Olivier Girardot
    java.lang.OutOfMemoryError: Java heap space
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Re: Spark Application Master on Yarn client mode - Virtual memory limit

    spark-user | 1 year ago | Olivier Girardot
    java.lang.OutOfMemoryError: Java heap space
  6. 0

    Re: Spark Application Master on Yarn client mode - Virtual memory limit

    spark-user | 1 year ago | Olivier Girardot
    java.lang.OutOfMemoryError: Java heap space

    1 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.OutOfMemoryError

      GC overhead limit exceeded

      at java.util.IdentityHashMap.resize()
    2. Java RT
      IdentityHashMap.put
      1. java.util.IdentityHashMap.resize(IdentityHashMap.java:471)
      2. java.util.IdentityHashMap.put(IdentityHashMap.java:440)
      2 frames
    3. Spark
      SizeEstimator$$anonfun$visitSingleObject$1.apply
      1. org.apache.spark.util.SizeEstimator$SearchState.enqueue(SizeEstimator.scala:159)
      2. org.apache.spark.util.SizeEstimator$$anonfun$visitSingleObject$1.apply(SizeEstimator.scala:203)
      3. org.apache.spark.util.SizeEstimator$$anonfun$visitSingleObject$1.apply(SizeEstimator.scala:202)
      3 frames
    4. Scala
      List.foreach
      1. scala.collection.immutable.List.foreach(List.scala:318)
      1 frame
    5. Spark
      RDD.iterator
      1. org.apache.spark.util.SizeEstimator$.visitSingleObject(SizeEstimator.scala:202)
      2. org.apache.spark.util.SizeEstimator$.org$apache$spark$util$SizeEstimator$$estimate(SizeEstimator.scala:186)
      3. org.apache.spark.util.SizeEstimator$.estimate(SizeEstimator.scala:54)
      4. org.apache.spark.util.collection.SizeTracker$class.takeSample(SizeTracker.scala:78)
      5. org.apache.spark.util.collection.SizeTracker$class.afterUpdate(SizeTracker.scala:70)
      6. org.apache.spark.util.collection.SizeTrackingVector.$plus$eq(SizeTrackingVector.scala:31)
      7. org.apache.spark.storage.MemoryStore.unrollSafely(MemoryStore.scala:278)
      8. org.apache.spark.CacheManager.putInBlockManager(CacheManager.scala:171)
      9. org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:78)
      10. org.apache.spark.rdd.RDD.iterator(RDD.scala:262)
      11. org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:87)
      12. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
      13. org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
      14. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      15. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
      16. org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)
      17. org.apache.spark.rdd.RDD.iterator(RDD.scala:262)
      18. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      19. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
      20. org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
      21. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      22. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
      23. org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
      24. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
      25. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300)
      26. org.apache.spark.rdd.RDD.iterator(RDD.scala:264)
      26 frames