java.lang.RuntimeException: Can't acquire 1049600 bytes memory to build hash relation, got 74332 bytes

Stack Overflow | Idris Hanafi | 4 months ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    Increase Java Memory on Spark for Building Large Hash Relations

    Stack Overflow | 4 months ago | Idris Hanafi
    java.lang.RuntimeException: Can't acquire 1049600 bytes memory to build hash relation, got 74332 bytes

    Root Cause Analysis

    1. org.apache.spark.SparkException

      Can't acquire 1049600 bytes memory to build hash relation, got 74332 bytes

      at org.apache.spark.sql.execution.joins.LongToUnsafeRowMap.ensureAcquireMemory()
    2. Spark Project SQL
      LongToUnsafeRowMap.init
      1. org.apache.spark.sql.execution.joins.LongToUnsafeRowMap.ensureAcquireMemory(HashedRelation.scala:414)
      2. org.apache.spark.sql.execution.joins.LongToUnsafeRowMap.init(HashedRelation.scala:424)
      2 frames