java.lang.ArrayIndexOutOfBoundsException: 8388607

spark-user | Ted Yu | 1 year ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    Re: java.lang.ArrayIndexOutOfBoundsException when attempting broadcastjoin

    spark-user | 1 year ago | Ted Yu
    java.lang.ArrayIndexOutOfBoundsException: 8388607

    1 unregistered visitors

    Root Cause Analysis

    1. java.lang.ArrayIndexOutOfBoundsException

      8388607

      at com.esotericsoftware.kryo.util.IdentityObjectIntMap.clear()
    2. Kryo
      Kryo.writeClassAndObject
      1. com.esotericsoftware.kryo.util.IdentityObjectIntMap.clear(IdentityObjectIntMap.java:345)
      2. com.esotericsoftware.kryo.util.MapReferenceResolver.reset(MapReferenceResolver.java:47)
      3. com.esotericsoftware.kryo.Kryo.reset(Kryo.java:804)
      4. com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:570)
      4 frames
    3. Spark
      SparkContext.broadcast
      1. org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194)
      2. org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:203)
      3. org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:102)
      4. org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:85)
      5. org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
      6. org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63)
      7. org.apache.spark.SparkContext.broadcast(SparkContext.scala:1326)
      7 frames
    4. Spark Project SQL
      BroadcastHashJoin$$anonfun$broadcastFuture$1.apply
      1. org.apache.spark.sql.execution.joins.BroadcastHashJoin$$anonfun$broadcastFuture$1$$anonfun$apply$1.apply(BroadcastHashJoin.scala:91)
      2. org.apache.spark.sql.execution.joins.BroadcastHashJoin$$anonfun$broadcastFuture$1$$anonfun$apply$1.apply(BroadcastHashJoin.scala:79)
      3. org.apache.spark.sql.execution.SQLExecution$.withExecutionId(SQLExecution.scala:100)
      4. org.apache.spark.sql.execution.joins.BroadcastHashJoin$$anonfun$broadcastFuture$1.apply(BroadcastHashJoin.scala:79)
      5. org.apache.spark.sql.execution.joins.BroadcastHashJoin$$anonfun$broadcastFuture$1.apply(BroadcastHashJoin.scala:79)
      5 frames
    5. Scala
      Future$PromiseCompletingRunnable.run
      1. scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
      2. scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
      2 frames
    6. Java RT
      Thread.run
      1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      3. java.lang.Thread.run(Thread.java:745)
      3 frames