java.lang.IllegalArgumentException: requirement failed at scala.Predef$.require(Predef.scala:221)

Stack Overflow | Frankie | 7 months ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    Spark Streaming "ERROR JobScheduler: error in job generator"

    Stack Overflow | 7 months ago | Frankie
    java.lang.IllegalArgumentException: requirement failed at scala.Predef$.require(Predef.scala:221)

    Root Cause Analysis

    1. java.lang.IllegalArgumentException

      requirement failed at scala.Predef$.require(Predef.scala:221)

      at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.cleanupOldBatches()
    2. Spark Project Streaming
      JobGenerator$$anon$1.onReceive
      1. org.apache.spark.streaming.scheduler.ReceivedBlockTracker.cleanupOldBatches(ReceivedBlockTracker.scala:166)
      2. org.apache.spark.streaming.scheduler.ReceiverTracker.cleanupOldBlocksAndBatches(ReceiverTracker.scala:223)
      3. org.apache.spark.streaming.scheduler.JobGenerator.clearMetadata(JobGenerator.scala:272)
      4. org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:182)
      5. org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:87)
      6. org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:86)
      6 frames
    3. Spark
      EventLoop$$anon$1.run
      1. org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
      1 frame