kafka.common.QueueFullException: Event queue is full of unsent messages, could not send event: KeyedMessage(logs.user-v2,system:services:appfabric,system:services:appfabric,[B@24471929)

Google Groups | ron cai | 3 months ago
  1. 0

    cdap-master can't be restarted

    Google Groups | 3 months ago | ron cai
    kafka.common.QueueFullException: Event queue is full of unsent messages, could not send event: KeyedMessage(logs.user-v2,system:services:appfabric,system:services:appfabric,[B@24471929)

    Root Cause Analysis

    1. kafka.common.QueueFullException

      Event queue is full of unsent messages, could not send event: KeyedMessage(logs.user-v2,system:services:appfabric,system:services:appfabric,[B@24471929)

      at kafka.producer.Producer$$anonfun$asyncSend$1.apply()
    2. Apache Kafka
      Producer$$anonfun$asyncSend$1.apply
      1. kafka.producer.Producer$$anonfun$asyncSend$1.apply(Producer.scala:113)[org.apache.kafka.kafka_2.10-0.8.2.2.jar:na]
      2. kafka.producer.Producer$$anonfun$asyncSend$1.apply(Producer.scala:91)[org.apache.kafka.kafka_2.10-0.8.2.2.jar:na]
      2 frames
    3. Scala
      WrappedArray.foreach
      1. scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)[org.scala-lang.scala-library-2.10.4.jar:na]
      2. scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)[org.scala-lang.scala-library-2.10.4.jar:na]
      2 frames
    4. Apache Kafka
      Producer.send
      1. kafka.producer.Producer.asyncSend(Producer.scala:91)[org.apache.kafka.kafka_2.10-0.8.2.2.jar:na]
      2. kafka.producer.Producer.send(Producer.scala:78)[org.apache.kafka.kafka_2.10-0.8.2.2.jar:na]
      3. kafka.javaapi.producer.Producer.send(Producer.scala:33)[org.apache.kafka.kafka_2.10-0.8.2.2.jar:na]
      3 frames
    5. co.cask.cdap
      LogAppender.append
      1. co.cask.cdap.logging.appender.kafka.SimpleKafkaProducer.publish(SimpleKafkaProducer.java:62)[co.cask.cdap.cdap-watchdog-3.5.0.jar:na]
      2. co.cask.cdap.logging.appender.kafka.KafkaLogAppender.append(KafkaLogAppender.java:61)[co.cask.cdap.cdap-watchdog-3.5.0.jar:na]
      3. co.cask.cdap.logging.appender.LogAppender.append(LogAppender.java:27)[co.cask.cdap.cdap-watchdog-3.5.0.jar:na]
      3 frames
    6. Logback Core Module
      AppenderAttachableImpl.appendLoopOnAppenders
      1. ch.qos.logback.core.AppenderBase.doAppend(AppenderBase.java:85)[ch.qos.logback.logback-core-1.0.9.jar:na]
      2. ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)[ch.qos.logback.logback-core-1.0.9.jar:na]
      2 frames
    7. Logback Classic Module
      Logger.info
      1. ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:280)[ch.qos.logback.logback-classic-1.0.9.jar:na]
      2. ch.qos.logback.classic.Logger.callAppenders(Logger.java:267)[ch.qos.logback.logback-classic-1.0.9.jar:na]
      3. ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:449)[ch.qos.logback.logback-classic-1.0.9.jar:na]
      4. ch.qos.logback.classic.Logger.filterAndLog_1(Logger.java:421)[ch.qos.logback.logback-classic-1.0.9.jar:na]
      5. ch.qos.logback.classic.Logger.info(Logger.java:611)[ch.qos.logback.logback-classic-1.0.9.jar:na]
      5 frames
    8. co.cask.cdap
      AbstractAuthorizationService.shutDown
      1. co.cask.cdap.security.authorization.AbstractAuthorizationService.shutDown(AbstractAuthorizationService.java:139)[co.cask.cdap.cdap-security-3.5.0.jar:na]
      1 frame
    9. Guava
      AbstractScheduledService$1$3.run
      1. com.google.common.util.concurrent.AbstractScheduledService$1$3.run(AbstractScheduledService.java:219)[com.google.guava.guava-13.0.1.jar:na]
      1 frame
    10. Java RT
      Thread.run
      1. java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[na:1.8.0_60]
      2. java.util.concurrent.FutureTask.run(FutureTask.java:266)[na:1.8.0_60]
      3. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)[na:1.8.0_60]
      4. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[na:1.8.0_60]
      5. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[na:1.8.0_60]
      6. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[na:1.8.0_60]
      7. java.lang.Thread.run(Thread.java:745)[na:1.8.0_60]
      7 frames