org.springframework.messaging.MessagingException: Consuming from Kafka failed

Spring JIRA | Jaroslaw Palka | 2 years ago
  1. 0

    I have found that when you define two consumer contexts and two inbound channel adapters, like this: {code:xml} <kafka:consumer-context id="response-consumer-context" zookeeper-connect="zookeeper-connect"> <kafka:consumer-configurations> <kafka:consumer-configuration group-id="response-gateway-group" key-decoder="string-decoder" value-decoder="string-decoder"> <kafka:topic streams="1" id="response" /> </kafka:consumer-configuration> </kafka:consumer-configurations> </kafka:consumer-context> <kafka:consumer-context id="error-consumer-context" zookeeper-connect="zookeeper-connect"> <kafka:consumer-configurations> <kafka:consumer-configuration group-id="error-gateway-group" key-decoder="string-decoder" value-decoder="string-decoder"> <kafka:topic streams="1" id="error" /> </kafka:consumer-configuration> </kafka:consumer-configurations> </kafka:consumer-context> <kafka:inbound-channel-adapter id="responseKafkaDestination" kafka-consumer-context-ref="response-consumer-context"> <int:poller fixed-delay="10" /> </kafka:inbound-channel-adapter> <kafka:inbound-channel-adapter id="errorKafkaDestination" kafka-consumer-context-ref="error-consumer-context"> <int:poller fixed-delay="10" /> </kafka:inbound-channel-adapter> {code} you get following error: {noformat} [task-scheduler-6] ERROR org.springframework.integration.handler.LoggingHandler - org.springframework.messaging.MessagingException: Consuming from Kafka failed at org.springframework.integration.kafka.support.ConsumerConfiguration.executeTasks(ConsumerConfiguration.java:110) at org.springframework.integration.kafka.support.ConsumerConfiguration.receive(ConsumerConfiguration.java:86) at org.springframework.integration.kafka.support.KafkaConsumerContext.receive(KafkaConsumerContext.java:56) at org.springframework.integration.kafka.inbound.KafkaHighLevelConsumerMessageSource.receive(KafkaHighLevelConsumerMessageSource.java:41) at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124) at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:190) at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:55) at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:149) at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146) at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:284) at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52) at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49) at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:278) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Iterator is in failed state at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) at java.util.concurrent.FutureTask.get(Unknown Source) at org.springframework.integration.kafka.support.ConsumerConfiguration.executeTasks(ConsumerConfiguration.java:97) ... 23 more Caused by: java.lang.IllegalStateException: Iterator is in failed state at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:54) at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) at org.springframework.integration.kafka.support.ConsumerConfiguration$1.call(ConsumerConfiguration.java:67) at org.springframework.integration.kafka.support.ConsumerConfiguration$1.call(ConsumerConfiguration.java:61) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) ... 3 more {noformat}

    Spring JIRA | 2 years ago | Jaroslaw Palka
    org.springframework.messaging.MessagingException: Consuming from Kafka failed
  2. 0

    I have found that when you define two consumer contexts and two inbound channel adapters, like this: {code:xml} <kafka:consumer-context id="response-consumer-context" zookeeper-connect="zookeeper-connect"> <kafka:consumer-configurations> <kafka:consumer-configuration group-id="response-gateway-group" key-decoder="string-decoder" value-decoder="string-decoder"> <kafka:topic streams="1" id="response" /> </kafka:consumer-configuration> </kafka:consumer-configurations> </kafka:consumer-context> <kafka:consumer-context id="error-consumer-context" zookeeper-connect="zookeeper-connect"> <kafka:consumer-configurations> <kafka:consumer-configuration group-id="error-gateway-group" key-decoder="string-decoder" value-decoder="string-decoder"> <kafka:topic streams="1" id="error" /> </kafka:consumer-configuration> </kafka:consumer-configurations> </kafka:consumer-context> <kafka:inbound-channel-adapter id="responseKafkaDestination" kafka-consumer-context-ref="response-consumer-context"> <int:poller fixed-delay="10" /> </kafka:inbound-channel-adapter> <kafka:inbound-channel-adapter id="errorKafkaDestination" kafka-consumer-context-ref="error-consumer-context"> <int:poller fixed-delay="10" /> </kafka:inbound-channel-adapter> {code} you get following error: {noformat} [task-scheduler-6] ERROR org.springframework.integration.handler.LoggingHandler - org.springframework.messaging.MessagingException: Consuming from Kafka failed at org.springframework.integration.kafka.support.ConsumerConfiguration.executeTasks(ConsumerConfiguration.java:110) at org.springframework.integration.kafka.support.ConsumerConfiguration.receive(ConsumerConfiguration.java:86) at org.springframework.integration.kafka.support.KafkaConsumerContext.receive(KafkaConsumerContext.java:56) at org.springframework.integration.kafka.inbound.KafkaHighLevelConsumerMessageSource.receive(KafkaHighLevelConsumerMessageSource.java:41) at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124) at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:190) at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:55) at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:149) at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146) at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:284) at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52) at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49) at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:278) at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Iterator is in failed state at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) at java.util.concurrent.FutureTask.get(Unknown Source) at org.springframework.integration.kafka.support.ConsumerConfiguration.executeTasks(ConsumerConfiguration.java:97) ... 23 more Caused by: java.lang.IllegalStateException: Iterator is in failed state at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:54) at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38) at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46) at org.springframework.integration.kafka.support.ConsumerConfiguration$1.call(ConsumerConfiguration.java:67) at org.springframework.integration.kafka.support.ConsumerConfiguration$1.call(ConsumerConfiguration.java:61) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) ... 3 more {noformat}

    Spring JIRA | 2 years ago | Jaroslaw Palka
    org.springframework.messaging.MessagingException: Consuming from Kafka failed
  3. 0

    Consuming from Kafka failed Iterator is in failed state

    Stack Overflow | 2 years ago | Amit
    java.lang.IllegalStateException: Iterator is in failed state
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    [Kafka-users] High Level Consumer Iterator IllegalStateException Issue - Grokbase

    grokbase.com | 8 months ago
    java.lang.IllegalStateException: Iterator is in failed state
  6. 0

    kafka.common.MessageSizeTooLargeException

    GitHub | 3 months ago | Tracy6465
    java.lang.IllegalStateException: Iterator is in failed state

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.IllegalStateException

      Iterator is in failed state

      at kafka.utils.IteratorTemplate.hasNext()
    2. Apache Kafka
      ConsumerIterator.next
      1. kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:54)
      2. kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
      3. kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
      3 frames
    3. org.springframework.integration
      ConsumerConfiguration$1.call
      1. org.springframework.integration.kafka.support.ConsumerConfiguration$1.call(ConsumerConfiguration.java:67)
      2. org.springframework.integration.kafka.support.ConsumerConfiguration$1.call(ConsumerConfiguration.java:61)
      2 frames
    4. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
      2. java.util.concurrent.FutureTask.run(Unknown Source)
      3. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
      4. java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      5. java.lang.Thread.run(Unknown Source)
      5 frames