kafka.message.InvalidMessageException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • Unfortunately I wasn't able to figure out exactly how I accomplished this, but somehow I managed to get one or more poison messages in my test topic using fairly simple single threaded SimpleProducer/SimpleConsumer code. In this state, I could see several hundreds of thousands of messages in the topic via JMX and adding new messages appeared to succeed from the API calls, however each add resulted in the following error: [2010-12-08 14:56:04,952] ERROR kafka.message.InvalidMessageException (kafka.network.Processor) kafka.message.InvalidMessageException at kafka.log.Log$$anonfun$append$1.apply(Log.scala:187) at kafka.log.Log$$anonfun$append$1.apply(Log.scala:185) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:29) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at kafka.message.MessageSet.foreach(MessageSet.scala:63) at kafka.log.Log.append(Log.scala:185) at kafka.server.KafkaRequestHandlers.handleProducerRequest(KafkaRequestHandlers.scala:56) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$1.apply(KafkaRequestHandlers.scala:40) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$1.apply(KafkaRequestHandlers.scala:40) at kafka.network.Processor.handle(SocketServer.scala:268) at kafka.network.Processor.read(SocketServer.scala:291) at kafka.network.Processor.run(SocketServer.scala:202) at java.lang.Thread.run(Thread.java:662) Additionally, no consumers were functional, they were all blocked and not draining the existing messages. Stopping the service did not get things healthy, but stopping and removing the logs fixed the problem. Not sure what the design intent is, but it would seem that failure to process a message internally within the server should cause the message to be rejected at the client layer.
    via by Erik Onnen,
  • Unfortunately I wasn't able to figure out exactly how I accomplished this, but somehow I managed to get one or more poison messages in my test topic using fairly simple single threaded SimpleProducer/SimpleConsumer code. In this state, I could see several hundreds of thousands of messages in the topic via JMX and adding new messages appeared to succeed from the API calls, however each add resulted in the following error: [2010-12-08 14:56:04,952] ERROR kafka.message.InvalidMessageException (kafka.network.Processor) kafka.message.InvalidMessageException at kafka.log.Log$$anonfun$append$1.apply(Log.scala:187) at kafka.log.Log$$anonfun$append$1.apply(Log.scala:185) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:29) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at kafka.message.MessageSet.foreach(MessageSet.scala:63) at kafka.log.Log.append(Log.scala:185) at kafka.server.KafkaRequestHandlers.handleProducerRequest(KafkaRequestHandlers.scala:56) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$1.apply(KafkaRequestHandlers.scala:40) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$1.apply(KafkaRequestHandlers.scala:40) at kafka.network.Processor.handle(SocketServer.scala:268) at kafka.network.Processor.read(SocketServer.scala:291) at kafka.network.Processor.run(SocketServer.scala:202) at java.lang.Thread.run(Thread.java:662) Additionally, no consumers were functional, they were all blocked and not draining the existing messages. Stopping the service did not get things healthy, but stopping and removing the logs fixed the problem. Not sure what the design intent is, but it would seem that failure to process a message internally within the server should cause the message to be rejected at the client layer.
    via by Erik Onnen,
    • kafka.message.InvalidMessageException at kafka.log.Log$$anonfun$append$1.apply(Log.scala:187) at kafka.log.Log$$anonfun$append$1.apply(Log.scala:185) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:29) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at kafka.message.MessageSet.foreach(MessageSet.scala:63) at kafka.log.Log.append(Log.scala:185) at kafka.server.KafkaRequestHandlers.handleProducerRequest(KafkaRequestHandlers.scala:56) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$1.apply(KafkaRequestHandlers.scala:40) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$1.apply(KafkaRequestHandlers.scala:40) at kafka.network.Processor.handle(SocketServer.scala:268) at kafka.network.Processor.read(SocketServer.scala:291) at kafka.network.Processor.run(SocketServer.scala:202) at java.lang.Thread.run(Thread.java:662)
    No Bugmate found.