org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8

JIRA | Lucas Bremgartner | 3 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    In my setup I use logstash to handle syslog input. One of my network appliances (mail-relay) sends logmessages with "strange" or may be wrong encoding, which leads to crashing logstash. I think, the above message is not validly encoded. In my opinion, if logstash is used in mission-critical environments, it should be tollerant to such "wrongly encoded" events. I checked with the file output and the redis output. Error Message (Redis): {noformat} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8 at json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71) at LogStash::Event.to_json(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:169) at LogStash::Outputs::Redis.receive(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/redis.rb:150) at RUBY.initialize((eval):17) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:255) at LogStash::Pipeline.output(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:247) at RUBY.outputworker(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:212) at RUBY.start_outputs(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:140) {noformat} Error Message (File): {noformat} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8 at json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71) at LogStash::Event.to_json(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:169) at LogStash::Outputs::File.receive(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/file.rb:68) at RUBY.initialize((eval):28) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:255) at LogStash::Pipeline.output(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:247) at RUBY.outputworker(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:212) at RUBY.start_outputs(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:140) {noformat} I recorded the corrupted messages with tcpdump and was able to reproduce the exception. Steps to reproduce: 1. Start logstash with following config-file: {noformat} input { udp { port => 514 type => syslog } } output { file { path => "/tmp/out.log" flush_interval => 0 } } {noformat} 2. Send correct message to syslog {noformat} echo -en "syslog correct" | nc -u localhost 514 {noformat} (Check /tmp/out.log for the message 3. Send corrupted message to syslog {noformat} echo -en "syslog corrupted: \xfc" | nc -u localhost 514 {noformat} (Nothing is written to /tmp/out.log, logstash crashes with above error-message) I double-checked with an alternative logstash configuration with charset "ASCII-8BIT". In this case the logline wasn't written to /tmp/out.log but logstash at least kept working normaly. Alternative logstash config: {noformat} input { udp { port => 515 type => syslog codec => plain { charset => "ASCII-8BIT" } } } output { file { path => "/tmp/fortimail-log/out.log" flush_interval => 0 } } {noformat}

    JIRA | 3 years ago | Lucas Bremgartner
    org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8

    Root Cause Analysis

    1. org.jruby.exceptions.RaiseException

      (GeneratorError) source sequence is illegal/malformed utf-8

      at json.ext.GeneratorMethods$RbHash.to_json()
    2. json.ext
      GeneratorMethods$RbHash.to_json
      1. json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71)
      1 frame