org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8

JIRA | Lucas Bremgartner | 3 years ago
  1. 0

    In my setup I use logstash to handle syslog input. One of my network appliances (mail-relay) sends logmessages with "strange" or may be wrong encoding, which leads to crashing logstash. I think, the above message is not validly encoded. In my opinion, if logstash is used in mission-critical environments, it should be tollerant to such "wrongly encoded" events. I checked with the file output and the redis output. Error Message (Redis): {noformat} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8 at json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71) at LogStash::Event.to_json(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:169) at LogStash::Outputs::Redis.receive(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/redis.rb:150) at RUBY.initialize((eval):17) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:255) at LogStash::Pipeline.output(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:247) at RUBY.outputworker(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:212) at RUBY.start_outputs(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:140) {noformat} Error Message (File): {noformat} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8 at json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71) at LogStash::Event.to_json(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:169) at LogStash::Outputs::File.receive(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/file.rb:68) at RUBY.initialize((eval):28) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:255) at LogStash::Pipeline.output(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:247) at RUBY.outputworker(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:212) at RUBY.start_outputs(file:/opt/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:140) {noformat} I recorded the corrupted messages with tcpdump and was able to reproduce the exception. Steps to reproduce: 1. Start logstash with following config-file: {noformat} input { udp { port => 514 type => syslog } } output { file { path => "/tmp/out.log" flush_interval => 0 } } {noformat} 2. Send correct message to syslog {noformat} echo -en "syslog correct" | nc -u localhost 514 {noformat} (Check /tmp/out.log for the message 3. Send corrupted message to syslog {noformat} echo -en "syslog corrupted: \xfc" | nc -u localhost 514 {noformat} (Nothing is written to /tmp/out.log, logstash crashes with above error-message) I double-checked with an alternative logstash configuration with charset "ASCII-8BIT". In this case the logline wasn't written to /tmp/out.log but logstash at least kept working normaly. Alternative logstash config: {noformat} input { udp { port => 515 type => syslog codec => plain { charset => "ASCII-8BIT" } } } output { file { path => "/tmp/fortimail-log/out.log" flush_interval => 0 } } {noformat}

    JIRA | 3 years ago | Lucas Bremgartner
    org.jruby.exceptions.RaiseException: (GeneratorError) source sequence is illegal/malformed utf-8
  2. 0

    GitHub comment 5#58538037

    GitHub | 2 years ago | rtyler
    org.jruby.embed.InvokeFailedException: (Encoding::UndefinedConversionError) ""\xC3\xA9"" from UTF-8 to US-ASCII
  3. 0

    Generating a .pom from a .gemspec with non-ASCII characters in it causes a 500

    GitHub | 2 years ago | rtyler
    org.jruby.embed.InvokeFailedException: (Encoding::UndefinedConversionError) ""\xC3\xA9"" from UTF-8 to US-ASCII
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Error after macOS Sierra upgrade

    GitHub | 2 months ago | diegoscl
    org.jruby.exceptions.RaiseException: (SyntaxError) <script>:2: invalid multibyte char (UTF-8)
  6. 0

    GitHub comment 156#152183948

    GitHub | 1 year ago | klokovas
    org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':aggregate'.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.jruby.exceptions.RaiseException

      (GeneratorError) source sequence is illegal/malformed utf-8

      at json.ext.GeneratorMethods$RbHash.to_json()
    2. json.ext
      GeneratorMethods$RbHash.to_json
      1. json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71)
      1 frame