org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `tv_sec' for ["2013-11-18T18:48:51.503Z", "2013-11-18T18:48:51.504Z"]:Array @40000000528a66c911da88a4 at LogStash::Event.sprintf(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/event.rb:226)

JIRA | Ryan Bellows | 3 years ago
  1. 0

    I've been having some issues getting the multiline filter to be stable over time. It will run ok for a day or two, or even 5 minutes, then crashes with this error: {code} @40000000528a66c911ac3d14 Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `tv_sec' for ["2013-11-18T18:48:51.503Z", "2013-11-18T18:48:51.504Z"]:Array @40000000528a66c911da88a4 at LogStash::Event.sprintf(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/event.rb:226) @40000000528a66c911da9074 at org.jruby.RubyString.gsub(org/jruby/RubyString.java:3065) @40000000528a66c911da9074 at LogStash::Event.sprintf(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/event.rb:212) @40000000528a66c911dac724 at LogStash::Outputs::ElasticSearch.receive(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/outputs/elasticsearch.rb:153) @40000000528a66c911dacef4 at LogStash::Outputs::Base.handle(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/outputs/base.rb:85) @40000000528a66c911dad6c4 at RUBY.worker_setup(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/outputs/base.rb:77) {code} I can comment out the redis input that is supplying the logs that go through the multiline filter, and then logstash is stable. Here are the relevant entries in my config file: logstash: 1.2.2 java: 1.7.0_05-b05 ruby: ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux] OS: CentOS 6.2 {code} redis { host => "my.redis.host.net" data_type => "list" type => "php_errors" key => "logstash-php-errors" threads => 4 } filer { ## start php error stuffs if [type] == "php_errors" { multiline { pattern => "^%{PHPTIME} " patterns_dir => [ "/usr/local/etc/cfg/logstash-indexer/patterns" ] negate => true what => previous } mutate { replace => [ "@source_host", "%{host}" ] add_tag => "php_errors" remove_tag => "multiline" } } } output { elasticsearch { cluster => "logs" node_name => "log1-logstash-es" workers => 10 } } {code}

    JIRA | 3 years ago | Ryan Bellows
    org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `tv_sec' for ["2013-11-18T18:48:51.503Z", "2013-11-18T18:48:51.504Z"]:Array @40000000528a66c911da88a4 at LogStash::Event.sprintf(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/event.rb:226)
  2. 0

    While backfilling Logstash with old logs, Logstash crashed with the following stack trace: {noformat} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `<<' for nil:NilClass at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:369) at Grok::Match.each_capture(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/jls-grok-0.10.10/lib/grok/pure/match.rb:23) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at Grok::Match.each_capture(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/jls-grok-0.10.10/lib/grok/pure/match.rb:21) at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:317) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:301) at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1257) at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:292) at LogStash::FilterWorker.flusher(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:50) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at LogStash::FilterWorker.flusher(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:48) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at LogStash::FilterWorker.flusher(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:45) at RUBY.run(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:29) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:249) at RUBY.interval(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/stud-0.0.8/lib/stud/interval.rb:11) at RUBY.run(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:29) {noformat} From what I can see, event.fields[key] ||= [] (grok.rb:368) isn't behaving as expected and when Logstash tries to push to event.fields[key] on the next line, it's still nil. Perhaps I'm wrong, I don't really know Ruby, I can just about read it, and Googled for information I didn't know. Thanks, Daniel EDIT: Just to add, those line numbers are from the grok.rb contained in the 1.1.9 jar and don't correspond with the current git tree.

    JIRA | 4 years ago | Daniel Brierton
    org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `<<' for nil:NilClass at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:369) at Grok::Match.each_capture(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/jls-grok-0.10.10/lib/grok/pure/match.rb:23)
  3. 0

    While backfilling Logstash with old logs, Logstash crashed with the following stack trace: {noformat} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `<<' for nil:NilClass at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:369) at Grok::Match.each_capture(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/jls-grok-0.10.10/lib/grok/pure/match.rb:23) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at Grok::Match.each_capture(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/jls-grok-0.10.10/lib/grok/pure/match.rb:21) at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:317) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:301) at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1257) at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:292) at LogStash::FilterWorker.flusher(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:50) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at LogStash::FilterWorker.flusher(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:48) at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1612) at LogStash::FilterWorker.flusher(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:45) at RUBY.run(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:29) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:249) at RUBY.interval(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/stud-0.0.8/lib/stud/interval.rb:11) at RUBY.run(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:29) {noformat} From what I can see, event.fields[key] ||= [] (grok.rb:368) isn't behaving as expected and when Logstash tries to push to event.fields[key] on the next line, it's still nil. Perhaps I'm wrong, I don't really know Ruby, I can just about read it, and Googled for information I didn't know. Thanks, Daniel EDIT: Just to add, those line numbers are from the grok.rb contained in the 1.1.9 jar and don't correspond with the current git tree.

    JIRA | 4 years ago | Daniel Brierton
    org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `<<' for nil:NilClass at LogStash::Filters::Grok.filter(file:/opt/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/grok.rb:369) at Grok::Match.each_capture(jar:file:/opt/logstash/logstash-1.1.9-monolithic.jar!/gems/jls-grok-0.10.10/lib/grok/pure/match.rb:23)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Hi, we very frequently (every couple of hours) run into the following error as soon as we allow more than one server to connect and deliver its logs via Lumberjack to logstash via the same port: {code} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (ConcurrencyError) Detected invalid array contents due to unsynchronized modifications with concurrent users at org.jruby.RubyArray.<<(org/jruby/RubyArray.java:1147) at LogStash::Codecs::Multiline.buffer(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:158) at LogStash::Codecs::Multiline.do_previous(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:179) at org.jruby.RubyMethod.call(org/jruby/RubyMethod.java:132) at LogStash::Codecs::Multiline.decode(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:153) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/inputs/lumberjack.rb:45) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:271) at Lumberjack::Connection.data(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:241) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:223) at Lumberjack::Parser.data_field_value(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:182) at Lumberjack::Parser.feed(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:93) at Lumberjack::Parser.compressed_payload(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:198) at Lumberjack::Parser.feed(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:93) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:220) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:59) {code} Lumberjack input configuration is {code} input { lumberjack { port => 3301 type => "tomcat" charset => "ISO-8859-1" ssl_certificate => "/etc/pki/logstash/logstash.crt" ssl_key => "/etc/pki/logstash/logstash.key" codec => multiline { charset => "ISO-8859-1" pattern => "^\[%{LOGLEVEL}%{SPACE}\]" negate => true what => "previous" } } } {code} We can omit the exception error only if we spent an extra logstash input configuration with another port for each delivering server but this is not really an option. We get this error as well with apache-Logs (even with two servers only). Any help is appreciated. Best regards, Andreas Huck

    JIRA | 3 years ago | A. Huck
    org.jruby.exceptions.RaiseException: (ConcurrencyError) Detected invalid array contents due to unsynchronized modifications with concurrent users at org.jruby.RubyArray.<<(org/jruby/RubyArray.java:1147) at LogStash::Codecs::Multiline.buffer(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:158) at LogStash::Codecs::Multiline.do_previous(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:179)
  6. 0

    Hi, we very frequently (every couple of hours) run into the following error as soon as we allow more than one server to connect and deliver its logs via Lumberjack to logstash via the same port: {code} Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (ConcurrencyError) Detected invalid array contents due to unsynchronized modifications with concurrent users at org.jruby.RubyArray.<<(org/jruby/RubyArray.java:1147) at LogStash::Codecs::Multiline.buffer(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:158) at LogStash::Codecs::Multiline.do_previous(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:179) at org.jruby.RubyMethod.call(org/jruby/RubyMethod.java:132) at LogStash::Codecs::Multiline.decode(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:153) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/inputs/lumberjack.rb:45) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:271) at Lumberjack::Connection.data(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:241) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:223) at Lumberjack::Parser.data_field_value(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:182) at Lumberjack::Parser.feed(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:93) at Lumberjack::Parser.compressed_payload(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:198) at Lumberjack::Parser.feed(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:93) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:220) at RUBY.run(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/lumberjack/server.rb:59) {code} Lumberjack input configuration is {code} input { lumberjack { port => 3301 type => "tomcat" charset => "ISO-8859-1" ssl_certificate => "/etc/pki/logstash/logstash.crt" ssl_key => "/etc/pki/logstash/logstash.key" codec => multiline { charset => "ISO-8859-1" pattern => "^\[%{LOGLEVEL}%{SPACE}\]" negate => true what => "previous" } } } {code} We can omit the exception error only if we spent an extra logstash input configuration with another port for each delivering server but this is not really an option. We get this error as well with apache-Logs (even with two servers only). Any help is appreciated. Best regards, Andreas Huck

    JIRA | 3 years ago | A. Huck
    org.jruby.exceptions.RaiseException: (ConcurrencyError) Detected invalid array contents due to unsynchronized modifications with concurrent users at org.jruby.RubyArray.<<(org/jruby/RubyArray.java:1147) at LogStash::Codecs::Multiline.buffer(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:158) at LogStash::Codecs::Multiline.do_previous(file:/opt/logstash/logstash-1.2.2-flatjar.jar!/logstash/codecs/multiline.rb:179)

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.jruby.exceptions.RaiseException

      (NoMethodError) undefined method `tv_sec' for ["2013-11-18T18:48:51.503Z", "2013-11-18T18:48:51.504Z"]:Array @40000000528a66c911da88a4 at LogStash::Event.sprintf(file:/usr/local/bin/logstash-1.2.2-flatjar.jar!/logstash/event.rb:226)

      at org.jruby.RubyString.gsub()
    2. JRuby Main Maven Artifact
      RubyString.gsub
      1. org.jruby.RubyString.gsub(org/jruby/RubyString.java:3065)
      1 frame