java.lang.RuntimeException: error rate while reading input records crossed threshold

Google Groups | varnit | 4 years ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    issue with ProtobufPigLoader loading lzo compressed file with one protobuf record per line

    Google Groups | 4 years ago | varnit
    java.lang.RuntimeException: error rate while reading input records crossed threshold

    Root Cause Analysis

    1. java.lang.Exception

      Unknown error

      at com.twitter.elephantbird.mapreduce.input.LzoRecordReader$InputErrorTracker.incErrors()
    2. Elephant Bird Core
      LzoBinaryB64LineRecordReader.nextKeyValue
      1. com.twitter.elephantbird.mapreduce.input.LzoRecordReader$InputErrorTracker.incErrors(LzoRecordReader.java:138)
      2. com.twitter.elephantbird.mapreduce.input.LzoBinaryB64LineRecordReader.nextKeyValue(LzoBinaryB64LineRecordReader.java:135)
      2 frames
    3. com.twitter.elephantbird
      ProtobufPigLoader.getNext
      1. com.twitter.elephantbird.pig.load.LzoBaseLoadFunc.getNextBinaryValue(LzoBaseLoadFunc.java:107)
      2. com.twitter.elephantbird.pig.load.ProtobufPigLoader.getNext(ProtobufPigLoader.java:62)
      2 frames
    4. org.apache.pig
      PigRecordReader.nextKeyValue
      1. org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
      1 frame
    5. Hadoop
      Child$4.run
      1. org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:540)
      2. org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
      3. org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
      4. org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
      5. org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
      6. org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      6 frames
    6. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:396)
      2 frames
    7. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
      1 frame
    8. Hadoop
      Child.main
      1. org.apache.hadoop.mapred.Child.main(Child.java:249)
      1 frame