java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1}

JIRA | Siyuan Zhou | 3 years ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    First reported here: https://groups.google.com/forum/#!topic/mongodb-user/lZaFgOdrnYk Steps to reproduce: Insert the below document into the persons collection {code} db.persons.insert({name: "John", age: 35.1}); {code} Create a table in hive from the persons collection in mongo {code} create table double (name STRING, age DOUBLE) STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler' TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/test.persons'); {code} Setup a table in hive to write to a mongo collection {code} create external table double_ext (name string, age double) stored by 'com.mongodb.hadoop.hive.MongoStorageHandler' TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/test.ext'); {code} Insert data into that table {code} insert into table double_ext select "John", 35.1 from double; {code} The operation fails and this stack trace is printed {noformat} Diagnostic Messages for this Task: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1} at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:394) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:671) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144) ... 8 more Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be cast to java.lang.Double at com.mongodb.hadoop.hive.BSONSerDe.serializePrimitive(BSONSerDe.java:592) at com.mongodb.hadoop.hive.BSONSerDe.serializeObject(BSONSerDe.java:454) at com.mongodb.hadoop.hive.BSONSerDe.serializeStruct(BSONSerDe.java:525) at com.mongodb.hadoop.hive.BSONSerDe.serialize(BSONSerDe.java:442) at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652) ... 9 more {noformat}

    JIRA | 3 years ago | Siyuan Zhou
    java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1}
  2. 0

    First reported here: https://groups.google.com/forum/#!topic/mongodb-user/lZaFgOdrnYk Steps to reproduce: Insert the below document into the persons collection {code} db.persons.insert({name: "John", age: 35.1}); {code} Create a table in hive from the persons collection in mongo {code} create table double (name STRING, age DOUBLE) STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler' TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/test.persons'); {code} Setup a table in hive to write to a mongo collection {code} create external table double_ext (name string, age double) stored by 'com.mongodb.hadoop.hive.MongoStorageHandler' TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/test.ext'); {code} Insert data into that table {code} insert into table double_ext select "John", 35.1 from double; {code} The operation fails and this stack trace is printed {noformat} Diagnostic Messages for this Task: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1} at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:162) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:394) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:671) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144) ... 8 more Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be cast to java.lang.Double at com.mongodb.hadoop.hive.BSONSerDe.serializePrimitive(BSONSerDe.java:592) at com.mongodb.hadoop.hive.BSONSerDe.serializeObject(BSONSerDe.java:454) at com.mongodb.hadoop.hive.BSONSerDe.serializeStruct(BSONSerDe.java:525) at com.mongodb.hadoop.hive.BSONSerDe.serialize(BSONSerDe.java:442) at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652) ... 9 more {noformat}

    JIRA | 3 years ago | Siyuan Zhou
    java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"John","age":35.1}

    Root Cause Analysis

    1. java.lang.ClassCastException

      org.apache.hadoop.hive.serde2.io.DoubleWritable cannot be cast to java.lang.Double

      at com.mongodb.hadoop.hive.BSONSerDe.serializePrimitive()
    2. com.mongodb.hadoop
      BSONSerDe.serialize
      1. com.mongodb.hadoop.hive.BSONSerDe.serializePrimitive(BSONSerDe.java:592)
      2. com.mongodb.hadoop.hive.BSONSerDe.serializeObject(BSONSerDe.java:454)
      3. com.mongodb.hadoop.hive.BSONSerDe.serializeStruct(BSONSerDe.java:525)
      4. com.mongodb.hadoop.hive.BSONSerDe.serialize(BSONSerDe.java:442)
      4 frames
    3. Hive Query Language
      ExecMapper.map
      1. org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:617)
      2. org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
      3. org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
      4. org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
      5. org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
      6. org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
      7. org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
      8. org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
      9. org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
      10. org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:652)
      11. org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
      11 frames
    4. Hadoop
      Child$4.run
      1. org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
      2. org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
      3. org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
      4. org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      4 frames
    5. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:394)
      2 frames
    6. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
      1 frame
    7. Hadoop
      Child.main
      1. org.apache.hadoop.mapred.Child.main(Child.java:249)
      1 frame