org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object

Cask Community Issue Tracker | Shankar Selvam | 2 years ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    [CDAP-2991] Explore doesn't work when it launches map-reduce job - Cask Community Issue Tracker

    cask.co | 1 year ago
    org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object
  2. 0

    Explore has an issue after the recent updates to twill-snapshot and dependency changes in cdap. {noformat} 2015-07-10 19:37:35,384 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) ... 9 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) ... 14 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) ... 17 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154) ... 22 more Caused by: java.lang.NoSuchMethodError: org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.addListener(Lcom/google/common/util/concurrent/Service$Listener;Ljava/util/concurrent/Executor;)V at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>(DefaultZKClientService.java:403) at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>(DefaultZKClientService.java:392) at org.apache.twill.internal.zookeeper.DefaultZKClientService.<init>(DefaultZKClientService.java:98) at org.apache.twill.zookeeper.ZKClientService$Builder.build(ZKClientService.java:101) at co.cask.cdap.common.guice.ZKClientModule.provideZKClientService(ZKClientModule.java:53) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.google.inject.internal.ProviderMethod.get(ProviderMethod.java:104) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:65) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024) at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974) at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013) at co.cask.cdap.hive.context.ContextManager.createContext(ContextManager.java:149) at co.cask.cdap.hive.context.ContextManager.getContext(ContextManager.java:85) at co.cask.cdap.hive.stream.StreamSerDe.initialize(StreamSerDe.java:88) at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:307) at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:353) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:123) ... 22 more {noformat} In org.apache.twill.internal.zookeeper.DefaultZKClientService, we use com.google.common.util.concurrent.AbstractService#addListener, which isn't present in older guava versions. the explore-query container's classpath has job.jar before the jar files in the container directory, and since job.jar has guava-11, we use that and get NoSuchMethodError exception. however, we did not update guava version from twill-0.5.0 to twill-0.6.0-SNAPSHOT, its just the new changes started exposing the issue.

    Cask Community Issue Tracker | 2 years ago | Shankar Selvam
    org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object
  3. 0

    Explore has an issue after the recent updates to twill-snapshot and dependency changes in cdap. {noformat} 2015-07-10 19:37:35,384 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) ... 9 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) ... 14 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) ... 17 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154) ... 22 more Caused by: java.lang.NoSuchMethodError: org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.addListener(Lcom/google/common/util/concurrent/Service$Listener;Ljava/util/concurrent/Executor;)V at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>(DefaultZKClientService.java:403) at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>(DefaultZKClientService.java:392) at org.apache.twill.internal.zookeeper.DefaultZKClientService.<init>(DefaultZKClientService.java:98) at org.apache.twill.zookeeper.ZKClientService$Builder.build(ZKClientService.java:101) at co.cask.cdap.common.guice.ZKClientModule.provideZKClientService(ZKClientModule.java:53) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.google.inject.internal.ProviderMethod.get(ProviderMethod.java:104) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:65) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024) at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974) at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013) at co.cask.cdap.hive.context.ContextManager.createContext(ContextManager.java:149) at co.cask.cdap.hive.context.ContextManager.getContext(ContextManager.java:85) at co.cask.cdap.hive.stream.StreamSerDe.initialize(StreamSerDe.java:88) at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:307) at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:353) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:123) ... 22 more {noformat} In org.apache.twill.internal.zookeeper.DefaultZKClientService, we use com.google.common.util.concurrent.AbstractService#addListener, which isn't present in older guava versions. the explore-query container's classpath has job.jar before the jar files in the container directory, and since job.jar has guava-11, we use that and get NoSuchMethodError exception. however, we did not update guava version from twill-0.5.0 to twill-0.6.0-SNAPSHOT, its just the new changes started exposing the issue.

    Cask Community Issue Tracker | 2 years ago | Shankar Selvam
    org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Error in configuring object
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Getting Stack trace: ExitCodeException exitCode=255 during MapReduce

    Stack Overflow | 9 months ago | Abhay Singh
    java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JVMId.(Lorg/apache/hadoop/mapred/JobID;ZJ)V

    Root Cause Analysis

    1. java.lang.NoSuchMethodError

      org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.addListener(Lcom/google/common/util/concurrent/Service$Listener;Ljava/util/concurrent/Executor;)V

      at org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>()
    2. org.apache.twill
      ZKClientService$Builder.build
      1. org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>(DefaultZKClientService.java:403)
      2. org.apache.twill.internal.zookeeper.DefaultZKClientService$ServiceDelegate.<init>(DefaultZKClientService.java:392)
      3. org.apache.twill.internal.zookeeper.DefaultZKClientService.<init>(DefaultZKClientService.java:98)
      4. org.apache.twill.zookeeper.ZKClientService$Builder.build(ZKClientService.java:101)
      4 frames
    3. co.cask.cdap
      ZKClientModule.provideZKClientService
      1. co.cask.cdap.common.guice.ZKClientModule.provideZKClientService(ZKClientModule.java:53)
      1 frame
    4. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    5. Google Guice - Core Library
      InjectorImpl.getInstance
      1. com.google.inject.internal.ProviderMethod.get(ProviderMethod.java:104)
      2. com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
      3. com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
      4. com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      5. com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
      6. com.google.inject.Scopes$1$1.get(Scopes.java:65)
      7. com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
      8. com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
      9. com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
      10. com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
      11. com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
      11 frames
    6. co.cask.cdap
      StreamSerDe.initialize
      1. co.cask.cdap.hive.context.ContextManager.createContext(ContextManager.java:149)
      2. co.cask.cdap.hive.context.ContextManager.getContext(ContextManager.java:85)
      3. co.cask.cdap.hive.stream.StreamSerDe.initialize(StreamSerDe.java:88)
      3 frames
    7. Hive Query Language
      ExecMapper.configure
      1. org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:307)
      2. org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:353)
      3. org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:123)
      3 frames
    8. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    9. Hadoop
      ReflectionUtils.newInstance
      1. org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
      2. org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
      3. org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
      3 frames
    10. Hadoop
      MapRunner.configure
      1. org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
      1 frame
    11. Java RT
      Method.invoke
      1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      4. java.lang.reflect.Method.invoke(Method.java:606)
      4 frames
    12. Hadoop
      ReflectionUtils.newInstance
      1. org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
      2. org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
      3. org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
      3 frames
    13. Hadoop
      YarnChild$2.run
      1. org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
      2. org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
      3. org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
      3 frames
    14. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:415)
      2 frames
    15. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
      1 frame
    16. Hadoop
      YarnChild.main
      1. org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
      1 frame