javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '9.3.168.192.in-addr.arpa' A dig and nslookup both show that the reverse and forward look-ups both get good responses with no errors from within the cluster. Shortly after these messages, the job runs...but every once in awhile we get a NPE: Exception in thread "main" java.lang.NullPointerException

Server Fault | phodamentals | 4 years ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    Reverse and Forward DNS set up correctly but sometimes MapReduce job fails

    Server Fault | 4 years ago | phodamentals
    javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '9.3.168.192.in-addr.arpa' A dig and nslookup both show that the reverse and forward look-ups both get good responses with no errors from within the cluster. Shortly after these messages, the job runs...but every once in awhile we get a NPE: Exception in thread "main" java.lang.NullPointerException

    Root Cause Analysis

    1. javax.naming.NameNotFoundException

      DNS name not found [response code 3]; remaining name '9.3.168.192.in-addr.arpa' A dig and nslookup both show that the reverse and forward look-ups both get good responses with no errors from within the cluster. Shortly after these messages, the job runs...but every once in awhile we get a NPE: Exception in thread "main" java.lang.NullPointerException

      at org.apache.hadoop.net.DNS.reverseDns()
    2. Hadoop
      DNS.reverseDns
      1. org.apache.hadoop.net.DNS.reverseDns(DNS.java:93)
      1 frame
    3. HBase
      TableInputFormatBase.getSplits
      1. org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:219)
      2. org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:184)
      2 frames
    4. Hadoop
      JobClient$2.run
      1. org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1063)
      2. org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1080)
      3. org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
      4. org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:992)
      5. org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
      5 frames
    5. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:415)
      2 frames
    6. Hadoop
      UserGroupInformation.doAs
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
      1 frame
    7. Hadoop
      Job.waitForCompletion
      1. org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
      2. org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
      3. org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
      3 frames